My adventures with *args and **kwargs in Python

The Protoss of Python (Warning: Starcraft References ahead!)

I first really truly came face-to-face with *args and **kwargs when working on a project where we were writing automated tests and were attempting to write our own Python Decorators on top of currently existing test cases to seamlessly be able to create a classification to specify whether to run them or not on the go. At the time, I admit, I didn’t really understand them fully, mostly I think because the concept of Decorators themselves was a bit new to me and the difficulty of grasping these overshadowed the realization of how easy understanding the usage of *args and **kwargs were.

Recently, I reunited with *args and **kwargs and have to say the concept of these are pretty easy to grasp. Really easy. I really like to compare them with the Protoss race in Starcraft. Easy to use and powerful, but so easy to OVERUSE and resort to certain-to-fail fallback strategies due to the deceiving ease of implementing. But with the right strategy and planning and full knowledge of these Pythonic creatures, they can definitely be the winner in your project.

What are you wearing? (Basic Function parameters)

I think, in this case, examples are probably the best way of describing WHAT they are.

First, here is a simple Python function/method that accepts items of clothing and spits out (returns) a list that comprises of the person’s outfit.

def outfit(self, socks_color, shoes_color, pants_color, shirt_color):
    return 'You are wearing %s socks, %s shoes, %s pants, and a %s shirt' % (socks_color, shoes_color, pants_color, shirt_color)

If you call this function with the following:

print outfit('grey','black', 'red', 'blue')

the output will be

>>>You are wearing grey socks, black shoes, red pants, and a blue shirt

Accessorize! (Optional Function parameters)

Some people (*ahem myself) are fashionably challenged and only wear a shirt, jeans, socks and shoes, but some others like to pretty themselves up. So we should let people wear a necklace or watch in this function.

The new function will be:

def outfit(self, socks_color, shoes_color, pants_color, shirt_color, accessory=None):
    #if accessory = None
    if not accessory:
        return 'You are wearing %s socks, %s shoes, %s pants, and a %s shirt' % (socks_color, shoes_color, pants_color, shirt_color)
    else:
        return 'You are wearing %s socks, %s shoes, %s pants, and a %s shirt and a nice %s' % (socks_color, shoes_color, pants_color, shirt_color, accessory)

and call it with

print outfit('grey','black', 'red', 'blue', 'watch')

which will produce…

>>>You are wearing grey socks, black shoes, red pants, and a blue shirt and a nice watch

GQ!!!

The beauty of this signature¬†is that you can either include or exclude “watch” into your function call parameters and the function (assuming it’s designed robustly) should be able to handle either case.

So, if you exclude the watch, as such:

print outfit('grey','black', 'red', 'blue')

you will get

>>>You are wearing grey socks, black shoes, red pants, and a blue shirt

Fashion Plate! (*args)

I’m not sure how I got from Starcraft to Fashion and Accessories, but we plug on!

Some people like to add on watches, sunglasses, pinky rings (FUHGETTABOUTIT) and earrings. Whoa, that’s a lot of accessories. But only one parameter for accessories! And with so many people dressing up (that’s what makes Humans so special and unique) we have to be able to accomodate them! Or for Halloween, maybe someone will come dressed with a Freddy Krueger mask on! Or maybe Freddy himself!!

So, for Freddy’s amazing outfit of brown socks, black shoes, brown pants, and red shirt, along with his claws, amazing signature hat, and let’s say he was going for some gold bling today in honor of his 50th Halloween, we’d love to pass this along:

>>>print outfit('brown', 'black', 'brown', 'red' 'claws', 'hat', 'bling-bling')

but remember, we only had ONE optional argument:

def outfit(self, socks_color, shoes_color, pants_color, shirt_color, accessory=None):
    #if accessory = None
    if not accessory:
        return 'You are wearing %s socks, %s shoes, %s pants, and a %s shirt' % (socks_color, shoes_color, pants_color, shirt_color)
    else:
        return 'You are wearing %s socks, %s shoes, %s pants, and a %s shirt and a nice %s' % (socks_color, shoes_color, pants_color, shirt_color, accessory)

so we would for sure get this exception:

>>> print outfit('brown', 'black', 'brown', 'red', 'claws', 'hat', 'bling-bling')
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
TypeError: outfit() takes at most 6 arguments (7 given)
>>>

Now we use our magical *args… (changed method a bit for simplicity)

def outfit(self, socks_color, shoes_color, pants_color, shirt_color, *args):
   return'%s, %s, %s, %s, %s' % (socks_color, shoes_color, pants_color, shirt_color, args)

and once again pass in the following items in Freddie’s ensemble… and…

>>> print outfit('brown', 'black', 'brown', 'red', 'claws', 'hat', 'bling-bling')
black, brown, red, claws, ('hat', 'bling-bling')
>>>

Notice the (‘hat’, ‘bling-bling’). Looks like a tuple! (feel free to confirm with the following:

def outfit(self, socks_color, shoes_color, pants_color, shirt_color, *args):
   print type(args)
>>> print outfit('brown', 'black', 'brown', 'red', 'claws', 'hat', 'bling-bling')
<type 'tuple'>
>>>

Wow! Looks like you can add any number of accessories (fo’ shizzle!) and this function still would accept all your bling bling!

Of course you need to prettify your usage of your *args

def outfit(self, socks_color, shoes_color, pants_color, shirt_color, *args):
   return'%s, %s, %s, %s, %s' % (socks_color, shoes_color, pants_color, shirt_color, args)
>>> print outfit('brown', 'black', 'brown', 'red', 'claws', 'hat', 'bling-bling', 'necklace', 'earrings', 'gold shoes', 'cracka-lacking pinky ring', 'chest hair exposed')
black, brown, red, claws, ('hat', 'bling-bling', 'necklace', 'earrings', 'gold shoes', 'cracka-lacking pinky ring', 'chest hair exposed')
>>>

Now that’s a SCARY Freddie Krueger.

Let’s also make use of the tuple and prettify it a bit.

def outfit(self, socks_color, shoes_color, pants_color, shirt_color, *args):
   print 'Looks like you have %s accessories' % len(args)
   outfit = ''
   outfit += 'You are wearing %s socks, %s shoes, %s pants, %s shirt. ' % (socks_color, shoes_color, pants_color, shirt_color)
   if len(args):
      outfit += 'You also have some nice accessories! I dig your '
      for item in args:
         outfit += '%s, ' % (item)
      outfit += 'outfit. Now dont haunt my dreams, Freddie!'
   return outfit
>>> print outfit('brown', 'black', 'brown', 'red', 'claws', 'hat', 'bling-bling', 'necklace', 'earrings', 'gold shoes', 'cracka-lacking pinky ring', 'chest hair exposed')
Looks like you have 7 accessories
You are wearing black socks, brown shoes, red pants, claws shirt. You also have some nice accessories! I dig your hat, bling-bling, necklace, earrings, gold shoes, cracka-lacking pinky ring, chest hair exposed, outfit. Now dont haunt my dreams, Freddie!
>>>

 Freddie is mad!

Now one thing I can think that may go wrong. Let’s modify the outfit method a bit (make it a little less lengthy)

def outfit(pants_color, *args):
   print args
   outfit = ''
   outfit += 'Nice outfit Freddie! You are wearing %s pants. ' % (pants_color)
   if len(args):
       outfit += 'Looks like you also have some %s underwear and a %s shirt' % (args[0], args[1])
   return outfit

Which is fine if you pass in the following:

>>> print outfit('black', 'pink', 'purple')
('pink', 'purple')
Nice outfit Freddie! You are wearing black pants. Looks like you also have some pink underwear and a purple shirt
>>>

Hey, a gentleman’s underwear color is his own business. However, the crux of the *args is that you need to make sure the order of arguments passed in is correct if you designed the method with the order of the arguments in mind. Quick simple example of what can go “wrong”:

Same function:

def outfit(pants_color, *args):
   print args
   outfit = ''
   outfit += 'Nice outfit Freddie! You are wearing %s pants. ' % (pants_color)
   if len(args):
       outfit += 'Looks like you also have some %s underwear and a %s shirt' % (args[0], args[1])
   return outfit

And let’s switch up the method call:

>>> print outfit('black', 'purple', 'pink')
('purple', 'pink')
Nice outfit Freddie! You are wearing black pants. Looks like you also have some purple underwear and a pink shirt
>>>

WHOA! Now you gave Freddie a pink shirt!!! He’s not happy! He has no issue with the color pink in and of itself, but it’s definitely less scary!! (ok a bit over the top, but see the complications?) Better not sleep (remember, he haunts your dreams) and read the rest of this blog entry instead!!

Some quick notes:

*args does not need to be your argument name. You can use *accessories, *bling, *yoyo, etc. The key part is the * (asterisk). Also, you should keep it at the end of the function signature.

Also note, on the example just above, the number of expected arbitrary (“extra”) arguments is only 2 for example purposes. If you plan to regularly use arbitrary (*arg) arguments, you should account for the fact that it is possible to pass an arbitrary number of arguments into the function.

Now onward.

**KWARGS = KeyWord ARGumentS

Hopefully this is becoming pretty clear now what **kwargs are/will be. Same concept as *args, except in the form of key/value format. A tiny bit more controlled.

Thus, using our previous example, the order doesn’t matter, and the method signature is still arbitrary to some degree. The method must still be written robustly enough to expect the correct arguments and handle it correctly if arguments differ then expected. Perhaps the best illustration is a simple example.

def outfit(pants_color, **kwargs):
   print kwargs
   outfit = ''
   outfit += 'Nice outfit Freddie! You are wearing %s pants. ' % (pants_color)
   if len(kwargs):
       outfit += 'Looks like you also have some %s underwear and a %s shirt' % (kwargs['underwear_color'],kwargs['shirt_color'])
   return outfit

Note the

       outfit += 'Looks like you also have some %s underwear and a %s shirt' % (underwear_color, shirt_color)

in the above.

Now, we pass in the arguments differently when calling this method.

>>> print outfit('red', shirt_color="red", underwear_color="polka-dot pink")
{'underwear_color': 'polka-dot pink', 'shirt_color': 'red'}
Nice outfit Freddie! You are wearing red pants. Looks like you also have some polka-dot pink underwear and a red shirt
>>>

Note that we passed in the underwear_color and shirt_color in a different order, yet, since we are relying on “keywords” to reference the expected value we need, order should not matter.

Freddie is happy you didn’t mix up his underwear color.

You can also pass in dictionaries as arguments. This is called unpacking an argument list.

def outfit(pants_color, underwear_color='red', shirt_color='red'):
   outfit = ''
   outfit += 'Nice outfit Freddie! You are wearing %s pants. ' % (pants_color)
   outfit += 'Looks like you also have some %s underwear and a %s shirt' % (underwear_color,shirt_color)
   return outfit
>>> new_dictionary = {'pants_color':'not-red', 'underwear_color':'not-red-underwear', 'shirt_color':'not-red-shirt'}
>>> print outfit(**new_dictionary)
Nice outfit Freddie! You are wearing not-red pants. Looks like you also have some not-red-underwear underwear and a not-red-shirt shirt
>>>

Note the “**” before the dictionary. This is a special symbol within your function call that designates that that dictionary should be passed as an argument. Of course, you need to be sure that the arguments (keys) match.

Conclusion/Wrap-up

So hopefully, this makes *args and *kwargs somewhat easier to comprehend. Once you do understand them, they are really easy to (over)use.

They are most commonly used in Python Decorators, and in functions that require a lot of arguments/settings (such as a DB connection string).

Again, do not overuse these. Best Practice dictates that your method definition is tight and well-defined to prevent undesired input from being received by the method during runtime.

But keep *args and **kwargs in mind when developing.

Comment below with any questions or if you need clarity on these. I may expound on this further in a future blog post.

Leave a Reply

Your email address will not be published. Required fields are marked *