Monday, July 16, 2007

Python: Aggregating function arguments

Python has three ways to pass arguments to functions: enumerated named arguments, unenumerated named arguments, and unnamed positional arguments. Enumerated named arguments are familiar to most programmers since most modern languages use this style of naming arguments (perl being a notable exception). For example, the following function specifies that it accepts 3 arguments, and assigns those arguments the names larry, moe, and curly for the scope of the function:
    def Stooge(larry, moe, curly):
...

If you call this function as Stooge(1, 2, 3) then the variable named larry equals 1, moe equals 2, and curly equals 3 when the function Stooge starts. Python, like C++ and Java, also allows you to specify the arguments explicitly; calling the function as Stooge(moe=2, curly=3, larry=1) or Stooge(1, 2, curly=3) again causes larry to equal 1, moe to equal 2, and curly to equal 3 when the function starts. I call this form of argument passing enumerated named arguments since names are assigned to each argument and all acceptable arguments are enumerated in the function declaration.

Python also supports unenumerated named arguments by specifying a "catch all" argument, prefixed with two asterisks. For example:
    def Stooge2(larry, moe, **kw):
...

In this case, Stooge2 accepts two arguments, larry and moe that may be specified either positionally or by name just as in the previous example. However, it also accepts any number of additional named arguments. For example, we could call the function as Stooge2(1, moe=2, shemp=3) or Stooge2(1, 2, shemp=3, curly=4). In both cases, as before, larry would start equal to 1 and moewould start equal to 2. However, now the kw argument would be populated with a dictionary mapping all other named parameters with their argument values. For example, it might contain {'shemp': 3, 'curly': 4}.

Before we move on to unnamed positional arguments, let me interrupt to touch on the point of this posting: how do you iterate over all named arguments whether they be enumerated or not?

If your function enumerates all accepted named arguments, then you can trivially get a dictionary mapping the argument names to their values if you call the builtin function locals() at the beginning of your function. For example:
    def WhichStooges(larry, moe, curly):
stooges = locals()
...

This would populate stooges with a dictionary with keys, "larry", "moe", and "curly". You could then iterate through the arguments and their values with a standard loop over stooges.items().

Now, if you add unenumerated named arguments into the picture, it gets a bit trickier. The most straightforward way is to use the fact that "catch all" argument is a standard dictionary and update it from locals() at the beginning of the function:
    def WhichStooges2(larry, moe, **stooges):
stooges.update(locals())
...

The only problem with this approach is that stooges still appears in the argument list, which is probably not what you want. This can be remedied like so:
    def WhichStooges2(larry, moe, **stooges):
stooges.update(locals())
del stooges['stooges']
...

Which only leaves the minor issue of the requirement for locals() to be called at the top of the function, before any other variables are defined in the function's scope. Wouldn't it be nice if we could enumerate the function arguments anywhere in the function? And wouldn't it be even better if we could encapsulate the logic for aggregating the function arguments into a utility function?

Before I get to the solution to those problems, for the sake of completeness I should cover unnamed positional arguments too. Unnamed positional arguments are additional positional arguments that are captured in a single list argument by prefixing the argument named with a single asterisk (*) in Python. For example:
    def WhichStooges3(larry, moe, *args):
...

In this case, larry and moe may still be passed values either by name or position as in previous examples. In addition, additional values may be specified but they cannot be named. Calling this function as WhichStooges3(1, 2, 3, 4) causes larryto start with the value 1, moe to start with the value 2, and args to start as a list containing (3, 4). The rules for mixing unnamed positional arguments and named arguments are non-trivial and covered in the Python documentation so I won't rehash them here.

Finally, he can construct one utility function that returns a dictionary of all named parameters (enumerated or not) as well as a list of all unnamed positional parameters. By using Python's inspect module we can encapsulate the logic into a single common routine that can be called anywhere within a function's scope.
    def arguments():
"""Returns tuple containing dictionary of calling function's
named arguments and a list of calling function's unnamed
positional arguments.
"""
from inspect import getargvalues, stack
posname, kwname, args = getargvalues(stack()[1][0])[-3:]
posargs = args.pop(posname, [])
args.update(args.pop(kwname, []))
return args, posargs

This routine removes the 'catch all' arguments (i.e. the positional catch all argument prefixed with a single asterisk and/or the keyword catch all argument prefixed with two asterisks) from the returned dictionary of named arguments for you.


Update 2009/09/29:
I updated the arguments() function to fix a bug that was brought to my attention by drewm1980's comment.

11 comments:

Anonymous said...

Excelent! Thanks a lot!

Unknown said...

Most helpfully useful.

Andrew Wagner said...

When I call this from a class's __init__(self,....)

I'm getting a:

KeyError: None

on:

posargs = args.pop(posname)

Any ideas? Thanks!

Kelly Yancey said...

@drewm1980: Sure enough, this was due to a bug in the arguments() function. Embarrassingly, it did not properly handle functions with no *args or **kwargs arguments. I just fixed it. Thanks!

Andrew Wagner said...

Thanks! I'll try it out.

Paddy3118 said...

Ouch,
Your use of the word enumerated grates as, for me, enumerated is strongly connected with "attaching numbers to", which is not what is happening. in what you use the word to describe.

This of course assumes that you were not using this argument by Lewis Carroll from Alice through the looking glass:

'When I use a word,' Humpty Dumpty said, in rather a scornful tone, 'it means just what I choose it to mean -neither more nor less.'
'The question is,' said Alice, 'whether you can make words mean so many different things.'
'The question is,' said Humpty Dumpty, 'which is to be master—that's all.'


Peace :-)

Kelly Yancey said...

@Paddy3118
http://www.merriam-webster.com/dictionary/enumerate

2 : to specify one after another : list

http://www.merriam-webster.com/thesaurus/enumerate

Meaning: 1 to specify one after another (I proceeded to enumerate the reasons why my allowance needed raising)
Synonyms detail, itemize, list

I'm sorry that the vernacular usage of an English word that happened to be borrowed for use as a Computer Science term is grating for you.

Wyatt said...

I wasn't aware that positional args could be called by name. I almost didn't believe you, but then I tried it myself:

>>> def func(a, b):
... print a, b
...
>>> func(b='B', a='A')
A B

tvn said...

This is really nice. Is it possible to also output the name of the function in addition to the arguments? Thanks ,

Mike said...

Thanks Kelly, This solved a messy problem where I have 65 registers I need to write one or more at a time. Where I thought I was going to have one function for each register (a bunch of functions), now I only need one.

I did have a minor gottcha where I want to call my function "mod_star" and feed binary writing format (ala bitstream).

mod_star(fyank= "0b0", jam="0b0", Jack= '0x 8 4 2 1 0 f 0 f')

It turns out that I could not write
mod_star(fyank=0b0) as python took "fyank=0" instead of "fyank=0b0" I guess the leading zero got interpreted somehow.

Christopher Anderson said...


I'm sorry that the vernacular usage of an English word that happened to be borrowed for use as a Computer Science term is grating for you.


Paddy3118 got burned!

And deservedly so, his criticism was not constructive. I had no trouble understanding your article, it was very legible. I unfortunately must agree with Paddy a tiny tiny bit however.

Whilst I would rather read your blog entry than the python canons, their lexicon must be Pythonic.

That said, even they seem to have trouble deciding if they're "arguments" or "parameters."


A function call always assigns values to all parameters mentioned in the parameter list, either from position arguments, from keyword arguments, or from default values. If the form “*identifier” is present, it is initialized to a tuple receiving any excess positional parameters, defaulting to the empty tuple. If the form “**identifier” is present, it is initialized to a new dictionary receiving any excess keyword arguments, defaulting to a new empty dictionary.