Dynamic python function creation overwriting function def

Question

I am trying to dynamically create functions in a class that are associated with names of functions in a module. The new functions must be able to call the original functions in the module. However, in the example code below, it appears there might be a memory/uniqueness issue in which the created functions are being overwritten. I'd like to know how to fix this.

I have a file mymod.py that I will import as a module:

def exec_test0():
    print "i am exec_test0"

def exec_test1():
    print "i am exec_test1"

and a file test.py with a class definition that needs to read the module:

import mymod as mm

class MyClass():
    def __init__(self):
        self.l = [fn for fn in dir(mm) if fn.startswith('exec_')]
        self.create_method()

    def create_method(self):
        # i expect this to create new methods with unique properties
        # as attributes of an object of this class
        for fn_exec in self.l:
            # this is the function template
            # that i want to call the relevant function from the
            # module mymod (mm)
            def method(self):
                method_to_call = getattr(mm, fn_exec)
                method_to_call()

            # this should create a new name based on the fn_exec
            # and assign method to it
            name_def = fn_exec.replace('exec_', 'do_')
            setattr(self.__class__, name_def, method)

if __name__ == '__main__':
    my_obj = MyClass()
    my_obj.do_test0()
    my_obj.do_test1()

The output of running test.py is:

i am exec_test1
i am exec_test1

I expect:

i am exec_test0
i am exec_test1

Any help on how to achieve this is greatly appreciated!

Update: the answer is given below, but I'm going to write a brief modification here and an extension for the new functions in the class to accept inputs.

The method create_method() in test.py should be updated as outlined in the accepted answer below:

def create_method(self):
    # i expect this to create new methods with unique properties
    # as attributes of an object of this class
    for fn_exec in self.l:
        # this is the function template
        # that i want to call the relevant function from the
        # module mymod (mm)
        # because this run at runtime, it requires a param instead
        def method(self, fn=fn_exec):
            method_to_call = getattr(mm, fn)
            method_to_call()

        # this should create a new name based on the fn_exec
        # and assign method to it
        name_def = fn_exec.replace('exec_', 'do_')
        setattr(self.__class__, name_def, method)

Furthermore, if one needs to pass parameters to the new method, such as an input some_input, create_method() would be updated as so:

def create_method(self):
    # i expect this to create new methods with unique properties
    # as attributes of an object of this class
    for fn_exec in self.l:
        # this is the function template
        # that i want to call the relevant function from the
        # module mymod (mm)
        # because this run at runtime, it requires a param instead
        def method(self, some_input, fn=fn_exec):
            method_to_call = getattr(mm, fn)
            method_to_call()
            print(some_input)

        # this should create a new name based on the fn_exec
        # and assign method to it
        name_def = fn_exec.replace('exec_', 'do_')
        setattr(self.__class__, name_def, method)

and the main block may look like:

if __name__ == '__main__':
    my_obj = MyClass()
    my_obj.do_test0('a')
    my_obj.do_test1('b')

which will have the following output:

i am exec_test0
a
i am exec_test1
b

chepner · Accepted Answer · 2014-03-18 18:43:35Z

2

The name fn_exec in the body of method is not evaluated until method is called, at which point fn_exec has the last value it took in the for loop. To make sure its value at the time method is defined is used, you'll need to make use of a default parameter value:

def method(self, fn=fn_exec):
    method_to_call = getattr(mm, fn)
    method_to_call()

When method is defined, fn is set equal to the current value of fn_exec, so that is the value used in defining method_to_call.

answered Mar 18, 2014 at 18:43

chepner

538k77 gold badges594 silver badges746 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

ohmu · Accepted Answer · 2014-03-18 18:55:14Z

0

This has to do with how closures work in Python. Add a method such as:

def make_wrapper(self, method_to_call):
    def method(self):
        method_to_call()

And change create_method() to:

def create_method(self):
    for fn_exec in self.l:
        method_to_call = getattr(mm, fn_exec)
        method = self.make_wrapper(method_to_call)

        name_def = fn_exec.replace('exec_', 'do_')
        setattr(self.__class__, name_def, method)

Since you are doing this on an instance, I would suggest setting the methods on the instance (and binding to self) instead of setting them directly on the class:

setattr(self, name_def, method.__get__(self, self.__class__))
# Instead of
setattr(self.__class__, name_def, method)

edited Mar 18, 2014 at 18:55

answered Mar 18, 2014 at 18:47

ohmu

19.9k44 gold badges114 silver badges151 bronze badges

Collectives™ on Stack Overflow

Dynamic python function creation overwriting function def

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related