Python: Dynamically import module's code from string with importlib

Question

I wish to dynamically import a module in Python (3.7), whereby the module's code is defined within a string.

Below is a working example that uses the imp module, which is deprecated in favour of importlib (as of version 3.4):

import imp

def import_code(code, name):
    # create blank module
    module = imp.new_module(name)
    # populate the module with code
    exec(code, module.__dict__)
    return module

code = """
def testFunc():
    print('spam!')
"""

m = import_code(code, 'test')
m.testFunc()

Python's documentation states that importlib.util.module_from_spec() should be used instead of imp.new_module(). However, there doesn't seem to be a way to create a blank module object using the importlib module, like I could with imp.

How can I use importlib instead of imp to achieve the same result?

This is also a neat approach to importing code : stackoverflow.com/a/62492464/12446456 — R_100
– R_100, Commented Oct 2, 2023 at 14:54

Right leg · Accepted Answer · 2019-04-29 14:28:15Z

10

You can simply instantiate types.Module:

import types
mod = types.ModuleType("mod")

Then you can populate it with exec just like you did:

exec(code, mod.__dict__)
mod.testFunc() # will print 'spam!'

So your code will look like this:

import types

def import_code(code, name):
    # create blank module
    module = types.ModuleType(name)
    # populate the module with code
    exec(code, module.__dict__)
    return module

code = """
def testFunc():
    print('spam!')
"""

m = import_code(code, 'test')
m.testFunc()

As commented by @Error - Syntactical Remorse, you should keep in mind that exec basically executes whatever code is contained in the string you give it, so you should use it with extra care. At least check what you're given, but it'd be good to use exclusively predefined strings.

edited Apr 29, 2019 at 14:28

answered Apr 29, 2019 at 14:19

Right leg

17k8 gold badges56 silver badges90 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Error - Syntactical Remorse Over a year ago

You may want to add a disclaimer that he should only use pre-specified strs or check them before calling the exec method.

Right leg Over a year ago

@Error-SyntacticalRemorse You're right, I added a side-note

Community · Accepted Answer · 2020-06-20 09:12:55Z

7

According to Python documentation module_from_spec()

importlib.util.module_from_spec(spec)

...

This function is preferred over using types.ModuleType to create a new module as spec is used to set as many import-controlled attributes on the module as possible.

Here is what I came up with to load the module from source code located in github repo. It is a way without writing the file to disk.

import requests
url = "https://github.com/udacity/deep-learning-v2-pytorch/raw/master/intro-to-pytorch/helper.py"
r = requests.get(url)

import importlib.util
spec = importlib.util.spec_from_loader('helper', loader=None, origin=url)
helper = importlib.util.module_from_spec(spec)
exec(r.content, helper.__dict__)

helper.view_classify() # executes function from github file

edited Jun 20, 2020 at 9:12

CommunityBot

11 silver badge

answered Feb 4, 2020 at 9:20

LVitya

5874 silver badges13 bronze badges

2 Comments

naisanza Over a year ago

exactly what I was looking for

boludoz Over a year ago

What a beauty, it was what I wanted, I can encrypt a code and execute it.

boludoz · Accepted Answer · 2023-11-12 17:04:51Z

import importlib.util
import urllib.request
import sys
from cryptography.fernet import Fernet

def import_code(url, import_as=None, key_string=None):
    
    # set import_as to the name of the url if it is None
    if import_as == None:
        import_as = url.split('/')[-1].split('.')[0]

    # throw error if import_as is not a string
    if not isinstance(import_as, str):
        raise TypeError('import_as must be a string')
    
    # throw error if url is not a string
    if not isinstance(url, str):
        raise TypeError('url must be a string')
    
    # throw error if url is empty
    if not url:
        raise ValueError('url must not be empty')
    
    try:
        # Download the module code from the URL
        response = urllib.request.urlopen(url)
        module_code = response.read().decode('utf-8')

        # Decrypt the module code if it is encrypted with Fernet and try except
        if key_string!=None:
            fernet_obj = Fernet(key_string)
            try:
                module_code = fernet_obj.decrypt(module_code)
            except:
                pass
        
        # Execute the module code in the current scope
        if import_as == '':
            exec(module_code, globals())
            return None

        # Create a new module object
        spec = importlib.util.spec_from_loader(import_as, loader=None)
        module = importlib.util.module_from_spec(spec)
        exec(module_code, module.__dict__)
        sys.modules[import_as] = module
        exec(f'{import_as} = module')
        return module
    except:
        raise

In that example I show you how to download a file from the internet, which may be encrypted with Fernet and I teach you how to execute it. I used it for a project, you can modify it and have it directly import a text string.

Ways to use:

# Take from url and file name
import_code(url, import_as=None)
import hello_word
hello_word.hello_word()

# Take custom name
hello_word = import_code(url)
hello_word.hello_word()

# Global import
import_code(url, import_as='')
hello_word()

Collectives™ on Stack Overflow

Python: Dynamically import module's code from string with importlib

3 Answers 3

2 Comments

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related