How to load a python module from a string?

Question

Given the string

optimizer = "tensorflow.train.ProximalAdagradOptimizer"

How can I achieve the following:

import tensorflow
optimizer = tensorflow.train.ProximalAdagradOptimizer()

Context

To add context to my specific use case (to address one of the comments): I'm trying to create a text-based config that describes how my model (specifically: estimator) was configured. That way I can easily re-instantiate them after training if I want to train it more or do other stuff with them. I haven't found a simple way to do this; I'm not looking for a saved_model for this. My use case is to easily reload models prior to committing them to a saved_model state. The config would look something like this:

model_config = {
  "type": "DNNClassifier",
  "n_classes": 10,
  "feature_columns": [
    {
      "numeric_column": [
        {
          "key": "x"
        },
        {
          "key": "y"
        }
      ]
    }
  ],
  "optimizer": {
    "AdamOptimizer": {
      "learning_rate": 1.0
    }
  }
}

Given that "config" I can instantiate my estimator with:

estimator = load_estimator(model_config, model_dir=model_dir)

The value of type would resolve to tensorflow.estimator.DNNClassifier. The value of feature_column[0].<key> would resolve to tensorflow.feature_column.numeric_column. Finally, the value of optimizer.<key> would resolve to tensorflow.train.AdamOptimizer.

you didn't explain how do you want to use this string in second code. There is no string in your code. Maybe you need only optimizer = tensorflow.train.ProximalAdagradOptimizer without () and later you can execute optimizer(). — furas
– furas, Commented Apr 18, 2019 at 20:01

javidcf · Accepted Answer · 2019-04-18 16:42:39Z

3

You can do something like this:

import importlib

def get_object_by_name(qualname):
    module, _, object = qualname.rpartition(".")
    if module:
        # package parameter is only necessary for relative imports
        # (here relative to this package)
        vs = vars(importlib.import_module(module, package=__package__))
    else:
        # If no module name we assume it is from the current module
        vs = globals()
    return vs[object]

optimizer_qualname = "tensorflow.train.AdamOptimizer"
optimizer_class = get_object_by_name(optimizer_qualname)
optimizer = optimizer_class()

I changed the optimizer to avoid the error due to the missing learning rate parameter in the example.

edited Apr 18, 2019 at 16:42

answered Apr 18, 2019 at 16:19

javidcf

59.9k7 gold badges87 silver badges134 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Frenchy · Accepted Answer · 2019-04-18 16:20:12Z

2

you could using the function eval its not the best answer but does the job

if "tensorflow" in optimizer:
    import tensorflow
    optimizer = eval(optimizer + '()')

you can"t do that : eval("import tensorflow")

edited Apr 18, 2019 at 16:20

answered Apr 18, 2019 at 16:12

Frenchy

17.1k3 gold badges21 silver badges42 bronze badges

Collectives™ on Stack Overflow

How to load a python module from a string?

Context

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

Context

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related