2

I want to write my own Keras layer which invokes a python function. So I came across tf.py_func which should actually do the job.

I tried the following: In the call-method of the MyLayer Class I wrapped my custom python function myFunc in tf.py_func. So the call-method returns tf.py_func(myFunc, [input], tf.float32).

I get the following error message:

TypeError: unsupported operand type(s) for %=: 'int' and 'NoneType'

What's the correct way of calling a python function in a custom Keras layer? What am I doing wrong?

4 Answers 4

1

You'd have to show the code of your layer and the function you pass to pyfunc for us to understand the error. However, although it is possible to embed tf functions in custom Keras layers, the best option in this case is probably to use a Keras lambda layer : https://keras.io/layers/core/#lambda

This directly takes a python function as a parameter. Like tf.py_func, it expects a function that takes a numpy array as input and outputs a numpy array.

Sign up to request clarification or add additional context in comments.

4 Comments

So I tried this using a very simplified example, a code snipped would look like this: inp = Input(shape=(1,24,16)) x = Conv2D(16, (3,3), activation='relu', padding='same')(inp) x = Lambda(tf.py_func(my_func, [x], tf.float32))(x) model = Model(inp, x) with def my_func(x): return 5*x I get this error message: target = K.placeholder(ndim=len(shape), TypeError: object of type 'NoneType' has no len() What am I doing wrong here?
@Pagna401 Here you should directly use my_func instead of wrapping it in tf.py_func. x = Lambda(my_func)(x).
When using tf.py_func, tensorflow needs to know the shape of the input at call time. This might cause the 'Nonetype' error, if you have, for instance, a batch of unknown size. Second, Lambda expects a function. This means you should not evaluate the function when you give it as a parameter.
@julhien: Slight correction: According to the Keras documentation, the Lambda layer "Takes input tensor or list of tensors as first argument." Therefore, the best one-liner for this situation would probably be Lambda(lambda input_: tf.py_func(function,[x],dtype)). Your point about the function needing to accept the numpy arrays remains valid, though.
1

If you declare weights at def build(self, input_shape):, then you have to "use" all the weights after the tf.py_func call.

import tensorflow as tf

from keras import backend as K
from keras.engine.topology import Layer

class MultiplyLayer(Layer):

    def __init__(self, **kwargs):

        super(MultiplyLayer, self).__init__(**kwargs)

    def build(self, input_shape):
        self.filters = self.add_weight(name='Filters', shape=(1, input_shape[1]), initializer='uniform', trainable=True)

        super(MultiplyLayer, self).build(input_shape)  # Be sure to call this at the end

    def call(self, x):
        xw = x * self.filters

        def my_func(x):
            return 5 * x

        X = tf.py_func(my_func, [xw], tf.float32)

        return X + (self.filters - self.filters)

    def compute_output_shape(self, input_shape):
        return (input_shape)

def get_model():
    model = Sequential()

    model.add(MultiplyLayer(input_shape=(num_features,)))

    model.add(Dense(128, activation='relu'))
    model.add(Dropout(0.25))
    model.add(Dense(64, activation='relu'))
    model.add(Dropout(0.4))
    model.add(Dense(num_classes, activation='softmax'))
    model.compile(loss=keras.losses.categorical_crossentropy, optimizer=keras.optimizers.Adadelta(), metrics=['accuracy'])
    return model

model = get_model()
model.summary()
model.fit(X_train, y_train_hot, batch_size=batch_size, epochs=epochs, verbose=verbose, validation_data=(X_test, y_test_hot))

Comments

0

You just need to wrap you function with a Lambda layer. Here is an example:

def complex_tf_fn(x):
    u, v = tf.nn.moments(x, axes=[1], keep_dims=True)
    return (x - u) / tf.sqrt(v)

with tf.device('/cpu:0'):
    model = tf.keras.Sequential([
        tf.keras.layers.Lambda(complex_tf_fn, input_shape=[784]),
        tf.keras.layers.Dense(1024, activation='relu'),
        tf.keras.layers.Lambda(complex_tf_fn),
        tf.keras.layers.Dense(10, activation='softmax')
    ])

1 Comment

So I tried this using a very simplified example but I am using pure Keras (is that a problem?)... a code snipped would look like this: inp = Input(shape=(1,24,16)) x = Conv2D(16, (3,3), activation='relu', padding='same')(inp) x = Lambda(tf.py_func(my_func, [x], tf.float32))(x) model = Model(inp, x) with def my_func(x): return 5*x I get this error message: target = K.placeholder(ndim=len(shape), TypeError: object of type 'NoneType' has no len() What am I doing wrong here? I do not get any errors if I just use your complex_tf_fn, so this might be due to tf.py_func
0

Wrap your function with Lambda layer and call py_func in your function.

def my_func(x):
    return 5 * x
def my_lambda_func(x):
    return py_func(my_func,[x],tf.float32)
x = Lambda(my_lambda_func)(x)

1 Comment

While this code may answer the question, providing information on how and why it solves the problem improves its long-term value.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.