Tensorflow 2.3: AttributeError: 'Tensor' object has no attribute 'numpy'

Question

I wanted to load the text file borrowed from here, where each line represent a json string like following:

{"overall": 2.0, "verified": true, "reviewTime": "02 4, 2014", "reviewerID": "A1M117A53LEI8", "asin": "7508492919", "reviewerName": "Sharon Williams", "reviewText": "DON'T CARE FOR IT.  GAVE IT AS A GIFT AND THEY WERE OKAY WITH IT.  JUST NOT WHAT I EXPECTED.", "summary": "CASE", "unixReviewTime": 1391472000}

I would like to extract only reviewText and overall feature from the dataset using tensorflow but facing following error.

AttributeError: in user code:

    <ipython-input-4-419019a35c5e>:9 None  *
        line_dataset = line_dataset.map(lambda row: transform(row))
    <ipython-input-4-419019a35c5e>:2 transform  *
        str_example = example.numpy().decode("utf-8")

    AttributeError: 'Tensor' object has no attribute 'numpy'

My code snippet looks like following:

def transform(example):
    str_example = example.numpy().decode("utf-8")
    json_example = json.loads(str_example)
    overall = json_example.get('overall', None)
    text = json_example.get('reviewText', None)
    return (overall, text)

line_dataset = tf.data.TextLineDataset(filenames = [file_path])
line_dataset = line_dataset.map(lambda row: transform(row))
for example in line_dataset.take(5):
    print(example)

I am using tensorflow 2.3.0.

javidcf · Accepted Answer · 2020-08-24 09:26:55Z

3

The input pipeline of a dataset is always traced into a graph (as if you used @tf.function) to make it faster, which means, among other things, that you cannot use .numpy(). You can however use tf.numpy_function to access the data as a NumPy array within the graph:

def transform(example):
    # example will now by a NumPy array
    str_example = example.decode("utf-8")
    json_example = json.loads(str_example)
    overall = json_example.get('overall', None)
    text = json_example.get('reviewText', None)
    return (overall, text)

line_dataset = tf.data.TextLineDataset(filenames = [file_path])
line_dataset = line_dataset.map(
    lambda row: tf.numpy_function(transform, row, (tf.float32, tf.string)))
for example in line_dataset.take(5):
    print(example)

answered Aug 24, 2020 at 9:26

javidcf

59.9k7 gold badges87 silver badges134 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

tornikeo · Accepted Answer · 2020-08-24 11:14:59Z

1

A bit wordy, but try it like this:

def transform(example):     
    str_example = example.numpy().decode("utf-8")     
    json_example = json.loads(str_example)     
    overall = json_example.get('overall', None)     
    text = json_example.get('reviewText', None)     
    return (overall, text)  

line_dataset = tf.data.TextLineDataset(filenames = [file_path]) 
line_dataset = line_dataset.map(
    lambda input:     
        tf.py_function(transform, [input], (tf.float32, tf.string))
)  
for example in line_dataset.take(5):     
    print(example)

This particular snippet works for any python function, not only the for numpy functions. So, if you need functions like print, input and so on, you can use this. You don't have to know all the details, but if you are interested, please ask me. :)

edited Aug 24, 2020 at 11:14

answered Aug 24, 2020 at 9:53

tornikeo

9828 silver badges21 bronze badges

3 Comments

Gaurav Gupta Over a year ago

What is the purpose of decorator used in tf_function?

Gaurav Gupta Over a year ago

I modified your snippet to cut it short and my version of your snippet works as well.

def transform(example):     str_example = example.numpy().decode("utf-8")     json_example = json.loads(str_example)     overall = json_example.get('overall', None)     text = json_example.get('reviewText', None)     return (overall, text)  line_dataset = tf.data.TextLineDataset(filenames = [file_path]) line_dataset = line_dataset.map(lambda input: tf.py_function(transform, [input], (tf.float32, tf.string)))  for example in line_dataset.take(5):     print(example)

tornikeo Over a year ago

With your permission, I'll update my answer with your snippet. Your code looks better :)

Collectives™ on Stack Overflow

Tensorflow 2.3: AttributeError: 'Tensor' object has no attribute 'numpy'

2 Answers 2

Comments

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related