How to set a numpy array in a pandas data frame cell?

Question

I have a pandas dataframe. I want to fill some of the cells with numpy array but I get the following ValueError.

I wil not fill with zero array in real life. This is the simplified example code to replicate the error

ValueError: could not broadcast input array from shape (10,) into shape (1,)

import pandas as pd
import numpy as np

df = pd.DataFrame(columns=['name1','name2','array1','array2' ])
df = df.append({'name1': 'aaaa','name2': 'bbbb','array1':np.nan,'array2': np.nan}, ignore_index=True)
df = df.append({'name1': 'cccc','name2': 'dddd','array1':np.nan,'array2': np.nan}, ignore_index=True)

df.loc[((df['name1']=='aaaa') & (df['name2']=='bbbb')),'array1']=np.zeros((10,1))

print(df)

SeaBean · Accepted Answer · 2021-05-19 10:07:33Z

1

One workaround solution is to use .map() with filtering of cell with .loc as you did, as follows:

This works since .map() works on transforming element-wise and would not try to broadcast array to the whole series.

df.loc[((df['name1']=='aaaa') & (df['name2']=='bbbb')),'array1'] = df.loc[((df['name1']=='aaaa') & (df['name2']=='bbbb')),'array1'].map(lambda x: np.zeros((10,1)))


print(df)

  name1 name2                                                                  array1 array2
0  aaaa  bbbb  [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]]    NaN
1  cccc  dddd                                                                     NaN    NaN


df.applymap(type)        #  to check the data type

           name1          name2                   array1           array2
0  <class 'str'>  <class 'str'>  <class 'numpy.ndarray'>  <class 'float'>
1  <class 'str'>  <class 'str'>          <class 'float'>  <class 'float'>

edited May 19, 2021 at 10:07

answered May 19, 2021 at 10:02

SeaBean

23.4k3 gold badges16 silver badges28 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

burcak Over a year ago

Sorry, I have not test it yet. But does is work for array of any shape?

SeaBean Over a year ago

@burcak Yes, it has no limitation on array shape.

burcak Over a year ago

I have tested meanwhile. Thanks.

Collectives™ on Stack Overflow

How to set a numpy array in a pandas data frame cell?

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related