Convert List object in a pandas dataframe to numpy array

Question

One of my pandas dataframe column is a list object of m columns, each row looks like this 'List(0.42, 0.24, 0.78,...)' with a list of n elements wrapped by quote marks. Dtype for this column is Object.

I need to convert this column into a m X n np array. So far I tried applying np.fromstring(col content) but it's mostly returning 'ValueError: string size must be a multiple of element size'. It did work for the first row though.

How to appropriately convert this List object column to an array?

How did you store you list string at the first place? I think it would be easier if you store your column values as actual lists instead of a string of your lists to begin with. — Ehsan
– Ehsan, Commented Aug 10, 2020 at 23:26
That's a string, not an actual list, or even a string that evals to list. It's missing the []. — hpaulj
– hpaulj, Commented Aug 11, 2020 at 0:36

BENY · Accepted Answer · 2020-08-10 23:37:09Z

1

We need trim your string , then split

np.array(s.str.strip('List').str.strip('(|)').str.split(', ').tolist())
Out[11]: 
array([['0.42', '0,24', '0.78,...'],
       ['0.42', '0,24', '0.78,...']], dtype='<U8')

Updated

s.str.strip('List').str.strip('(|)').str.split(',',expand=True).apply(lambda x : x.str.strip()).values
Out[18]: 
array([['0.42', '0', '24', '0.78', '...'],
       ['0.42', '0', '24', '0.78', '...']], dtype=object)

edited Aug 10, 2020 at 23:37

answered Aug 10, 2020 at 23:26

BENY

324k22 gold badges176 silver badges250 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Ehsan Over a year ago

Note 0,24 in the list. I think one need more careful trimming. And this is a pandas data frame column from what I understand and not an array of strings.

Ehsan Over a year ago

@Fast response. But OP edited that to 0.24 :D . This solution will still work of course.

santoku Over a year ago

thanks to your answer @BEN_YO (1) I got it working with strip() and split instead of str.split, as it would return 'str' object has no attribute 'str' error (2) updated the 0,24 typo, it's just 0.24 (3) somehow the result dimension can be different from np.fromstring() could there be something wrong with np.fromstring?

BENY Over a year ago

@santoku I think the numpy from string should be return the correct result ~

BENY Over a year ago

@santoku then you may check your real data and see the different ~

|

Collectives™ on Stack Overflow

Convert List object in a pandas dataframe to numpy array

1 Answer 1

7 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Related