How to create a 2D numpy array from a block of strings

Question

With list comprehension, I am able to take a 20x20 block of numbers in string format, and convert it to a list of lists of integers. The numbers are seperated by white space and the lines are seperated by a newline.

grid = [[int(x) for x in line.split()] for line in nums.split('\n')]

However, what I want is to use numpy for its speed. I could use np.asarray() with my intermediate list, but I don't think that is efficient use of numpy.

I also tried using np.fromstring(), but I can't figure out the logic to make it work for a 2D array.

Is there any way to accomplish this task without the use of creating intermediate python lists?

The numpy csv loaders, loadtxt and genfromtxt operate as you do - read each line and from that make a list of lists. Then at the end create an array from that. We use np.array(list_of_lists) all the time, at least for small(er) arrays. — hpaulj
– hpaulj, Commented Oct 9, 2019 at 16:30

yatu · Accepted Answer · 2022-04-20 05:57:06Z

5

You could use np.fromstring setting a space as separator and reshape to the desired shape:

np.fromstring(s, sep=' ').reshape(20, 20)

Or as a more general solution:

rows = s.count('\n') + 1
np.fromstring(s, sep=' ').reshape(-1, rows)

edited Apr 20, 2022 at 5:57

answered Oct 9, 2019 at 15:36

yatu

88.6k12 gold badges93 silver badges148 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

rocksNwaves Over a year ago

excellent! I was afraid that wouldn't be able to handle the newlines, but it seems to ignore them.

Mohammad Zeineldeen · Accepted Answer · 2019-10-09 16:30:32Z

3

More general for any 2D grid:

rows = s.count('\n') + 1
np.fromstring(s, sep=' ').reshape(rows, -1)

edited Oct 9, 2019 at 16:30

answered Oct 9, 2019 at 15:42

Mohammad Zeineldeen

3121 silver badge5 bronze badges

3 Comments

rocksNwaves Over a year ago

What does the -1 argument mean? I can't find it's use in documentation.

Mohammad Zeineldeen Over a year ago

It simply means that we want numpy to figure out the required dimension. For example, if you have a 2D array x of shape (3, 4) and you do x.reshape(-1) then you will get a 1D array of size 12 (numpy will figure out that this should contains 3x4 = 12 elements).

Mohammad Zeineldeen Over a year ago

@rocksNwaves It was wrong before as the rows and cols where flipped. Now it is correct.

Collectives™ on Stack Overflow

How to create a 2D numpy array from a block of strings

2 Answers 2

1 Comment

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related