Numpy Array2string just writing ... in string? [duplicate]

Question

I have a simple thing to do, read some vectors and write them in a file.

The vectors are 1024 dimensional.

 for emb in src:
            print(len(emb[0].detach().cpu().numpy()))  #--> prints 1024!
            f.write(np.array2string(emb[0].detach().cpu().numpy(), separator=', ') + " \n")

My file looks like this:

[-0.18077464, -0.02889516,  0.33970496, ..., -0.28685367,  0.00343359,
 -0.00380083] 
[-0.04554039,  0.0891239 ,  0.0457519 , ..., -0.02622034,  0.04410202,
 -0.03626832] 
[ 0.2415923 ,  0.36748591, -0.10974079, ...,  0.06169772,  0.0134424 ,
  0.01647076] 
[ 0.019123  ,  0.00409475,  0.03623311, ..., -0.13063622,  0.02434589,
  0.00400023] 
[ 0.01281842,  0.00028924,  0.03185712, ..., -0.062907  ,  0.02143336,
 -0.00206215] 
[ 0.01748654,  0.00136842, -0.01337154, ..., -0.04148545,  0.00875527,
 -0.03006736]

So, I just can't access my vectors, 1024 dimension is transformed to whatever 6 or 7 dimensional vector + .... :(

How can I write vectors to my file correctly?

Cheers :)

Question has nothing to do with machine-learning - kindly do not spam the tag (removed). — desertnaut
– desertnaut, Commented Nov 23, 2018 at 23:25
Do you also want those brackets? That will make reading the file more complicated. array2string is doing the same thing as str(x) or print(x). It's the summarized output intended primarily for quick interactive views. It's not intended saving and reloading. — hpaulj
– hpaulj, Commented Nov 23, 2018 at 23:36
@hpaulj no, it makes reading tougher...oh, so setting the flag and just doing f.write(str(x)) is okay? — tstseby
– tstseby, Commented Nov 23, 2018 at 23:43
If you want the common csv format, np.savetxt is good basic choice for writing a 2d array. The result should be readable with np.genfromtxt. — hpaulj
– hpaulj, Commented Nov 24, 2018 at 0:18

hpaulj · Accepted Answer · 2018-11-24 00:23:53Z

The normal way of writing a 2d array to a text file (so it can be read back) is with np.savetxt:

In [309]: src = np.random.rand(6,4)
In [310]: src
Out[310]: 
array([[0.78756364, 0.11385762, 0.16631052, 0.10987765],
       [0.59954504, 0.80417064, 0.22461205, 0.47827772],
       [0.10993457, 0.11650874, 0.55887911, 0.71854456],
       [0.53572426, 0.55055622, 0.25423811, 0.46038837],
       [0.05418115, 0.50696182, 0.31515915, 0.65310375],
       [0.81168653, 0.81063907, 0.95371101, 0.11875685]])

write:

In [311]: np.savetxt('test.txt', src, fmt='%10.6f',delimiter=',')
In [312]: cat test.txt
  0.787564,  0.113858,  0.166311,  0.109878
  0.599545,  0.804171,  0.224612,  0.478278
  0.109935,  0.116509,  0.558879,  0.718545
  0.535724,  0.550556,  0.254238,  0.460388
  0.054181,  0.506962,  0.315159,  0.653104
  0.811687,  0.810639,  0.953711,  0.118757

Test the loading:

In [314]: np.genfromtxt('test.txt',delimiter=',')
Out[314]: 
array([[0.787564, 0.113858, 0.166311, 0.109878],
       [0.599545, 0.804171, 0.224612, 0.478278],
       [0.109935, 0.116509, 0.558879, 0.718545],
       [0.535724, 0.550556, 0.254238, 0.460388],
       [0.054181, 0.506962, 0.315159, 0.653104],
       [0.811687, 0.810639, 0.953711, 0.118757]])

savetxt does a formatted write, row by row, roughly like:

In [315]: fmt = ','.join(['%10.6f']*4)
In [316]: fmt
Out[316]: '%10.6f,%10.6f,%10.6f,%10.6f'
In [317]: for row in src:
     ...:     print(fmt%tuple(row))    # f.write(...)
     ...:     
  0.787564,  0.113858,  0.166311,  0.109878
  0.599545,  0.804171,  0.224612,  0.478278
  0.109935,  0.116509,  0.558879,  0.718545
  0.535724,  0.550556,  0.254238,  0.460388
  0.054181,  0.506962,  0.315159,  0.653104
  0.811687,  0.810639,  0.953711,  0.118757

In fact I can wrap that in file write:

In [318]: with open('test1.txt','w') as f:
     ...:     for row in src:
     ...:         print(fmt%tuple(row), file=f)
     ...:     
In [319]: cat test1.txt
  0.787564,  0.113858,  0.166311,  0.109878
  0.599545,  0.804171,  0.224612,  0.478278
 ...

berkelem · Accepted Answer · 2018-11-23 23:39:17Z

0

The vectors are still 1024 dimensional, but the display is only showing a curtailed view of the array.

You can view the whole array by setting print options:

import numpy as np
np.set_printoptions(threshold=np.nan)

edited Nov 23, 2018 at 23:39

answered Nov 23, 2018 at 23:32

berkelem

2,1453 gold badges21 silver badges41 bronze badges

2 Comments

tstseby Over a year ago

threshold=numpy.nan or threshold=np.nan...any difference ?

berkelem Over a year ago

That was a typo sorry. If you import numpy as np you should use np throughout. If you just import numpy then use numpy. They are exactly equivalent, so just a matter of preference.

Collectives™ on Stack Overflow

Numpy Array2string just writing ... in string? [duplicate]

2 Answers 2

Comments

2 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Linked

Related