dealing with numpy array and dataframe columns

Question

I have the following dataframe:

dates,values
2014-10-01 00:00,10.606
2014-10-01 01:00,10.595
2014-10-01 02:00,10.583
2014-10-01 03:00,10.572
2014-10-01 04:00,10.56
2014-10-01 05:00,10.564
2014-10-01 06:00,10.65
2014-10-01 07:00,10.801
2014-10-01 08:00,10.977
2014-10-01 09:00,11.316
2014-10-01 10:00,11.88
2014-10-01 11:00,12.427
2014-10-01 12:00,12.751
2014-10-01 13:00,12.863
2014-10-01 14:00,12.823
2014-10-01 15:00,12.686
2014-10-01 16:00,12.499
2014-10-01 17:00,12.293
2014-10-01 18:00,12.086
2014-10-01 19:00,11.89
2014-10-01 20:00,11.712
2014-10-01 21:00,11.552
2014-10-01 22:00,11.413
2014-10-01 23:00,11.292
2014-10-02 00:00,11.188
2014-10-02 01:00,11.1

Let's say that I want to select all the data related to a specific day. In this case. For example 2014-10-01. These are the operation being used in my code:

dfr       =  pd.read_csv(f_name, parse_dates=True,index_col=0,
                                       infer_datetime_format=True)

yy  = dfr [dfr.index.floor('D')  == ' 2014-10-01 00:00:00'].to_numpy()

This is what I get:

array([[10.606],
       [10.595],
       [10.583],
       [10.572],
       [10.56 ],
       [10.564],
       [10.65 ],
       [10.801],
       [10.977],
       [11.316],
       [11.88 ],
       [12.427],
       [12.751],
       [12.863],
       [12.823],
       [12.686],
       [12.499],
       [12.293],
       [12.086],
       [11.89 ],
       [11.712],
       [11.552],
       [11.413],
       [11.292]])

However, I would like to have yy in the following form:

array([10.606,10.595,10.583,10.572,10.56 ,10.564,10.65 ,10.801,10.977, 11.316,11.88 ,12.427,12.751,12.863,12.823,12.686,12.499,12.293,12.086,11.89 ,11.712,11.552,11.413,11.292])

Indeed I have to work with another vector xx which is:

xx=array([ 2.91833891,  2.84972246,  0.50386982,  5.35302713,  4.81822114,
        3.33330121,  5.63819964, 11.20447123, 12.98512414,  9.95449998,
        5.78945234,  9.90594599,  1.25708361,  3.02603884,  1.02683686,
        3.84912813,  1.55641116, 13.04097404,  9.6277124 , 10.73849736,
        5.39958019,  3.43633323, 13.5965677 ,  7.31914519])

This would help me in using np.sum and so on without deal with cycle.

Thanks in advance

you need to specify the column you want as an array i.e dfr[dfr.index.floor('D') == ' 2014-10-01 00:00:00']['values'].to_numpy() — Umar.H
– Umar.H, Commented Aug 16, 2021 at 16:57
filter using datetime: df[df['dates'].dt.date == datetime.date(year=2014, month=10, day=1)]['values'].tolist() — It_is_Chris
– It_is_Chris, Commented Aug 16, 2021 at 16:58

Anurag Dabas · Accepted Answer · 2021-08-16 17:09:40Z

1

use loc:

yy=dfr.loc[dfr.index.floor('D')  == ' 2014-10-01 00:00:00','values'].to_numpy()

OR

use flatten():

yy=dfr[dfr.index.floor('D')  == ' 2014-10-01 00:00:00'].to_numpy().flatten()
#yy=dfr[dfr.index.floor('D')  == ' 2014-10-01 00:00:00'].to_numpy().ravel()

edited Aug 16, 2021 at 17:09

answered Aug 16, 2021 at 16:57

Anurag Dabas

24.3k9 gold badges25 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Andrej Kesely · Accepted Answer · 2021-08-16 16:58:47Z

0

Another solution, use df.loc to select only one column:

yy = dfr.loc[
    dfr.index.floor("D") == " 2014-10-01 00:00:00", "values"
].to_numpy()
print(yy)

Prints:

[10.606 10.595 10.583 10.572 10.56  10.564 10.65  10.801 10.977 11.316
 11.88  12.427 12.751 12.863 12.823 12.686 12.499 12.293 12.086 11.89
 11.712 11.552 11.413 11.292]

answered Aug 16, 2021 at 16:58

Andrej Kesely

196k15 gold badges60 silver badges105 bronze badges

1 Comment

Anurag Dabas Over a year ago

so fast I just change my answer and sir you posted :)

Corralien · Accepted Answer · 2021-08-16 21:03:49Z

0

In fact what you need is a Series not a DataFrame:

At file level, use squeeze=True parameter to read csv:

dfr = pd.read_csv(f_name, parse_dates=True,index_col=0,
                  infer_datetime_format=True, squeeze=True)

Use numpy ravel function:

>>> dfr[dfr.index.floor('D')  == ' 2014-10-01 00:00:00'].to_numpy().ravel()

array([10.606, 10.595, 10.583, 10.572, 10.56 , 10.564, 10.65 , 10.801,
       10.977, 11.316, 11.88 , 12.427, 12.751, 12.863, 12.823, 12.686,
       12.499, 12.293, 12.086, 11.89 , 11.712, 11.552, 11.413, 11.292])

Use one of solutions proposed by @AnuragDabas or @AndrejKesely.

answered Aug 16, 2021 at 21:03

Corralien

121k8 gold badges43 silver badges68 bronze badges

Collectives™ on Stack Overflow

dealing with numpy array and dataframe columns

3 Answers 3

Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related