Error when creating dataframe from two columns in Python panda

Question

when i try to create dataframe from two columns i.e. pids and SalePrice I get error "Exception: Data must be 1-dimensional". I think the error is coming because these two data series are in different format like below. Please help how can i make these data series same

ksubmission = pd.DataFrame({'Id':pids,'SalePrice':predictions_kaggle})

Exception: Data must be 1-dimensional

pids.shape

(1459,)

predictions_kaggle.shape

(1459, 1)

predictions_kaggle is in below format

array([[115901.20520943],
       [144313.70246636],
       [165320.94012928],
       ...,
       [155759.14767572],
       [111175.64223766],
       [249104.99042467]])

while pids is in below format

0       1461
1       1462
2       1463
3       1464
4       1465
        ... 
1454    2915
1455    2916
1456    2917
1457    2918
1458    2919
Name: Id, Length: 1459, dtype: int64

Or have you tried to use predictions_kaggle.flatten() to make it 1-dimensional? — Seraph Wedd
– Seraph Wedd, Commented Dec 17, 2019 at 1:14

oppressionslayer · Accepted Answer · 2019-12-17 01:21:23Z

1

I think you need to do this if the lengths are the same:

import pandas as pd
import numpy as np
pd.DataFrame(predictions_kaggle, index=pids).reset_index().rename(columns={'index': 'Id', 0:'SalePrice'}) 

or

pd.DataFrame({'Id':pids,'SalePrice':np.ndarray.flatten(predictions_kaggle)})

edited Dec 17, 2019 at 1:21

answered Dec 17, 2019 at 1:15

oppressionslayer

7,2242 gold badges11 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

user2774120 Over a year ago

but pids shouldnt be index.. instead it should be a column

oppressionslayer Over a year ago

does it work though, we can rename it a column by reseting index. It also let me know how to format it

oppressionslayer Over a year ago

@user2774120 i changed it. It might not be the end solution, but it might work. Otherwise i believe you have to flatten the list with np.ndarray.flatten

Seraph Wedd · Accepted Answer · 2019-12-17 01:27:13Z

1

The problem here is that your predictions_kaggle array is not a 1-D array but rather a 2-D one. As proof, the shape of a 1-D array should be in the form (n,) but instead you have (n,1) which indicates that each line of your array is a single value inside an array. A quick fix to this is by flattening the array, which will turn it into a 1-D array:

ksubmission = pd.DataFrame({'Id':pids,'SalePrice':predictions_kaggle.flatten()})

Hope this helps.

answered Dec 17, 2019 at 1:27

Seraph Wedd

8546 silver badges14 bronze badges

Collectives™ on Stack Overflow

Error when creating dataframe from two columns in Python panda

2 Answers 2

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related