pandas, convert DataFrame to MultiIndex'ed DataFrame

Question

I have a pandas.DataFrame that I want to convert to a MultiIndexed pandas.DataFrame.

import numpy
import pandas
import itertools

xs = numpy.linspace(0, 10, 100)
ys = numpy.linspace(0, 0.1, 20)
zs = numpy.linspace(0, 5, 200)

def func(x, y, z):
    return x * y / z

vals = list(itertools.product(xs, ys, zs))
result = [func(x, y, z) for x, y, z in vals]

# Original DataFrame.
df = pandas.DataFrame(vals, columns=['x', 'y', 'z'])
df = pandas.concat((pandas.DataFrame(result, columns=['result']), df), axis=1)

# I want to turn `df` into this `df2`.
index = pandas.MultiIndex.from_tuples(vals, names=['x', 'y', 'z'])
df2 = pandas.DataFrame(result, columns=['result'], index=index)

Note that in this example I create what I want and what I have.

So, IRL I would start with df and want to turn it into df2 (and don't have access to vals and result), how do I do this?

jezrael · Accepted Answer · 2016-11-09 13:04:12Z

15

You need set_index:

print (df2.head())
                  result
x   y   z               
0.0 0.0 0.000000     NaN
        0.025126     0.0
        0.050251     0.0
        0.075377     0.0
        0.100503     0.0

print (df.set_index(['x','y','z']).head())

                  result
x   y   z               
0.0 0.0 0.000000     NaN
        0.025126     0.0
        0.050251     0.0
        0.075377     0.0
        0.100503     0.0

If need compare both DataFrames, need replace NaN to same values, else get False:

print (df.set_index(['x','y','z']).eq(df2).all())
result    False
dtype: bool

print (np.nan == np.nan)
False

print (df.fillna(1).set_index(['x','y','z']).eq(df2.fillna(1)).all())
result    True
dtype: bool

edited Nov 9, 2016 at 13:04

answered Nov 9, 2016 at 12:54

jezrael

868k103 gold badges1.4k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Murtaza Haji Over a year ago

Can we do this for non numerical values? Like have the whitespaces for all common values after the first value in index x and y column?

jezrael Over a year ago

@MurtazaHaji - Do you think empty strings under x,y in answer? It is only for display

Collectives™ on Stack Overflow

pandas, convert DataFrame to MultiIndex'ed DataFrame

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related