How do you efficiently access multiple slices from numpy array

Question

Currently I am accessing multiple slices as follows:

First, I allocate an array that will be re-assigned many times

X = np.zeros( (batch_size, window, 5) )

This is the assignment loop that will be run multiple times (batch_indices has different indices each time but the same shape):

for i, b in enumerate(batch_indices):
    X[i] = Xs[b:b+window]

Is there a more efficient way? I feel like there should be syntax similar to:

X = Xs[ [slice(b,b+window) for b in batch_indices] ]

While the shape of Xs is 2-dimensional, the final shape of X should be a 3-dimensional np.array. Think of it as follows: Xs is one long multi-dimensional time-series, and X needs to be a numpy array containing many slices of the multi-dimensional time-series.

have you looked into numpy iterations?

Francesco Montesano
– Francesco Montesano

2017-07-17 20:04:44 +00:00
Commented Jul 17, 2017 at 20:04 — Francesco Montesano
– Francesco Montesano, Commented Jul 17, 2017 at 20:04
have you try boolean indexes? worth a performance check...

Raf
– Raf

2017-07-17 20:09:20 +00:00
Commented Jul 17, 2017 at 20:09 — Raf
– Raf, Commented Jul 17, 2017 at 20:09
Did the posted solution work for you?

Divakar
– Divakar

2017-07-25 11:58:59 +00:00
Commented Jul 25, 2017 at 11:58 — Divakar
– Divakar, Commented Jul 25, 2017 at 11:58

Divakar · Accepted Answer · 2017-07-17 20:13:48Z

2

Approach #1

One vectorized approach would be to create all those sliding windowed indices and index into Xs with those, like so -

X = Xs[np.asarray(batch_indices)[:,None] + np.arange(window)]

Approach #2

Another memory efficient approach would be to create sliding-windows with np.lib.stride_tricks.as_strided, thus avoiding the creation of the sliding windowed indices as done in the previous approach and simply index with batch_indices, like so -

X = strided_axis0(Xs,window)[np.asarray(batch_indices)]

Strides based function strided_axis0 is from here.

answered Jul 17, 2017 at 20:13

Divakar

222k19 gold badges273 silver badges374 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How do you efficiently access multiple slices from numpy array

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related