Summation of only consecutive values in a python array

Question

I am new on python (and even programing!), so I will try to be as clear as I can to explain my question. For you guys it could be easy, but I have not found a satisfactory result on this yet.

Here is the problem:

I have an array with both negative and positive values, say:

x = numpy.array([1, 4, 2, 3, -1, -6, -6, 5, 6, 7, 3, 1, -5, 4, 9, -5, -2, -1, -4])

I would like to sum ONLY the negative values that are continuous, i.e. only sum(-1, -6, -6), sum(-5, -2, -1, -4) and so on. I have tried using numpy.where, as well as numpy.split based on the condition.

For example:

 for i in range(len(x)):
     if x[i] < 0.:
         y[i] = sum(x[i])

However, as you can expect, I just got the summation of all negative values in the array instead. In this case sum(-1, -6, -6, -5, -5, -2, -1, -4) Could guys share with me an aesthetic and efficient way to solve this problem? I will appreciate any response on this.

Thank you very much

Are you looking for a numpy solution only or pure python solution too? — Akavall
– Akavall, Commented Sep 24, 2015 at 20:52
For me it is fine with a pythonic solution. I am trying to follow the advice Kasramvd. However, if you recommend me an other option besides using itertools, that would be fine too. — hurrdrought
– hurrdrought, Commented Sep 24, 2015 at 21:11
@AshwiniChaudhary, the results expected should be like Kasramvd have showed. But I am open to try with other options besides itertools as well. Thank you! — hurrdrought
– hurrdrought, Commented Sep 24, 2015 at 21:27
@hurrdrought if any of these answers resolved your question, you should mark it as accepted — Azad
– Azad, Commented Nov 7, 2015 at 15:54

Kasravnd · Accepted Answer · 2015-09-24 21:14:56Z

7

You can use itertools module, here with using groupby you can grouping your items based on those sign then check if it meet the condition in key function so it is contains negative numbers then yield the sum else yield it and at last you can use chain.from_iterable function to chain the result :

>>> from itertools import groupby,tee,chain
>>> def summ_neg(li):
...     for k,g in groupby(li,key=lambda i:i<0) :
...           if k:
...              yield [sum(g)]
...           yield g
... 
>>> list(chain.from_iterable(summ_neg(x)))
[1, 4, 2, 3, -13, 5, 6, 7, 3, 1, -5, 4, 9, -12]

Or as a more pythonic way use a list comprehension :

 list(chain.from_iterable([[sum(g)] if k else list(g) for k,g in groupby(x,key=lambda i:i<0)]))
[1, 4, 2, 3, -13, 5, 6, 7, 3, 1, -5, 4, 9, -12]

edited Sep 24, 2015 at 21:14

answered Sep 24, 2015 at 20:57

Kasravnd

108k19 gold badges167 silver badges195 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Ashwini Chaudhary Over a year ago

You don't need two iterators, simply check if the value of _(i.e key) is True or False. If it's True then it's a group of positive numbers.

Kasravnd Over a year ago

@AshwiniChaudhary Ooo yeah sure, as I always use _ instead of k I just forgot it's use ;-) thanks for reminding that!

hurrdrought Over a year ago

@Kasramvd, thank you very much for this option. It is working as expected. Also, do you know a way to know the sum of those negative values but knowing how many numbers (i.e. the shape of those chunks) were added? I know this could be complex. Thanks in advance.

Kasravnd Over a year ago

@hurrdrought Welcome! in that case you can convert g to a list and put it in a variable after the for loop and after the condition you can just yield it and its length using len(var).

Divakar · Accepted Answer · 2015-09-24 23:04:54Z

1

Here's a vectorized NumPythonic solution -

# Mask of negative numbers
mask = x<0

# Differentiation between Consecutive mask elements. We would look for 
# 1s and -1s to detect rising and falling edges in the mask corresponding 
# to the islands of negative numbers.
diffs = np.diff(mask.astype(int))

# Mask with 1s at start of negative islands
start_mask = np.append(True,diffs==1) 

# Mask of negative numbers with islands of one isolated negative numbers removed
mask1 = mask & ~(start_mask & np.append(diffs==-1,True))

# ID array for IDing islands of negative numbers
id = (start_mask & mask1).cumsum()

# Finally use bincount to sum elements within their own IDs
out = np.bincount(id[mask1]-1,x[mask1])

You can also use np.convolve to get mask1, like so -

mask1 = np.convolve(mask.astype(int),np.ones(3),'same')>1

You can also get the count of negative numbers in each "island" with a little tweak to existing code -

counts = np.bincount(id[mask1]-1)

Sample run -

In [395]: x
Out[395]: 
array([ 1,  4,  2,  3, -1, -6, -6,  5,  6,  7,  3,  1, -5,  4,  9, -5, -2,
       -1, -4])

In [396]: out
Out[396]: array([-13., -12.])

In [397]: counts
Out[397]: array([3, 4])

edited Sep 24, 2015 at 23:04

answered Sep 24, 2015 at 21:18

Divakar

222k19 gold badges273 silver badges374 bronze badges

5 Comments

hurrdrought Over a year ago

Thank you very much! I am also trying this option with good results too. It is good because I can see how many numbers I have added (i.e. counts).

hurrdrought Over a year ago

I wanted to let you know that, thanks to your post, I have (almost) finished my program. I really like this simple but efficient way to do this. Thank you very much.

Divakar Over a year ago

@hurrdrought Awesome! I would think this solution to be efficient in terms of performance as this avoids loops and that's where NumPy based solutions shine. Were you also able to time these approaches for your dataset?

hurrdrought Over a year ago

Yes, I got the expected results from my datasets using your recommended code. Now, I am trying to do the same but using a larger dataset instead (>1000 of single data). Thanks!

hurrdrought Over a year ago

I am sorry for the delay. Yes! It is working with 1D arrays. Now, I am trying to do the same but applied to 3D arrays (netcdf files). I guess that it will work. However, my script is very slow running in Python. I do not know whether it is related to the efficiency of my program (for sure), the performance of python or both. Maybe later I will submit a new issue about this. Thank you very much for your suggestions!

LetzerWille · Accepted Answer · 2015-09-24 23:37:50Z

you can flag negative values .... and do this with plain python

prev = False

    for i,v in enumerate(a):
            j = i + 1     
            if j < len(a):
                if a[i] < 0 and  a[j] < 0:
                    temp.append(v)
                    prev = True
                elif a[i] < 0 and prev:
                    temp.append(v)
                    prev = True
                elif a[i] > 0:
                    prev = False
            else:
                if prev and v < 0:
                    temp.append(v)

output

print(temp)

[-1, -6, -6, -5, -2, -1, -4]

with intertools i would do just that

def sum_conseq_negative(li):
    neglistAll = []
    for k, g in groupby(li, key=lambda i:i<0):
        negList = list(g)
        if k and len(negList) > 1:
            neglistAll.extend(negList)
    return sum(negList), len(negList)

 sumOf, numOf = sum_conseq_negative(li)

print("sum of negatives {} number of summed {}".format(sumOf,numOf))

sum of negatives -25 number of summed 7

Collectives™ on Stack Overflow

Summation of only consecutive values in a python array

3 Answers 3

4 Comments

5 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

4 Comments

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related