Geek asks and answers

Python - alignment of time series and functions "for today"

I have a dataset with the following first three columns. Include a basket identifier (unique identifier), sale amount (dollars), and transaction date. I want to calculate the next column for each row of a dataset, and I would like to do this in Python.

Previous sale of the same basket (if any); Sales schedule to the current basket; Average time for the current basket (if available); Maximum time for the current basket (if available)

Basket  Sale   Date       PrevSale SaleCount MeanToDate MaxToDate
88      $15 3/01/2012                1      
88      $30 11/02/2012      $15      2         $23        $30
88      $16 16/08/2012      $30      3         $20        $30
123     $90 18/06/2012               1      
477     $77 19/08/2012               1      
477     $57 11/12/2012      $77      2         $67        $77
566     $90 6/07/2012                1

I am new to Python and I am really struggling to find something to really do this. I sorted the data (as indicated above) using BasketID and Date, so I can get the previous sale in bulk by moving forward one for each individual basket. I don’t know how to get MeanToDate and MaxToDate in an efficient way, except for loops ... any ideas?

+5
source share
2 answers

This should do the trick:

from pandas import concat
from pandas.stats.moments import expanding_mean, expanding_count

def handler(grouped):
    se = grouped.set_index('Date')['Sale'].sort_index()
    # se is the (ordered) time series of sales restricted to a single basket
    # we can now create a dataframe by combining different metrics
    # pandas has a function for each of the ones you are interested in!
    return  concat(
        {
            'MeanToDate': expanding_mean(se), # cumulative mean
            'MaxToDate': se.cummax(),         # cumulative max
            'SaleCount': expanding_count(se), # cumulative count
            'Sale': se,                       # simple copy
            'PrevSale': se.shift(1)           # previous sale
        },
        axis=1
     )

# we then apply this handler to all the groups and pandas combines them
# back into a single dataframe indexed by (Basket, Date)
# we simply need to reset the index to get the shape you mention in your question
new_df = df.groupby('Basket').apply(handler).reset_index()

Read more about grouping / aggregation here .

+4
source 



import pandas as pd
pd.__version__  # u'0.24.2'

from pandas import concat

def handler(grouped):
    se = grouped.set_index('Date')['Sale'].sort_index()
    return  concat(
        {
            'MeanToDate': se.expanding().mean(),   # cumulative mean
            'MaxToDate': se.expanding().max(),  # cumulative max
            'SaleCount': se.expanding().count(),   # cumulative count
            'Sale': se,                # simple copy
            'PrevSale': se.shift(1)   # previous sale
        },
        axis=1
     )

###########################
from datetime import datetime  
df = pd.DataFrame({'Basket':[88,88,88,123,477,477,566],
                  'Sale':[15,30,16,90,77,57,90],
                  'Date':[datetime.strptime(ds,'%d/%m/%Y') 
                          for ds in ['3/01/2012','11/02/2012','16/08/2012','18/06/2012',
                                    '19/08/2012','11/12/2012','6/07/2012']]})
#########

new_df = df.groupby('Basket').apply(handler).reset_index()
0

More articles:


All Articles