Python / Data Science Essentials Interview Questions
How do rolling and expanding window functions work in Pandas?
Window functions compute statistics over a sliding or expanding subset of rows, essential for time-series smoothing, trend detection, and feature engineering. Unlike groupby aggregations, window functions return a result for every row, preserving the original index.
import pandas as pd import numpy as np ts = pd.DataFrame({ 'date': pd.date_range('2024-01-01', periods=10, freq='D'), 'sales': [100, 120, 90, 150, 200, 130, 110, 180, 160, 140], }) ts = ts.set_index('date') # --- Rolling window (fixed-size, slides one step at a time) --- ts['ma3'] = ts['sales'].rolling(window=3).mean() # 3-day moving avg ts['std3'] = ts['sales'].rolling(window=3).std() ts['min3'] = ts['sales'].rolling(window=3).min() # First window-1 values are NaN (not enough history) # min_periods: require fewer observations before computing ts['ma3_mp'] = ts['sales'].rolling(window=3, min_periods=1).mean() # --- Expanding window (grows to include all rows so far) --- ts['cum_max'] = ts['sales'].expanding().max() ts['cum_mean'] = ts['sales'].expanding().mean() # --- Exponentially weighted moving average (more weight on recent data) --- ts['ewma'] = ts['sales'].ewm(span=3).mean() # --- Lag / shift features (common in time-series forecasting) --- ts['lag1'] = ts['sales'].shift(1) # yesterday's sales ts['lag7'] = ts['sales'].shift(7) # last week's sales ts['pct_change'] = ts['sales'].pct_change() # % change from previous row
Moving averages (rolling mean) smooth out noise to reveal trends. Exponentially weighted moving averages give more influence to recent observations, making them responsive to recent changes while still smoothing. Lag features turn a time-series prediction problem into a supervised learning problem where past values predict future ones.
Why do the first (window - 1) rows of a rolling() calculation contain NaN?
What does ts['sales'].shift(1) produce?
More Related questions...
