Python / Data Science Essentials Interview Questions
When should you use df.apply() versus vectorised Pandas operations?
apply() runs a Python function on every row or column of a DataFrame. It is the most flexible transformation tool in Pandas but also the slowest because it falls back to a Python-level loop under the hood.
import pandas as pd import numpy as np df = pd.DataFrame({'price': [10.5, 20.0, 8.75, 35.0], 'qty': [3, 5, 2, 1 ]}) # --- Slow: apply with a Python lambda --- df['revenue'] = df.apply(lambda row: row['price'] * row['qty'], axis=1) # --- Fast: vectorised arithmetic (always prefer this) --- df['revenue'] = df['price'] * df['qty'] # --- apply on a single column (Series.apply) --- df['price_cat'] = df['price'].apply(lambda x: 'high' if x > 20 else 'low') # --- Faster alternative: np.where --- df['price_cat'] = np.where(df['price'] > 20, 'high', 'low') # --- Multi-condition: np.select --- conditions = [df['price'] > 25, df['price'] > 15, df['price'] > 0] choices = ['premium', 'mid', 'budget'] df['tier'] = np.select(conditions, choices, default='unknown') # When apply is genuinely needed: # â calling a function that returns a list/dict/Series per row # â complex multi-column logic that cannot be expressed as vectorised ops df.apply(lambda row: pd.Series({'x': row['price']+1, 'y': row['qty']*2}), axis=1)
Performance hierarchy for transformations (fastest to slowest): vectorised arithmetic > NumPy ufuncs > df.eval() string expressions > map() on a Series > apply() > explicit Python for-loop. Use apply only when no vectorised alternative exists; for simple conditions always use np.where or boolean indexing instead.
Why is df['price'] * df['qty'] faster than df.apply(lambda row: row['price']*row['qty'], axis=1)?
Which NumPy function is the recommended vectorised replacement for a simple if/else transformation on a column?
More Related questions...
