Pandas — Selecting & Filtering Data
Master data slicing in Pandas: select specific columns, extract rows via label-based loc and positional iloc, create Boolean filter masks with &, |, and ~, use isin(), and combine row filtering with column projection.
1. Why Select & Filter Data?
A corporate sales database contains millions of transactions spanning years. An analyst rarely studies the entire raw table at once. Instead, business questions demand precision:
Isolating specific variables or index positions (e.g. keeping only Product and Sales while dropping 40 irrelevant columns).
Retaining only records that satisfy analytical criteria (e.g. Sales >= 50000 or Region == "North").
2. Selecting Columns: Single vs. Double Brackets
Pandas provides two column selection modes based on bracket count:
sales_series = df["Sales"]
# 2. Double brackets: Returns a 2D DataFrame
sales_df = df[["Sales"]]
# 3. Multiple columns: Returns a 2D DataFrame
subset_df = df[["Product", "Sales"]]
3. 🔥 Live Column Selection Lab
Given an inventory table with Product, Category, Price, and Sales. Complete the 3 column projection tasks:
4. Selecting Rows: loc vs. iloc
Row selection requires choosing between label coordinates and integer memory positions:
- Selects by row and column labels
df.loc[0]finds the row labeled0- Inclusive Slicing:
df.loc[0:2]includes rows0, 1, and 2!
- Selects by integer positions (0 to length-1)
df.iloc[0]finds the first physical row- Exclusive Slicing:
df.iloc[0:2]returns rows0 and 1(standard Python rules).
6. 🔥 Live Row Selection Lab
Use iloc and loc on an employee table:
7. Boolean Filtering: The Mask Mechanism
Filtering in Pandas works in two connected phases:
mask = df["Sales"] > 50000
# [False, True, False, True]
# Phase 2: Passing mask to df[...] retains only True rows
high_sales = df[df["Sales"] > 50000]
9. Multiple Conditions: Bitwise &, |, and ~
When combining conditions, standard Python and/or keywords fail. You must use:
(df["Sales"] > 50000) & (df["Age"] < 25).10. 🔥 Live Multi-Condition Filtering Lab
Filter the department records below:
12. Select Columns + Filter Rows with df.loc
In data analytics, you often need to filter rows and project specific columns simultaneously:
result = df.loc[df["Sales"] >= 60000, ["Name", "Sales"]]
print(result)
14. 🔥 Final Live Challenge: Employee Dataset Filtering
Perform 5 targeted queries on employee performance:
- Select
EmployeeandSalarycolumns - Filter employees with
Salary >= 55000 - Find Sales employees with
Performance >= 90 - Find employees from HR or IT using
isin() - Select only
EmployeeandPerformancefor employees withPerformance >= 85
15. Knowledge Assessment Quiz
Test your understanding of loc, iloc, Boolean masks, and multi-condition subsetting:
Pandas Selecting & Filtering Certification Quiz
Test your understanding with real-world query prediction and syntax questions.
What is the primary difference between df.loc and df.iloc in Pandas?