Python EDA: Descriptive Statistics & Numerical Summaries
Transform raw tables into actionable business intelligence. Master central tendency (mean, median, mode), quantify dispersion (range, variance, standard deviation), analyze quartile distributions (Q1, Q2, Q3, IQR), and leverage describe() to rapidly extract the statistical story behind your data.
1. What is Descriptive Statistics?
Numerical summaries that describe essential data characteristics
Consider this simple salary column: 30000, 32000, 35000, 40000, 45000. What can an analyst learn without manually inspecting every individual row?
Center
What is a typical salary?
Spread
How dispersed are values?
Range
What are min & max?
Middle 50%
Where do most peers sit?
Anomalies
Are there unusual spikes?
• Descriptive: Summarizes and characterizes the dataset you currently possess (what we teach here).
• Inferential: Uses sample data to draw probabilistic conclusions about an unobserved wider population.
2. The Mean: Arithmetic Average
Summing values divided by observation count
The mean is the most common measure of center. In Pandas:
Pandas Syntax
df["Salary"].mean()Sum of values ÷ number of values. For 30k, 32k, 35k, 40k, 45k, the mean is ₹36,400.
Extreme Value Sensitivity
30k, 32k, 35k, 40k, 350kReplace 45k with 350k: the mean jumps to ₹97,400! Not a single typical employee earns anywhere near ₹97,400.
3. The Median: Robust Middle Value
Position-based center that resists extreme outliers
Sort your data in ascending order; the median is the exact physical midpoint:
Exact middle element is the 3rd value: ₹35,000.
Average of the two middle elements (32k + 36k) / 2 = ₹34,000.
Vectorized, highly efficient, and automatically ignores NaNs.
• For symmetrical, bell-shaped data without extreme spikes: Mean is standard.
• For skewed distributions (income, real estate, net worth): Median is far more honest.
4. The Mode: Most Frequent Value
Indispensable for categorical columns and product popularity
The mode is the observation that appears with the highest frequency:
Categorical Example
["Mumbai", "Delhi", "Mumbai", "Pune", "Mumbai"]"Mumbai" appears 3 times. Mode = "Mumbai".
Pandas Syntax
df["City"].mode()Returns a Series (because datasets can be bimodal with ties for 1st place).
5. Minimum, Maximum & Range
Establishing boundaries and overall data boundaries
Smallest observed value in the column.
Largest observed value in the column.
Total distance spanned between lowest and highest point.
6. Quartiles & the Interquartile Range (IQR)
Dividing ordered data into four quarters
Quartile Benchmarks
• Q1 (25th percentile): df["Salary"].quantile(0.25)
• Q2 (50th percentile / Median): df["Salary"].quantile(0.50)
• Q3 (75th percentile): df["Salary"].quantile(0.75)
The Interquartile Range (IQR)
IQR = Q3 - Q1Measures the spread of the middle 50% of your data. Completely immune to extreme minimums and maximums, serving as the statistical backbone for outlier detection.
7. Variance & Standard Deviation
Quantifying how observations scatter around the mean
Why isn’t the mean enough? Compare these two groups of test scores:
Group A: Clustered
48, 49, 50, 51, 52• Mean: 50.0
• Standard Deviation: 1.58
• Highly consistent; scores are clustered tightly near 50.
Group B: Dispersed
20, 35, 50, 65, 80• Mean: 50.0 (Identical mean!)
• Standard Deviation: 23.18
• Highly volatile; scores are widely scattered from the center.
•
df["Salary"].var() → Sample variance (in squared units).•
df["Salary"].std() → Standard deviation (in original units, e.g. Rupees or Years). A larger standard deviation indicates greater variability.8. df.describe(): The EDA Swiss Army Knife
Extracting an instant 8-point numerical overview of every column
Calling df.describe() computes all essential metrics in a single line:
| Metric | Meaning | Analyst Takeaway |
|---|---|---|
count | Total non-null rows | Quick check for missing data |
mean | Arithmetic average | General center |
std | Sample standard deviation | Spread around average |
min | Lowest observed value | Absolute floor |
25% | First quartile (Q1) | 25% of data sits below here |
50% | Median (Q2) | Exact physical middle |
75% | Third quartile (Q3) | 75% of data sits below here |
max | Highest observed value | Absolute ceiling |
9. 🔥 Live describe() Practice
Generate statistics and evaluate whether Salary has an extreme right-skew
10. Mean vs. Median: Real Impact of an Outlier
Observe how an outlier shifts the mean by 70% while leaving median untouched
11. Descriptive Statistics for Categorical Data
How describe() summarizes text columns
When invoked on non-numeric columns like df["City"].describe(), Pandas produces:
Total non-null string entries.
Count of distinct categories/values.
The mode (most frequent string).
Number of times the "top" string occurred.
12. Practical Enterprise EDA: Employee Salary Audit
Synthesizing statistics across multiple business dimensions
| Employee | Department | Age | Salary | Performance |
|---|---|---|---|---|
| Amit | Sales | 24 | ₹42,000 | 78 |
| Priya | HR | 27 | ₹50,000 | 85 |
| Rahul | Sales | 25 | ₹45,000 | 82 |
| Neha | IT | 29 | ₹60,000 | 91 |
| Karan | Sales | 31 | ₹48,000 | 80 |
| Riya | IT | 26 | ₹65,000 | 88 |
| Vikas | HR | 35 | ₹120,000 | 86 |
Key Analytical Takeaways:
- Mean vs Median: Mean salary is ₹61,428, but median is ₹50,000. Vikas’s executive salary pulls the mean up by ₹11,428.
- Spread: Performance scores are tightly clustered (Range: 78 to 91, std ~ 4.5), while salaries span from ₹42k to ₹120k (Range: ₹78k).
- Recommendation: When reporting to staff, present the median salary (₹50,000) as the true representative figure.
13. Descriptive Statistics vs. Full EDA
Numerical summaries are step one in an ongoing discovery cycle
Descriptive Statistics
The mathematical numbers that quantify center, spread, percentiles, and boundaries.
Answers:"What are the numbers?"
Full Exploratory Data Analysis (EDA)
The investigative holistic process combining descriptive statistics, charts, scatter plots, correlations, and business questions.
Answers:"What business patterns do these numbers reveal?"
14. 🔥 Final Live Challenge: Retail Catalog EDA
Analyze product pricing, sales volume dispersion, and extract business insights
You are auditing an e-commerce catalog with 8 products. Generate the full descriptive statistics table, compare mean vs median price, calculate range and IQR, and examine UnitsSold standard deviation:
15. Common Mistakes to Avoid
Pitfalls made when interpreting descriptive statistics
In right-skewed data, the mean is pulled upwards. Always compare mean with median to check for skewness.
Range only measures the difference between the two most extreme points (Max - Min). Standard deviation accounts for how all points scatter around the mean.
Descriptive statistics are merely the numerical summary phase. Full EDA requires visual exploration, cross-tabulation, and investigating business causality.
Q1 and Q3 mark the 25th and 75th percentiles (the middle 50% boundary). Min and Max represent the absolute single lowest and highest points.
16. Knowledge Check Quiz
Validate your descriptive statistics mastery
Python Descriptive Statistics Knowledge Check
Validate your understanding of mean, median, mode, range, quartiles, IQR, standard deviation, and describe().
Why is the sample median often preferred over the sample mean when analyzing income or real-estate prices?
17. Key Takeaways Checklist
Skills you have mastered in Descriptive Statistics
.mean(), .median(), and .mode()..min(), .max(), and Range..quantile() and determine the IQR..var()) and standard deviation (.std()).df.describe().