EDA and Data Visualisation Tutorial | Matplotlib, Seaborn, Plotly | Free

.ipynb

Follow along in JupyterDownload the complete Week 5 notebook — every code example ready to run.

Data Science Fundamentals Course

Week 5 Overview

Exploratory Data Analysis (EDA) is the critical phase where you get to know your data deeply before building any models. It involves using statistics and visualizations to understand patterns, distributions, relationships, and anomalies in your data. Effective EDA can reveal insights, inform feature engineering decisions, and prevent costly mistakes in modeling. This week focuses on:

Calculating descriptive statistics (mean, median, mode, variance, etc.)
Understanding distributions and their properties
Creating compelling visualizations with Matplotlib and Seaborn
Analyzing relationships between variables
Creating professional-quality plots suitable for reports and presentations

By the end of Week 5, you will be able to:

Calculate and interpret comprehensive summary statistics
Create distributions and identify skewness and outliers
Build beautiful, informative visualizations
Analyze correlations between variables
Communicate data insights through effective visualizations
Create publication-quality plots for reports and presentations

Week 5 is divided into three 2-hour sessions:

Session 1: Descriptive Statistics and Univariate Analysis
Session 2: Matplotlib and Seaborn for Data Visualization
Session 3: Correlation Analysis, Advanced Visualizations, and Plotly

SESSION 1: Descriptive Statistics and Univariate Analysis

Duration: 2 hours

1.1 What is Exploratory Data Analysis?

EDA is the process of analyzing datasets to summarize their main characteristics, often using statistical graphics and other data visualization methods. The goals of EDA are to:

Understand data structure and content
Identify patterns, trends, and anomalies
Check data quality (missing values, outliers)
Generate hypotheses for testing
Inform feature engineering decisions
Detect relationships between variables
Prepare data for modeling

EDA is NOT about finding the final answer, but about asking questions and exploring what the data tells you.

1.2 Descriptive Statistics

import pandas as pd
import numpy as np

# Create sample dataset
df = pd.DataFrame({
'Age': [22, 25, 28, 30, 32, 35, 38, 40, 42, 45, 48, 50],
'Salary': [35000, 42000, 48000, 52000, 58000, 62000, 68000, 72000, 78000, 85000, 92000, 100000],
'YearsExp': [1, 2, 3, 4, 5, 7, 8, 10, 12, 15, 18, 20]
})

print("Dataset:")
print(df)
print()

# Basic statistics
print("="*50)
print("DESCRIPTIVE STATISTICS")
print("="*50)

# Mean (average)
print(f"Mean Age: {df['Age'].mean():.2f}")
print(f"Mean Salary: {df['Salary'].mean():.2f}")

# Median (middle value)
print(f"Median Age: {df['Age'].median():.2f}")
print(f"Median Salary: {df['Salary'].median():.2f}")

# Mode (most frequent value)
print(f"Mode Age: {df['Age'].mode().values[0] if len(df['Age'].mode()) > 0 else 'No mode'}")

# Standard deviation (spread)
print(f"Std Dev Age: {df['Age'].std():.2f}")
print(f"Std Dev Salary: {df['Salary'].std():.2f}")

# Variance (squared standard deviation)
print(f"Variance Age: {df['Age'].var():.2f}")

# Range
print(f"Range Age: {df['Age'].max() - df['Age'].min()}")

# Min and Max
print(f"Min Age: {df['Age'].min()}")
print(f"Max Age: {df['Age'].max()}")

# Quartiles
print(f"\n25th percentile (Q1) Age: {df['Age'].quantile(0.25):.2f}")
print(f"50th percentile (Q2/Median) Age: {df['Age'].quantile(0.50):.2f}")
print(f"75th percentile (Q3) Age: {df['Age'].quantile(0.75):.2f}")

# IQR (Interquartile Range)
Q1 = df['Age'].quantile(0.25)
Q3 = df['Age'].quantile(0.75)
IQR = Q3 - Q1
print(f"IQR Age: {IQR:.2f}")

# Use pandas describe() for quick summary
print("\n" + "="*50)
print("PANDAS describe() METHOD")
print("="*50)
print(df.describe())
# Shows: count, mean, std, min, 25%, 50%, 75%, max

# describe() with more percentiles
print("\n" + "="*50)
print("WITH CUSTOM PERCENTILES")
print("="*50)
print(df.describe(percentiles=[0.1, 0.25, 0.5, 0.75, 0.9]))

# Correlation matrix
print("\n" + "="*50)
print("CORRELATION")
print("="*50)
print(df.corr())

# Data types and memory
print("\n" + "="*50)
print("DATA INFO")
print("="*50)
print(df.info())

# Value counts (for categorical)
df_cat = pd.DataFrame({
'Department': ['Sales', 'IT', 'Finance', 'Sales', 'IT', 'Sales', 'HR', 'Finance', 'Sales', 'IT'],
'Performance': ['Good', 'Excellent', 'Good', 'Fair', 'Excellent', 'Good', 'Fair', 'Excellent', 'Good', 'Excellent']
})

print("\n" + "="*50)
print("VALUE COUNTS")
print("="*50)
print(df_cat['Department'].value_counts())
print("\nDepartment percentages:")
print(df_cat['Department'].value_counts(normalize=True) * 100)

1.3 Understanding Distributions

import pandas as pd
import numpy as np
from scipy import stats

# Create datasets with different distributions
np.random.seed(42)

# Normal distribution
normal_data = np.random.normal(loc=100, scale=15, size=1000)

# Right-skewed distribution (e.g., income)
right_skewed = np.random.exponential(scale=2, size=1000)

# Left-skewed distribution
left_skewed = -np.random.exponential(scale=2, size=1000)

df = pd.DataFrame({
'Normal': normal_data,
'Right_Skewed': right_skewed,
'Left_Skewed': left_skewed
})

# Skewness (measure of asymmetry)
# Positive skew: tail on right (mean > median)
# Negative skew: tail on left (mean < median)
# Skewness ~ 0: fairly symmetric
print("="*50)
print("SKEWNESS")
print("="*50)
print(f"Normal Skewness: {stats.skew(df['Normal']):.3f}")
print(f"Right-Skewed Skewness: {stats.skew(df['Right_Skewed']):.3f}")
print(f"Left-Skewed Skewness: {stats.skew(df['Left_Skewed']):.3f}")

# Kurtosis (measure of tail weight/peakedness)
# High kurtosis: heavy tails (more outliers)
# Low kurtosis: light tails (fewer outliers)
print("\n" + "="*50)
print("KURTOSIS")
print("="*50)
print(f"Normal Kurtosis: {stats.kurtosis(df['Normal']):.3f}")
print(f"Right-Skewed Kurtosis: {stats.kurtosis(df['Right_Skewed']):.3f}")
print(f"Left-Skewed Kurtosis: {stats.kurtosis(df['Left_Skewed']):.3f}")

# Real-world example: Analyze test scores
scores = pd.DataFrame({
'Test1': np.random.normal(75, 10, 100),
'Test2': np.array([45]*10 + [90]*50 + [75]*40), # Bimodal
'Test3': np.random.normal(85, 5, 100) # Most pass
})

print("\n" + "="*50)
print("TEST SCORES ANALYSIS")
print("="*50)
print(scores.describe())
print("\nSkewness:")
print(scores.skew())
print("\nKurtosis:")
print(scores.kurtosis())

# Interpretation
print("\n" + "="*50)
print("INTERPRETATION")
print("="*50)
print("Skewness > 0: Right-skewed (positive skew)")
print("Skewness < 0: Left-skewed (negative skew)")
print("Skewness ~ 0: Symmetric/Normal")
print()
print("Kurtosis > 0: Heavy-tailed (leptokurtic)")
print("Kurtosis < 0: Light-tailed (platykurtic)")
print("Kurtosis ~ 0: Normal-like (mesokurtic)")

1.4 Bivariate and Multivariate Analysis

import pandas as pd
import numpy as np

# Create sample data
df = pd.DataFrame({
'Age': [22, 25, 28, 30, 32, 35, 38, 40, 42, 45],
'Salary': [35000, 42000, 48000, 52000, 58000, 62000, 68000, 72000, 78000, 85000],
'YearsExp': [1, 2, 3, 4, 5, 7, 8, 10, 12, 15],
'Department': ['Sales', 'IT', 'Sales', 'Finance', 'IT', 'Sales', 'HR', 'Finance', 'IT', 'Sales']
})

print("Dataset:")
print(df)

# Correlation between two variables
print("\n" + "="*50)
print("BIVARIATE ANALYSIS: CORRELATION")
print("="*50)

correlation = df['Age'].corr(df['Salary'])
print(f"Correlation between Age and Salary: {correlation:.3f}")

correlation2 = df['YearsExp'].corr(df['Salary'])
print(f"Correlation between Experience and Salary: {correlation2:.3f}")

# Covariance
print("\n" + "="*50)
print("COVARIANCE")
print("="*50)
cov = df[['Age', 'Salary']].cov()
print(cov)

# Correlation matrix (all numeric columns)
print("\n" + "="*50)
print("CORRELATION MATRIX")
print("="*50)
print(df.corr())

# Group by analysis
print("\n" + "="*50)
print("GROUP ANALYSIS")
print("="*50)

# Average salary by department
dept_stats = df.groupby('Department')['Salary'].agg(['mean', 'std', 'count', 'min', 'max'])
print("\nSalary statistics by Department:")
print(dept_stats)

# Multiple aggregations
agg_stats = df.groupby('Department').agg({
'Salary': ['mean', 'min', 'max'],
'Age': ['mean', 'std'],
'YearsExp': 'mean'
})
print("\nMultiple statistics by Department:")
print(agg_stats)

# Pivot table
print("\n" + "="*50)
print("PIVOT TABLE")
print("="*50)

# Create sample with more data
df_large = pd.DataFrame({
'Month': ['Jan', 'Jan', 'Feb', 'Feb', 'Mar', 'Mar'] * 2,
'Region': ['North', 'South', 'North', 'South', 'North', 'South'] * 2,
'Sales': [1000, 1200, 1500, 1300, 1800, 1600, 1100, 1250, 1550, 1350, 1850, 1650]
})

pivot = df_large.pivot_table(values='Sales', index='Region', columns='Month', aggfunc='mean')
print(pivot)

# Real-world: Customer analysis
customers = pd.DataFrame({
'CustomerID': range(1, 11),
'Age': [25, 32, 28, 45, 38, 52, 29, 35, 41, 48],
'Spending': [1000, 2500, 1800, 5000, 3500, 6000, 2000, 2800, 4200, 5500],
'Region': ['North', 'South', 'North', 'South', 'East', 'West', 'North', 'South', 'East', 'West'],
'Visits': [5, 10, 8, 15, 12, 20, 6, 9, 14, 18]
})

print("\n" + "="*50)
print("CUSTOMER ANALYSIS")
print("="*50)

# Correlation with spending
print(f"\nAge vs Spending correlation: {customers['Age'].corr(customers['Spending']):.3f}")
print(f"Visits vs Spending correlation: {customers['Visits'].corr(customers['Spending']):.3f}")

# By region
print("\nAverage spending by region:")
print(customers.groupby('Region')['Spending'].agg(['mean', 'count']))

SESSION 2: Matplotlib and Seaborn for Data Visualization

Duration: 2 hours

2.1 Introduction to Matplotlib

import matplotlib.pyplot as plt
import pandas as pd
import numpy as np

# Create sample data
np.random.seed(42)
data = pd.DataFrame({
'Month': ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun'],
'Sales': [1000, 1200, 950, 1400, 1600, 1800],
'Expenses': [600, 700, 650, 800, 900, 950]
})

# Basic line plot
plt.figure(figsize=(10, 6))
plt.plot(data['Month'], data['Sales'], marker='o', linewidth=2, markersize=8)
plt.title('Monthly Sales', fontsize=16, fontweight='bold')
plt.xlabel('Month', fontsize=12)
plt.ylabel('Sales ($)', fontsize=12)
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()

# Multiple lines
plt.figure(figsize=(10, 6))
plt.plot(data['Month'], data['Sales'], marker='o', label='Sales', linewidth=2)
plt.plot(data['Month'], data['Expenses'], marker='s', label='Expenses', linewidth=2)
plt.title('Sales vs Expenses', fontsize=16, fontweight='bold')
plt.xlabel('Month', fontsize=12)
plt.ylabel('Amount ($)', fontsize=12)
plt.legend(loc='best')
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()

# Scatter plot
np.random.seed(42)
x = np.random.randn(100)
y = 2 * x + np.random.randn(100)

plt.figure(figsize=(10, 6))
plt.scatter(x, y, alpha=0.6, s=100, edgecolors='black', linewidth=0.5)
plt.title('Relationship between X and Y', fontsize=16, fontweight='bold')
plt.xlabel('X values', fontsize=12)
plt.ylabel('Y values', fontsize=12)
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()

# Bar plot
categories = ['A', 'B', 'C', 'D', 'E']
values = [25, 45, 30, 50, 35]

plt.figure(figsize=(10, 6))
bars = plt.bar(categories, values, color=['red', 'blue', 'green', 'orange', 'purple'], alpha=0.7)
plt.title('Sales by Category', fontsize=16, fontweight='bold')
plt.xlabel('Category', fontsize=12)
plt.ylabel('Sales ($)', fontsize=12)
# Add value labels on bars
for bar, value in zip(bars, values):
height = bar.get_height()
plt.text(bar.get_x() + bar.get_width()/2., height,
f'{int(value)}', ha='center', va='bottom', fontsize=10)
plt.grid(True, alpha=0.3, axis='y')
plt.tight_layout()
plt.show()

# Histogram
np.random.seed(42)
data = np.random.normal(100, 15, 1000)

plt.figure(figsize=(10, 6))
plt.hist(data, bins=30, color='skyblue', edgecolor='black', alpha=0.7)
plt.title('Distribution of Test Scores', fontsize=16, fontweight='bold')
plt.xlabel('Score', fontsize=12)
plt.ylabel('Frequency', fontsize=12)
plt.axvline(np.mean(data), color='red', linestyle='--', linewidth=2, label=f'Mean: {np.mean(data):.1f}')
plt.axvline(np.median(data), color='green', linestyle='--', linewidth=2, label=f'Median: {np.median(data):.1f}')
plt.legend()
plt.grid(True, alpha=0.3, axis='y')
plt.tight_layout()
plt.show()

# Subplots
fig, axes = plt.subplots(2, 2, figsize=(12, 10))

# Subplot 1: Line plot
axes[0, 0].plot(data['Month'], data['Sales'], marker='o')
axes[0, 0].set_title('Sales Over Time')
axes[0, 0].grid(True, alpha=0.3)

# Subplot 2: Bar plot
axes[0, 1].bar(data['Month'], data['Expenses'])
axes[0, 1].set_title('Expenses by Month')
axes[0, 1].grid(True, alpha=0.3, axis='y')

# Subplot 3: Histogram
axes[1, 0].hist(np.random.randn(1000), bins=30, color='skyblue', edgecolor='black')
axes[1, 0].set_title('Distribution')

# Subplot 4: Scatter
axes[1, 1].scatter(np.random.randn(100), np.random.randn(100))
axes[1, 1].set_title('Scatter Plot')

plt.suptitle('Dashboard Overview', fontsize=16, fontweight='bold')
plt.tight_layout()
plt.show()

# Save figure
plt.figure(figsize=(10, 6))
plt.plot(data['Month'], data['Sales'], marker='o')
plt.title('Sales')
plt.savefig('sales_plot.png', dpi=300, bbox_inches='tight')
print("Figure saved as 'sales_plot.png'")

2.2 Advanced Visualization with Seaborn

import seaborn as sns
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np

# Set style
sns.set_style("whitegrid")
sns.set_palette("husl")

# Load built-in dataset
iris = sns.load_dataset("iris")
print(iris.head())

# Histogram with KDE
plt.figure(figsize=(10, 6))
sns.histplot(data=iris, x="sepal_length", kde=True, bins=20)
plt.title("Sepal Length Distribution", fontsize=14, fontweight='bold')
plt.tight_layout()
plt.show()

# Box plot (shows quartiles and outliers)
plt.figure(figsize=(10, 6))
sns.boxplot(data=iris, x="species", y="sepal_length")
plt.title("Sepal Length by Species", fontsize=14, fontweight='bold')
plt.tight_layout()
plt.show()

# Violin plot (shows full distribution)
plt.figure(figsize=(10, 6))
sns.violinplot(data=iris, x="species", y="sepal_length")
plt.title("Sepal Length Distribution by Species", fontsize=14, fontweight='bold')
plt.tight_layout()
plt.show()

# Strip plot (individual points)
plt.figure(figsize=(10, 6))
sns.stripplot(data=iris, x="species", y="sepal_length", size=8, jitter=True)
plt.title("Sepal Length by Species (Individual Points)", fontsize=14, fontweight='bold')
plt.tight_layout()
plt.show()

# Count plot (categorical frequencies)
plt.figure(figsize=(10, 6))
sns.countplot(data=iris, x="species")
plt.title("Count of Each Species", fontsize=14, fontweight='bold')
plt.tight_layout()
plt.show()

# Scatter plot with hue
plt.figure(figsize=(10, 6))
sns.scatterplot(data=iris, x="sepal_length", y="sepal_width", hue="species", s=100)
plt.title("Sepal Length vs Width by Species", fontsize=14, fontweight='bold')
plt.tight_layout()
plt.show()

# Regression plot (scatter + trend line)
plt.figure(figsize=(10, 6))
sns.regplot(data=iris, x="sepal_length", y="sepal_width")
plt.title("Sepal Length vs Width (with Trend)", fontsize=14, fontweight='bold')
plt.tight_layout()
plt.show()

# Multiple plots with hue and col
plt.figure(figsize=(14, 6))
sns.lmplot(data=iris, x="sepal_length", y="sepal_width", hue="species", col="species")
plt.suptitle("Sepal Relationships by Species", fontsize=14, fontweight='bold', y=1.01)
plt.tight_layout()
plt.show()

# Real-world: Sales analysis
sales_data = pd.DataFrame({
'Date': pd.date_range('2023-01-01', periods=100),
'Region': np.random.choice(['North', 'South', 'East', 'West'], 100),
'Sales': np.random.randint(1000, 5000, 100),
'Category': np.random.choice(['Electronics', 'Clothing', 'Food'], 100)
})

# Box plot by region
plt.figure(figsize=(10, 6))
sns.boxplot(data=sales_data, x="Region", y="Sales")
plt.title("Sales Distribution by Region", fontsize=14, fontweight='bold')
plt.tight_layout()
plt.show()

# Violin plot with split
plt.figure(figsize=(12, 6))
sns.violinplot(data=sales_data, x="Region", y="Sales", hue="Category", split=False)
plt.title("Sales by Region and Category", fontsize=14, fontweight='bold')
plt.tight_layout()
plt.show()

SESSION 3: Correlation Analysis, Advanced Visualizations, and Plotly

Duration: 2 hours

3.1 Correlation Analysis and Heatmaps

import seaborn as sns
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np

# Create sample data
np.random.seed(42)
df = pd.DataFrame({
'Age': np.random.randint(20, 60, 100),
'Salary': np.random.randint(30000, 150000, 100),
'YearsExp': np.random.randint(0, 30, 100),
'Performance': np.random.randint(1, 5, 100),
'Satisfaction': np.random.randint(1, 5, 100)
})

# Add correlation between some variables
df['Salary'] = df['Age'] * 2000 + df['YearsExp'] * 3000 + np.random.randint(-10000, 10000, 100)
df['Performance'] = (df['YearsExp'] / 2 + np.random.randint(-2, 2, 100)).astype(int)
df['Satisfaction'] = (df['Performance'] + np.random.randint(-1, 1, 100)).astype(int)

print("Data shape:", df.shape)
print(df.head())

# Correlation matrix
print("\n" + "="*50)
print("CORRELATION MATRIX")
print("="*50)
corr_matrix = df.corr()
print(corr_matrix)

# Heatmap of correlations
plt.figure(figsize=(10, 8))
sns.heatmap(corr_matrix, annot=True, fmt='.2f', cmap='coolwarm', center=0,
square=True, linewidths=1, cbar_kws={"shrink": 0.8})
plt.title("Correlation Heatmap", fontsize=14, fontweight='bold')
plt.tight_layout()
plt.show()

# Focused correlation with one variable
salary_corr = df.corr()['Salary'].sort_values(ascending=False)
print("\n" + "="*50)
print("CORRELATION WITH SALARY")
print("="*50)
print(salary_corr)

# Plot correlation with salary
plt.figure(figsize=(10, 6))
salary_corr.drop('Salary').plot(kind='barh', color=['green' if x > 0 else 'red' for x in salary_corr.drop('Salary')])
plt.title("Correlation with Salary", fontsize=14, fontweight='bold')
plt.xlabel("Correlation Coefficient", fontsize=12)
plt.tight_layout()
plt.show()

# Pairplot (all relationships)
print("\nGenerating pairplot (takes a moment)...")
# Using iris dataset for cleaner example
iris = sns.load_dataset("iris")
pairplot = sns.pairplot(iris, hue="species", diag_kind="kde")
pairplot.fig.suptitle("Pairplot of Iris Dataset", fontsize=16, fontweight='bold', y=0.995)
plt.tight_layout()
plt.show()

# Real-world: Employee dataset
employees = pd.DataFrame({
'Age': [25, 28, 32, 35, 38, 42, 45, 48, 52, 55],
'Salary': [45000, 52000, 60000, 68000, 75000, 85000, 95000, 105000, 120000, 135000],
'YearsExp': [1, 3, 5, 8, 10, 15, 18, 20, 25, 28],
'Performance': [3, 3.5, 4, 4, 4.5, 4, 4.5, 5, 5, 4.5],
'Absences': [8, 6, 4, 3, 2, 1, 0, 1, 0, 1]
})

plt.figure(figsize=(10, 8))
corr = employees.corr()
sns.heatmap(corr, annot=True, fmt='.3f', cmap='RdYlGn', center=0,
square=True, linewidths=1, cbar_kws={"shrink": 0.8})
plt.title("Employee Data Correlation Matrix", fontsize=14, fontweight='bold')
plt.tight_layout()
plt.show()

3.2 Advanced Visualization Techniques

import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
import numpy as np

# Faceted plot (multiple subplots by category)
iris = sns.load_dataset("iris")

g = sns.FacetGrid(iris, col="species", height=4)
g.map(sns.scatterplot, "sepal_length", "sepal_width")
plt.suptitle("Sepal Dimensions by Species", fontsize=14, fontweight='bold', y=1.00)
plt.tight_layout()
plt.show()

# Distribution plot with categorization
sales_data = pd.DataFrame({
'Region': np.repeat(['North', 'South', 'East', 'West'], 50),
'Sales': np.concatenate([
np.random.normal(5000, 1000, 50),
np.random.normal(4500, 1200, 50),
np.random.normal(6000, 800, 50),
np.random.normal(5200, 1100, 50)
])
})

plt.figure(figsize=(12, 6))
for region in sales_data['Region'].unique():
data = sales_data[sales_data['Region'] == region]['Sales']
plt.hist(data, alpha=0.5, label=region, bins=20)
plt.title("Sales Distribution by Region", fontsize=14, fontweight='bold')
plt.xlabel("Sales ($)")
plt.ylabel("Frequency")
plt.legend()
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()

# Joint plot (bivariate + marginal distributions)
plt.figure()
joint = sns.jointplot(data=iris, x="sepal_length", y="sepal_width", kind="hex")
joint.fig.suptitle("Sepal Dimensions Joint Distribution", fontsize=14, fontweight='bold', y=0.995)
plt.tight_layout()
plt.show()

# Clustermap (hierarchical clustering visualization)
np.random.seed(42)
data = np.random.rand(10, 12)
df_cluster = pd.DataFrame(data, columns=[f'Feature_{i}' for i in range(12)])

plt.figure(figsize=(10, 8))
sns.clustermap(df_cluster.corr(), cmap='coolwarm', center=0, figsize=(8, 8))
plt.suptitle("Hierarchical Clustering of Features", fontsize=14, fontweight='bold')
plt.tight_layout()
plt.show()

# KDE plot (kernel density estimation)
np.random.seed(42)
x = np.random.normal(100, 15, 1000)
y = np.random.normal(100, 15, 1000)

plt.figure(figsize=(10, 6))
sns.kdeplot(x=x, y=y, cmap="YlOrRd", fill=True, thresh=0, levels=10)
plt.title("2D Density Plot", fontsize=14, fontweight='bold')
plt.xlabel("X values")
plt.ylabel("Y values")
plt.tight_layout()
plt.show()

# Real-world: Business analytics dashboard
np.random.seed(42)
months = pd.date_range('2023-01-01', periods=12, freq='M')
dashboard_data = pd.DataFrame({
'Month': months,
'Revenue': np.random.randint(50000, 150000, 12),
'Customers': np.random.randint(1000, 5000, 12),
'Expenses': np.random.randint(30000, 80000, 12),
'Region': np.random.choice(['North', 'South', 'East', 'West'], 12)
})

fig, axes = plt.subplots(2, 2, figsize=(14, 10))

# Revenue trend
axes[0, 0].plot(dashboard_data['Month'], dashboard_data['Revenue'], marker='o', linewidth=2)
axes[0, 0].set_title('Monthly Revenue', fontweight='bold')
axes[0, 0].grid(True, alpha=0.3)
axes[0, 0].set_ylabel('Revenue ($)')

# Customer count
axes[0, 1].bar(dashboard_data['Month'].dt.month, dashboard_data['Customers'], color='skyblue')
axes[0, 1].set_title('Monthly Customers', fontweight='bold')
axes[0, 1].set_ylabel('Customer Count')
axes[0, 1].grid(True, alpha=0.3, axis='y')

# Profit
profit = dashboard_data['Revenue'] - dashboard_data['Expenses']
axes[1, 0].fill_between(range(len(dashboard_data)), profit, alpha=0.3, color='green')
axes[1, 0].plot(profit, marker='o', color='green', linewidth=2)
axes[1, 0].set_title('Monthly Profit', fontweight='bold')
axes[1, 0].set_ylabel('Profit ($)')
axes[1, 0].grid(True, alpha=0.3)

# Revenue by region
region_revenue = dashboard_data.groupby('Region')['Revenue'].sum().sort_values(ascending=False)
axes[1, 1].barh(region_revenue.index, region_revenue.values, color=['red', 'blue', 'green', 'orange'])
axes[1, 1].set_title('Total Revenue by Region', fontweight='bold')
axes[1, 1].set_xlabel('Revenue ($)')

plt.suptitle('Business Dashboard', fontsize=16, fontweight='bold', y=0.995)
plt.tight_layout()
plt.show()

3.3 Interactive Visualizations with Plotly

# Install plotly: pip install plotly

import plotly.express as px
import plotly.graph_objects as go
import pandas as pd
import numpy as np

# Load sample data
iris = px.data.iris()

# Interactive scatter plot
fig = px.scatter(iris, x="sepal_width", y="sepal_length",
color="species", size="petal_length",
hover_name="species",
title="Iris Dataset - Interactive Scatter Plot",
labels={"sepal_width": "Sepal Width", "sepal_length": "Sepal Length"})
fig.show()

# Interactive box plot
fig = px.box(iris, x="species", y="sepal_length",
title="Sepal Length by Species",
color="species")
fig.show()

# Interactive line chart
df_stocks = pd.DataFrame({
'Date': pd.date_range('2023-01-01', periods=100),
'Stock_A': np.cumsum(np.random.randn(100)) + 100,
'Stock_B': np.cumsum(np.random.randn(100)) + 100,
'Stock_C': np.cumsum(np.random.randn(100)) + 100
})

fig = px.line(df_stocks, x='Date', y=['Stock_A', 'Stock_B', 'Stock_C'],
title="Stock Price Trends",
labels={'value': 'Price ($)'})
fig.show()

# Interactive bar chart
fruit_data = pd.DataFrame({
'Fruit': ['Apple', 'Banana', 'Orange', 'Grape', 'Mango'],
'Sales': [1000, 1500, 1200, 800, 1100],
'Region': ['North', 'South', 'North', 'East', 'West']
})

fig = px.bar(fruit_data, x='Fruit', y='Sales', color='Region',
title="Fruit Sales by Region",
barmode='group')
fig.show()

# 3D scatter plot
fig = px.scatter_3d(iris, x='sepal_length', y='sepal_width', z='petal_length',
color='species', size='petal_width',
title="3D Iris Dataset")
fig.show()

# Animated visualization
gapminder = px.data.gapminder().query("year >= 2000")
fig = px.scatter(gapminder, x="gdpPercap", y="lifeExp",
animation_frame="year", animation_group="country",
size="pop", color="continent", hover_name="country",
log_x=True, size_max=60,
range_x=[100, 100000], range_y=[25, 90],
title="Economic Growth and Life Expectancy Over Time")
fig.show()

# Custom plotly figure
fig = go.Figure()

# Add trace 1
x = np.linspace(0, 10, 100)
y1 = np.sin(x)
y2 = np.cos(x)

fig.add_trace(go.Scatter(x=x, y=y1, mode='lines', name='sin(x)'))
fig.add_trace(go.Scatter(x=x, y=y2, mode='lines', name='cos(x)'))

fig.update_layout(
title='Trigonometric Functions',
xaxis_title='X values',
yaxis_title='Y values',
hovermode='x unified',
height=600,
width=1000
)
fig.show()

# Real-world: Sales dashboard
np.random.seed(42)
sales_data = pd.DataFrame({
'Date': pd.date_range('2023-01-01', periods=365),
'Sales': np.random.randint(1000, 5000, 365),
'Region': np.random.choice(['North', 'South', 'East', 'West'], 365),
'Category': np.random.choice(['Electronics', 'Clothing', 'Food'], 365)
})

# Daily sales trend
daily_sales = sales_data.groupby('Date')['Sales'].sum()
fig = px.line(x=daily_sales.index, y=daily_sales.values,
title='Daily Sales Trend',
labels={'x': 'Date', 'y': 'Sales ($)'})
fig.show()

# Sales by region
region_sales = sales_data.groupby('Region')['Sales'].sum()
fig = px.pie(values=region_sales.values, names=region_sales.index,
title='Sales Distribution by Region')
fig.show()

# Sales by category
category_sales = sales_data.groupby('Category')['Sales'].sum()
fig = px.bar(x=category_sales.index, y=category_sales.values,
title='Sales by Category',
labels={'x': 'Category', 'y': 'Total Sales ($)'})
fig.show()

Best Practices for Data Visualization

Choose appropriate chart type for your data: use bar charts for categories, line charts for time series, scatter plots for relationships
Keep it simple: avoid clutter and unnecessary elements
Use clear titles and labels: make sure axes are labeled and units are clear
Choose colors wisely: use contrasting colors, avoid red-green combinations for colorblind viewers
Include legends when necessary: explain what colors/shapes represent
Highlight key insights: use annotations, trends, or highlighting to draw attention to important findings
Maintain consistency: use same colors, fonts, and styles across related visualizations
Consider your audience: technical reports need different style than business presentations
Provide context: show baselines, averages, or benchmarks for comparison
Test readability: ensure text is large enough and colors are distinguishable

Week 5 Summary

By completing Week 5, you have learned:

What EDA is and why it's critical in data science
Descriptive statistics: mean, median, mode, std, variance, range
Understanding distributions: skewness, kurtosis
Quartiles, percentiles, and IQR for understanding data spread
Bivariate analysis: correlations, covariance, group statistics
Pivot tables for summarizing data by categories
Matplotlib basics: line plots, scatter plots, bar charts, histograms
Subplots for creating multi-chart figures
Seaborn for statistical visualizations: box plots, violin plots, distribution plots
Heatmaps for visualizing correlations
Pairplots for exploring all relationships in dataset
Advanced plots: faceted plots, joint plots, density plots
Business dashboards combining multiple visualizations
Interactive visualizations with Plotly
Best practices for effective data visualization

Week 5 Assignments

Assignment 1: Comprehensive EDA Report

Perform complete exploratory analysis on a dataset:

Calculate all descriptive statistics (mean, median, std, quartiles, etc.)
Analyze distributions: skewness, kurtosis
Create 10+ visualizations using Matplotlib and Seaborn
Include: histograms, box plots, scatter plots, distribution plots
Analyze correlations and create heatmap
Group analysis by categories
Generate a written report with key insights and findings
Suggest features for modeling based on EDA findings

Assignment 2: Interactive Dashboard with Plotly

Create an interactive dashboard:

Load a multi-dimensional dataset (sales, customer, or similar)
Create at least 5 interactive visualizations with Plotly
Include different chart types: line, bar, scatter, pie, box
Make visualizations interactive: hover info, filters, etc.
Create a narrative that tells a story with the data
Include annotations and highlights for key findings
Save as HTML file for sharing

Assignment 3: Exploratory Analysis Report

Download a real dataset and produce professional analysis:

Use a dataset from Kaggle or similar with 500+ rows
Perform complete EDA with statistics and visualizations
Create a professional PDF/HTML report with:
Executive summary
Descriptive statistics
10-15 high-quality visualizations
Correlation analysis
Key findings and recommendations
Data quality assessment
Format for business presentation

Additional Resources

Books

Chapter 9: Plotting and Visualization - "Python for Data Analysis" by Wes McKinney
Fundamentals of Data Visualization by Claus Wilke (free online)
The Visual Display of Quantitative Information by Edward Tufte

Online Documentation

Matplotlib Documentation: https://matplotlib.org/
Seaborn Documentation: https://seaborn.pydata.org/
Plotly Documentation: https://plotly.com/python/
pandas Visualization: https://pandas.pydata.org/docs/user_guide/visualization.html

Gallery and Examples

Matplotlib Gallery: https://matplotlib.org/stable/gallery/index
Seaborn Gallery: https://seaborn.pydata.org/examples.html
Plotly Gallery: https://plotly.com/python/

← Previous Week Next Week →

Week 5: Exploratory Data Analysis and Visualisation

In this week

Week 5 Overview

SESSION 1: Descriptive Statistics and Univariate Analysis

Duration: 2 hours

1.1 What is Exploratory Data Analysis?

1.2 Descriptive Statistics

1.3 Understanding Distributions

1.4 Bivariate and Multivariate Analysis

SESSION 2: Matplotlib and Seaborn for Data Visualization

Duration: 2 hours

2.1 Introduction to Matplotlib

2.2 Advanced Visualization with Seaborn

SESSION 3: Correlation Analysis, Advanced Visualizations, and Plotly

Duration: 2 hours

3.1 Correlation Analysis and Heatmaps

3.2 Advanced Visualization Techniques

3.3 Interactive Visualizations with Plotly

Best Practices for Data Visualization

Week 5 Summary

Week 5 Assignments

Assignment 1: Comprehensive EDA Report

Assignment 2: Interactive Dashboard with Plotly

Assignment 3: Exploratory Analysis Report

Recommended Practice Problems

Additional Resources

Books

Online Documentation

Gallery and Examples