Capstone: End-to-End Data Science Project

Your capstone demonstrates mastery across the entire data science lifecycle. This is your portfolio centerpiece.

Capstone Workflow

1. Problem Selection Criteria

2. Project Structure

capstone/
├── README.md
├── proposal.md
├── data/
│   ├── raw/
│   ├── processed/
│   └── data_dictionary.md
├── notebooks/
│   ├── 01_EDA.ipynb
│   ├── 02_Feature_Engineering.ipynb
│   ├── 03_Modeling.ipynb
│   └── 04_Evaluation.ipynb
├── src/
│   ├── data/
│   │   ├── collect.py
│   │   └── clean.py
│   ├── features/
│   │   └── build_features.py
│   ├── models/
│   │   ├── train.py
│   │   └── predict.py
│   ├── api/
│   │   └── app.py
│   └── utils.py
├── configs/
│   └── params.yaml
├── tests/
├── Dockerfile
├── docker-compose.yml
├── requirements.txt
└── .github/
    └── workflows/
        └── ci.yml

3. Timeline

4. Evaluation Template

Model comparison uses statistical tests to validate improvements:

Wilcoxon signed-rank test for significance:

# Model comparison framework
results = {
    "logistic_regression": {"accuracy": 0.82, "f1": 0.79, "auc": 0.87},
    "random_forest": {"accuracy": 0.87, "f1": 0.85, "auc": 0.92},
    "xgboost": {"accuracy": 0.91, "f1": 0.89, "auc": 0.95},
    "lightgbm": {"accuracy": 0.90, "f1": 0.88, "auc": 0.94}
}

# Statistical significance
from scipy.stats import wilcoxon
stat, p_value = wilcoxon(xgb_scores, rf_scores)
print(f"p-value: {p_value:.4f}")  # Significant if p < 0.05

5. Writeup Structure

1. Executive Summary (1 paragraph)
2. Problem Definition (business context, objectives)
3. Data Description (source, size, features, quality)
4. Methodology (approach, models, evaluation)
5. Results (metrics, visualizations, insights)
6. Limitations (what the model cannot do)
7. Future Work (improvements, extensions)
8. Appendix (code references, hyperparameters)

6. Presentation Structure (15 minutes)

Key Takeaways

• Problem first: Choose a problem that matters, not just interesting data
• Document everything: Trade-offs, decisions, and failures are valuable
• Deploy it: A running application demonstrates end-to-end capability
• Tell the story: Your presentation skills matter as much as your code