Ensemble Learning & Advanced Model Techniques

Course outline and learning expectations

This is a tutor-led course. The outline shows what your tutor will cover; teaching materials and examinations are provided directly to enrolled students.

What you will learn

• Explain why combining diverse models can improve predictive performance
• Build and tune bagging and Random Forest models
• Implement AdaBoost, gradient boosting and XGBoost workflows
• Compare XGBoost, LightGBM and CatBoost for speed, data characteristics and practical use
• Create stacking models using out-of-fold predictions while avoiding data leakage
• Evaluate ensemble models using suitable classification or regression metrics
• Compare predictive performance, computing cost and interpretability
• Decide when a simpler model is more appropriate for a small African dataset
• Improve and present a crop-disease-risk model as a capstone project

Modules

1. 1

   Why Combine Models

   Understand ensemble learning, model diversity and the wisdom-of-crowds idea

   Ensemble intuitionBias and varianceModel diversityVoting and averaging
2. 2

   Bagging and Random Forests

   Revisit bootstrap aggregation and Random Forests in depth

   Bootstrap samplesRandom feature selectionOut-of-bag evaluationFeature importanceHyperparameter tuning
3. 3

   Boosting and AdaBoost

   Learn how sequential learners correct earlier errors

   Weak learnersSample weightingAdaBoost classificationLearning rateOverfitting checks
4. 4

   Gradient Boosting and XGBoost

   Build and tune gradient-boosted tree models

   Gradient boosting intuitionXGBoost workflowRegularisationEarly stoppingCross-validation
5. 5

   LightGBM and CatBoost

   Compare practical boosting libraries for speed, memory use and categorical data

   LightGBM workflowCatBoost workflowCategorical featuresTraining efficiencyModel comparison
6. 6

   Stacking Models

   Combine base learners with a meta-model without data leakage

   Base modelsOut-of-fold predictionsMeta-learnersStacking classifiersStacking regressors
7. 7

   Choosing Ensembles for African Datasets

   Decide when ensembles add value and when simpler models are more suitable

   Small datasetsClass imbalanceData qualityCompute constraintsInterpretability trade-offs
8. 8

   Capstone: Crop-Disease-Risk Model

   Improve the Course 10 crop-disease-risk model with XGBoost and compare it with earlier models

   Baseline reviewFeature preparationXGBoost trainingPerformance comparisonInterpretability trade-offsCapstone presentation