Reinforcement Learning Fundamentals

Course outline and learning expectations

This is a tutor-led course. The outline shows what your tutor will cover; teaching materials and examinations are provided directly to enrolled students.

What you will learn

• Explain the reinforcement learning agent-environment framework and core terminology
• Represent sequential decision problems as Markov Decision Processes
• Implement and tune tabular Q-learning with exploration strategies
• Build and evaluate a simple agent in a Gymnasium environment
• Explain the main components and training logic of Deep Q-Networks
• Compare reinforcement learning with supervised learning, optimisation and rule-based approaches
• Identify practical, ethical and operational limitations of reinforcement learning
• Develop a capstone agent for simulated retail restocking decisions

Modules

1. 1

   Reinforcement learning foundations

   Understand how agents learn through interaction and rewards

   Agent-environment loopStates and observationsActionsRewards and returnsPoliciesEpisodesRL compared with supervised learning
2. 2

   Markov Decision Processes

   Model sequential decisions with states, actions, transitions and rewards

   Markov propertyTransition dynamicsReward functionsDiscount factorValue functionsBellman intuition
3. 3

   Q-learning basics

   Implement value-based learning for discrete environments

   Q-tablesTemporal-difference updatesLearning rateDiscount factorEpsilon-greedy explorationTraining loopsConvergence intuition
4. 4

   Building a simple RL agent

   Create, train and evaluate an agent in a game-style environment

   Gymnasium APIReset and step cycleEnvironment wrappersLogging rewardsEvaluation episodesReproducibility and random seeds
5. 5

   Deep Q-Networks intuition

   Understand how neural networks extend Q-learning to larger state spaces

   Function approximationReplay bufferTarget networkLoss calculationExploration schedulingTraining stabilityPyTorch DQN workflow
6. 6

   Real-world RL use cases

   Assess RL applications through Africa-relevant examples and simulations

   Routing and dispatchInventory and restockingResource allocationDynamic pricingEnergy and network optimisationSimulation requirements
7. 7

   Limitations and responsible use

   Decide when RL is useful, risky or unnecessary

   Sample inefficiencyReward designSafety and testingDistribution shiftCompute and data constraintsEthical considerationsSimpler alternatives
8. 8

   Capstone retail restocking agent

   Design and evaluate an agent for a simulated small-shop inventory problem

   Problem definitionState and action designReward functionDemand simulationBaseline policyTraining and evaluationResults presentation