Module Search

MATH2707: Markov Chains II

Please ensure you check the module availability box for each module outline, as not all modules will run in each academic year. Each module description relates to the year indicated in the module availability box, and this may change from year to year, due to, for example: changing staff expertise, disciplinary developments, the requirements of external bodies and partners, and student feedback. Current modules are subject to change in light of the ongoing disruption caused by Covid-19.

Type	Open
Level	2
Credits	10
Availability	Available in 2023/24
Module Cap	None.
Location	Durham
Department	Mathematical Sciences

Prerequisites

Calculus I (Maths Hons) (MATH1081) or Calculus I (MATH1061), Probability I (MATH1597) and Linear Algebra I (Maths Hons) (MATH1091) or Linear Algebra I (MATH1071)

Corequisites

None

Excluded Combinations of Modules

None

Aims

To introduce and develop the concept of a Markov chain, as a fundamental type of stochastic process, and to study key features of Markov models using probabilistic tools such as generating functions.

Content

Markov property
Stationary distributions
Classification of states
Hitting probabilities and expected hitting times
Convergence to equilibrium
Applications to random walks
Generating functions
Further topics to be chosen from: Gibbs sampler, mixing and card shuffling, stochastic epidemics, discrete renewal theory, Markov decision processes

Learning Outcomes

Subject-specific Knowledge:

Be able to solve seen and unseen problems involving Markov chains.
Have a knowledge and understanding of this subject demonstrated through an ability to identify stationary distributions, classify states, and compute hitting probabilities and expected hitting times, and a working knowledge of generating functions and their computational and theoretical power.
Reproduce theoretical mathematics concerning Markov chains to a level appropriate to Level 2, including key definitions and theorems.

Subject-specific Skills:

In addition students will have enhanced mathematical skills in the following areas: intuition for key features of probabilistic systems that evolve in time.

Key Skills:

Students will have basic mathematical skills in the following areas: problem solving, modelling, computation.

Modes of Teaching, Learning and Assessment and how these contribute to the learning outcomes of the module

Lectures demonstrate what is required to be learned and the application of the theory to practical examples.
Problem classes show how to solve example problems in an ideal way, revealing also the thought processes behind such solutions.
Tutorials provide active problem-solving engagement and immediate feedback to the learning process.
Formative assessments provide feedback to guide students in the correct development of their knowledge and skills in preparation for the summative assessment.
The end-of-year examination assesses the knowledge acquired and the ability to solve predictable and unpredictable problems.

Teaching Methods and Learning Hours

Activity	Number	Frequency	Duration	Total	Monitored
Lectures	21	2 per week in Michaelmas; 1 in Easter	1 hour	21
Tutorials	5	Fortnightly in Michaelmas; 1 in Easter	1 hour	5	Yes
Problem Classes	4	Fortnightly in Michaelmas	1 hour	4
Preparation and reading				70
Total				100

Summative Assessment

Component: Examination		Component Weighting: 100%
Element	Length / Duration	Element Weighting	Resit Opportunity
Written Examination	2 hours	100

Formative Assessment

Weekly written or electronic assignments to be assessed and returned.

More information

If you have a question about Durham's modular degree programmes, please visit our FAQ webpages, Help page or our glossary of terms. If you have a question about modular programmes that is not covered by the FAQ, or a query about the on-line Undergraduate Module Handbook, please contact us.

Prospective Students: If you have a query about a specific module or degree programme, please Ask Us.

Current Students: Please contact your department.