FY823: Bayesian inference and information theory (5 ECTS)

STADS: 07013201

Level
Master's level course

Teaching period
The course is offered in the spring semester.

Teacher responsible
Email: mlomholt@sdu.dk

Additional teachers
krog@sdu.dk

Timetable
Show entire timetable
Show personal time table for this course.

Prerequisites:
None

Academic preconditions:
Students taking the course are expected to:

• Have knowledge of probability theory

Course introduction
NOTE that the teaching method in the course is special – see the elaboration on the teaching method below.

The aim of the course is to give the students knowledge of the Bayesian way of thinking and solving inference problems as well as the thoughts behind information theory (Bayesian inference is a method where regular probability theory is extended to apply to hypothesis, whereby it constitutes an alternative to orthodox statistics. Information theory is about efficient storage and communication of data).

The course gives a foundation for applying Bayesian methods to analyse data, as for instance obtained through experiments, simulations or registration. This can for instance be used in projects in the rest of the education. The course also gives knowledge of the connection between information theory and the concept of entropy in statistical physics.

In relation to the competence profile of the degree it is the explicit focus of the course to:

• give skills to analyse data and evaluate the plausibility of different physical theories.
• give knowledge and understanding of Bayesian inference and information theory.
• give ability to acquire new knowledge in an effective and autonomous way and communicate this knowledge to colleagues

Expected learning outcome
The learning objective of the course is that the student demonstrates the ability to:

• Autonomously acquire the knowledge about Bayesian inference and information theory as described under Contents
• Solve problems regarding fitting of parameters and model selection
• Present theory and exercises
• Apply the acquired knowledge within a chosen subject (final project)

Subject overview
The following main topics are contained in the course:

• Interpretation of probability
• Model selection
• Fitting of parameters
• Bayes’ theorem
• Shannon entropy
• Coding theory

Literature

• David J. C. Mackay: Information Theory, Inference, and Learning Algorithms, Cambridge University Press.

Website
This course uses e-learn (blackboard).

Prerequisites for participating in the exam

1. Active participation in the teaching. Passed/failed, examination by teacher. The prerequisite examination is a prerequisite for participation in exam element a). (07013212).

Assessment and marking:

1. Oral exam. Passed/not passed, internal marking. The exam takes its starting point in a written report about the final project. It takes the form of a presentation followed by questions. (5 ECTS). (07013202).

Expected working hours
The teaching method is based on three phase model.
Intro phase: 16 hours
Skills training phase: 15 hours, hereof:
 - Tutorials: 15 hours

Educational activities

• Study of the textbook
• Solving of exercises
• Preparation of presentations
• Undertaking of project

Educational form
The teaching method in the course is build around the students presenting the material for eachother. At every class (2 hours) there will be typically 3 students who each spend approximately 10 min. on presenting af section from the textbook (intro phase), as well as 3 students presenting an exercise (training phase). The teachers will be available in between the classes if the presenters have questions to their material or exercise. Due to the additional time needed for preparing the presentations then the number of classes is lower than usual.

Language
This course is taught in English, if international students participate. Otherwise the course is taught in Danish.

Course enrollment
See deadline of enrolment.

Tuition fees for single courses
See fees for single courses.