| Course Type | Elective Course |
|---|
| Course Level | Master's Degree With Thesis |
|---|
| Year of Study | - |
|---|
| Semester | - |
|---|
| ECTS | 10 |
|---|
| Name of Lecturer(s) |
- |
| Learning Outcomes |
At the end of this course, the students;
- Learn entropy, relative entropy and mutual information.
- Learn data compression. Markov chains.
- Learn entropy rates, Hidden Markov models.
- Learn Huffman, Hamming, zero-error codes.
- Learn feedback capacity, differential entropy, maximum entropy
- Learn rate distortion theory, network information theory.
|
| Mode of Delivery | Face to face |
|---|
| Prerequisites | No |
|---|
| Recommended Optional Programme Components | None |
|---|
| Course Description | Information theory and its relationship with Statistics. Shannon information theory, memoryless sources and channels. Entropy and information. Data compression and coding for secure communications. |
|---|
| Course Contents |
| WEEK |
TOPICS |
|
1st Week |
Entropy, relative entropy and mutual information. Entropy rates, |
|
2nd Week |
Hidden Markov models. Markov chains. |
|
3rd Week |
Data compression. |
|
4th Week |
Huffman codes. Arithmetic coding. |
|
5th Week |
Channel capacity. Channel coding theorem. |
|
6th Week |
Zero-error codes. |
|
7th Week |
Hamming codes. |
|
8th Week |
Midterm Exam |
|
9th Week |
Feedback capacity. |
|
10th Week |
Differential entropy. |
|
11th Week |
Gaussian channel, bandlimited channels. |
|
12th Week |
Maximum entropy and spectral estimation. |
|
13th Week |
Rate distortion theory. |
|
14th Week |
Network information theory. |
|
| Recommended or Required Reading | T. M. Cover (1991), Elements of Information Theory, John Wiley;
D.J.C. Mackay (2005), Information Theory, Interference and Learning Algorithms, Cambridge.
|
|---|
| Planned Learning Activities and Teaching Methods |
- Lecture
- Questions/Answers
- Presentation
|
|---|
| Assessment Methods and Criteria |
|
Quantity |
Percentage (%) |
| Mid-term |
1 |
35 |
| Assignment |
1 |
12 |
| Quiz |
2 |
13 |
| Total(%) |
|
60 |
| Contribution of In-term Studies to Overall Grade(%) |
|
60 |
| Contribution of Final Examination to Overall Grade(%) |
|
40 |
| Total(%) |
|
100 |
|
| ECTS Allocated Based on Student Workload |
| Activities |
Quantity |
Duration (Hour) |
Total Workload (Hour) |
| Midterm exam |
1 |
3 |
3 |
| Preparation for Quiz |
0 |
0 |
0 |
| Individual or group work |
14 |
8 |
112 |
| Preparation for Final exam |
1 |
40 |
40 |
| Course hours |
14 |
3 |
42 |
| Preparation for Midterm exam |
1 |
30 |
30 |
| Laboratory (including preparation) |
0 |
0 |
0 |
| Final exam |
1 |
3 |
3 |
| Homework |
2 |
35 |
70 |
| Total Work Load | 300 |
| Total Work Load / 30 (h) | 10 |
| ECTS Credit of the Course | 10 |
|
| Language of Instruction | Turkish |
|---|
| Work Placement(s) | No |
|---|
