Lecture

• homeworks
• attendance to practicals is mandatory, 3 absences allowed at maximum
• final test

$A\subseteq\Omega$

• we divide the number of $A$ occurring by the number of experiments
• maximum likelihood estimation

• $p(A)\in[0,1]$
• $p(\Omega)=1$
• $p(\bigcup A_i)=\sum p(A_i)$

nothing can be more uncertain than the uniform distribution

$G(p)=2^{H(p)}$

$H(X,Y)=H(Y\mid X)+H(X)$

entropy … the least average number of bits needed to encode a message

• $I(X,Y)=D(p(x,y)\Vert p(x)p(y))$
• we can derive that $I(X,Y)=H(X)-H(X\mid Y)$
  • by symmetry $I(X,Y)=H(Y)-H(Y\mid X)$