Shannon entropy

qif.measure.shannon

Shannon entropy \(H(\pi) = -\sum_x \pi_x \log_2 \pi_x\), an uncertainty measure (in bits). Only double is supported.

Functions

prior(pi: ndarray) → float

The Shannon entropy \(H(\pi)\).

posterior(pi: ndarray, C: ndarray) → float

The posterior Shannon entropy \(H[\pi\triangleright C]\), that is the conditional entropy \(H(X|Y)\).

add_leakage(pi: ndarray, C: ndarray) → float

The additive leakage \(H(\pi) - H[\pi\triangleright C]\), that is the mutual information \(I(X;Y)\).

mult_leakage(pi: ndarray, C: ndarray) → float

The multiplicative leakage \(H(\pi) / H[\pi\triangleright C]\).

add_capacity(C: ndarray, md: float = ..., mrd: float = ...) → Tuple[float, ndarray]

The Shannon capacity \(\max_\pi I(X;Y)\), computed with the Blahut–Arimoto algorithm. Iterates until the lower and upper bounds of the algorithm are equal up to md, mrd. Returns (capacity, pi), where pi is a prior achieving it.