scipy.special.entr#

scipy.special.entr(x, out=None) = <ufunc 'entr'>#

Elementwise function for computing entropy.

\[\begin{split}\text{entr}(x) = \begin{cases} - x \log(x) & x > 0 \\ 0 & x = 0 \\ -\infty & \text{otherwise} \end{cases}\end{split}\]

Parameters:

xndarray: Input array.
outndarray, optional: Optional output array for the function values

Returns:

resscalar or ndarray: The value of the elementwise entropy function at the given points x.

See also

kl_div, rel_entr, scipy.stats.entropy

Notes

Added in version 0.15.0.

This function is concave.

The origin of this function is in convex programming; see [1]. Given a probability distribution \(p_1, \ldots, p_n\), the definition of entropy in the context of information theory is

\[\sum_{i = 1}^n \mathrm{entr}(p_i).\]

To compute the latter quantity, use scipy.stats.entropy.

Array API Standard Support

entr has support for Python Array API Standard compatible backends in addition to NumPy. The following combinations of backend and device (or other capability) are supported.

Library	CPU	GPU
NumPy	✅	n/a
CuPy	n/a	✅
PyTorch	✅	✅
JAX	✅	✅
Dask	✅	n/a

For the NumPy backend, this function supports all NumPy ufunc keyword arguments. Other backends may support out, but none of the other ufunc kwargs. out is typically supported for CuPy and PyTorch, but not currently in cases where SciPy relies on a generic Array API implementation or, for PyTorch on CPU, falls back to the NumPy backend. out is never supported for JAX because JAX arrays are immutable.

See Support for the array API standard for more information.

References

[1]

Boyd, Stephen and Lieven Vandenberghe. Convex optimization. Cambridge University Press, 2004. DOI:10.1017/CBO9780511804441.

Examples

>>> import numpy as np
>>> from scipy.special import entr

Calculate the entropy (in nats) of a 3-outcome probability distribution

>>> p = np.array([0.2, 0.5, 0.3])
>>> entr(p)
array([0.32188758, 0.34657359, 0.36119184])
>>> entr(p).sum()
1.0296530140645737