Intuitively Understanding the Shannon Entropy

Information theory is a subfield of mathematics concerned with transmitting data across a noisy channel.

A cornerstone of information theory is the idea of quantifying how much information there is in a message. More generally, this can be used to quantify the information in an event and a random variable, called entropy, and is calculated using probability.

Calculating information and entropy is a useful tool in machine learning and is used as the basis for techniques such as feature selection, building decision trees, and, more generally, fitting classification models. As such, a machine learning practitioner requires a strong understanding and intuition for information and entropy.

(Visited 2 times, 1 visits today)

You might be interested in

LEAVE YOUR COMMENT

Your email address will not be published. Required fields are marked *

eos
EOS (EOS) 0,825380 0,18%
aave
Aave (AAVE) 58,82 2,37%
the-graph
The Graph (GRT) 0,125928 4,35%
kusama
Kusama (KSM) 24,91 4,14%
waves
Waves (WAVES) 1,59 0,38%
dash
Dash (DASH) 39,09 2,22%
bitcoin
Bitcoin (BTC) 25.094,72 0,62%
ethereum
Ethereum (ETH) 1.736,24 0,27%
cardano
Cardano (ADA) 0,338410 3,09%
tether
Tether (USDT) 0,931850 0,04%
xrp
XRP (XRP) 0,472474 0,32%
solana
Solana (SOL) 19,28 0,07%
polkadot
Polkadot (DOT) 4,88 1,59%
usd-coin
USD Coin (USDC) 0,931634 0,08%
dogecoin
Dogecoin (DOGE) 0,066837 0,30%
uniswap
Uniswap (UNI) 4,68 1,22%
terra-luna
Terra Luna Classic (LUNC) 0,000079 0,23%
litecoin
Litecoin (LTC) 86,56 4,06%
chainlink
Chainlink (LINK) 5,99 0,73%
bitcoin-cash
Bitcoin Cash (BCH) 105,46 0,39%
algorand
Algorand (ALGO) 0,137446 0,52%
matic-network
Polygon (MATIC) 0,834028 0,63%
stellar
Stellar (XLM) 0,086048 2,80%
cosmos
Cosmos Hub (ATOM) 9,62 2,69%
filecoin
Filecoin (FIL) 4,36 0,06%
tron
TRON (TRX) 0,069386 1,47%
ethereum-classic
Ethereum Classic (ETC) 16,63 0,71%
dai
Dai (DAI) 0,931728 0,11%
tezos
Tezos (XTZ) 0,841385 1,30%
monero
Monero (XMR) 137,87 1,99%