Information theory is a branch of applied mathematics and engineering involving the quantification of information. Historically, information theory developed to find fundamental limits on compressing and reliably communicating data. Since its inception it has broadened to find applications in statistical inference, networks other than communication networks, biology, quantum information theory, data analysis, and other areas, although it is still widely used in the study of communication.
A key measure of information that comes up in the theory is known as information entropy, which is usually expressed by the average number of bits needed for storage or communication. Intuitively, entropy quantifies the uncertainty involved in a random variable. For example, a fair coin flip will have less entropy than a roll of a die.
Applications of fundamental topics of information theory include lossless data compression (e.g. ZIP files), lossy data compression (e.g. MP3s), and channel coding (e.g. for DSL lines). The field is at the crossroads of mathematics, statistics, computer science, physics, neurobiology, and electrical engineering. Its impact has been crucial to success of the Voyager missions to deep space, the invention of the CD, the feasibility of mobile phones, the development of the Internet, the study of linguistics and of human perception, the understanding of black holes, and numerous other fields. Important sub-fields of information theory are source coding, channel coding, algorithmic complexity theory, algorithmic information theory, and measures of information.
The most important quantities of information are entropy, the information in a random variable, and mutual information, the amount of information in common between two random variables.
Entropy of a Bernoulli trial as a function of success probability, often called the binary entropy function, Hb(p). The entropy is maximized at 1 bit per trial when the two possible outcomes are equally probable, as in an unbiased coin toss.
Hiç yorum yok:
Yorum Gönder