用位思考 (Thinking in terms of Bits)

Imagine you want to send outcomes of 3 coin flips to your friend's house. Your friend knows that you want to send him those messages but all he can do is get the answer of Yes/No questions arranged by him. Let's assume the arranged question: Is it head? You will send him the sequence of zeros or ones as an answer to those questions which is commonly known as a bit(binary digit). If Zero represents No and one represents Yes and the actual outcome of the toss was Head, Head, and Tail. Then you would need to send him 1 1 0 to convey your facts(information). so it costs us 3 bits to send those messages.

我想像一下，您想將3次硬幣翻轉(zhuǎn)的結(jié)果發(fā)送到您朋友的房子。 您的朋友知道您想向他發(fā)送這些消息，但是他所能做的就是得到他安排的是/否問(wèn)題的答案。 讓我們假設(shè)一個(gè)安排好的問(wèn)題：是頭嗎？ 您將把零或一的序列發(fā)送給他，作為對(duì)這些問(wèn)題的解答(通常稱為位(二進(jìn)制數(shù)字))。 如果零代表否，一個(gè)代表是，并且折騰的實(shí)際結(jié)果是頭，頭和尾。 然后，您需要向他發(fā)送1 1 0來(lái)傳達(dá)您的事實(shí)(信息)。 因此發(fā)送這些消息要花費(fèi)我們3位。

How many bits does it take to make humans?

做一個(gè)人需要多少位？

Our entire genetic code is contained in a sequence of 4 states T, A, G, C in DNA. Now we would need 2 bits (00,01,10,11) to encode these states. And multiplied by 6 billion letters of genetic code in the genome yields 1.5 GB of information. So we can fit our entire genetic code on a single DVD.

我們的整個(gè)遺傳密碼包含在DNA的4種狀態(tài)T，A，G，C中。 現(xiàn)在我們將需要2位(00,01,10,11)來(lái)編碼這些狀態(tài)。 乘以基因組中60億個(gè)字母的遺傳密碼可產(chǎn)生1.5 GB的信息。 因此，我們可以將整個(gè)遺傳密碼放在一張DVD中。

多少信息？ (How much Information?)

Suppose I flip a fair coin with a 50% chance of getting head and a 50% chance of getting tails. Similarly now instead of a fair coin, I flip a biased coin with head on both sides. What do you think in which case I am more certain about the outcome of the toss? Obviously the answer would be a biased coin. It is because in the case of the fair coin I am uncertain about the outcome because none of the possibilities are more likely to happen than the other while in biased coin I am not uncertain about the outcome because I know I will get heads.

小號(hào) uppose我翻轉(zhuǎn)一個(gè)公平的硬幣獲得頭部的幾率為50％和獲得尾巴的幾率為50％。 現(xiàn)在，類似地，我用偏頭硬幣在兩頭都翻轉(zhuǎn)而不是公平的硬幣。 在這種情況下，您認(rèn)為我對(duì)投擲的結(jié)果有何把握？ 答案顯然是有偏見的。 這是因?yàn)樵诠接矌诺那闆r下，我不確定結(jié)果，因?yàn)闆](méi)有一種可能性比另一種可能性更大；而在有偏見的硬幣中，我不確定結(jié)果，因?yàn)槲抑牢視?huì)獲得正面的。

Let's look at the other way. Would you be surprised if I told you the coin with head on both sides gave head as an outcome? No. Because you did not learn anything new with that statement. Outcomes did not give you any further information. On the other hand, when the coin is fair you have the least knowledge of what will happen next. So each toss gives you the new information.

讓我們看看另一種方式。 如果我告訴您兩側(cè)都帶硬幣的硬幣作為結(jié)果給了您，您會(huì)感到驚訝嗎？ 不。因?yàn)槟鷽](méi)有從該聲明中學(xué)到任何新知識(shí)。 結(jié)果未提供任何進(jìn)一步的信息。 另一方面，當(dāng)代幣公平時(shí)，您對(duì)接下來(lái)將要發(fā)生的事情的了解最少。 因此，每次折騰都會(huì)為您提供新的信息。

So the intuition behind quantifying information is the idea of measuring how much surprise there is in an event. Those events that are rare (low probability) are more surprising and therefore have more information than those events that are common (high probability).

因此，量化信息背后的直覺(jué)是測(cè)量事件中有多少驚喜的想法。 那些罕見的事件(低概率)比那些常見的事件(高概率)更令人驚訝，因此具有更多的信息。

Low Probability Event: High Information (surprising)

低概率事件 ：高信息( 令人驚訝)

High Probability Event: Low Information (unsurprising)

高概率事件 ：低信息( 不足為奇 )

As a prereqisite, If you want to learn about basic probability theory, I wrote about that here.

作為先決條件，如果您想了解基本概率論，我在這里寫過(guò)。

So information seems to be randomness. So if we want to know how much information does something contains we need to know how random and unpredictable it is. Mathematically, Information gained by observing an event X with probability P is given by:

因此，信息似乎是隨機(jī)的。 因此，如果我們想知道某物包含多少信息，就需要知道它是多么隨機(jī)和不可預(yù)測(cè)。 在數(shù)學(xué)上， 通過(guò)觀察概率為P的事件X獲得的信息由下式給出 ：

By plugging the values in the formula, we can clearly see information contained in certain events like observing head by tossing a coin with heads on both sides is 0 while the uncertain event leads to more information after it is observed. So this definition satisfies the basic requirement that it is a decreasing function of p.

通過(guò)插入公式中的值，我們可以清楚地看到某些事件中包含的信息，例如通過(guò)擲一個(gè)硬幣，使硬幣的正反兩面都是正面，而觀察正面，則不確定事件會(huì)導(dǎo)致更多信息。 因此，該定義滿足了它是p的遞減函數(shù)的基本要求。

But You may have the question …………..

但是您可能有一個(gè)問(wèn)題…………..

Why the logarithmic function?

為什么是對(duì)數(shù)函數(shù)？

And what is the base of the logarithm?

對(duì)數(shù)的底是什么？

As an answer to the second question, You can use any base of the logarithm. In information theory, we use base 2 in which case the unit of information is called a bit.

作為第二個(gè)問(wèn)題的答案，您可以使用任何對(duì)數(shù)的底數(shù)。 在信息論中，我們使用以2為底的情況，在這種情況下，信息的單位稱為位。

The answer to the first question is that the resulting definition has several elegant resulting properties, and it is the simplest function that provides these properties. One of these properties is additivity. If you have two independent events (i.e., events that have nothing to do with each other), then the probability that they both occur is equal to the product of the probabilities with which they each occur. What we would like is for the corresponding information to add up.

第一個(gè)問(wèn)題的答案是，生成的定義具有多個(gè)優(yōu)美的生成屬性，而這是提供這些屬性的最簡(jiǎn)單函數(shù)。 這些屬性之一是可加性 。 如果您有兩個(gè)獨(dú)立的事件(即彼此無(wú)關(guān)的事件)，則它們都發(fā)生的概率等于它們各自發(fā)生的概率的乘積 。 我們希望將相應(yīng)的信息加起來(lái)。

For instance, the event that it rained in Kathmandu yesterday and the event that the number of views in this post is independent, and if I am told something about both events, the amount of information I now have should be the sum of the information in being told individually of the occurrence of the two events.

例如，昨天在加德滿都下雨的事件，以及該帖子中的視圖數(shù)量是獨(dú)立的事件，并且如果被告知有關(guān)這兩個(gè)事件的信息，那么我現(xiàn)在擁有的信息量應(yīng)該是被分別告知兩個(gè)事件的發(fā)生。

The logarithmic definition provides us with the desired additivity because given two independent events A and B

對(duì)數(shù)定義為我們提供了所需的可加性，因?yàn)榻o定了兩個(gè)獨(dú)立的事件A和B

熵 (Entropy)

Entropy is the average(expected) amount of information conveyed by identifying the outcome of some random source.

?ntropy是信息的平均(預(yù)期)量傳送通過(guò)識(shí)別一些隨機(jī)源的結(jié)果。

It is simply the sum of several terms, each of which is the information of a given event weighted by the probability of that event occurring.

它只是幾項(xiàng)的總和，每一項(xiàng)都是給定事件的信息，由該事件發(fā)生的概率加權(quán)。

Like information, it is also measured in bits. If we use log?2?? for our calculation we can interpret entropy as the number of bits it would take us on average to encode our information.

像信息一樣，它也以位為單位。 如果我們使用l og 2進(jìn)行計(jì)算，我們可以將熵解釋為編碼我們的信息平均需要的位數(shù)。

In the important special case of two mutually exclusive events (i.e., exactly one of the two events can occur), occurring with probabilities p and 1 ? p, respectively, the entropy is:

在兩個(gè)互斥事件的重要特殊情況下(即恰好可以發(fā)生兩個(gè)事件之一)，它們分別以概率p和1-p發(fā)生，熵為：

帶走 (Takeaway)

• Information is randomness. The more uncertain we are about the outcome of an event the more information we will get after observing an event.

  信息就是隨機(jī)性。 我們對(duì)事件的結(jié)果越不確定，觀察事件后我們將獲得更多的信息。

• The equation for calculating the range of Entropy: 0 ≤ Entropy ≤ log(n), where n is the number of outcomes
  計(jì)算熵范圍的公式：0≤熵≤log(n)，其中n是結(jié)果數(shù)
• Entropy 0(minimum entropy) occurs when one of the probabilities is 1 and the rest are 0’s.
  當(dāng)其中一個(gè)概率為1且其余概率為0時(shí)，發(fā)生熵0(最小熵)。
• Entropy log(n)(maximum entropy) occurs when all the probabilities have equal values of 1/n.
  當(dāng)所有概率都具有等于1 / n的值時(shí)，就會(huì)出現(xiàn)熵log(n)(最大熵)。

翻譯自: https://medium.com/@regmi.sobit/why-randomness-is-information-f2468966b29d

總結(jié)

以上是生活随笔為你收集整理的为什么随机性是信息的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。