This way, we remove any unsupported assumptions from our distribution and keep ourselves honest. Higher entropy means that we are less sure about what will happen next.Īs such, we should maximize the entropy of our probability distribution as long as all required conditions (constraints) are satisfied. The information entropy is the measure of uncertainty. 3 Maximum Entropy Distribution with no Information That is where the concept of information entropy comes in. The question is how to express what we’ve said in mathematics. So, we should use a distribution expressing what we know and don’t know.īy using the uniform distribution, we tell the world that we are maximally ignorant about the weather and we are sincere. If we claim that we have some information without proof or statistics, we would be dishonest, and no one should ever believe our predictions. In other words, when we use a particular probability distribution (other than the uniform distribution) for the weather, we claim that we have some information about the weather. We could use a different probability distribution if we have some weather information.įor example, if we know the mean and standard deviation, we could use the normal distribution, which would be more beneficial than random guesses by the uniform distribution. Given that we have no information, our most unassuming model is a coin flip.Īlthough this keeps us honest, it would not be a very accurate weather model because the uniform distribution is entirely ignorant. That’d be like a coin flip, right? 2 Maximum Ignorance and Honesty Not really - we don’t have the mean and standard deviation of the weather, so we cannot use the normal distribution.Īs we don’t know anything, we can only randomly guess, can’t we? What kind of probability distribution should we use? So, our model should reflect that we are entirely ignorant about the weather. We don’t want to introduce bias or assumption into our weather model since we know nothing about the weather. How can we develop a probability distribution model to predict the probability of “Fine”? ![]() We don’t know if it’s in Japan, the US, or a country on a planet in a galaxy far, far away. However, we have no information about the place. Suppose we want to predict if the weather of a place is fine or not. ![]() If any of the above questions make you wonder, you are in the right place. Why do many probability distributions have the exponential term? ![]() Have you ever wondered why we often use the normal distribution?
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |