The binomial and multinomial distribution

In order to discuss the statistical view of entropy we require a means to determine how many ways it is possible to divide N distinguishable particles into groups such that there are n₁ particles in the first group, n₂ in the second group, and so on, and such that n₁ + n₂ + … = N. Consider first the number of permutations possible for N particles. If there are N locations for the particles we can place the first one in any of N places, the second in any of N-1 places, the third in any of N-2, etc. The total number of ways of placing all of the particles is the product

N(N-1)(N-2)…(2)(1) = N!

Now consider that the particles are in two groups. In this case we have n₁ + n₂ = N. We can place particles in group 1 in any of n₁ positions, then n₁-1, n₁-2 etc. and we can do the same for group 2. You might be tempted to conclude that the number of ways of placing the particles in two groups is n₁!n₂!. However, all n₁! orders of the first group and n₂! orders of the second group correspond to just one division of N! objects in n₁ objects and n₂ objects. Therefore, the total number of ways of placing N particles in two groups is

W = N!/n₁!n₂!

The combination of factorials above appears in the binomial expansion

and is therefore called the binomial coefficient.
The multinomial coefficient applies to a situation where there are more than two groups. In general, if there are M different groups that we can place N objects in we have as the total number of ways

Application to permutations of letters

We have already seen that the permutations of the indices i,j,k, etc. gives rise to N! different combinations. The permutation of indices in the sum over Boltzmann factors lead to the factor of N! in the partition function Q = q^N/N! for indistinguishable particles. In this application we are assuming that all of the indices are unique and therefore the number of ways of arranging them is given by

i.e. there is only one index in each group.

Suppose we asked how many ways there are of arranging the letters in the word MINIMUM. There are three Ms and two Is with a total of seven letters. In this case then there are four groups, the group of M, N, I, and U. The total number of ways is

Application to a game of chance

Many of you have probably played a game called YAHTZEE. On each turn you role five dice. You obtain points based on the configuration of the dice. The following point scheme is made in the instructions.

We define a configuration as {[1],[2],[3],[4],[5],[6]} where [1] is number of ones etc. rolled on a particular cast of the dice.

Consider how many ways there are to get a YAHTZEE of all ones. In this case each of the five die must have the same number. The configuration is {5,0,0,0,0,0} and the number of ways is given by 5!/5!0!0!0!0! = 1. We all know that there is only one way to get all ones. But what about a full house? We must be specific since there are a number of full houses possible (how many?). Let us take two twos and three sixes. The configuration is {0,2,0,0,0,3} and the number of ways this can be achieved is 5!/0!2!0!0!0!3! = 120/(2x6) = 10. There are two possible long straights. The configuration is {1,1,1,1,1,0} and the number of ways this can be achieved is 5!/1!1!1!1!0! = 120/5 = 24. At this point you can calculate the number of ways a particular three-of-a-kind can be obtained.

Note that if you really want to calculate the total probability you need to take into account the fact that there are multiple configurations that have the same name. A YAHTZEE can be all ones, all twos etc. and thus there are six possible YAHTZEE’s with configurations

{5,0,0,0,0,0} {0,5,0,0,0,0} {0,0,5,0,0,0} {0,0,0,5,0,0} {0,0,0,0,5,0} {0,0,0,0,0,5}.