Dirichlet-multinomial Distribution - Probability Mass Function

Probability Mass Function

Conceptually, we are doing N independent draws from a categorical distribution with K categories. Let us represent the independent draws as random categorical variables for . Let us denote the number of times a particular category has been seen (for ) among all the categorical variables as . Note that . Then, we have two separate views onto this problem:

1. A set of N categorical variables .
2. A single vector-valued variable, distributed according to a multinomial distribution.

The former case is a set of random variables specifying each individual outcome, while the latter is a variable specifying the number of outcomes of each of the K categories. The distinction is important, as the two cases have correspondingly different probability distributions.

In both cases, the parameter of the categorical distribution is where is the probability to draw value k. p is likewise the parameter of the multinomial distribution P(x|p). Rather than specifying p directly, we give it a conjugate prior distribution, and hence it is drawn from a Dirichlet distribution with parameter vector .

By integrating out p, we obtain a compound distribution. However, the form of the distribution is different depending on which view we take.