Covariance - Definition

Definition

The covariance between two jointly distributed real-valued random variables x and y with finite second moments is defined as

\sigma(x,y) = \operatorname{E}{\big)(y - \operatorname{E})\big]},

where E is the expected value of x, also known as the mean of x. By using the linearity property of expectations, this can be simplified to

\begin{align} \sigma(x,y) &= \operatorname{E}\left\right) \left(y - \operatorname{E}\left\right)\right] \\ &= \operatorname{E}\left - \operatorname{E}\left y + \operatorname{E}\left \operatorname{E}\left\right] \\ &= \operatorname{E}\left - \operatorname{E}\left \operatorname{E}\left - \operatorname{E}\left \operatorname{E}\left + \operatorname{E}\left \operatorname{E}\left \\ &= \operatorname{E}\left - \operatorname{E}\left \operatorname{E}\left, \end{align}

where is the correlation between x and y; if this term is zero then the random variables are orthogonal.

For random vectors and (of dimension m and n respectively) the m×n covariance matrix is equal to

\begin{align} \sigma(\mathbf{x},\mathbf{y}) & = \operatorname{E} \left) (\mathbf{y} - \operatorname{E})^\mathrm{T}\right]\\ & = \operatorname{E}\left - \operatorname{E}\operatorname{E}^\mathrm{T}, \end{align}

where mT is the transpose of the vector (or matrix) m.

The (i,j)-th element of this matrix is equal to the covariance Cov(xi, yj) between the i-th scalar component of x and the j-th scalar component of y. In particular, Cov(y, x) is the transpose of Cov(x, y).

For a vector \mathbf{x}= \begin{bmatrix}x_1 & x_2 & \dots & x_m\end{bmatrix}^\mathrm{T} of m jointly distributed random variables with finite second moments, its covariance matrix is defined as

Random variables whose covariance is zero are called uncorrelated.

The units of measurement of the covariance Cov(x, y) are those of x times those of y. By contrast, correlation coefficients, which depend on the covariance, are a dimensionless measure of linear dependence. (In fact, correlation coefficients can simply be understood as a normalized version of covariance.)

Read more about this topic:  Covariance

Famous quotes containing the word definition:

    “The definition of good prose is proper words in their proper places; of good verse, the most proper words in their proper places. The propriety is in either case relative. The words in prose ought to express the intended meaning, and no more; if they attract attention to themselves, it is, in general, a fault.”
    —Samuel Taylor Coleridge (1772–1834)

    “According to our social pyramid, all men who feel displaced racially, culturally, and/or because of economic hardships will turn on those whom they feel they can order and humiliate, usually women, children, and animals—just as they have been ordered and humiliated by those privileged few who are in power. However, this definition does not explain why there are privileged men who behave this way toward women.”
    —Ana Castillo (b. 1953)

    “No man, not even a doctor, ever gives any other definition of what a nurse should be than this—”devoted and obedient.” This definition would do just as well for a porter. It might even do for a horse. It would not do for a policeman.”
    —Florence Nightingale (1820–1910)