Recently, I strolled around an exciting fact about a difference of meaning of covariant and contravariant words in mathematics (category theory) and physics (tensor analysis). Well, that makes it harder for a mathematician and a physicist to talk about these two words without knowing in what sense.

In category theory, we can think of functors as the mapping of objects between categories. We can say that if a functor preserves the direction of morphism, then the functor is a covariant one. If it reverses the direction of the morphism, then it is a contravariant functor. John Baez has briefly mentioned them in his book "Gauge Fields, Knots and Gravity". An identity functor is a covariant functor, and so are tangent vectors. While cotangent vectors and 1-forms are contravariant. (1-form in this case is differential of a function, however, if a differential of a function is to be thought as a vector field then the vector fields are covariant.)

Suppose we have a map $\phi:M \rightarrow N$ from one manifold to another. On $N$, we have real valued functions defined from $\psi:N\rightarrow \mathbb{R}^n$. To get real valued functions on $M$ we have pullback $\psi$ from $N$ to $M$ by $\phi$. 

$$\phi * \psi = \psi \circ \phi$$

We see that real-valued functions on $M$ suffer a change of direction in their morphism. So they are contravariant.

In tensor analysis, one can say that $X_\mu$ is covariant and $X^\mu$ is contravariant. (It is important to dodge that $\partial_\mu$ is covariant while its component $v^\mu$ can be contravariant.)