Finite State Space Markov Chains Let $S$ be a finite set and $X_n$ be a Markov chain taking values in $S$ with transition matrix $P$. Let $S'$ be another finite set with $|S'| < |S|$ and $\varphi : S \to S'$, which we will assume is onto without loss of generality.