This paper investigates the distributed mean square output tracking
problem of heterogeneous multi-agent systems with Markovian switching
topologies and infinite transmission delays. The main challenge of the
concerned problem arises from how to deal with Markovian switching
topologies and infinite transmission delays simultaneously. A novel
distributed observer is developed based on a newly proposed Lyapunov
functional method. Then, a distributed controller based on the
distributed observer is developed. It is shown that the stochastic
distributed tracking problem is solved in the sense of mean square if
the union graph of the underlying Markovian switching topology contains
a spanning tree. A distinctive feature of the proposed controller is
that the infinite delays are not required to be known. Finally, the
effectiveness of the proposed controller is illustrated by two numerical
examples.