We provide a sufficient condition for the existence of a Markov perfect equilibrium for pure strategies in a class of Markov games where each stage has strategic complementarities. We assume that both the sets of actions for all players and the set of states are finite and that the horizon is also finite, while the past studies examined Markov games with infinite horizons where the sets of actions and states are assumed to be infinite. We give an elementary proof of the existence and apply the result to a game of Bertrand oligopoly with investment.