In this paper, we propose a new hierarchical pulse neural network and its reinforcement learning algorithm with network extension. The proposed pulse neural network has three layers, and all of the neurons are pulse neurons. This network learns relations between input pulse sequences and the desired outputs by updating connection weights and by adding neurons dynamically. We carried out the computer simulation to confirm the performance of the proposed algorithm.