© 2020 IEEE. This paper presents a navigation system for mobile robots in outdoor environments and the preliminary robot implementation results. Objectives of the proposed navigation system include path generation on the map (2D binary image) and path following of the robot to reach the goal location. In our method there is no waypoint in the generated paths and the map. The A-Star search algorithm is employed to plan paths on the map, and the q-learning is used to train the robot to follow the generated paths. The difference between the robot positions and A-star generated random paths is used to evaluate the performance of the proposed method. Preliminary simulation results revealed the potentials of the cooperation between reinforcement learning-based algorithms and conventional path planning algorithms for robot navigation.