We study Langevin dynamics describing nonequilibirum steady states. Employing the phenomenological framework of steady-state thermodynamics constructed by Oono and Paniconi [Prog. Theor. Phys. Suppl. 130, 29 (1998)], we find that the extended form of the second law which they proposed holds for transitions between steady states and that the Shannon entropy difference is related to the excess heat produced in an infinitely slow operation. A generalized Version of the Jarzynski work relation plays an important role in our theory.