This paper shows structural optimal design of optical waveguide components utilizing an efficient 3D frequency-domain and 2D time-domain beam propagation method (BPM) with an alternating direction implicit (ADI) scheme. Usual optimal design procedure is based on iteration of numerical simulation, and total computational cost of the optimal design mainly depends on the efficiency of numerical analysis method. Since the system matrices are tridiagonal in the ADI-based BPM, efficient analysis and optimal design are available. Shape and topology optimal design shown in this paper is based on optimization of density distribution and sensitivity analysis to the density parameters. Computational methods of the sensitivity are shown in the case of using the 3D semi-vectorial and 2D time-domain BPM based on ADI scheme. The validity of this design approach is shown by design of optical waveguide components: mode converters, and a polarization beam splitter.