OpenCL and OpenACC are generic frameworks for heterogeneous programming combining CPU and accelerator devices such as GPUs. We test incorporation of these two frameworks into the Bridge++, a general-purpose code set for numerical simulations of lattice QCD, to offload the time-consuming fermion matrix inversion to accelerator devices. We discuss advantages and dis-Advantages of these frameworks from the viewpoints of constructing reusable components based on the object-oriented programming and of tuning the code to achieve a high performance.