Aims: To improve glycemic variability (GV) is crucial in the management of multiple daily insulin (MDI) treatment in diabetes. To evaluate the GV improvement in MDI treated type 2 diabetes (T2D) with low-dose metformin 750 mg/day (LMET), which was popular in the clinical practice in Japan, we compared the effect of adding vildagliptin 100 mg/day (LMET + DPP4i treatment) or increased metformin dose to 1500 mg/day (HMET treatment), in the setting of continuous glucose monitoring (CGM) analysis. Materials and methods: Single-center, open-label, 12 weeks-two period cross-over design. Twenty T2D with inadequately controlled (7.0% < HbA1c ≤ 9.0%) with MDI + LMET were enrolled. Primary endpoints were GV and hypoglycemia derived from CGM indices, performed after each 12 week treatment periods. Results: There was no significant difference in both LMET + DPP4i treatment and HMET treatment, in terms of HbA1c, body weight changes, and total daily dose of insulin to achieve the targeted glycemia. LMET + DPP4i treatment compared to HMET treatment, significantly reduced the calculated GV value, mean (7.15 ± 1.30 vs 7.82 ± 1.60, p = 0.04), standard deviation (1.78 ± 0.55 vs 2.27 ± 1.11, p = 0.03), continuous overlapping net glycemic action (6.44 ± 1.28 vs 7.12 ± 1.69, p < 0.05), J-Index (26.7 ± 11.0 vs 34.9 ± 19.8, p < 0.05), high blood glucose index (3.01 ± 1.96 vs 6.73 ± 4.85, p = 0.02), and mean amplitude of glycemic excursions (4.53 ± 1.35 vs 5.50 ± 2.34, p = 0.03). Conclusion: The GV metrics regarding daily and nocturnal hypoglycemia were not significantly different between LMET + DPP4i treatment and HMET treatment. LMET + DPP4i treatment decreased GV associated with hyperglycemia. Adding DPP-4-inhibitor to the lower dose of metformin is an alternative approach to the stable GV in MDI compared to additional high-dose metformin. National Clinical Trial registration in Japan, number is JPRN-UMIN000024663. Supplementary Information: The online version contains supplementary material available at 10.1007/s13340-021-00513-6.