Accretion on to seed black holes (BHs) is believed to play a crucial role in formation of supermassive BHs observed at high-redshift (z > 6). Here, we investigate the combined effect of gas angular momentum and radiation feedback on the accretion flow, by performing 2D axially symmetric radiation hydrodynamics simulations that solve the flow structure across the Bondi radius and the outer part of the accretion disc simultaneously. The accreting gas with finite angular momentum forms a rotationally supported disc inside the Bondi radius, where the accretion proceeds by the angular momentum transport due to assumed α-type viscosity. We find that the interplay of radiation and angular momentum significantly suppresses accretion even if the radiative feedback is weakened in an equatorial shadowing region. The accretion rate is O(α) ̃ O(0.01 - 0.1) times the Bondi value, where α is the viscosity parameter. By developing an analytical model, we show that such a great reduction of the accretion rate persists unless the angular momentum is so small that the corresponding centrifugal radius is ≲ 0.04 times the Bondi radius. We argue that BHs are hard to grow quickly via rapid mass accretion considering the angular momentum barrier presented in this paper....