Isolated silica concretions in calcareous sediments have unique shapes and distinct sharp boundaries and are considered to form by diagenesis of biogenic siliceous grains. However, the details and rates of syngenetic formation of these spherical concretions are still not fully clear. Here we present a model for concretion growth by diffusion, with chemical buffering involving decomposition of organic matter leading to a pH change in the pore-water and preservation of residual bitumen cores in the concretions. The model is compatible with some pervasive silica precipitation. Based on the observed elemental distributions, C, N, S, bulk carbon isotope and carbon preference index (CPI) measurements of the silica-enriched concretions, bitumen cores and surrounding calcareous rocks, the rate of diffusive concretion growth during early diagenesis is shown using a diffusion-growth diagram. This approach reveals that ellipsoidal SiO2 concretions with a diameter of a few cm formed rapidly and the precipitated silica preserved the bitumen cores. Our work provides a generalized chemical buffering model involving organic matter that can explain the rapid syngenetic growth of other types of silica accumulation in calcareous sediments.