Organic farming has the potential to reverse biodiversity loss in farmland and benefit agriculture by enhancing ecosystem services. Although the mixed success of organic farming in enhancing biodiversity has been attributed to differences in taxa and landscape context, no studies have focused on the effect of macro-scale factors such as climate and topography. This study provides the first assessment of the impact of macro-scale factors on the effectiveness of within-farm management on biodiversity, using spiders in Japan as an example. A multilevel modelling approach revealed that reducing pesticide applications increases spider abundance, particularly in areas with high precipitation, which were also associated with high potential spider abundance. Using the model we identified areas throughout Japan that can potentially benefit from organic farming. The alteration of local habitat-abundance relations by macro-scale factors could explain the reported low spatial generality in the effects of organic farming and patterns of habitat association.