We present a first principle scheme to compute the rigidity, i.e., the shear-modulus of structural glasses at finite temperatures using the cloned liquid theory, which combines the replica theory and the liquid theory. With the aid of the replica method which enables disentanglement of thermal fluctuations in liquids into intra-state and inter-state fluctuations, we extract the rigidity of metastable amorphous solid states in the supercooled liquid and glass phases. The result can be understood intuitively without replicas. As a test case, we apply the scheme to the supercooled and glassy state of a binary mixture of soft-spheres. The result compares well with the shear-modulus obtained by a previous molecular dynamic simulation. The rigidity of metastable states is significantly reduced with respect to the instantaneous rigidity, namely, the Born term, due to non-affine responses caused by displacements of particles inside cages at all temperatures down to T = 0. It becomes nearly independent of temperature below the Kauzmann temperature T-K. At higher temperatures in the supercooled liquid state, the non-affine correction to the rigidity becomes stronger suggesting melting of the metastable solid state. Inter-state part of the static response implies jerky, intermittent stress-strain curves with static analogue of yielding at mesoscopic scales. (C) 2012 American Institute of Physics. [http://dx.doi.org/10.1063/1.4722343]