We investigate the properties of a sample of 35 galaxies, detected with ALMA at 1.1 mm in the GOODS-ALMA field (area of 69 arcmin$^2$, resolution = 0.60", RMS $\simeq$ 0.18 mJy beam$^{-1}$). Using the UV-to-radio deep multiwavelength coverage of the GOODS-South field, we fit the spectral energy distributions of these galaxies to derive their key physical properties. The galaxies detected by ALMA are among the most massive at $z$ = 2-4 (M$_{\star,med}$ = 8.5$ \times$ 10$^{10}$ M$_\odot$) and are either starburst or located in the upper part of the galaxy star-forming main sequence. A significant portion of our galaxy population ($\sim$ 40%), located at $z\sim$ 2.5-3, exhibits abnormally low gas fractions. The sizes of these galaxies, measured with ALMA, are compatible with the trend between $H$-band size and stellar mass observed for $z\sim2$ elliptical galaxies suggesting that they are building compact bulges. We show that there is a strong link between star formation surface density (at 1.1 mm) and gas depletion time: the more compact a galaxy's star-forming region is, the shorter its lifetime will be (without gas replenishment). The identified compact sources associated with relatively short depletion timescales ($\sim$100 Myr), are the ideal candidates to be the progenitors of compact elliptical galaxies at $z$ $\sim$ 2....