Beam interactions with surface waves and higher order modes in oversized G-band slow-wave structures (SWSs) are studied in a coaxial SWS. The coaxial SWS reduces the number of higher order mode and eases the mode competition problem. The SWS is excited by an annular electron beam. The slow cyclotron (SC) interaction as well as the Cherenkov interaction occurs due to 3-D beam perturbations of the electron beam. Growth rates for the surface waves are larger than those for the higher order modes and increase with increasing beam current. The SC interaction merges with the Cherenkov interaction for a sufficiently high beam current and a relatively low magnetic field. And the merged growth rate also increases with the beam current.