Consider a complete non-compact (weighted) Riemannian manifold M which can be decomposed as a connected sum of finitely many compete non-compact manifolds, each of which satisfies a two-sided Gaussian heat kernel bound. We review what is known concerning heat kernel estimates on M, including some recent progresses in the case M is parabolic, and discuss the difficulties remaining to obtain a complete result. We also discuss implication for the lowest eigenvalue (with Neumann boundary condition) of large balls centered at a fixed central point.