Foreground object identification can be considered as anomaly detection in a redundant background. This paper proposes unsupervised deep learning of foreground objects on the basis of the prior knowledge about spatio-temporal sparseness and low-rankness of foreground objects and background scenes. The proposed framework trains a U-Net model to encode and decode the sparse foreground objects in batches of input images with low-rank backgrounds, by minimizing a combination of nuclear and ℓ1 norms as a loss function. This approach is similar to background subtraction based on robust principal component analysis (RPCA): an iterative method that detects sparse foreground objects as outliers while learning the principal components of the linearly dependent background. In contrast, the proposed method is advantageous over RPCA in that once the U-Net model has learned enough features common to the foreground objects, it can robustly detect them from any single image regardless of the low-rankness and sparseness. The U-Net also enables online object segmentation with much less computational expense than that of RPCA. These advantages are illustrated with background subtraction in video surveillance. It is also shown that the proposed method can build up a well-generalized cell segmentation model from only a few dozen unannotated training images.