In this paper, we generalize previous results showing connections between inductive inference from positive data and algebraic structures by using tools from universal algebra. In particular, we investigate the inferability from positive data of language classes defined by closure operators. We show that some important properties of language classes used in inductive inference correspond closely to commonly used properties of closed set systems. We also investigate the inferability of algebraic closed set systems, and show that these types of systems are inferable from positive data if and only if they contain no infinite ascending chain of closed sets. This generalizes previous results concerning the inferability of various algebraic classes such as the class of ideals of a ring. We also show the relationship with algebraic closed set systems and approximate identifiability as introduced by Kobayashi and Yokomori [11]. We propose that closure operators offer a unifying framework for various approaches to inductive inference from positive data.