What is abstraction? In our view, abstraction is generalization. Specifically, we propose that abstract concepts emerge as the natural product of associative learning and generalization by similarity. We support this proposal by presenting evidence for two ideas: first, that children's knowledge about how categories are organized and how words refer to them can be explained as learned generalizations over specific experiences of words referring to categories; and second, that the path of concepts from concrete to more abstract can be observed throughout development and that even in their more abstract form, concepts retain some of their original sensory basis. We illustrate these two facts by examining, in two kinds of learners—networks and young children—the development of three abstract ideas: (i) the idea of word; (ii) the idea of object; and (iii) the idea of substance.