Research with adults has shown that a person's internal context, or state, influences how memory functions. This factor is rarely considered in research on infant memory, in part because of the practical and ethical difficulties of manipulating these variables in infants. In this article, we argue that models of infant memory will remain limited in scope and accuracy if the internal context of participants is not considered. As a case in point, we present emerging literature on sleep-dependent memory. Our review shows that for infants, timely sleep after a learning experience helps them retain and further process new memories. Studies need to explore the role of prior sleep for encoding, and to tease apart the contributions to infant memory of different types, features, and stages of sleep. We conclude that considering internal states, such as sleep, is necessary for developing a deeper understanding of early human memory.