Animals perform a complex array of behaviors, from changes in body posture to vocalizations to other dynamic outputs. Far from being a disordered collection of actions, however, there is thought to be an intrinsic structure to the set of behaviors and their temporal and functional organization. In this talk, I will introduce a novel method for mapping the behavioral space of organisms using unsupervised machine learning techniques. This method relies only upon the underlying structure of postural movement data to organize and classify behavior, eschewing ad hoc behavioral definitions. Applying this method to videos of freely-behaving fruit flies (D. melanogaster), I will show that the organisms’ behavioral repertoire consists of a hierarchically-organized set of stereotyped behaviors. This hierarchical patterning results in the emergence of long time scales of memory in the system, providing insight into the mechanisms of behavioral control over that occur over seconds, minutes, hours, days, and the entire lifetime of the fly and pointing to potential neurobiological implementations. Lastly, I will show the generality of this approach to behavioral analysis — its applicability to other species, alternative behavioral modalities, and high-throughput screens investigating the underlying neurobiology of behavior.