Learning by Experiment: Continually Evolving Machines

Evolution always presented life forms with new challenges -- due to changes in weather, terrain, competition between different organisms, and other reasons. To increase the chance of survival, instead of solely optimizing current performance, it is in an agent's interest to maximize its ability to adapt to changes. Possibly this old evolutionary trait manifests itself in modern humans in their ability to adapt to new tasks and challenges quickly. Even if we consider a lifetime of a human, the ability to adapt is critical. An open question is what enables humans to adapt, a trait that modern AI systems lack. A prominent theory in developmental psychology suggests that "seemingly" frivolous play is a mechanism for infants to experiment to increase their knowledge incrementally. Play prepares infants for future life by laying down the foundation of a high-level experimentation framework to quickly understand how things work in new environments for constructing goal-directed plans. I will discuss how the idea of experimentation can be leveraged to construct robots that improve with experience and solve novel problems presented to them.