Prospective Optimization
Human performance approaches that of an ideal
observer and optimal actor in some perceptual and motor
tasks. These optimal abilities depend on the capacity of the
cerebral cortex to store an immense amount of information
and to flexibly make rapid decisions. However, behavior only
approaches these limits after a long period of learning while
the cerebral cortex interacts with the basal ganglia, an ancient
part of the vertebrate brain that is responsible for learning
sequences of actions directed toward achieving goals. Progress
has been made in understanding the algorithms used by the
brain during reinforcement learning, which is an online
approximation of dynamic programming. Humans also make
plans that depend on past experience by simulating different
scenarios, which is called prospective optimization. The same
brain structures in the cortex and basal ganglia that are active
online during optimal behavior are also active offline during
prospective optimization. The emergence of general principles
and algorithms for goal-directed behavior has consequences
for the development of autonomous devices in engineering
applications.
