Kenji Doya



Research Interest

I enjoy playing many sports like tennis, snowboarding and windsurfing, and always wonder why people and animals can learn such a variety of skillful movements through random trials and repeated practices. This is why I started to build robots that learn to move in a way now known as "reinforcement learning," and have been studying the brain's mechanisms for motor control and learning.

And now as a triathlete, I face with daily decisions on my training menu: should I practice swimming, cycling, or running? And even a harder daily decision is: how much time should I spend for training, research, and family life? That gives me the motivation to work on the brain's mechanisms for decision making, by predicting the goodness and badness of choice options.

What makes our decisions difficult is lots of trade-offs, such as sticking to a certain choice or trying a new one, and focusing on what happens now or in the long run. I am very much interested in the brain's chemical systems that appear to play a role in controlling the balance in such trade-offs. A better understanding of the functions of "neuromodulators," such as serotonin, can be a key to unlocking the wonders of our emotional system and to the therapies of psychiatric disorders like depression and impulsivity.

I am very happy to be have this opportunity to run my lab in Okinawa with active young scientists from variety of backgrounds in computation, neurobiology, and robotics. I also enjoy taking a part in building a totally new graduate institute literally from the ground up. And it is needless to say that Okinawa gives me the best environment for triathlon training and all kinds of marine sports!

Publication List [English] [Japanese]

Representative Publications

• Doya K (2008). Modulators of decision making. Nature Neuroscience, 11, 410-416.
• Doya K (2007). Invitation to Computational Neuroscience: Toward Understanding the Brain's Learning Mechanisms (in Japanese). Saiensu-Sha.
• Doya K (2007). Reinforcement learning: Computational theory and biological mechanisms. HFSP Journal, 1, 30-40.
• Doya K, Ishii S, Pouget A, Rao RPN (2007). Bayesian Brain: Probabilistic Approaches to Neural Coding, MIT Press.
• Elfwing S, Doya K, Christensen HI (2007). Evolutionary development of hierarchical learning structures. IEEE Transactions on Evolutionary Computations, 11, 249-264.
• Tanaka SC, Schweighofer N, Asahi S, Shishida K, Okamoto Y, Yamawaki S, Doya K (2007). Serotonin differentially regulates short- and long-term prediction of rewards in the ventral and dorsal striatum. PLoS ONE, 2, e1333.
• Doya K, Uchibe E (2005). The Cyber Rodent project: Exploration of adaptive mechanisms for self-preservation and self-reproduction. Adaptive Behavior, 13, 149-160.
• Morimoto J, Doya K (2005). Robust reinforcement learning. Neural Computation, 17, 335-359.
• Samejima K, Ueda K, Doya K, Kimura M (2005). Representation of action-specific reward values in the striatum. Science, 301, 1337-1340.
• Schweighofer N, Doya K, Fukai H, Chiron JV, Furukawa T, Kawato M (2004). Chaos may enhance information transmission in the inferior olive. Proceedings of the National Academy of Sciences, USA, 101, 4655-4660.
• Tanaka SC, Doya K, Okada G, Ueda K, Okamoto Y, Yamawaki S (2004). Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nature Neuroscience, 7, 887-893.
• Wolpert DM, Doya K, Kawato M (2003). A unifying computational framework for motor control and social interaction. Philosophical Transactions of the Royal Society, 358, 593 -602.
• Doya K (2002). Metalearning and neuromodulation. Neural Networks, 15, 495-506.
• Doya K., Kimura H., Kawato M. (2001). Neural mechanisms of learning and control. IEEE Control Systems Magazine, 21, 42-54.
• Doya K (2000). Complementary roles of basal ganglia and cerebellum in learning and motor control. Current Opinion in Neurobiology, 10, 732-739.
• Doya K (2000). Reinforcement learning in continuous time and space. Neural Computation, 12, 219-245.
• Doya K (1999). What are the computations of the cerebellum, the basal ganglia, and the cerebral cortex. Neural Networks, 12, 961-974.
• Doya K, Sejnowski TJ (1999). A computational model of avian song learning. Gazzaniga M.S., The New Cognitive Neurosciences, MIT Press, 469-482.
• Doya K, Selverston AI (1994). Dimension reduction of biological neuron models by artificial neural networks. Neural Computation, 6, 696-717.
• Doya K, Yoshizawa S (1989). Adaptive neural oscillator using continuous-time back-propagation learning. Neural Networks, 2, 375-386.

Biography

• Education:
  • 1980-1984 B.S. in Engineering, University of Tokyo
  • 1984-1986 M.S. in Engineering, University of Tokyo
  • 1994 Ph.D. in Engineering, University of Tokyo
• Professional Positions:
  • 1986-1991 Instructor, University of Tokyo
  • 1991-1993 Visiting Researcher, University of California, San Diego
  • 1993-1994 Research Associate, The Salk Institute
  • 1994-2003 Senior Researcher, Advanced Telecommunication Research Institute International (ATR)
  • 1995-2006 Visiting Associate Professor, Nara Institute of Science and Technology (NAIST)
  • 1996-1999 Group Leader, Dynamic Brain Project, Japan Science and Technology Corporation (JST)
  • 1999-2005 Research Director, Metalearning and Neuromodulation Project, JST
  • 2003- Department Head, ATR Computational Neuroscience Laboratories
  • 2004- Principal Investigator, Okinawa Institute of Science and Technology
  • 2007- Visiting Professor, NAIST
• Social Services:
  • 1999-2002 Vice President, Japanese Neural Network Society
  • 1999-2003 Director, Neuro-Informatics Summer School
  • 2004-2008 Co-organizer, Okinawa Computational Neuroscience Course
  • 2008- Co-editor in Chief, Neural Networks

Awards

• 2000, 2003, 2005, 2006 Best Paper Awards, Japanese Neural Network Society
• 2007 JSPS Award, Japan Society for Promotion of Science
• 2008 Tsukahara Award, Brain Science Foundation

back

---