論文

査読有り
2016年

Reinforcement learning for stabilizing an inverted pendulum naturally leads to intermittent feedback control as in human quiet standing

2016 38TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY (EMBC)
  • Kenjiro Michimoto
  • ,
  • Yasuyuki Suzuki
  • ,
  • Ken Kiyono
  • ,
  • Yasushi Kobayashi
  • ,
  • Pietro Morasso
  • ,
  • Taishin Nomura

2016-October
開始ページ
37
終了ページ
40
記述言語
英語
掲載種別
研究論文(国際会議プロシーディングス)
DOI
10.1109/EMBC.2016.7590634
出版者・発行元
IEEE

Intermittent feedback control for stabilizing human upright stance is a promising strategy, alternative to the standard time-continuous stiffness control. Here we show that such an intermittent controller can be established naturally through reinforcement learning. To this end, we used a single inverted pendulum model of the upright posture and a very simple reward function that gives a certain amount of punishments when the inverted pendulum falls or changes its position in the state space. We found that the acquired feedback controller exhibits hallmarks of the intermittent feedback control strategy, namely the action of the feedback controller is switched-off intermittently when the state of the pendulum is located near the stable manifold of the unstable saddle-type upright equilibrium of the inverted pendulum with no active control: this action provides an opportunity to exploit transiently converging dynamics toward the unstable upright position with no help of the active feedback control. We then speculate about a possible physiological mechanism of such reinforcement learning, and suggest that it may be related to the neural activity in the pedunculopontine tegmental nucleus (PPN) of the brainstem. This hypothesis is supported by recent evidence indicating that PPN might play critical roles for generation and regulation of postural tonus, reward prediction, as well as postural instability in patients with Parkinson's disease.

リンク情報
DOI
https://doi.org/10.1109/EMBC.2016.7590634
DBLP
https://dblp.uni-trier.de/rec/conf/embc/MichimotoSKKMN16
Web of Science
https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcAuth=JSTA_CEL&SrcApp=J_Gate_JST&DestLinkType=FullRecord&KeyUT=WOS:000399823500010&DestApp=WOS_CPL
URL
http://www.scopus.com/inward/record.url?eid=2-s2.0-85009079617&partnerID=MN8TOARS
URL
http://orcid.org/0000-0002-2690-6527
ID情報
  • DOI : 10.1109/EMBC.2016.7590634
  • ISSN : 1557-170X
  • DBLP ID : conf/embc/MichimotoSKKMN16
  • ORCIDのPut Code : 30716790
  • SCOPUS ID : 85009079617
  • Web of Science ID : WOS:000399823500010

エクスポート
BibTeX RIS