PubMed · 14692633
Inter-module credit assignment in modular reinforcement learning.
Abstract
Critical issues in modular or hierarchical reinforcement learning (RL) are (i) how to decompose a task into sub-tasks, (ii) how to achieve independence of learning of sub-tasks, and (iii) how to assure optimality of the composite policy for the entire task. The second and last requirements are often under trade-off. We propose a method for propagating the reward for the entire task achievement between modules. This is done in the form of a 'modular reward', which is calculated from the temporal difference of the module gating signal and the value of the succeeding module. We implement modular reward for a multiple model-based reinforcement learning (MMRL) architecture and show its effectiveness in simulations of a pursuit task with hidden states and a continuous-time non-linear control task.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Kazuyuki Samejima, Kenji Doya, Mitsuo Kawato. 2003. Inter-module credit assignment in modular reinforcement learning.. https://doi.org/10.1016/s0893-6080(02)00235-6. https://pubmed.health/record/pubmed%3A14692633/inter-module-credit-assignment-in-modular-reinforcement-learning
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite PubMed Health by Indiciae: BibTeX · RIS.