Sample Efficient Hierarchical Reinforcement Learning via Best Policy Identification
Mirrored from arXiv — Machine Learning for archival readability. Support the source by reading on the original site.
Computer Science > Machine Learning
Title:Sample Efficient Hierarchical Reinforcement Learning via Best Policy Identification
Abstract:We present HBPI-UCRL, a model-based algorithm for hierarchical reinforcement learning (HRL) that learns high-level and low-level policies in parallel. HBPI-UCRL exploits the fact that a high-level transition corresponds to a multi-step transition at the low level. We introduce two conditions on the low-level dynamics that are sufficient to make parallel HRL learnable. When these conditions hold, we prove that HBPI-UCRL has a polynomial sample complexity in the problem parameters. In the sparse-reward, goal-directed setting, our sample complexity upper bound for HBPI-UCRL is strictly lower than that of its non-hierarchical counterpart, providing theoretical justification for the empirical success of HRL.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2607.29294 [cs.LG] |
| (or arXiv:2607.29294v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2607.29294
arXiv-issued DOI via DataCite (pending registration)
|
Access Paper:
- View PDF
- TeX Source
References & Citations
Bookmark
Bibliographic and Citation Tools
Code, Data and Media Associated with this Article
Demos
Recommenders and Search Tools
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
More from arXiv — Machine Learning
-
Application of Artificial Intelligence for Fraudulent Banking Operations Recognition
Aug 11
-
Data-Driven Fire-Zone Segmentation for Improved Short-Term Wildfire Prediction
Aug 11
-
Evolving Safety Landscape of Multi-modal Large Language Models: A Survey of Emerging Threats and Safeguards
Aug 11
-
Tracing sources of epistemic uncertainty in deep learning predictions: homo- and hetero-scedastic linearized estimators
Aug 11
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.