Anc-VI Sets New Standards in Speed for Bellman Consistency in Reinforcement Learning

Anc-VI accelerates Bellman consistency, achieving faster convergence rates for reinforcement learning, particularly when the discount factor is close to 1, outpacing standard value iteration.


This content originally appeared on HackerNoon and was authored by Anchoring

:::info Authors:

(1) Jongmin Lee, Department of Mathematical Science, Seoul National University;

(2) Ernest K. Ryu, Department of Mathematical Science, Seoul National University and Interdisciplinary Program in Artificial Intelligence, Seoul National University.

:::

Abstract and 1 Introduction

1.1 Notations and preliminaries

1.2 Prior works

2 Anchored Value Iteration

2.1 Accelerated rate for Bellman consistency operator

2.2 Accelerated rate for Bellman optimality opera

3 Convergence when y=1

4 Complexity lower bound

5 Approximate Anchored Value Iteration

6 Gauss–Seidel Anchored Value Iteration

7 Conclusion, Acknowledgments and Disclosure of Funding and References

A Preliminaries

B Omitted proofs in Section 2

C Omitted proofs in Section 3

D Omitted proofs in Section 4

E Omitted proofs in Section 5

F Omitted proofs in Section 6

G Broader Impacts

H Limitations

2.1 Accelerated rate for Bellman consistency operator

First, for general state-action spaces, we present the accelerated convergence rate of Anc-VI for the Bellman consistency operator.

\

\

\

:::info This paper is available on arxiv under CC BY 4.0 DEED license.

:::

[1] Arguably, T π is affine, not linear, but we follow the convention of [69] say T π is linear.


This content originally appeared on HackerNoon and was authored by Anchoring


Print Share Comment Cite Upload Translate Updates
APA

Anchoring | Sciencx (2025-01-14T22:56:20+00:00) Anc-VI Sets New Standards in Speed for Bellman Consistency in Reinforcement Learning. Retrieved from https://www.scien.cx/2025/01/14/anc-vi-sets-new-standards-in-speed-for-bellman-consistency-in-reinforcement-learning/

MLA
" » Anc-VI Sets New Standards in Speed for Bellman Consistency in Reinforcement Learning." Anchoring | Sciencx - Tuesday January 14, 2025, https://www.scien.cx/2025/01/14/anc-vi-sets-new-standards-in-speed-for-bellman-consistency-in-reinforcement-learning/
HARVARD
Anchoring | Sciencx Tuesday January 14, 2025 » Anc-VI Sets New Standards in Speed for Bellman Consistency in Reinforcement Learning., viewed ,<https://www.scien.cx/2025/01/14/anc-vi-sets-new-standards-in-speed-for-bellman-consistency-in-reinforcement-learning/>
VANCOUVER
Anchoring | Sciencx - » Anc-VI Sets New Standards in Speed for Bellman Consistency in Reinforcement Learning. [Internet]. [Accessed ]. Available from: https://www.scien.cx/2025/01/14/anc-vi-sets-new-standards-in-speed-for-bellman-consistency-in-reinforcement-learning/
CHICAGO
" » Anc-VI Sets New Standards in Speed for Bellman Consistency in Reinforcement Learning." Anchoring | Sciencx - Accessed . https://www.scien.cx/2025/01/14/anc-vi-sets-new-standards-in-speed-for-bellman-consistency-in-reinforcement-learning/
IEEE
" » Anc-VI Sets New Standards in Speed for Bellman Consistency in Reinforcement Learning." Anchoring | Sciencx [Online]. Available: https://www.scien.cx/2025/01/14/anc-vi-sets-new-standards-in-speed-for-bellman-consistency-in-reinforcement-learning/. [Accessed: ]
rf:citation
» Anc-VI Sets New Standards in Speed for Bellman Consistency in Reinforcement Learning | Anchoring | Sciencx | https://www.scien.cx/2025/01/14/anc-vi-sets-new-standards-in-speed-for-bellman-consistency-in-reinforcement-learning/ |

Please log in to upload a file.



There are no updates yet.
Click the Upload button above to add an update.

You must be logged in to translate posts. Please log in or register.

Related Posts