Finite-sample Guarantees of Stochastic Approximation Algorithms in Average-reward Reinforcement Learning and Decentralized Stochastic Optimization