Learning in markov game for femtocell power allocation with limited coordination

In this paper, we study the power allocation problem for the downlink transmission in a set of closed-access femtocells which underlay a number of macrocells. We introduce a mutli-step pricing mechanism for the macrocells to control the cross- tier interference by femtocell transmissions without explicit coordination. We model the cross- tier joint power allocation process in the heterogeneous network as a non-cooperative, average-reward Markov game. By investigating the structure of the instantaneous payoff functions in the game, we propose a self-organized strategy learning scheme based on learning automata for both the macrocell base stations and the femtocell access points to adapt their transmit power simultaneously. We prove that the proposed learning scheme is able to find a pure-strategy Nash equilibrium of the game without the need for the femtocell access points to share any local information. Simulation results show the efficiency of the proposed learning scheme.