Reinforcement Learning for Self-Organizing Wake-Up Scheduling in Wireless Sensor Networks

TitleReinforcement Learning for Self-Organizing Wake-Up Scheduling in Wireless Sensor Networks
Publication TypeJournal Article
Year of Publication2012
AuthorsMihaylov, M, Le Borgne, Y-A, Tuyls, K, Nowé, A
Secondary AuthorsFilipe, J, Fred, A
JournalCommunications in Computer and Information Science
Start Page382
ISBN Number978-3-642-29965-0

Wake-up scheduling is a challenging problem in wireless sen- sor networks. It was recently shown that a promising approach for solv- ing this problem is to rely on reinforcement learning (RL). The RL approach is particularly attractive since it allows the sensor nodes to coordinate through local interactions alone, without the need of cen- tral mediator or any form of explicit coordination. This article extends previous work by experimentally studying the behavior of RL wake-up scheduling on a set of three di erent network topologies, namely line, mesh and grid topologies. The experiments are run using OMNET++, a the state-of-the-art network simulator. The obtained results show how simple and computationally bounded sensor nodes are able to coordinate their wake-up cycles in a distributed way in order to improve the global system performance. The main insight of these experiments is to show that sensor nodes learn to synchronize if they have to cooperate for for- warding data, and learn to desynchronize in order to avoid interferences. This synchronization/desynchronization behavior, referred to for short as (de)synchronicity, allows to improve the message throughput even for very low duty cycles.