The central research question is: “How can complex networks become self-organising while ensuring stability and without sacrificing on performance. Moreover the decisions taken by the system should be understand- able and guidable.”
More precisely, the project aims to develop a framework for studying and managing modern distributed networked systems that contain a large number of entities or agents, both machine and human, which strive to achieve their personal objectives. The framework developed within the proposal will guide these entities, either through direct control or by way of incentives, in order to achieve system-wide optimal behaviour, satisfy global objectives and adhere to the system’s operational constraints in the face of diverging and incompatible personal goals. Software language abstractions will be identified and developed, to support the ease of the deployment of the framework on a wide variety of networks.
The framework will build on the expertise of the teams in machine learning (including game theory, self-organisation of complex systems, large-scale multi-agent systems and emergent social behaviour), network management and modelling, and software language design. The key idea of the framework is that the context within which intelligent decision making components or agents operate may depend on spatial and temporal factors. As such, they should be able to adapt their behaviour and goals as a function of space and time.
The framework should satisfy the following requirements: It should be generic so as to be applicable to a wide range of networks, it should be scalable with respect to the size of the network, the resulting behaviour should be (near) optimal and at all times minimal performance should be guaranteed, also in unexpected situations. Several fundamental scientific challenges remain to be solved before this high-level objective can be achieved. They can be summarised as follows:
• Complex multi-agent control
The SMILE-IT project will develop programming abstractions for distributed network control, that allow agents to be configured and controlled in a network-, rather than agent-centric manner. Moreover, it will provide abstractions to efficiently query and control the state of large-scale complex networks, as reinforcement learning techniques continuously require a view on the current state of the environment.
• Fast and stable convergence towards an acceptable solution
SMILE-IT aims to guarantee acceptable performance as soon as the management agents go operational. The project will investigate how to combine learning with heuristic knowledge and existing control strategies, in order to guarantee performance during learning. Moreover, solutions are needed that allow agents with (partially) conflicting goals to collaborate and jointly achieve a converged policy that leads to acceptable performance for all.
• Robust and adaptive management under unexpected conditions
Novel learning-based techniques that can cope with large degrees of uncertainty, and select suitable actions even if the network’s state is only partially known will be developed. Moreover, unexpected situations, such as failures or faults, may occur during operations. SMILE-IT will investigate how reinforcement learning techniques can be applied to detecting and recovering from such unexpected situations.
The development of the SMILE-IT framework will be guided by 2 driving cases in the smart grids and telecom networks application domains. Advanced prototypes for these cases will be developed to support an extensive evaluation of the SMILE-IT technologies. In a later phase, applications to a number of additional domains proposed by the user committee (such as traffic networks) will be examined, in order to demonstrate the general applicability of SMILE-IT methods.