In order to manage the ever-growing number of devices present in modern and future ambient environments, as well as their dynamics and openness, we aim to propose a distributed multi-agent system that learns, in interaction with a human user, what would be their preferred applications given the services available. The goal of this Ph.D. thesis is to focus on the interaction between a reinforcement learning system and the human user, to improve the system's learning capabilities as well as the user's ease with the system, and ultimately build a working prototype, usable by end-users.