Book excerpt: This book describes how to create robots capable to develop the behavioral and cognitive skills required to perform a task through machine learning methods. It focuses on model-free approaches with minimal human intervention in which the behavior used by the robots to solve their task and the way in which such behavior is produced is discovered by the adaptive process automatically, i.e. it is not specified by the experimenter. The book, which is targeted toward researchers, PhD and Master students with an interest in machine learning and robotics: (i) introduces autonomous robots, evolutionary algorithms, reinforcement learning algorithms, and learning by demonstration methods, (ii) uses concrete experiments to illustrate the fundamental aspects of embodied intelligence, (iii) provides theoretical and practical knowledge, including tutorials and exercises, and (iv) provides an integrated review of recent research in this area carried within partially separated research communities.