Book excerpt: This book integrates a wide range of research topics related to and necessary for the development of proactive, smart, computers in the human interaction loop, including the development of audio-visual perceptual components for such environments; the design, implementation and analysis of novel proactive perceptive services supporting humans; the development of software architectures, ontologies and tools necessary for building such environments and services, as well as approaches for the evaluation of such technologies and services. The book is based on a major European Integrated Project, CHLI (Computers in the Human Interaction Loop), and throws light on the paradigm shift in the area of HCI that rather than humans interactive directly with machines, computers should observe and understand human interaction, and support humans during their work and interaction in an implicit and proactive manner.