Book excerpt: The advent of adversarial machine learning fundamentally challenges the widespread adoption of Convolutional Neural Networks (CNNs), Vision Transformers and other deep neural networks. Addressing adversarial machine learning attacks are of paramount importance to ensure such systems can be safely deployed in sensitive areas like health care and security. In this dissertation, we focus on developing three key concepts in adversarial machine learning: defense analysis for CNNs, defense design for CNNs and the robustness of the new Vision Transformer architecture. From the analysis side, we develop a new adaptive black-box attack and test eight recent defenses under this threat model. Next, we specifically focus on the black-box threat model and design a novel defense which oers significant improvements in robustness over state-of-the-art defenses. Lastly, we study the robustness of Vision Transformers, a new alternative to CNNs. We propose a new attack on Vision Transformers as well as a new CNN/transformer hybrid defense.