Book excerpt: Mechanical systems often involve multi-physics interactions and complex nonlinearities due to which design optimization and diagnostics become challenging. The inherent complexity of the processes, along with limitations in data availability, mandates principled uncertainty estimates in modeling, something which probabilistic machine learning techniques offer. Budget restrictions pose serious data limitations in the development and testing phases of the product pipeline, particularly in the presence of a hierarchy in the associated multi-physics process models with respect to their fidelity levels. This thesis focuses on some fundamental developments and applications of probabilistic machine learning strategies in design optimization and advanced diagnostics. Novel multi-fidelity surrogate modeling and optimization strategies will be discussed with respect to data-driven engineering design optimization problems, for example, process parameter optimization in additive manufacturing, phase-field model calibration and compressor rotor design. Even in applications involving high data volumes, quantification of aleatoric and epistemic uncertainties in the dataset often becomes useful in having conservative estimates of the predictions, an application of which has been discussed with respect to optimal selection of chemistry solvers for IC engine simulations. During the product testing phase, advanced diagnostics in performance critical applications mandate the detection of anomalies with as little data as possible. The thesis also focuses on some of the applications of a class of probabilistic sequential models, called hidden Markov models in the detection of thermoacoustic instabilities and lean blow-out in combustion systems using acoustic and chemiluminescence sensor data. The framework achieves computationally efficient and robust predictions of regime changes with parsimonious data requirements, which deems it suitable for online applications.