Gaussian process regression underpins a myriad of academic and industrial applications in machine learning and statistics, with maximum likelihood estimation routinely used to select appropriate parameters for covariance kernels. However, it is an open question to establish the circumstances under which maximum likelihood estimation is well done, that is, the predictions of regression models are immune to small perturbations in the data. This article identifies scenarios in which the maximum likelihood estimator is not well set. These failure cases occur in a noiseless data setting for any Gaussian process with a stationary covariance function whose length-scale parameter is estimated using the maximum likelihood method. Although maximum likelihood failures are part of Gaussian process folklore, these rigorously theoretical results appear to be the first of their kind. The implication of these negative results is that when training Gaussian process models using maximum likelihood estimation, fitness may need to be evaluated a posteriori on a case-by-case basis.
