Bayesian Additive Regression Trees (BART) is a general Bayesian nonparametric regression algorithm. The posterior distribution is the distribution for the sum of the decision trees, and predictions are made by averaging fitted samples from the posterior distribution.
The combination of strong predictive performance and ability to provide uncertainty measures has made BART popularly used in social sciences, biostatistics, and causal inference.
BART uses Markov Chain Monte Carlo (MCMC) to obtain approximate posterior samples in the space of parameterized tree sums, but it is often observed that chain mixing is slow.
In this paper, we reduce the total to a single tree and provide an initial lower bound on the mixing time for a simplified version of BART that uses a subset of the possible migrations of the MCMC proposal distribution. The lower mixing time bound increases exponentially with the number of data points.
Inspired by this new relationship between mixing time and number of data points, we run rigorous simulations with BART. We qualitatively show that the BART mixing time increases with the number of data points.
The slow mixing time of simplified BART suggests large variations between different runs of the simplified BART algorithm, and similar large variations are known for BART in the literature. This large variation can lead to a loss of stability in models, predictions, and posterior intervals obtained from BART MCMC samples.
Lower bounds and simulations recommend increasing the number of chains depending on the number of data points.
