Numerical models of ultrasound propagation have applications across many fields including medical ultrasound. When used for treatment planning or computational dosimetry, their accuracy is particularly important in ensuring patient safety. Careful validation of the models is therefore required.Model validation can include comparison with analytical solutions or other models, and validation against measurements of ultrasound fields. To determine the level of agreement between measurements and simulations it is necessary to establish the expected uncertainties arising both during the measurement of acoustic pressure, and in the simulations, e.g. from the definition of medium properties and geometry.Here we describe a program of experimental validation of the open source k-Wave Matlab toolbox, which includes comparison of measurements and simulations of ultrasound fields in water and heterogeneous fluid and elastic absorbing media, with a discussion of the associated uncertainties.In both water, and with glycerol filled phantoms or bone mimicking phantoms, measured and simulated spatial peak pressure amplitudes agreed to within 6%, within the expected uncertainty, with close agreement in the size and position of the focal region. To help enable future validation of other codes, the measurement data generated will be made available in an open source database hosted by UCL.