SAMSI Working Group on Probabilistic Numerics

Duration: This project officially ended on May 31st 2018, as part of the Program on Quasi-Monte Carlo and High-Dimensional Sampling Methods for Applied Mathematics.

Group Leaders: Tim Sullivan and Chris Oates

Description: The accuracy and robustness of numerical predictions that are based on mathematical models depend critically upon the construction of accurate discrete approximations to key quantities of interest. The exact error due to approximation will be unknown to the analyst, but worst-case upper bounds can often be obtained. This working group aims, instead, to develop Probabilistic Numerical Methods, which provide the analyst with a richer, probabilistic quantification of the numerical error in their output, thus providing better tools for reliable statistical inference.

Research Topics:

  • Reference priors for the probabilistic solution of differential equations.

  • Heavy-tailed stable distributions for robust uncertainty quantification.

  • Statistical estimation with multi-resolution operator decompositions.

  • Probabilistic numerical methods as Bayesian inversion methods.

Group Members: 

Workshops and Visits: 

  • July and August, 2017: F Schaefer visited M Girolami and F-X Briol @ Alan Turing Institute and Imperial College London.

  • August and September, 2017: F-X Briol and A Barp visited H Owhadi, A Stuart and F Schaefer @ Caltech.

  • August 28 - Sept 1, 2017: Group meeting at the SAMSI Program on Quasi Monte Carlo Opening Workshop in Duke, NC, USA.

  • April 11-13, 2018: Meeting of the working group at the Alan Turing Institute, London. Supported by SAMSI (10,000 USD) and the Lloyds Register Foundation Programme on Data-Centric Engineering at the Alan Turing Institute (3,000 GBP). [web] [web2]

  • April 16-18, 2018: Minisymposium on Probabilistic Numerical Methods for Quantification of Discretisation Error @ SIAM UQ, CA, USA. [Part I] [Part II] [Part III]

  • May 7-9, 2018: Session on Probabilistic Numerics @ SAMSI QMC Transition Workshop, NC, USA. [web]

Reading Group: (currently organised by Han Cheng Lie, formerly organised by F-X Briol)

  • 24-01-2017: Louis Ellam - A statistical model of urban retail structure.

  • 07-02-2017: Jon Cockayne - Discussion of "A probabilistic model for the numerical solution of initial value problems" by Schober et al. [slides]

  • 21-02-2017: Chris Oates - Discussion of "Probabilistic interpretation of linear solvers" by P. Hennig. [slides]

  • 07-03-2017: Francois-Xavier Briol - Discussion of "An introduction to sampling via measure transport" by Marzouk et al. [slides]

  • 21-03-2017: Tim Sullivan - Discussion of "MAP estimators and their consistency in Bayesian nonparametric inverse problems" by Dashti et al and "Maximum a posteriori probability estimates in infinite-dimensional Bayesian inverse problems", by Helin and Burger.

  • 04-04-2017: Han Cheng Lie - Discussion of "Why does Monte Carlo Fail to Work Properly in High-Dimensional Optimization Problems?", by Polyak and Shcherbakov.

  • 18-04-2017: Jon Cockayne - Linear Algebra for Probabilistic Numerics. [slides]

  • 02-05-2017: Louis Ellam - Pre-conditioned Ensemble Monte Carlo.

  • 16-05-2017: Onur Teymur - Discussion of "Bayesian Inference of Log Determinants" by Fitzsimons et al.

  • 11-07-2017: Toni Karvonen - Discussion of "Fully symmetric kernel quadrature" by Karvonen and Särkkä. [slides]

  • 25-07-2017: Discussion of Mike Larkin's work:

    • Chris Oates to discuss "Estimation of a non-negative function". [slides]

    • Tim Sullivan to discuss "Optimal approximation in Hilbert spaces with reproducing kernel functions".

    • Han Cheng-Lie to discuss "Gaussian measure in Hilbert space and applications in numerical analysis".

    • Jon Cockayne to discuss "Weak probability distributions on reproducing kernel Hilbert spaces" [slides]

  • 08-08-2017: Tom Rainforth - Discussion of "Bayesian Optimization for Probabilistic Programs" by Rainforth et al.

  • 02-10-2017 (3pm UK): Tim Sullivan and Chris Oates - (Re)introduction to the SAMSI Working Group.

  • 16-10-2017 (3pm UK): Motonobu Kanagawa - Discussion of "Convergence Analysis of Deterministic Kernel-Based Quadrature Rules in Misspecified Settings" by Kanagawa et al. [slides]

  • 30-10-2017 (3pm UK): Jon Cockayne - Discussion of "Bayesian Probabilistic Numerical Methods for Industrial Process Monitoring" by Oates et al. [slides]

  • 13-11-2017 (4pm UK): Jens Oettershagen - Discussion of "Construction of Optimal Cubature Algorithms with Applications to Econometrics and Uncertainty Quantification", PhD thesis. [slides]

  • 27-11-2017 (4pm UK): Florian Schaefer - Discussion of "Compression, Inversion and Approximate PCA of Dense Kernel Matrices at Near-Linear Computational Complexity", by Schaeffer et al. 

  • 11-12-2017 (4pm UK): Chris Oates - Discussion of "Better Together? Statistical Learning in Models Made of Modules", by Jacob et al. [slides]

  • 29-01-2018 (3pm UK): Tim Sullivan - Discussion of "Random forward models and log-likelihoods in Bayesian inverse problems" by Lie et al. [slides]

  • 12-02-2018 (4pm UK): Reimar Leike - Discussion of "Towards information optimal forward simulation of partial differential equations" by Leike and Enßlin

  • 26-02-2018 (4pm UK): Francois-Xavier Briol - Discussion of "Bayesian Quadrature for Multiple Related Integrals" by Xi et al. [slides]

  • 12-03-2018 (4pm UK): Henry Chai - Discussion of "An Improved Bayesian Framework for Quadrature of Constrained Integrands" by Chai and Garnett. [slides]

Publications:

  • Kersting H, Sullivan TJ, Hennig P (2018) Convergence Rates of Gaussian ODE Filters. [arXiv]

  • Teymur O, Calderhead B, Lie HC, Sullivan TJ (2018) Implicit Probabilistic Integrators for ODEs. NeurIPS 2018. [arXiv]

  • Wang J, Cockayne J, Oates CJ (2018) On the Bayesian Solution of Differential Equations. [arXiv]

  • Rathinavel J and Hickernell FJ (2018+) Automatic Bayesian Cubature. [in preparation]

  • Karvonen T, Oates CJ, Särkkä S (2018) A Bayes-Sard Cubature Method. NeurIPS 2018. [arXiv]

  • Cockayne J, Oates CJ, Girolami M (2018) A Bayesian Conjugate Gradient Method. [arXiv] [Software]

  • Xi X, Briol F-X, Girolami M (2018) Bayesian Quadrature for Multiple Related Integrals. ICML 2018. [arXiv]

  • Dukic V, Bortz DM (2018) Uncertainty Quantification Using Probabilistic Numerics: Application to Models in Mathematical Epidemiology. Inverse Problems in Science and Engineering, 28(2):223-232. [Journal]

  • Lie HC, Sullivan T (2017) Equivalence of Weak and Strong Modes of Measures on Topological Vector Spaces. [arXiv]

  • Lie HC, Sullivan T, Teckentrup, AL (2017) Random Forward Models and Log-Likelihoods in Bayesian Inverse Problems. [arXiv]

  • Oates CJ, Cockayne J, Aykroyd RG (2017) Bayesian Probabilistic Numerical Methods for Industrial Process Monitoring. [arXiv]

  • Oates CJ, Niederer S, Lee A, Briol F-X, Girolami M. (2017) Probabilistic Models for Integration Error in Assessment of Functional Cardiac Models. Advances in Neural Information Processing Systems (NIPS 2017). [Journal] [arXiv] [Video] [Poster] [Blog]

  • Schaefer F, Sullivan TJ, Owhadi H (2017) Compression, inversion, and approximate PCA of dense kernel matrices at near-linear computational complexity. [arXiv]

  • Lie HC, Stuart AM, Sullivan TJ (2017) Strong Convergence Rates of Probabilistic Integrators for Ordinary Differential Equations. [arXiv]

  • Cockayne J, Oates CJ, Sullivan T, Girolami M. (2017) Bayesian Probabilistic Numerical Methods. [arXiv] [Best Student Paper Award, ASA Section on Bayesian Statistical Science]

  • Briol F-X, Cockayne J, Teymur, O. (2016) Contributed Discussion on Article by Chkrebtii, Campbell, Calderhead, and Girolami. Bayesian Analysis, 11(4):1285-1293. [Journal] [arXiv]

Presentations:

  • 23-01-2017: FX Briol to speak @ Stochastic Analysis Seminar, Mathematics Institute, University of Oxford, UK.

  • 31-01-2017: FX Briol to speak @ Workshop on the Mathematics for Measurement, Edinburgh, UK.

  • 16-02-2017: C Oates to speak @ the UNSW Workshop on High-Dimensional Approximation. Talk title: Bayesian Probabilistic Numerical Computation.

  • 22-02-2017: J Cockayne to speak @ Computer Science Department, Imperial College London, UK.

  • 01-03-2017: J Cockayne to speak @ SIAM Conference on Computation Science and Engineering. Talk title: Probabilistic Meshless Methods for Partial Differential Equations and Bayesian Inverse Problems. [Video]

  • 24-03-2017: C Oates to give a tutorial on Probabilistic Numerics at the University of New South Wales, Australia.

  • 05-05-2017: J Cockayne to speak @ Statistics Seminar Series, Imperial College London, UK.

  • 15-05-2017: J Cockayne to present a poster @ Advances in Data Science, University of Manchester, UK.

  • June 5-9, 2017: T Sullivan, J Cockayne, F Schaefer, H Owhadi, O Chkrebtii and C Oates to speak @ ICERM workshop on Probabilistic Scientific Computing: Statistical inference approaches to numerical analysis and algorithm design, organised by P Hennig, H Owhadi and others. [Videos]

  • 13-06-2017: J Cockayne to speak @ Max Planck Institute for Intelligent Systems, Tuebingen, Germany.

  • June 18-23, 2017: P Hennig and C Oates to run the Dobbiaco Summer School on Probabilistic Numerics in Bolzano, Italy.

  • 05-07-2017: FX Briol to speak @ Statistical Data Science Workshop, Imperial College London and Winton Capital.

  • July 10-14, 2017: FX Briol to speak @ SIAM Annual Meeting in Pittsburgh, PA, USA. [Abstract]

  • July 29 - August 4, 2017: M Girolami to deliver Medallion Lecture @ Joint Statistical Meeting, Baltimore, USA. [Media]

  • August 28 - Sept 1, 2017: T Sullivan and C Oates to speak at the SAMSI Program on Quasi Monte Carlo Opening Workshop in Duke, NC, USA.

  • September 2017: C Oates to speak @ Turing Data Science Classes, Alan Turing Institute, London, UK. [Slides]

  • November 29, 2017: C Oates to speak @ Cantab Capital Institute for the Mathematics of Information, University of Cambridge, UK.

  • January 10, 2018: C Oates to speak @ Isaac Newton Institute Workshop on Key UQ Methodologies and Motivating Applications. [Workshop] [Video]

  • February 21, 2018: F-X Briol to speak @ Isaac Newton Institute, Cambridge, UK.

  • February 28, 2018: F-X Briol to speak @ Machine Learning Group, University of Sheffield, UK.

  • March 2018: C Oates to speak @ Bayes Comp, Barcelona, Spain. [Conference]

  • 8 May 2018: T Sullivan, C Oates, O Chkrebtii, H Lie and J Cockayne to speak @ SAMSI QMC Transition Workshop, NC, USA. [web]

stco.jpg

Proceedings of Prob Num 2018:

The participants of the Prob Num 2018 workshop are invited to submit research, that was either presented at the workshop or discussed in the collaborative sessions, to a special edition of the Springer journal Statistics and Computing. The special edition will be edited by Mark Girolami, Ilse Ipsen, Chris Oates, Art Owen and Tim Sullivan.

Deadline for submission: 3rd September 2018

Author style guide for the journal: [link] 

LaTeX template, with acknowledgement text included: [link]

Submissions should be emailed in pdf format: probnum2018@turing.ac.uk

For full instructions, please refer to the website for Prob Num 2018.

Crediting SAMSI:

“This material was based upon work partially supported by the National Science Foundation under Grant DMS-1127914 to the Statistical and Applied Mathematical Sciences Institute. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.” [details]