Awesome Conformal Prediction

Slack for Awesome Conformal Prediction Please reach out to [email protected] if you have issues with Slack link and need invite to the Slack.

The most comprehensive professionally curated resource on Conformal Prediction including the best tutorials, videos, books, papers, articles, courses, websites, conferences and open-source libraries code in Python, R and Julia.

I have created this resource after completing my PhD in Machine Learning specialising in Conformal Prediction (supervised by the creator of Conformal Prediction Prof. Vladimir Vovk).

The resources were meticulously collected since 2015 and after completing my PhD (thesis "Machine Learning for Probabilistic Prediction is available in the "Theses" section) I have decided to share them with the global machine learning community.

Having done research in Conformal Prediction since 2015, I am still amazed about how robust, powerful and flexible this best framework for Uncertainty Qualification is and how much it has to offer to solve most of the problems involving uncertainty. Conformal Prediction is no longer aniche area of research like it was just a few years ago, it has seen exponential growth during the last 2-3 years due to the work of amazing ambassadors of Conformal Prediction such as Prof. Larry Wasserman and Anastasious Angelopolous in academia.

I am actively promoting Conformal Prediction (because it is well Awesome) on social media incluidng LinkedIn and Twitter. ResearchGate has all my research and I occasionally publish in Medium sharing insights from the data science trenches in the industry.

Please feel free to connect and also help spread the word about Conformal Prediction.

Awesome Conformal Prediction has been cited in the best book on Machine Learning (aka the machine learning 'bible') "Probabilistic Machine Learning: Advanced Topics" by the leading research scientist at Google (over 80K Google Scholar citations) and Bestselling Machine Learning Book Author Kevin Murphy (starting at page 555).

Please star 🌟 the repo and spread the word. If you use the repository in a scientific publication, please cite Awesome Conformal Prediction to help promote this amazing framework in academia and industry:

Manokhin, Valery. (2022). Awesome Conformal Prediction (v1.0.0). Zenodo. https://zenodo.org/record/6467205 https://doi.org/10.5281/zenodo.6467205

Bibtex entry export https://zenodo.org/record/6467205/export/hx

@software{manokhin_valery_2022_6467205, author = {Manokhin, Valery}, title = {Awesome Conformal Prediction}, month = apr, year = 2022, note = {{"If you use Awesome Conformal Prediction, please cite it as below."}}, publisher = {Zenodo}, version = {v1.0.0}, doi = {10.5281/zenodo.6467205}, url = {https://doi.org/10.5281/zenodo.6467205} }

Slack for Awesome Conformal Prediction

Please reach out to [email protected] if you have issues with Slack link and need invite to the Slack.

Why Conformal Prediction?

One of the most influential and celebrated machine learning researchers - Professor Michael I. Jordan:

'𝗖𝗼𝗻𝗳𝗼𝗿𝗺𝗮𝗹 𝗣𝗿𝗲𝗱𝗶𝗰𝘁𝗶𝗼𝗻 𝗶𝗱𝗲𝗮𝘀 𝗮𝗿𝗲 𝗧𝗛𝗘 𝗮𝗻𝘀𝘄𝗲𝗿 𝘁𝗼 𝗨𝗤 (𝘂𝗻𝗰𝗲𝗿𝘁𝗮𝗶𝗻𝘁𝘆 𝗾𝘂𝗮𝗻𝘁𝗶𝗳𝗶𝗰𝗮𝘁𝗶𝗼𝗻), 𝗜 𝘁𝗵𝗶𝗻𝗸 𝗶𝘁'𝘀 𝘁𝗵𝗲 𝗯𝗲𝘀𝘁 𝗜 𝗵𝗮𝘃𝗲 𝘀𝗲𝗲𝗻 - 𝗶𝘁𝘀 𝘀𝗶𝗺𝗽𝗹𝗲, 𝗴𝗲𝗻𝗲𝗿𝗮𝗹𝗶𝘀𝗮𝗯𝗹𝗲 𝗲𝘁𝗰.' (ICML 2021 UQ workshop). 🔥🔥🔥🔥🔥

One the most influential statistics Professors - Larry Wasserman (Carnegie Mellon):

'𝗦𝗼 𝘁𝗵𝗲 𝗯𝗲𝗮𝘂𝘁𝘆 𝗼𝗳 𝘁𝗵𝗲 𝗰𝗼𝗻𝗳𝗼𝗿𝗺𝗮𝗹 𝘁𝗵𝗶𝗻𝗴 𝗶𝘀 𝗵𝗼𝘄 𝘀𝗶𝗺𝗽𝗹𝗲 𝗶𝘁 𝗶𝘀 𝘁𝗼 𝗱𝗼 𝗶𝘁 𝗮𝗻𝗱 𝗵𝗼𝘄 𝗴𝗲𝗻𝗲𝗿𝗮𝗹 𝗶𝘁 𝗶𝘀. 𝗦𝗼 𝗜 𝘁𝗵𝗶𝗻𝗸 𝘆𝗼𝘂 𝗸𝗻𝗼𝘄 𝗶𝗱𝗲𝗮𝘀 𝘁𝗵𝗮𝘁 𝗰𝗮𝘁𝗰𝗵 𝗼𝗻, 𝗴𝗲𝗻𝗲𝗿𝗮𝗹 𝗶𝗱𝗲𝗮𝘀 𝘁𝗵𝗮𝘁 𝗮𝗿𝗲 𝗽𝗿𝗲𝘁𝘁𝘆 𝗴𝗲𝗻𝗲𝗿𝗮𝗹 𝗮𝗻𝗱 𝐞𝐚𝐬𝐲 𝐭𝐨 𝐢𝐦𝐩𝐥𝐞𝐦𝐞𝐧𝐭 𝐭𝐡𝐚𝐭 𝐲𝐨𝐮 𝐜𝐚𝐧 𝐩𝐢𝐜𝐭𝐮𝐫𝐞 𝐲𝐨𝐮𝐫𝐬𝐞𝐥𝐟 𝐮𝐬𝐢𝐧𝐠 𝐢𝐧 𝐫𝐞𝐚𝐥 𝐚𝐩𝐩𝐥𝐢𝐜𝐚𝐭𝐢𝐨𝐧𝐬 𝐚𝐫𝐞 𝐭𝐡𝐞 𝐫𝐞𝐚𝐬𝐨𝐧 𝐭𝐡𝐚𝐭 𝐩𝐞𝐨𝐩𝐥𝐞 𝐮𝐬𝐢𝐧𝐠 𝐜𝐨𝐧𝐟𝐨𝐫𝐦𝐚𝐥 𝐩𝐫𝐞𝐝𝐢𝐜𝐭𝐢𝐨𝐧.' 🚀🚀🚀🚀🚀

'Conformal inference methods are becoming all the rage in academia and industry alike. In a nutshell, these methods deliver exact prediction intervals for future observations without making any distributional assumption whatsoever other than having iid, and more generally, exchangeable data.'

Prof. Emmanual Candes (Stanfor) - Neurips 2022 key talk.

https://slideslive.com/icml-2021/workshop-on-distributionfree-uncertainty-quantification

When prominent professors from the best research labs in the world say this about conformal prediction it is quite an endorsement.

What about the industry one might ask - well Conformal Prediction already for several years powers the main anomaly detection proposition in Microsoft Azure and Data Robot (AutoML) uses Conformal Prediction to generate robust prediction intervals for its models

In 2021 conformal prediction research experienced exponential growth in academia and with the availability of open-source libraries the industry is positioned to replicate this growth in the industry.

📢📢Industry take notice. The revolution in Uncertainty Quantification / Probabilistic Prediction / Forecasting is already here 📢📢 A big one 🔥🔥🔥🔥🔥

🌟 🌟 🌟 🌟 🌟

Featured resources:

A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification Colab 🔥🔥🔥🔥🔥 TIME SERIES 🚀🚀🚀🚀🚀

This is newest version of the super-popular tutorial on Conformal Prediction (over 10K+ stars on YouTube) now significantly expanded (2x), including advanced techniques such as covariate shift conformal, as well as a super fun history and literature review in Section 7.

A Tutorial on Conformal Prediction by Anastasios Angelopoulos and Stephen Bates (2021) 🔥🔥🔥🔥🔥

A Tutorial on Conformal Prediction Part 2: Conditional Coverage and Diagnostics by Anastasios Angelopoulos and Stephen Bates (2022)

⭐⭐️⭐️⭐️⭐️

Table of Contents

• Events

• Books

• PhD and MSc Theses

• Videos

• Papers

• Papers Time Series

• Articles

• Tutorials

• Courses

• Presentation_slides

• Researchers

• Websites

• Twitter

• TikTok

• Conferences

• Python

• R

• Julia

• Code Other Languages

• AI platforms

• Patents

• Miscellaneous

• Contributing

Events

11th Symposium on Conformal and Probabilistic Prediction with Applications

Books

1. Algorithmic Learning in a Random World by Vladimir Vovk and Alex Gammerman, also Glenn Shafer (2005). Second edition in progress. 🔥🔥🔥🔥🔥
2. Conformal Prediction for Reliable Machine Learning by Vineeth Balasubramanian, Shen-Shyang Ho, Vladimir Vovk (2014) 🔥🔥🔥🔥🔥
3. Conformal predictive distributions with kernels by Vladimir Vovk, Ilia Nouretdinov, Valery Manokhin, Alex Gammerman (Chapter in 'Braverman Readings in Machine Learning. Key Ideas from Inception to Current State', Springer, 2018). 🔥🔥🔥🔥🔥
4. Confidence, Likelihood, Probability by Tore Schweder and Nils Lid Hjort (University of Oslo, 2016). Not directly about Conformal Prediction but a great book about modern frequentist methods. 🔥🔥🔥🔥🔥

Theses

1. Machine Learning for Probabilistic Prediction, PhD Thesis, Valery Manokhin (Royal Holloway, UK, 2022)
2. Conformal and Venn Predictors for large, imbalanced and sparse chemoinformatics data, PhD Thesis, Paolo Toccaceli (Royal Holloway, UK, 2021)
3. Competitive online algorithms for probabilistic prediction, PhD Thesis, Raisa Dzhamtyrova (Royal Holloway, UK, 2020)
4. Conformal Prediction and Testing under On-line Compression Models, PhD Thesis, Valentina Fedorova (Royal Holloway, UK, 2014)
5. Adaptive Online Learning, PhD Thesis, Dmitry Adamskiy (Royal Holloway, UK, 2013)
6. On discovery and exploitation of temporal structure in data sets, PhD Thesis, Tim Scarfe, (Royal Holloway, UK, 2015)
7. Black-box Security Measuring Black-box Information Leakage via Machine Learning,PhD Thesis, Giovanni Cherubin (Royal Holloway, UK, 2019)
8. Small and Large Scale Probabilistic Classifiers with Guarantees of Validity, PhD Thesis, Ivan Petej (Royal Holloway, UK, 2019)
9. Confidence and Venn Machines and Their Applications to Proteomics by Devetyarov, Dmitry (Royal Holloway, UK, 2019)
10. Conformal Anomaly Detection - detecting abnormal trajectories in surveillance applications by Rikard Laxhammar (University of Skoeve, Sweden, 2014)
11. Inductive Confidence Machine for Pattern Recognition - is it the next step towards AI by David Surkov (Royal Holloway, UK, 2004)
12. Distribution Free Prediction Intervals for Multiple Functional Regression by Ryan Kelly (University of Pittsburgh, 2020).
13. Probabilistic Load Forecasting with Deep Conformalized Quantile Regression by Vilde Jensen (Artcic University of Norway, 2021)
14. Model-free methods for multiple testing and predictive inference, PhD Thesis, Zhimei Ren (Stanford, 2021)
15. Comparison of Support Vector Machines and Deep Learning For QSAR with Conformal Prediction by Deligianni Maria, MSc thesis, Universit of Uppsala (2022)
16. Predictive Maintenance with Conformal and Probabilistic Prediction: A Commercial Case Study by James Gammerman (2022)
17. Risk-Sensitive Decision-Making for Autonomous-Driving by Hardy Hasan (University of Uppsala, 2022)
18. Distribution-Free Finite-Sample Guarantees and Split Conformal Prediction, MSc thesis by Roel Hulsman, University of Oxford (2022)

Tutorials

1. A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification by Anastasios N. Angelopoulos and Stephen Bates (2021) Video Code 🔥🔥🔥🔥🔥
2. A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification by Anastasios N. Angelopoulos and Stephen Bates (2021)
3. A Tutorial on Conformal Predictive Distributions by Paolo Toccaceli (2020) 🔥🔥🔥🔥🔥
4. Conformal Predication Tutorial by Henrik Linusson (2021) 🔥🔥🔥🔥🔥
5. Henrik Linusson: Conformal Prediction by Henrik Linusson (2020)
6. Predicting with Confidence - Henrik Boström by Henrik Boström (2016)
7. Venn Predictors Tutorial by Ulf Johansson, Cecilia Sönströd, Tuve Löfström, and Henrik Boström (2021)
8. Ulf Johansson: Venn Predictors by Ulf Johansson (2020)
9. A Tutorial on Conformal Prediction by Glenn Shafer and Vladimir Vovk (2008) 🔥🔥🔥🔥🔥
10. Conformal Prediction: a Unified Review of Theory and New Challenges by Gianluca Zeni, Matteo Fontana1 and Simone Vantini (Politecnico di Milano, Italy, 2021) 🔥🔥🔥🔥🔥
11. Conformal Prediction in Spark by Marco Capuccini (Uppsala University, 2017)
12. Tutorial on Venn-ABERS prediction by Paolo Toccaceli (Royal Holloway, UK, 2019) 🔥🔥🔥🔥🔥
13. An Introduction to Conformal Prediction by Henrik Linusson (2017) 🔥🔥🔥🔥🔥
14. Introduction to Conformal Prediction by Vineeth N Balasubramanian (Indian Institue of Technology, Hyderabad, 2015)
15. Conformal prediction A Tiny Tutorial on Predicting with Confidence by Henrik Linusson and Ulf Johansson (2014)
16. Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification by Anastasios Angelopoulos (Berkeley, 2022)
17. A Tutorial on Conformal Prediction Part 2: Conditional Coverage and Diagnostics by Anastasios Angelopoulos and Stephen Bates (2022) 🔥🔥🔥🔥🔥
18. Beyond Conformal Prediction: Tutorial on Conformal Part 3 by Anastasios Angelopoulos and Stephen Bates (2022) 🔥🔥🔥🔥🔥
19. Getting predictions intervals with conformal inference by Rajiv Shah (2022) YouTube Code 🔥🔥🔥🔥🔥

Courses

1. Uncertain: Modern topics in uncertainty estimation YouTube Course notes by Aaron Roth (University of Pennsylvania, 2022)
2. Topics in Modern Statistical Learning

Videos

1. Treatment of Uncertainty in the Foundations of Probability by Vladimir Vovk (Royal Holloway, UK, 2017)
2. Large-Scale Probabilistic Prediction With and Without Validity Guarantees by Vladimir Vovk (Royal Holloway, UK, NeurIPS 2015) 🔥🔥🔥🔥🔥
3. Conformal testing in a binary model situation by Vladimir Vovk (Royal Holloway, UK, 2021)
4. Protected probabilistic classification by Vladimir Vovk (Royal Holloway, UK, 2021)
5. Retrain or not retrain: conformal test martingales for change-point detection by Vladimir Vovk (Royal Holloway, UK, 2021) 🔥🔥🔥🔥🔥
6. A Tutorial on Conformal Prediction by Anastasios Angelopoulos and Stephen Bates (Berkeley, ICML 2021) 🔥🔥🔥🔥🔥
7. Steps Toward Trustworthy Machine Learning by Tom Dietterich (2021)
8. A Tutorial on Conformal Predictive Distributions by Paolo Toccaceli (Royal Holloway, UK, 2020) 🔥🔥🔥🔥🔥
9. Conformal Prediction Tutorial by Henrik Linusson (2021) 🔥🔥🔥🔥🔥
10. Henrik Linusson: Conformal Prediction by Henrik Linusson (2020)
11. Predicting with Confidence - Henrik Boström by Henrik Boström (2016)
12. How to increase certainty in predictive modeling by Emmanuel Candes (Stanford, 2021) 🔥🔥🔥🔥🔥
13. Recent Progress in Predictive Inference by Emmanuel Candes (Stanford, 2020) 🔥🔥🔥🔥🔥
14. Some recent progress in predictive inference" (Stanford) @ MAD+ by Emmanuel Candes (Stanford, 2020)
15. Conformal Prediction in 2020 by Emmanuel Candes (Stanford, 2020) 🔥🔥🔥🔥🔥
16. Assumption-free prediction intervals for black-box regression algorithms by Aaditya Ramdas (Carnegie Mellon, 2020) 🔥🔥🔥🔥🔥
17. Maria Navarro: Quantifying uncertainty in Machine Learning predictions | PyData London 2019 by Maria Navarro (2019)
18. Conformal Prediction: Enhanced Method for Understanding the Prediction Quality by Artem Ryasik and Greg Landrum
19. Venn Predictors Tutorial by Ulf Johansson, Cecilia Sönströd, Tuwe Löfström, and Henrik Boström (2021)
20. Mondrian conformal predictive distributions by Henrik Boström, Ulf Johansson, and Tuwe Löfström (2021) 🔥🔥🔥🔥🔥
21. Calibrating Multi-Class Models by Ulf Johansson, Tuwe Löfström, and Henrik Boström (2021)
22. Conformal testing in a binary model situation by Vladimir Vovk (Royal Holloway, UK, 2021)
23. Conformal prediction in Orange by Tomaž Hočevar and Blaž Zupan (2021)
24. Distribution-Free, Risk-Controlling Prediction Sets by Anastasios Angelopoulos (Stanford, 2021) 🔥🔥🔥🔥🔥
25. Conformal Prediction and Distribution-Free Calibration by Aaditya Ramdas (Carnegie Mellon, 2021) 🔥🔥🔥🔥🔥
26. Reliable Diagnostics by Conformal Predictors by Alexander Gammerman (Royal Holloway, UK, 2015)
27. Distribution-Free, Risk-Controlling Prediction Sets by Anastasios Angelopoulos (Stanford, 2021) 🔥🔥🔥🔥🔥
28. Conformal Inference of Counterfactuals and Time-to-event Outcomes by Lihua Lei (Stanford, 2021)
29. Algo Hour – Conformal Inference of Counterfactuals and Individual Treatment Effect by Lihua Lei (Stanford, 2021)
30. Conformal Inference of Counterfactuals and Individual Treatment effects(Stanford) by Lihua Lei (Stanford, 2021)
31. Approximation to object conditional validity with inductive conformal predictors by Anthony Bellotti (University of Nottingham Ningbo, China, 2021)
32. Ulf Johansson: Venn Predictors by Ulf Johansson (Jönköping University, Sweden, 2021) 🔥🔥🔥🔥🔥
33. Transformer-based conformal predictors for paraphrase detection by Patrizio Giavannotti and Prof. Alexander Gammerman (Royal Holloway, UK, 2021)
34. Conformal Inference of Counterfactuals and Individual Treatment Effects by Lihua Lei (Stanford, 2020)
35. Model-Free Predictive Inference by Larry Wasserman (Carnegie Mellon, 2020) 🔥🔥🔥🔥🔥
36. Shapley-value based inductive conformal prediction by William Lopez Jaramillo (2021)
37. Conformal Training: Learning Optimal Conformal Classifiers | DeepMind by David Stutz (2021) 🔥🔥🔥🔥🔥
38. Distribution-Free, Risk-Controlling Prediction Sets by Anastasios Angelopoulos (2021) 🔥🔥🔥🔥🔥
39. Assumption-Free, High-Dimensional Inference by Larry Wasserman (2016)
40. Neural Predictive Monitoring under Partial Observability by Francesca Cairolli (2021)
41. Conformalized Kernel Ridge Regression and Its Efficiency by Evgeny Burnaev (Skolkovo, Russia, 2015)
42. Fast conformal classification using influence functions by Giovanni Cherubin (Alan Turing Institute, UK, 2021)
43. Valid inferential models and conformal prediction by Ryan Martin (North Carolina State University, USA, 2021)
44. Mondrian conformal predictive distributions by Henrik Boström, Ulf Johansson and Tuwe Löfström (KTH Royal Institute of Technology, Sweden, 2021) 🔥🔥🔥🔥🔥
45. Evaluation of updating strategies for conformal predictive systems in the presence of extreme events by Hugo Werner, Lars Carlsson, Ernst Ahlberg and and Henrik Boström (KTH Royal Institute of Technology, Sweden, 2021)
46. Exact and Robust Conformal Inference Methods for Predictive Machine Learning With Dependent Data by Victor Chernozhukov (MIT, USA, 2019) 🔥🔥🔥🔥🔥
47. Ulf Johansson: Venn Predictors by Ulf Johansson (Jönköping University, Sweden, 2020) 🔥🔥🔥🔥🔥
48. Class-wise confidence for debt prediction in real estate management by Soundouss Messoudi (2021)
49. How Nonconformity Functions and Difficulty of Datasets Impact the Efficiency of Conformal Classifiers by Marharyta Aleksandrova (2021)
50. Nested conformal prediction and quantile out-of-bag ensemble methods by Chirag Gupta (Carnegie Mellon, 2020) 🔥🔥🔥🔥🔥
51. Panel with Michael I. Jordan, Vladimir Vovk, and Larry Wasserman, moderated by Aaditya Ramdas by Vladimir Vovk, Larry Wasserman, Michael I. Jordan, Aaditya Ramdas, ICML 2021 🔥🔥🔥🔥🔥 🔥🔥🔥🔥🔥
52. Black-box uncertainty - Anastasios Angelopoulos by Anastasios Angelopoulos (Berkeley, USA, 2021) 🔥🔥🔥🔥🔥
53. P.C. Mahalanobis Memorial Lectures 2020-21 by Vladimir Vovk (Royal Holloway, UK, 2021)
54. Rahul Vishwakarma: New Perspective on Machine Learning Predictions Under Uncertainty | SNIA Storage Developer Conference, Santa Clara 2019 by Rahul Vishwakarma (2019)
55. Fast conformal classification using influence functions by Umang Bhatt, Adrian Weller and Giovanni Cherubin (Cambridge / Alan Turinig Institute, 2021).
56. Adaptive Conformal Predictions for Time Series | ISDFS by Margaux Zaffran (2022) TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
57. Recent progress in predictive inference by Emmanuel Candes, Stanford University (2022)
58. Conformalized Survival Analysis with Adaptive Cutoffs by Rina Foygel Barber, Zhimei Ren, Yu Gui and Rohan Hore, University of Chicago (2022)
59. Calibrating probabilistic hierarchical forecasts with conformal predictions by Daan Ferdinandusse (University of Amsterdam, 2022)
60. Michael I. Jordan on Conformal Prediction by Michael I. Jordan (Berkeley, 2022)
61. Distribution-free Prediction: Exchangeability and Beyond by Rina Foygel Barber (University of Chicago, 2022)
62. Purdue Statistics Theme Seminar, Conformal Prediction in 2022 by Emmanuel Candes (Stanford, 2022)

Papers

1. Introducing Conformal Prediction in Predictive Modeling. A Transparent and Flexible Alternative to Applicability Domain Determination by Ulf Norinder, Lars Carlsson, Scott Boyer, and Martin Eklund (2014)
2. Uncertainty Sets for Image Classifiers using Conformal Prediction by Anastasios N. Angelopoulos, Stephen Bates, Jitendra Malik, & Michael I. Jordan (Berkeley, 2021) 🔥🔥🔥🔥🔥
3. Conformal Prediction Under Covariate Shift by Ryan Tibshirani, Rina Foygel Barber, Emmanuel Candes, Aaditya Ramdas (Carnegie Mellon, Stanford, Chicago, 2019) 🔥🔥🔥🔥🔥
4. Regression Conformal Prediction with Nearest Neighbours by Harris Papadopoulos, Vladimir Vovk and Alex Gammerman (Royal Holloway, UK, 2014)
5. Nested conformal prediction and quantile out-of-bag ensemble methods by Chirag Gupta, Arun Kuchibhotla and Aaditya Ramdas (Carnegie Mellon, 2021) 🔥🔥🔥🔥🔥
6. Cross-conformal predictive distributions by Vladimir Vovk, Ilia Nouretdinov, Valery Manokhin and Alexander Gammerman (Royal Holloway, UK, 2018) 🔥🔥🔥🔥🔥 🔥🔥🔥🔥🔥
7. Criteria of Efficiency for Conformal Prediction by Vladimir Vovk, Ilia Nouretdinov, Valentina Fedorova, Ivan Petej, and Alex Gammerman ((Royal Holloway, UK, 2016)
8. Conformal Prediction for Simulation Models by Benjamin LeRoy and Chad Schafer (Carnegie Mellon, 2021)
9. Distribution-free, risk-controlling prediction sets Stephen Bates, Anastasios Angelopoulos, Lihua Lei, Jitendra Malik and Michael I Jordan (Berkeley, 2021) 🔥🔥🔥🔥🔥
10. Conditional calibration for false discovery rate control under dependence by William Fithian and Lihua Lei (Stanford, 2021)
11. Conformal Prediction: a Unified Review of Theory and New Challenges by Gianluca Zeni, Matteo Fontana and S. Vantini (2021) 🔥🔥🔥🔥🔥
12. Regression conformal prediction with random forests by Ulf Johansson, Henrik Boström, Tuve Löfström and Henrik Linusson (2014)
13. A conformal prediction approach to explore functional data by Jing Lei, Alessandro Rinaldo, Larry Wasserman (Carnegie Mellon, 2013)
14. An electronic nose-based assistive diagnostic prototype for lung cancer detection with conformal prediction by Xianghao Zhana,c, Zhan Wanga, Meng Yangb, Zhiyuan Luod, You Wanga, Guang Li (2020)
15. Predicting the Rate of Skin Penetration Using an Aggregated Conformal Prediction Framework by Martin Lindh, A. Karlén, Ulf Norinder (2017)
16. The application of conformal prediction to the drug discovery process by Martin Eklund, Ulf Norinder, Scott Boyer & Lars Carlsson (2014) 🔥🔥🔥🔥🔥
17. Distributional conformal prediction by Victor Chernozhukov, Kaspar Wüthrich, Yinchu Zhu (2021) 🔥🔥🔥🔥🔥
18. Anomaly Detection of Trajectories with Kernel Density Estimation by Conformal Prediction by James Smith, Ilia Nouretdinov, Rachel Craddock, Charles Offer, and Alexander Gammerman (2009)
19. Conformal prediction interval estimation and applications to day-ahead and intraday power markets by Christopher Kath, Florian Ziel (2019) 🔥🔥🔥🔥🔥
20. The application of conformal prediction to the drug discovery process by Martin Eklund, Ulf Norinder, Scott Boyer & Lars Carlsson (2013)
21. Anomaly Detection of Trajectories with Kernel Density Estimation by Conformal Prediction by James Smith, Ilia Nouretdinov, Rachel Craddock, Charles Offer, Alexander Gammerman (Royal Holloway, UK, 2014)
22. Conformal Prediction: a Unified Review of Theory and New Challenges by Gianluca Zeni, Matteo Fontana1 and Simone Vantini (Politecnico di Milano, Italy, 2021)
23. Exchangeability, Conformal Prediction, and Rank Tests by Arun Kuchibhotla (Carnegie Mellon, 2021)
24. Conformal prediction with localization by Leying Guan (Yale, 2020)
25. Predicting skin sensitizers with confidence - Using conformal prediction to determine applicability domain of GARD by Andy Forreryd, Ulf Norinder, Tim Lindberg, Malin Lindstedt (2018)
26. Binary classification of imbalanced datasets using conformal prediction by Ulf Norinder, Scott Boyer (2017)
27. Discretized conformal prediction for efficient distribution-free inference by Wenyu Chen, Kelli-Jean Chun, and Rina Foygel Barber (2017)
28. Validity, consonant plausibility measures, and conformal prediction by Leonardo Cella. and Ryan Martin (2021)
29. Conformal Prediction Classification of a Large Data Set of Environmental Chemicals from ToxCast and Tox21 Estrogen Receptor Assays by Ulf Norinder, Scott Boyer (2016)
30. Conformal prediction to define applicability domain – A case study on predicting ER and AR binding by U. Norinder, A. Rybacka, P.Andersson (2016)
31. Conformal prediction of biological activity of chemical compounds by Paolo Toccaceli, Ilia Nouretdinov, Alex Gammerman (Royal Holloway, UK, 2017) 🔥🔥🔥🔥🔥
32. Introducing conformal prediction in predictive modeling for regulatory purposes. A transparent and flexible alternative to applicability domain determination by Ulf Norinder, Lars Carlsson, Scott Boyer, Martin Eklund (2015)
33. Aggregated Conformal Prediction by Lars CarlssonMartin EklundUlf Norinder (2014)
34. Interpretation of Conformal Prediction Classification Models by Ernst Ahlberg, Ola Spjuth, Catrin Hasselgren, Lars Carlsson (2015)
35. Cross-Conformal Prediction with Ridge Regression by Harris Papadopoulos (2015)
36. Sparse conformal prediction for dissimilarity data by Frank-Michael Schleif, Xibin Zhu and Barbara Hammer (2015)
37. Effective utilization of data in inductive conformal prediction using ensembles of neural networks by Tuve Löfström, Ulf Johansson and Henrik Boström (2013)
38. Beyond the Basic Conformal Prediction Framework by Vladimir Vovk (2014)
39. An electronic nose-based assistive diagnostic prototype for lung cancer detection with conformal prediction by Xianghao Zhan, Zhan Wang, Meng Yang, Zhiyuan Luo, You Wang, Guang Li (Stanford, Royal Holloway, China University of Mining and Technology, 2020)
40. Predicting with confidence: Using conformal prediction in drug discovery by Jonathan Alvarsson, Staffan Arvidsson McShane, Ulf Norinder, Ola Spjuth (2021)
41. Inductive conformal prediction for silent speech recognition by Ming Zhang, You Wang, Zhang Wei, Meng Yang, Zhiyuan Luo, Guang Li (2020)
42. Large scale comparison of QSAR and conformal prediction methods and their applications in drug discovery by Nicolas Bosc, Francis Atkinson, Eloy Felix, Anna Gaulton, Anne Hersey and Andrew R. Leach (2019)
43. Deep Conformal Prediction for Robust Models by Soundouss Messoudi, Sylvain Rousseau and Sébastien Destercke (2020)
44. Strong validity, consonance, and conformal prediction by Leonardo Cella and Ryan Martin (2020)
45. Skin Doctor CP: Conformal Prediction of the Skin Sensitization Potential of Small Organic Molecules by Anke Wilm, U. Norinder, M. Agea, Christina de Bruyn Kops, Conrad Stork, J. Kühnl, J. Kirchmair (2020)
46. Conformal prediction based active learning by linear regression optimization by Sergio Matiz, Kenneth E.Barner (2020)
47. Conformal prediction intervals for the individual treatment effect by Danijel Kivaranovic, Robin Ristl, Martin Poschb, Hannes Leeb (2020)
48. Nearest neighbor based conformal prediction by László Györfi and Harro Walk (2020)
49. Concepts and Applications of Conformal Prediction in Computational Drug Discovery by Isidro Cortés-Ciriano and Andreas Bender (2019) 🔥🔥🔥🔥🔥
50. Predicting Ames Mutagenicity Using Conformal Prediction in the Ames/QSAR International Challenge Project by Ulf Norinder, Ernst Ahlberg, Lars Carlsson (2018)
51. Nested Conformal Prediction and the Generalized Jackknife by Arun Kuchibhotla and Aaditya Ramdas (Carnegie Mellon, 2019)
52. Predictive inference with the jackknife+ by Rina Foygel Barber, Emmanuel Candès, Aaditya Ramdas, and Ryan Tibshirani (2020) 🔥🔥🔥🔥🔥
53. Nonparametric predictive distributions based on conformal prediction by Vladimir Vovk, Jieli Shen, Valery Manokhin and Min-ge Xie (Royal Holloway, UK, Rutgers, USA, 2018) 🔥🔥🔥🔥🔥
54. A Distribution-Free Test of Covariate Shift Using Conformal Prediction by Xiaoyu Hu and Jing Lei (Peking Univerity, China and Carnegie Mellon, USA, 2020) 🔥🔥🔥🔥🔥
55. Exchangeability, Conformal Prediction, and Rank Tests by Arun Kuchibhotla (Carnegie Mellon, 2021)
56. Conformal prediction with localization by Leying Guan (2020)
57. Multitask Modeling with Confidence Using Matrix Factorization and Conformal Prediction by Ulf Norinder, Fredrik Svensson
58. Conformal prediction of HDAC inhibitors by U. Norinder, J.J.Navaka, E. Lopez-Lopez, D. Mucs & J.L. Medina-Franco (2019)
59. Computing Full Conformal Prediction Set with Approximate Homotopy by Eugene Ndiaye, Ichiro Takeuchi (2019)
60. Conformal Prediction Based on Raman Spectra for the Classification of Chinese Liquors by Jiao Gu, Huaibo Liu, Chaoqun Ma, Lei Li, Chun Zhu, Christ Glorieux, Guoqing Chen (2019)
61. Efficient and minimal length parametric conformal prediction regions by Daniel Eck and Forrest Crawford (2019)
62. Conformal Prediction for Students' Grades in a Course Recommender System by Raphael Morsomme and Evgueni Smirnov (2019)
63. Efficient iterative virtual screening with Apache Spark and conformal prediction by Laeeq Ahmed, Valentin Georgiev, Marco Capuccini, Salman Toor, Wesley Schaal, Erwin Laure and Ola Spjuth (2018)
64. Predicting Off-Target Binding Profiles With Confidence Using Conformal Prediction by Samuel Lampa, Jonathan Alvarsson, Staffan Arvidsson Mc Shane, Arvid Berg, Ernst Ahlberg, Ola Spjuth (2018)
65. Maximizing gain in high-throughput screening using conformal prediction by Fredrik Svensson, Avid M. Afzal1, Ulf Norinder and Andreas Bender (2018)
66. Conformalized Survival Analysis by Emmanuel Candès, Lihua Lei and Zhimei Ren (2021) R-Code 🔥🔥🔥🔥🔥
67. Random Forest Prediction Intervals by Haozhe Zhang†, Joshua Zimmerman†, Dan Nettleton† and Daniel J. Nordman† (Iowa State University, USA, 2019)
68. Conformal Training: Learning Optimal Conformal Classifiers | DeepMind by David Stutz (DeepMind), Krishnamurthy Dvijotham, Ali Taylan Cemgil and Arnaud Doucet (2021)
69. Comparing the Bayes and typicalness frameworks by Thomas Melluish, Craig Saunders, Ilia Nouretdinov, and Volodya Vovk (Royal Holloway, UK, 2001). 🔥🔥🔥🔥🔥
70. Large-scale probabilistic predictors with and without guarantees of validity by Vladimir Vovk, Ivan Petej, and Valentina Fedorova (Royal Holloway, Yandex, NeurIPS) 🔥🔥🔥🔥🔥
71. Inductive conformal prediction for silent speech recognition by Ming Zhang, You Wang, Wei Zhang, Meng Yang, Zhiyuan Luo and Guang Li (2020)
72. Conformal Prediction using Conditional Histograms by Matteo Sesia and Yaniv Romano (NeurIPS 2021 paper). 🔥🔥🔥🔥🔥
73. Valid prediction intervals for regression problems by Nicolas Dewolf, Bernard De Baets, Willem Waegeman (2021) 🔥🔥🔥🔥🔥
74. Application of conformal prediction interval estimations to market makers’ net positions by Wojciech Wisniewski, David Lindsay, Sian Lindsay (Royal Holloway, UK, 2020)
75. Locally Valid and Discriminative Prediction Intervals for Deep Learning Models by Zhen Lin, Shubhendu Trivedi, Jimeng Sun (NeurIPS, 2021) 🔥🔥🔥🔥🔥
76. Distribution-Free Federated Learning with Conformal Predictions by Charles Lu and Jayashree Kalpathy-Cramer (2022)
77. Coreset-based Conformal Prediction for Large-scale Learning by Nery Riquelme-Granada, Khuong Nguyen, Zhiyuan Luo (Royal Holloway, UK, 2019)
78. Fast probabilistic prediction for kernel SVM via enclosing balls by Nery Riquelme-Granada, Khuong Nguyen, Zhiyuan Luo (Royal Holloway, UK, 2020)
79. Conformalized density- and distance-based anomaly detection in time-series data by Evgeny Burnaev, Vladislav Ishimtsev (2016)
80. Predictive Inference with Weak Supervision by Maxime Cauchois, Suyash Gupta, Alnur Ali and John Duchi (Stanford, 2022)
81. Conformal Prediction in Clinical Medical Sciences by Janette Vazquez and Julio C. Facelli University of Utah, 2022)
82. Provably Improving Expert Predictions with Conformal Prediction by Eleni Straitouri, Lequng Wang, Nastaran Okati and Manuel Gomez Rodriguez (Max Planck Institute for Software Systems / Cornell University, 2021).
83. Conformal predictive distributions with kernels by Vladimir Vovk, Ilia Nouretdinov, Valery Manokhin, Alex Gammerman (Royal Holloway, UK, 2017)
84. Multi-class probabilistic classification using inductive and cross Venn–Abers predictors by Valery Manokhin (Royal Holloway, UK, 2017). 🔥🔥🔥🔥🔥
85. Computationally efficient versions of conformal predictive distributions by Vladimir Vovk, Ivan Petej, Ilia Nouretdinov, Valery Manokhin, Alex Gammerman (Royal Holloway, UK, 2019).
86. Cover your cough: detection of respiratory events with confidence using a smartwatch by Khuong An Nguyen, Zhiyuan Luo (Royal Holloway, 2019).
87. Predicting Amazon customer reviews with deep confidence using deep learning and conformal prediction by Ulf Norinder and Petra Norinder (2022)
88. Conformal Prediction for the Design Problem by Clara Fannjianga, Stephen Batesa, Anastasios Angelopoulosa, Jennifer Listgartena and Michael I. Jordan (Berkeley, 2022) 🔥🔥🔥🔥🔥
89. Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging by Anastasios N. Angelopoulos, Amit Kohli, Stephen Bates, Michael I. Jordan, Jitendra Malik, Thayer Alshaabi, Srigokul Upadhyayula, Yaniv Romano (Berkeley and Technion, 2022) 🔥🔥🔥🔥🔥
90. Conformal predictive decision making by Vladimir Vovk and Claus Bendtsen (2018).
91. The Lifecycle of a Statistical Model: Model Failure Detection, Identification, and Refitting by Alnur Ali1, Maxime Cauchois and John C. Duchi (Stanford, 2022)
92. E-values: Calibration, combination, and applications by Vladimir Vovk (Royal Holloway) and Ruodu Wang (University of Waterloo) (2019)
93. Conformal Prediction Sets with Limited False Positives by Adam Fisch, Tal Schuster, Tommi Jaakkola and Regina Barzilay (MIT / Google Research, 2022)
94. Adaptive Conformal Predictions for Time Series by Margaux Zaffran, Aymeric Dieuleveut, Olivier Fe ́ron, Yannig Goude, and Julie Josse (EDF / INRIA / CMAP, France, 2022) 🔥🔥🔥🔥🔥
95. Ensemble Conformalized Quantile Regression for Probabilistic Time Series Forecasting by Vilde Jensen, Filippo Maria Bianchi, Stian Norman Anfinsen (Arctic University of Norway, 2022) TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥 Python Code
96. Prediction of Metabolic Transformations using Cross Venn-ABERS Predictors by Staffan Arvidsson, Ola Spjuth, Lars Carlsson and Paolo Toccaceli (University of Uppsala, Astra Zeneca, Royal Holloway, 2017)
97. Probabilistic Prediction in scikit-learn by Sweidan, Dirar and Ulf Johansson. 🔥🔥🔥🔥🔥
98. Conformalized Online Learning: Online Calibration Without a Holdout Set by Shai Feldman, Stephen Bates and Yaniv Romano (2022). TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
99. Valid model-free spatial prediction by Huiying Mao, Ryan Martin and Brian J Reich (2020)
100. Conformal Prediction with Temporal Quantile Adjustments by Zhen Lin, Shubhendu Trivedi, Jimeng Sun (2022) TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
101. Calibration of Natural Language Understanding Models with Venn–ABERS Predictors by Patrizio Giovannotti (Royal Holloway, UK, 2022) NLP
102. Conformal prediction interval for dynamic time-series by Chen Xu, Yao Xie (Georgia Tech, 2021) TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
103. Conformal prediction set for time-series by Chen Xu, Yao Xie (Georgia Tech, 2022) TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
104. Adaptive Conformal Predictions for Time Series by Margaux Zaffran, Aymeric Dieuleveut, Olivier Fe ́ron, Yannig Goude, and Julie Josse (2022) TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
105. Conformal Time-Series Forecasting by Kamile Stankeviciu te and Ahmed M. Alaa (2021) TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
106. Efficient Conformal Prediction via cascaded inference with expanded admission by Adam Fisch, Tal Schuster, Tommi Jaakkola, Regina Barzilay (MIT 2021]. Python Code
107. Split Localized Conformal Prediction by Xing Han, Ziyang Tang, Joydeep Ghosh, Qiang Liu (University of Texas, 2022). Python Code
108. Three Applications of Conformal Prediction for Rating Breast Density in Mammography by Charles Lu, Ken Chang, Praveer Singh, Jayashree Kalpathy-Crame (2022)
109. Conformal prediction set for time-series by Chen Xu, Yao Xie (Georgia Tech, 2022) Python Code TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
110. Recommendation systems with distribution-free reliability guarantees) by Anastasious Angelopolous, Karl Krauth, Stephen Bates, Yixin Wang and Michael I. Jordan (Berkeley 2022)
111. Model Agnostic Conformal Hyperparameter Optimization by Riccardo Doyle (Spotify, 2022)
112. Improving Trustworthiness of AI Disease Severity Rating in Medical Imaging with Ordinal Conformal Prediction Sets by Charles Lu, Anastasios N. Angelopoulos, Stuart Pomerantz (2022)
113. Conformal Off-Policy Prediction in Contextual Bandits by Muhammad Faaiz Taufiq, Jean-François Ton, Rob Cornish, Yee Whye Teh, Arnaud Doucet (Oxford, 2022) Video presentation
114. Training Uncertainty-Aware Classifiers with Conformalized Deep Learning by Bat-Sheva Einbinder, Yaniv Romano, Matteo Sesia, Yanfei Zhou (Technion, USCLA 2022) Video presentation; Code
115. Semantic uncertainty intervals for disentangled latent space by Swami Sankaranarayanan, Anastasios N. Angelopoulos, Stephen Bates, Yaniv Romano, and Phillip Isola (Unversity of Berkeley, Technion, 2022) 🔥🔥🔥🔥🔥
116. MAPIE: an open-source library for distribution-free uncertainty quantification by Vianney Taquet, Vincent Blot, Thomas Morzadec, Louis Lacombe, Nicolas Brunel (Quantmetry, France, 2022)
117. CODiT: Conformal Out-of-Distribution Detection in Time- Series Data by Ramneet Kaur et.al., Unibersity of Pensylvania (2022). Code TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
118. Confident Adaptive Language Modeling by Tal Schuster, Adam Fisch, Jai Gupta, Mostafa Dehghani, Dara Bahri, Vinh Q. Tran, Yi Tay, Donald Metzler (Google, MIT, 2022j
119. Probabilistic Conformal Prediction Using Conditional Random Samples by Zhendong Wang, Ruijiang Gao, Mingzhang Yin, Mingyuan Zhou, David M. Blei (Columbia University, 2020) Code
120. A general framework for multi-step ahead adaptive conformal heteroscedastic time series forecasting by Martim Sousa, Ana Maria Tome and Jose Moreira (University of Aveiro, 2022) TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
121. A novel Deep Learning approach for one-step Conformal Prediction approximation by Julia A. Meister, Khuong An Nguyen, Stelios Kapetanakis and Zhiyuan Luo (University of Brighton, UK, 2022) 🔥🔥🔥🔥🔥
122. Conformal Risk Control by Anastasious Angelopolous, Stephen Bates, Adam Fisch, Lihua Lei and Tal Schuster (Berkeley, Stanford, MIT and Google Research, 2022) 🔥🔥🔥🔥🔥
123. CD-split and HPD-split: Efficient Conformal Regions in High Dimensions by Rafael Izbicki, Gilson Shimizu, Rafael B. Stern (San Carlos University Brazil, 2022)
124. Flexible distribution-free conditional predictive bands using density estimators by Rafael Izbicki, Gilson Shimizu, and Rafael B. Stern (San Carlos University Brazil, 2020)
125. Split Conformal Prediction for Dependent Data by Roberto I. Oliveira, Paulo Orenstein, Thiago Ramos, João Vitor Romano (IMPA, Rio de Janeiro, Brazil, 2022)
126. Conformal Inference for Online Prediction with Arbitrary Distribution Shifts by Isaac Gibbs and Emmanual Candes (Stanford, 2022) 🔥🔥🔥🔥🔥
127. A General Framework For Multi-step Ahead Adaptive Conformal Heteroscedastic Time Series Forecasting by Martim Sousa, Ana Maria Tomé, University of Aveiro (2022) Code TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
128. Cough-based COVID-19 detection with audio quality clustering and confidence measure based learning by Alice E. Ashby, Julia A. Meister, Khuong An Nguyen, Zhiyuan Luo, Werner Gentzk (University of Brighton, 2022)
129. Assessing Explanation Quality by Venn Prediction by Amr Alkhatib, Henrik Bostroem and Ulf Johansson (2022)
130. Conformal prediction for hypersonic flight vehicle classification by Zepu Xi, Xuebin Zhuang, Hongbo Chen (Yat-sen University, Guangzhou, China, 2022) Slides
131. Robust Gas Demand Forecasting With Conformal Prediction by Mouhcine Mendil, Luca Mossina, Marc Nabhan, Kevin Pasini (2022)
132. Conformal Prediction Interval Estimations with an Application to Day-Ahead and Intraday Power Markets by Christopher Kath and Florian Ziel (2020)
133. Conformal Prediciton beyond exchangeability by Rina Foygel Barber, Emmanuel J. Candes, Aaditya Ramdas, Ryan J. Tibshirani (2022)
134. Robust Gas Demand Forecasting with Conformal Prediction by Mouhcine Mendil, Luca Mossina, Marc Nabhan, Kevin Pasini (2022)
135. Split conformal prediction for dependant data by Roberto I. Oliveira, Paulo Orenstein, Thiago Ramos and João Vitor Romano (2022)
136. Integrative conformal p-values for powerful out-of-distribution testing with labeled outliers by Ziyi Liang, Matteo Sesia, Wenguang Sun (UCLA, 2022)
137. Conformal Prediction Bands for Two-Dimensional Functional Time Series by Niccolo` Ajroldia, Jacopo Diquigiovannib, Matteo Fontanac, Simone Vantinia (2022)
138. Conformal prediction of small-molecule drug resistance in cancer cell lines by Saiveth Hernandez-Hernandez, Sachin Vishwakarma and Pedro Ballester (2022)
139. Ellipsoidal conformal inference for Multi-Target Regression by Soundouss Messoudi, Sebastien Destercke, Sylvain Rousseau (2022) Slides
140. Conformal Methods for Quantifying Uncertainty in Spatiotemporal Data: A Survey by Sophia Sun (UCLA, 2022)
141. [Deep Learning With Conformal Prediction for Hierarchical Analysis of Large-Scale Whole-Slide Tissue Images(https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9103229) by Håkan Wieslander , Philip J. Harrison, Gabriel Skogberg, Sonya Jackson, Markus Fridén, Johan Karlsson, Ola Spjuth, and Carolina Wählby (2021)
142. Audio–visual domain adaptation using conditional semi-supervised Generative Adversarial Networks by Christos Athanasiadis, Enrique Hortal, Stylianos Asteriadis (2022)
143. Conformal Prediction is Robust to Label Noise by Bat-Sheva Einbinder, Stephen Bates, Anastasios N. Angelopoulos, Asaf Gendler, Yaniv Romano (2022)
144. Copula Conformal Prediction for Multi-step Time Series Forecasting TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
145. Batch Multivalid Conformal Prediction by Christopher Jung, Georgy Noarov, Ramya Ramalingam, Aaron Roth (Stanford, university of Pensylvania, 2022)
146. Selection by Prediction with Conformal p-values by Ying Jin1 and Emmanuel J. Candes, (Stanford, 2022]
147. Few-Shot Calibration of Set Predictors via Meta-Learned Cross-Validation-Based Conformal Prediction by Sangwoo Park, Kfir M. Cohen, Osvaldo Simeone (2022)
148. Conformalized Fairness via Quantile Regression by Meichen Liu, Lei Ding, Dengdeng Yu, Wulong Liu, Linglong Kong, Bei Jiang (University of Alberta, Noah Arc Huawei, 2022)
149. Test-time recalibration of conformal predictors under distribution shift based on unlabeled examples by Fatih Furkan Yilmaz, and Reinhard Heckel (Rice University / University of Munuch, 2022)
150. Extending Conformal Prediction to Hidden Markov Models with Exact Validity via de Finetti's Theorem for Markov Chains by Buddhika Nettasinghe, Samrat Chatterjee, Ramakrishna Tipireddy, Mahantesh Halappanavar (2022)
151. Predictive inference with feature conformal prediction (2022) 🔥🔥🔥🔥🔥
152. Constructing Prediction Intervals with Neural Networks: An Empirical Evaluation of Bootstrapping and Conformal Inference Methods by Alex Contarino, Christine Schubert Kabban, Chancellor Johnstone and Fairul Mohd-Zaid (2022)
153. Spatio-Temporal Wildfire Prediction using Multi-Modal Data by Chen Xu1, Yao Xie, Daniel A. Zuniga Vazquez, Rui Yao, and Feng Qiu (2022)
154. Calibrating AI models for few-shot demodulation via conformal prediction Kfir M. Cohen1, Sangwoo Park, Osvaldo Simeone, Shlomo Shamai (2022)
155. Test-time recalibration of conformal predictors under distribution shift based on unlabeled examples Code
156. Nonparametric Quantile Regression: Non-Crossing Constraints and Conformal Prediction by Wenlu Tang, Guohao Shen, Yuanyuan Lin and Jian Huang (The Hong Kong Polytechnic University, 2022)
157. Safe Planning in Dynamic Environments using Conformal Prediction by Lars Lindemann, Matthew Cleaveland∗, Gihyun Shim, and George J. Pappas (University of Pensylvania, 2022)
158. Conformal prediction under feedback covariate shift for biomolecular design by Clara Fannjiang, Stephen Bates, Anastasios N. Angelopoulos,and Michael I. Jordan (2022) 🔥🔥🔥🔥🔥
159. Conformal Predictor for Improving Zero-shot Text Classification Efficiency by Prafulla Kumar Choubey, Yu Bai, Chien-Sheng Wu, Wenhao Liu, Nazneen Rajani (Saleforce AI Research and Hugging Face, 2022)
160. Bayesian Optimization with Conformal Coverage Guarantees by Samuel Stanton, Wesley Maddox and Andrew Gordon Wilson (Genentech, New York University, 2022) Code 🔥🔥🔥🔥🔥
161. Measuring the Confidence of Traffic Forecasting Models: Techniques, Experimental Comparison and Guidelines towards Their Actionability by Ibai Lanaa, Ignacio (In ̃aki) Olabarrietaa, Javier Del Sera (2022)
162. Training Uncertainty-Aware Classifiers with Conformalized Deep Learning by Bat-Sheva Einbinder, Yaniv Romano, Matteo Sesia and Yanfei Zhou (Technion/UCLA, NeurIPS 2022 paper) 🔥🔥🔥🔥🔥
163. Conformalized Fairness via Quantile Regression by Meichen Liu, Lei Ding, Dengdeng Yu, Wulong Liu, Linglong Kong, Bei Jiang (University of Alberta, Huawei Noah’s Ark Lab Canada, NeurIPS 2022 paper) Code 🔥🔥🔥🔥🔥
164. Engineering Uncertainty Representations to Monitor Distribution Shifts by Thomas Bonnier and Benjamin Bosch (Société Générale, 2022)
165. CONffusion: CONFIDENCE INTERVALS FOR DIFFUSION MODELS Project 🔥🔥🔥🔥🔥

Papers_Time_Series

1. Conformal prediction interval for dynamic time-series by Chen Xu, Yao Xie (Georgia Tech, 2021) TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥 Python Code Video Video ICML
2. Conformal prediction set for time-series by Chen Xu, Yao Xie (Georgia Tech, 2022) Python Code TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
3. Conformal Time-Series Forecasting by Kamile Stankeviciu te and Ahmed M. Alaa (2021) TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
4. Ensemble Conformalized Quantile Regression for Probabilistic Time Series Forecasting by Vilde Jensen, Filippo Maria Bianchi, Stian Norman Anfinsen (2022). TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
5. Adaptive Conformal Predictions for Time Series by Margaux Zaffran, Aymeric Dieuleveut, Olivier Fe ́ron, Yannig Goude, and Julie Josse (2022) Python Code TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥 Video
6. Conformalized Online Learning: Online Calibration Without a Holdout Set by Shai Feldman, Stephen Bates and Yaniv Romano (2022). TIME SERIES 🔥🔥🔥🔥🔥 Code
7. Conformal Prediction with Temporal Quantile Adjustments by Zhen Lin, Shubhendu Trivedi, Jimeng Sun (2022) TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
8. Exact and Robust Conformal Inference Methods for Predictive Machine Learning with Dependent Data by Victor Chernozhukov (MIT), Kaspar Wuethrich (University of California, San Diego) and Yinchu Zhu (University of Oregon) (2018)
9. Distributional Conformal Prediction by Chernozhukov (MIT), Kaspar Wuethrich (University of California, San Diego) and Yinchu Zhu (University of Oregon) (2022)
10. Distribution-Free Prediction Bands for Multivariate Functional Time Series: an Application to the Italian Gas Market by Jacopo Diquigiovanni (University of Padua) Matteo Fontana (Joint Research Centre - European Commission) Simone Vantini (Politecnico di Milano) (2021)
11. CODiT: Conformal Out-of-Distribution Detection in Time- Series Data by Ramneet Kaur et.al., Unibersity of Pensylvania (2022). Code TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
12. A General Framework For Multi-step Ahead Adaptive Conformal Heteroscedastic Time Series Forecasting by Martim Sousa, Ana Maria Tomé, University of Aveiro (2022) Code TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
13. Conformal Prediction Interval Estimations with an Application to Day-Ahead and Intraday Power Markets by Christopher Kath and Florian Ziel (2020)
14. Robust Gas Demand Forecasting With Conformal Prediction by Mouhcine Mendil, Luca Mossina, Marc Nabhan, Kevin Pasini (2022)
15. Conformal Prediction Bands for Two-Dimensional Functional Time Series by Niccolo` Ajroldia, Jacopo Diquigiovannib, Matteo Fontanac, Simone Vantinia (2022)
16. Copula Conformal Prediction for Multi-step Time Series Forecasting TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥

Presentation_slides

1. Machine Learning for Probabilistic Prediction by Valery Manokhin, 2022
2. Adaptive Conformal Anomaly Detection for Time-series by Evgeny Burnaev, Alexander Bernstein, Vlad Ishimtsev and Ivan Nazarov (Skoltech, Moscow, Russia, 2017)
3. Nonparametric predictive distributions based on conformal prediction by Vladimir Vovk, Jieli Shen, Valery Manokhin, Min-ge Xie, Ilia Nouretdinov and Alex Gammerman (Royal Holloway, University of London Rutgers University, 2017)
4. What Can Conformal Inference Offer to Statistics? by Lihua Lei, Stanford University
5. Conformal Regressors and Predictive Systems – a Gentle Introduction by Henrik Bostroem (KTH, Sweden, 2022)
6. Applications of Conformal Predictors by Ernst Ahlberg and Lars Carlsson (Stena Line, 2022)
7. crepes: a Python Package for Conformal Regressors and Predictive Systems by Henrik Bostroem (KTH, Sweden, 2022)
8. Assessing Explanation Quality by Venn Prediction by Amr Alkhatib, Henrik Boström and Ulf Johansson (2022)
9. Well-Calibrated Rule Extractors by Ulf Johansson, Tuwe Löfström, Niclas Ståhl (2022)
10. [Calibration of Natural Language Understanding Models with Venn-ABERS Predictors](Calibration of Natural Language Understanding Models with Venn-ABERS Predictors](https://copa-conference.com/presentations/patrizio.pdf) by Patrizio Giovannotti (2022)
11. Reinforcement Learning Prediction Intervals with Guaranteed Fidelity by Thomas Dietterich (University of Oregon, 2022)
12. Conformal Prediction beyond exchangeability by Rina Foygel Barber (University of Chicago, 2022)
13. Split conformal prediction for dependant data by Roberto I. Oliveira, Paulo Orenstein, Thiago Ramos and João Vitor Romano (2022)
14. Conformal prediction of small-molecule drug resistance in cancer cell lines by Saiveth Hernandez-Hernandez, Sachin Vishwakarma and Pedro Ballester

Researchers

1. Vladimir Vovk, Royal Holloway, United Kingdom
2. Alexander Gammerman, Royal Holloway, United Kingdom
3. Glenn Shafer, Rutgers University, USA
4. Emmanuel Candès, Stanford, USA
5. Ryan Tibshiriani, Carnegie Mellon, USA
6. Yaniv Romano, Technion—Israel Institute of Technology
7. Michael I. Jordan, Berkeley, USA
8. Jitendra Malik, Berkeley, USA
9. Anastasios Angelopoulos, Berkeley, USA
10. Lihua Lei, Stanford, USA
11. Henrik Boström, KTH, Sweden
12. Ulf Johansson, Jönköping University, Sweden
13. Henrik Linusson, University of Borås, Sweden
14. Harris Papadopoulos, Frederick University, Cyprus
15. Vladimir V'yugin, Institute for Information Transmission Problems (IITP), Russia
16. Evgeny Burnaev, Skoltech, Russia
17. Aaditya Ramdas, Carnegie Mellon, USA
18. Benjamin LeRoy, Carnegie Mellon, USA
19. Victor Chernozhukov, MIT, USA
20. Ulf Norinder, Stockholm University, Sweden
21. Ola Spjuth, Uppsala University, Sweden
22. Ilia Nouretdinov, Royal Holloway, United Kingdom
23. Matteo Fontana, Joint Research Centre - European Commission
24. Yao Xie, Georgia Institute of Technology
25. Zhimeo Ren, University of Chicago
26. Rafael Izbicki, Federal University of São Carlos (UFSCar) Brazil
27. Rina Foygel Barber University of Chicago

Articles

1. Measuring Models' Uncertainty: Conformal Prediction by Leo Dreyfus-Schmidt (Dataiku, 2020). 🔥🔥🔥🔥🔥
2. Conformal Prediction for Neural Regression Model by Pranab Ghosh (2021). 🔥🔥🔥🔥🔥
3. How to Handle Uncertainty in Forecasts by Michael Berk (2021)
4. How to Add Uncertainty Estimation to your Models with Conformal Prediction by Zachary Warnes (2021)
5. nonconformist: An easy way to estimate prediction intervals by Maria Jesus Ugarte (2021).
6. Detecting Weird Data: Conformal Anomaly Detection by Matthew Burruss (2020).
7. “MAPIE” Explained Exactly How You Wished Someone Explained to You by Samuele Mazzanti (2022). 🔥🔥🔥🔥🔥
8. With MAPIE, uncertainties are back in machine learning by Vianney Taquet (2021) 🔥🔥🔥🔥🔥
9. How to Predict Risk-Proportional Intervals with Conformal Quantile Regression by Samuele Mazzanti (2022). 🔥🔥🔥🔥🔥
10. Stanford statisticians and Washington Post data scientists build more honest prediction models Stanford (2021) 🔥🔥🔥🔥🔥
11. How to Detect Anomalies — state-of-the-art methods using Conformal Prediction by Valery Manokhin (2021) 🔥🔥🔥🔥🔥
12. How to calibrate your classifier in an intelligent way by Valery Manokhin (2022) 🔥🔥🔥🔥🔥
13. Conformal Prediction forecasting with MAPIE by Valery Manokhin (2022) 🔥🔥🔥🔥🔥
14. How to predict full probability distribution using machine learning Conformal Predictive Distributions by Valery Manokhin (2022) 🔥🔥🔥🔥🔥
15. How to predict quantiles in a more intelligent way (or ‘Bye-bye quantile regression, hello Conformal Quantile Regression by Valery Manokhin (2022) 🔥🔥🔥🔥🔥
16. Conformal Prediction in Julia by Patrick Altmeyer (2022)
17. Getting predictions intervals with conformal inference by Rajiv Shah (2022)

Websites

1. Main website with research from Prof. Vladimir (Volodya) Vovk 🔥🔥🔥🔥🔥
2. Conformal Prediction - Prediction with guaranteed performance Royal Holloway, United Kingdom
3. A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification by Anastasios N. Angelopoulos 🔥🔥🔥🔥🔥
4. Reliable Predictive Inference by Yaniv Romano 🔥🔥🔥🔥🔥

Twitter

1. Title: What Can Conformal Inference Offer to Statistics? by Lihua Lei, Stanford, 2022
2. Conformalized survival analysis by Lihua Lei, Stanford, 2021
3. Conformal Risk Control by Anastasious Angelopolous, Berkeley, 2022
4. Stable Conformal Prediction Sets by Eugene Ndiaye (Georgia Tech, 2022)
5. Machine learning sucks at uncertainty quantification. But there is a solution that almost sounds too good to be true: conformal prediction by Cristoph Molnar (2022).
6. How to correctly, yet efficiently model the uncertainty on predictions by Nico Wolf (2022)

TikTok

1. Getting prediction intervals with conformal prediction by Rajiv Shah (Hugging Face,2022)
2. Why you want prediction intervals instead of point predictions by Rajiv Shah (Hugging Face,2022)
3. It’s important to make sure your model is well calibrated by Rajiv Shah (Hugging Face,2022)

Conferences & Workshops

1. 11th Symposium on Conformal and Probabilistic Prediction with Applications 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
2. IFDS Workshop on Conformal Prediction 🔥🔥🔥🔥🔥
3. Workshop on Distribution-Free Uncertainty Quantification at ICML 2022 🔥🔥🔥🔥🔥
4. Workshop on Distribution-Free Uncertainty Quantification at ICML 2021🔥🔥🔥🔥🔥
5. 10th Symposium on Conformal and Probabilistic Prediction with Applications 🔥🔥🔥🔥🔥
6. 9th Symposium on Conformal and Probabilistic Prediction with Applications 🔥🔥🔥🔥🔥
7. 8th Symposium on Conformal and Probabilistic Prediction with Applications 🔥🔥🔥🔥🔥
8. 7th Symposium on Conformal and Probabilistic Prediction with Applications 🔥🔥🔥🔥🔥
9. 6th Symposium on Conformal and Probabilistic Prediction with Applications 🔥🔥🔥🔥🔥

Python

1. MAPIE - Model Agnostic Prediction Interval Estimator by Quantmetry team (2021) Paper Includes TIME SERIES (EnbPI) 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
2. 'Crêpes' - Conformal regressors and predictive systems by Henrik Boström (2021) Paper 🔥🔥🔥🔥🔥 Presentation by Henrik Bostroem (KTH, Sweden, 2022)
3. EnbPI by Chen Xu (2021) TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥 Paper
4. Nonconformist by Henrik Linusson (2015) 🔥🔥🔥🔥🔥
5. Venn-ABERS Predictor by Paolo Toccaceli (2019) Paper 🔥🔥🔥🔥🔥
6. Conformalized Quantile Regression by Yaniv Romano (2019) 🔥🔥🔥🔥🔥
7. Conformal Classification by Anastasios N. Angelopoulos (2021)
8. Orange3 Conformal Prediction
9. Conformal: an R package to calculate prediction errors in the conformal prediction framework by Isidro Cortes, 2019
10. Uncertainty Toolbox by Youngseog Chung, Willie Neiswanger, Ian Char and Han Guo (2021)
11. TorchUQ is an extensive library for uncertainty quantification (UQ) based on pytorch (2022)
12. Multi-class-probabilistic-classification using Venn-ABERS (Conformal) prediction by Valery Manokhin (Royal Holloway, 2022)
13. Copula Conformal Multi Target Regression by Soundouss Messoudi (2021)
14. Uncertainty Toolbox by Youngseog Chung, Willie Neiswanger, Ian Char and Han Guo (2021)
15. Conformal Histogram Regression: efficient conformity scores for non-parametric regression problems by Mateo Sesia and Yaniv Romano (NeurIPS 2021). 🔥🔥🔥🔥🔥
16. Conformalized density- and distance-based anomaly detection in time-series data (KNN-CAD) by Evgeny Burnaev, Vladislav Ishimtsev (2016). Top #3 winning solution in Numenta competition 🔥🔥🔥🔥🔥
17. Conformal time-series forecasting by Kamile ̇ Stankeviciute (Cambridge, NeurIPS 2021) TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
18. Valid Prediction Intervals by Nicolas Dewolf, Bernard DeBaets, Willem Waegeman (2022) 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
19. Adaptive Conformal Predictions for Time Series by Margaux Zaffran, Aymeric Dieuleveut, Olivier Fe ́ron, Yannig Goude, and Julie Josse (2022) TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥 Video Code
20. Conformal learning from scratch by Marharyta Aleksandrova (2021)
21. Ensemble Conformalized Quantile Regression for Probabilistic Time Series Forecasting Vilde Jensen, Filippo Maria Bianchi and Stian Norman Anfinsen (2022) TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
22. Conformalized Online Learning: Online Calibration Without a Holdout Set by Shai Feldman, Stephen Bates and Yaniv Romano (2022). TIME SERIES 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
23. PySloth - Python package for Probabilistic Prediction by Valery Manokhin 🚀🚀🚀🚀🚀 🔥🔥🔥🔥🔥
24. Conformal Prediction in KNIME
25. Nonconformist by Henrik Linusson (2015) 🔥🔥🔥🔥🔥
26. SKTime by Franz Kiraly (2022)
27. NeuralProphet (2022)

Code-R

1. Conformal Inference R Project maintained by Ryan Tibshirani (2016) 🔥🔥🔥🔥🔥
2. Prediction Bands by Rafael Izbicki and Benjamin LeRoy (2019)
3. Conformal: an R package to calculate prediction errors in the conformal prediction framework by Isidro Cortes, 2019
4. Online Time Series Anomaly Detectors by Alaine Iturria, 2021 🔥🔥🔥🔥🔥
5. piRF - Prediction Intervals for Random Forests by Chancellor Johnstone and Haozhe Zhang (2019)
6. conformalClassification: Transductive and Inductive Conformal Predictions for Classification Problems by Niharika Gauraha and Ola Spjuth (2019)
7. R Package for Spatial Conformal Prediction
8. [conformalInference.multi: Conformal Inference Tools for Regression with Multivariate Response](conformalInference.multi: Conformal Inference Tools for Regression with Multivariate Response](https://cran.r-project.org/web/packages/conformalInference.multi/index.html) by Jacopo Diquigiovanni, Matteo Fontana, Aldo Solari, Simone Vantini, Paolo Vergottini, Ryan Tibshirani (2021) 🔥🔥🔥🔥🔥
9. Conformal: an R package to calculate prediction errors in the conformal prediction framework by Isidro Cortes, 2019
10. cfsurvival - An R package that implements the conformalized survival analysis methodology Paper

Code-Julia

1. ConformalPrediction.jl by Patrick Altmeyer (2022)
2. RandomForest by Henrik Boström (2017) 🔥🔥🔥🔥🔥

Code-Other

1. LibCP -- A Library for Conformal Prediction 🔥🔥🔥🔥🔥
2. An Implementation of Venn-ABERS predictor 🔥🔥🔥🔥🔥
3. LibVM -- A Library for Venn Machine
4. Scala-CP by Marco Capuccini (2017)' 🔥🔥🔥🔥🔥 (see tutorial section 'Conformal Prediction in Spark')

AI-platforms

1. Conformal Prediction in Knime Presentation 🔥🔥🔥🔥🔥
2. Data Robot (https://docs.datarobot.com/en/docs/release/public-preview/mlops-preview/prediction-intervals-regression.html#the-trumpet-chart)

Patents

1. Rahul Vishwakarma, Method and system for reliably forecasting storage disk failure. US 2021/0034450 A1 United States Patent and Trademark Office, Feb 2021
2. Rahul Vishwakarma, Analyzing Time Series Data for Sets of Devices Using Machine Learning Techniques. US 2021/0241929 A1 United States Patent and Trademark Office, Aug 2021
3. Rahul Vishwakarma, System and method for prioritizing and preventing backup failures. US 2021/0374568 A1 United States Patent and Trademark Office, Dec 2021