Hi, I’m a senior researcher in machine learning and computer vision at the Torr Vision Group (University of Oxford) and a research director at Five AI Oxford, a cool startup recently acquired by Bosch. At Five, we are helping in building the technology for driverless cars.
At Oxford University, I primarily supervise PhD (DPhil) students working on topics related to robustness, continual learning, efficient methods for neural network training etc. At Five AI Oxford office, I lead a small team of cool research scientists, and also help the company in shaping their research agenda related to the perception side of their underlying AI systems. I also help the applied engineering team of Five AI in building research-based products.
During my PhD, I was part of CVN group of INRIA and CentraleSupélec where I was supervised by Prof. M. Pawan Kumar and Prof. Nikos Paragios from October 2012 to March 2016.
I am excited and humbled to serve as a Programme Chair at BMVC 2022 (The 33rd British Machine Vision Conference)
Research Interests
I work in the field of machine learning (optimization, deep learning, generative models etc.) and its applications to vision and language. Please find below a few research areas (and related articles) that I am excited about.
- NN Robustness: RegMixup (NeurIPS22), Make some noise (NeurIPS22), CNNs vs Transformers Contest (ECCV22), Making better mistakes (ICLR21), Benign Overfitting (ICLR21), Low Rank Feature, Stable Rank (ICLR20), Focal Loss Calibration (NeurIPS20)
- Incremental Learning: RWalk (ECCV18), Tiny Episodic (ICML19-WS), GDumb (ECCV20), Orthog-CL (NeurIPS20), Hindsight (AAAI 21)
- Neural Network Discretization/Pruning: ProxMeanField (ICCV19), MD (AISTATS21), FORCE (ICLR21)
- Generative Models: MAD-GAN (CVPR18), Interactive Sketch (ICCV19)
- Vision and Language: Flip-Dial (CVPR18), CCA-VD (NIPS-WS18)
- Weakly supervised semantic segmentation: BMVC17, EMMCVPR17
- High-Order Inference: Parsimonious Labeling (ICCV15)
- Structured Prediction: HOAP-SVM (ECCV14), ECCV16, ICML16
Research Collaborators
I have had the opportunity to learn from and collaborate with with many brilliant people from a variety of institutions such as UC Berkeley, INRIA, Adobe research, Meta, IIIT Hyderabad, IIT Kanpur, Huawei, KAUST, Naver Labs, ANU Australia, EPFL etc.
Amazing PhD students I closely work with
(officially – implies PhD students I formally co-supervise at Oxford with Phil, write regular reviews/reports for them etc.)
- Francesco Pinto (since 2020)
- Jishnu Mukhoti (intern student Sept 2018 to Aug 2019, PhD student since Oct 2019)
- Pau de Jorge (officially since Oct 2020)
- Chen Lin (officially since Oct 2020)
- Ameya Prabhu (working as a PhD student since Oct 2020)
Past students I worked with
- Viveka Kulharia (officially, 2017 to 2021), [PhD Thesis]
- Amartya Sanyal (2018 to 2021)
- Arslan Chaudhry (PhD student (official), Jan 2017 to Nov 2020. joined DeepMind), [PhD Thesis]
- Arnab Ghosh (Intern Jan-July 2017, then joined as a PhD student in TVG and collaborated until the mid of 2019)
- Daniela Massiceti (from October 2016 until Dec 2018. Moved to MSR Cambridge)
- Anuj Sharma (Masters thesis intern (official), Feb-Aug 2017, moved to Five AI), [Masters Thesis]
Preprints
05) [NEW]
What Makes and Breaks Safety Fine-tuning? A Mechanistic Study
Samyak Jain, Ekdeep Singh Lubana, Kemal Oksuz, Tom Joy, Philip HS Torr, Amartya Sanyal, Puneet K Dokania
arXiv 2024
04) [NEW]
MoCaE: Mixture of Calibrated Experts Significantly Improves Object Detection
Kemal Oksuz, Selim Kuzucu, Tom Joy, and Puneet K. Dokania
arXiv 2024
03) [NEW]
Segment, Select, Correct: A Framework for Weakly-Supervised Referring Segmentation
Francisco Eiras, Kemal Oksuz, Adel Bibi, Philip H.S. Torr, Puneet K. Dokania
arXiv 2023
02) Online Continual Learning Without the Storage Constraint
Ameya Prabhu, Zhipeng Cai, Puneet Dokania , Philip Torr, Vladlen Koltun, and Ozan Sener
arXiv 2023
01) Robustness via Deep Low-Rank Representations
Amartya Sanyal, Varun Kanade, Philip H. S. Torr, Puneet K. Dokania
Project Page
Publications (Conferences)
34) [NEW]
On Calibration of Object Detectors: Pitfalls, Evaluation and Baselines
Selim Kuzucu, Kemal Oksuz, Jonathan Sadeghi, and Puneet K. Dokania
In ECCV 2024 (Milan, Italy), Oral presentation
33) [NEW]
Placing Objects in Context via Inpainting for Out-of-distribution Segmentation
Pau de Jorge, Riccardo Volpi, Puneet K Dokania, Philip HS Torr, Grégory Rogez
In ECCV 2024 (Milan, Italy)
32) [NEW]
Fine-tuning can cripple your foundation model; preserving features may be the solution
Jishnu Mukhoti, Yarin Gal, Philip HS Torr, Puneet K Dokania
In Transactions on Machine Learning Research (TMLR) 2024, Featured Certification
31) Graph Inductive Biases in Transformers without Message Passing
Liheng Ma, Chen Lin, Derek Lim, Adriana Romero-Soriano, Puneet K. Dokania, Mark Coates, Philip Torr, Ser-Nam Lim
In ICML 2023 (Hawaii)
30) Towards Building Self-Aware Object Detectors via Reliable Uncertainty Quantification and Calibration
Kemal Oksuz, Tom Joy, Puneet K. Dokania
In CVPR 2023 (Vancouver Canada)
29) Computationally Budgeted Continual Learning: What Does Matter?
Ameya Prabhu, Hasan Abed Al Kader Hammoud, Puneet K. Dokania, Philip HS Torr, Ser-Nam Lim, Bernard Ghanem, Adel Bibi
In CVPR 2023 (Vancouver Canada)
28) Sample-dependent Adaptive Temperature Scaling for Improved Calibration
Tom Joy, Francesco Pinto, Ser-Nam Lim, Philip HS Torr, Puneet K. Dokania
In AAAI 2023 (Safe and Robust AI track)
27) Query-based Hard-Image Retrieval for Object Detection at Test Time
Edward Ayers, Jonathan Sadeghi, John Redford, Romain Mueller, Puneet K. Dokania
In AAAI 2023 (Safe and Robust AI track)
26) RegMixup: Mixup as a Regularizer Can Surprisingly Improve Accuracy & Out Distribution Robustness
Francesco Pinto, Harry Yang, Ser-Nam Lim, Philip HS Torr, Puneet K. Dokania
In NeurIPS 2022
25) Make Some Noise: Reliable and Efficient Single-Step Adversarial Training
Pau de Jorge, Adel Bibi, Riccardo Volpi, Amartya Sanyal, Philip HS Torr, Gregory Rogez, Puneet K. Dokania
In NeurIPS 2022
24) An Impartial Take to the CNN vs Transformer Robustness Contest
Francesco Pinto, Philip H. S. Torr, Puneet K. Dokania
In ECCV 2022
23) A Continuous Mapping For Augmentation Design
Keyu Tian*, Chen Lin*, Ser-Nam Lim, Wanli Ouyang, Puneet K. Dokania, Philip H. S. Torr
In NeurIPS 2021
22) Mirror Descent View for Neural Network Quantization
Thalaiyasingam Ajanthan*, Kartik Gupta*, Philip H. S. Torr, Richard Hartley, Puneet K. Dokania
In AISTATS 2021
21) Progressive Skeletonization: Trimming more fat from a network at initialization
Pau de Jorge, Amartya Sanyal, Harkirat S. Behl, Gregory Rogez, Puneet K. Dokania
In ICLR 2021
20) How benign is benign overfitting?
Amartya Sanyal, Puneet K. Dokania, Varun Kanade, Philip H. S. Torr
In ICLR 2021 Spotlight
19) No Cost Likelihood Manipulation at Test Time For Making Better Mistakes in Deep Networks
Shyamgopal Karthik, Ameya Prabhu, Puneet K. Dokania , Vineet Gandhi
In ICLR 2021
18) Using Hindsight to Anchor Past Knowledge in Continual Learning
Arslan Chaudhry, Albert Gordo, Puneet K. Dokania , Philip Torr, David Lopez-Paz
In AAAI 2021
17) Calibrating Deep Neural Networks using Focal Loss
Jishnu Mukhoti*, Viveka Kulharia*, Amartya Sanyal, Stuart Golodetz, Philip H.S. Torr, Puneet K. Dokania
In NeurIPS 2020
16) Continual Learning in Low-rank Orthogonal Subspaces
Arslan Chaudhry, Naeemullah Khan, Puneet K. Dokania, Philip H.S. Torr
In NeurIPS 2020
15) GDumb: A Simple Approach that Questions Our Progress in Continual Learning
Ameya Prabhi, Philip H. S. Torr, Puneet K. Dokania
In ECCV 2020 (Oral)
14) Stable Rank Normalization for Improved Generalization in Neural Networks and GANs
Amartya Sanyal, Philip H. S. Torr, Puneet K. Dokania
In ICLR 2020 (Spotlight), Addis Ababa Ethiopia
13) Interactive Sketch & Fill: Multiclass Sketch-to-Image Translation
Arnab Ghosh, Richard Zhang, Puneet K. Dokania, Oliver Wang, Alexei A. Efros, Philip H. S. Torr, Eli Shechtman
In ICCV 2019, Seoul, Korea
Project Page
12) Proximal Mean-field for Neural Network Quantization
Thalaiyasingam Ajanthan, Puneet K. Dokania, Richard Hartley, Philip H. S. Torr
In ICCV 2019, Seoul, Korea
11) Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
Arslan Chaudhry*, Puneet K. Dokania*, Thalaiyasingam Ajanthan*, Philip H. S. Torr
In ECCV 2018, Munich, Germany
Slides
10) Multi-Agent Diverse Generative Adversarial Networks
Arnab Ghosh, Viveka Kulharia, Vinay Namboodiri, Philip H. S. Torr, Puneet K. Dokania
In CVPR 2018 (Spotlight), Salt Lake, USA
09) FlipDial: A Generative Model for Two-Way Visual Dialogue
Daniela Massiceti, N. Siddharth, Puneet K. Dokania, Philip H.S. Torr
In CVPR 2018 (Oral), Salt Lake, USA
Project Page
08) Discovering Class-Specific Pixels for Weakly-Supervised Semantic Segmentation A. Chaudhry, Puneet K. Dokania, Philip H. S. Torr
In BMVC 2017 (Oral), London, U.K.
07) Bottom-Up Top-Down Cues for Weakly-Supervised Semantic Segmentation
Q. Hou, D. Massiceti, Puneet K. Dokania, Y. Wei, M-M. Cheng, Philip H. S. Torr
In EMMCVPR 2017, Venice, Italy.
06) Partial Linearization based Optimization for Multi-class SVM
P. Mohapatra, Puneet K. Dokania, C. V. Jawahar, M. P. Kumar
In ECCV 2016, Amsterdam, the Netherlands.
Supplementary | Poster
05) Minding the Gaps for Block Frank-Wolfe Optimization of Structured SVMs
A. Osokin, JB Alayrac, I. Lukasewitz, Puneet K. Dokania, S. Lacoste-Julien
In ICML 2016, New York City, USA.
Project Page
04) Parsimonious Labeling.
Puneet K. Dokania and M. Pawan Kumar
In ICCV 2015, Santiago, Chile.
Report (refer Chapter 3 of my PhD thesis)
03) Learning to Rank using High-Order Information
Puneet K. Dokania , A. Behl, C. V. Jawahar and M. Pawan Kumar
In ECCV 2014, Zurich, Switzerland.
Code
02) Learning-Based Approach for Online Lane Change Intention Prediction
Puneet Kumar , M. Perrollaz, S. Lefevre and C. Laugier
In IEEE Intelligent Vehicle Symposium (IV) 2013, Gold Coast City, Australia.
Video-1 (Best viewed in VLC) | Video-2 (Best viewed in VLC)
01) Discriminative parameter estimation for random walks segmentation
P. Y. Baudin, D. Goodman, Puneet Kumar , N. Azzabou, P. G. Carlier, N. Paragios, M. Pawan Kumar
In MICCAI 2013, Nagoya, Japan.
Technical Report
Publications (Workshops)
08) On Batch Normalisation for Approximate Bayesian Inference
Jishnu Mukhoti, Puneet K. Dokania, Philip H.S. Torr, Yarin Gal
In 3rd Symposium on Advances in Approximate Bayesian Inference 2020
07) Choice of Representation Matter for Adversarial Robustness
Amartya Sanyal, Varun Kanade, Puneet K. Dokania, Philip H.S. Torr
In NeurIPS 2020 Workshop, Interpretable Inductive Biases and Physically Structured Learning
06) Interpolation Noisy Datasets hurt Adversarial Robustness
Amartya Sanyal, Varun Kanade, Puneet K. Dokania, Philip H.S. Torr
In NeurIPS 2020 Workshop, Dataset Curation and Security
05) On using Focal Loss for Neural Network Calibration
Jishnu Mukhoti*, Viveka Kulharia*, Amartya Sanyal, Stuart Golodetz, Philip H.S. Torr, Puneet K. Dokania
In ICML 2020 Workshop (Spotlight), Uncertainty and Robustness in Deep Learning
04) Stable Rank Normalization for Improved Generalization in Neural Networks
Amartya Sanyal, Philip H. S. Torr, Puneet K. Dokania
In ICML 2019 Workshop, Understanding and Improving Generalization in Deep Learning, Long Beach, USA
03) Continual Learning with Tiny Episodic Memories
Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K. Dokania, Philip H. S. Torr, Marc’Aurelio Ranzato
In ICML 2019 Workshop, MTLRL2019: Workshop on Multi-Task and Lifelong Reinforcement Learning, Long Beach, USA
02) Visual Dialogue without Vision or Dialogue
Daniela Massiceti*, Puneet K. Dokania*, N. Siddharth*, Philip H.S. Torr
In NeurIPS 2018 Workshop, Critiquing and Correcting Trends in Machine Learning, Montréal, Canada
01) Deformable Registration through Learning of Context-Specific Metric Aggregation
E. Ferrante*, Puneet K. Dokania*, R. Marini, N. Paragios
In MLMI MICCAI 2017, Quebec, Canada.
Publications (Journals)
07) Catastrophic overfitting can be induced with discriminative non-robust features
Guillermo Ortiz-Jiménez, Pau de Jorge, Amartya Sanyal, Adel Bibi, Puneet K. Dokania, Pascal Frossard, Gregory Rogéz, Philip H.S. Torr
In Transactions on Machine Learning Research (TMLR), 2023.
06) Ancer: Anisotropic certification via sample-wise volume maximization
Francisco Eiras, Motasem Alfarra, M Pawan Kumar, Philip HS Torr, Puneet K Dokania, Bernard Ghanem, Adel Bibi
In Transactions on Machine Learning Research (TMLR), 2022.
05) Diagnosing and Preventing Instabilities in Recurrent Video Processing
Thomas Tanay, Aivar Sootla, Matteo Maggioni, Puneet K. Dokania, Philip Torr, Ales Leonardis, Gregory Slabaugh
In IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022.
04) Weakly-Supervised Learning of Metric Aggregations for Deformable Image Registration
Enzo Ferrante, Puneet K. Dokania, Rafael M. Silva, N. Paragios
In IEEE Journal of Biomedical and Health Informatics, 2018.
03) Rounding-based Moves for Semi-Metric Labeling
M. Pawan Kumar and Puneet K. Dokania
In JMLR 2016.
02) High Dynamic Range Fuzzy Color Image Enhancement Using Ant Colony System
O. P. Verma, P. Kumar , M. Hanmandlu, S. Chhabra
In Journal of Applied Soft Computing, 2012.
01) A Novel Bacterial Foraging Technique for Edge Detection
O. P. Verma, M. Hanmandlu, P. Kumar , S. Chhabra, A. Jindal
In Pattern Recognition Letters, 2011.
Technical Reports and Thesis
02) High-Order Inference, Ranking, and Regularization Path for Structured SVM
Puneet Kumar Dokania, PhD Thesis 2016
Slides
01) Learning-Based Approach for Online Lane Change Intention Prediction
Puneet Kumar, Master Thesis 2012.