DualPM: Dual Posed-Canonical Point Maps for 3D Shape and Pose Reconstruction

CVPR 2025 - Conference Highlight

Ben Kaye*, Tomas Jakab*, Shangzhe Wu, Christian Rupprecht, Andrea Vedaldi
(*equal contribution)
University of Oxford

teaser.mp4

📖 Overview

DualPM is a novel approach for 3D shape and pose reconstruction using dual posed-canonical point maps. Dual Point Maps reduce tasks in analysis of deformable objects to mapping an image of an object to its Dual Point Maps—a pair of point maps defined in camera space and canonical space. DualPMs are easy to predict with a neural network, enabling effective 3D object reconstruction and other tasks.

🚀 Installation

Note: Requires a valid CUDA install to use nvdiffrast.

git clone https://github.com/ben-kaye/DualPM_Paper.git
cd DualPM_Paper
pip install (-e) .

📚 Downloads

See DOWNLOAD.md.

📚 Usage

We provide a separate library, dualpm_lib, for computing Dual Point Maps, which is automatically installed and is a thin wrapper built upon nvdiffrast.

Example usage is given in datasets.py > MeshToDualPointmap.calculate_dual_pointmap(.).

We train our example model on paired input images and rasterized point map targets.

Training

1. Download the dataset
2. Configure weight path and data locations in configs/main.yaml
3. Run the training script:

python scripts/train.py

Inference

Inference requires our feature extraction code (a thin wrapper of sd-dino) to obtain the ODISE-DINO features. This must be computed offline as the models are large.

Complete the feature extraction then use our provided script to obtain predictions:

python scripts/infer.py

Benchmarking

Our main benchmarking code is available at AnimodelPoints. This is a derivative of the Animodel benchmark but configured for point clouds.

📄 Citing

If you find this work useful, please cite our paper:

@InProceedings{kaye2025dualpm,
  author    = {Ben Kaye and Tomas Jakab and Shangzhe Wu and Christian Rupprecht and Andrea Vedaldi},
  title     = {{DualPM}: Dual {Posed-Canonical} Point Maps for {3D} Shape and Pose Reconstruction},
  booktitle = {Proceedings of the {IEEE/CVF} Conference on Computer Vision and Pattern Recognition ({CVPR})},
  month     = {June},
  year      = {2025},
  pages     = {6425--6435}
}