Version 0.6.1

This release brings PyTorch 1.10 builds and numerical fixes and improvements

Large fixes

• Raysampling now works with non-square image conventions commit
• Perspective_correct mesh rasterization calculation is protected against divide-by-zero. This fixes quite a number of bug reports, e.g. #561. commit

Breaking changes

• This commit makes camera code more consistent in behaving like align_corners=False everywhere, by removing some extra -1’s in the code for screen space conversion. (1) If you create a camera in screen coordinates, i.e. setting in_ndc=False, then anything you do with the camera which touches NDC space may be affected, including trying to use renderers. The transform_points_screen function will not be affected. (2) If you call the function “transform_points_screen” on a camera defined in NDC space then results will be different.

Bug fixes

• Raysampling now works with cameras defined in screen space. commit.
• Pointclouds.inside_box is now properly reduced rather than returning separate results per axis. commit
• PulsarPointsRenderer fixed with non-square cameras commit
• Functions clone and detach on TexturesUV properly propagate align_corners and padding_mode options. commit
• Fixing default arguments of add_points_features_to_volume_densities_features commit
• Fix opencv camera conversion for non-square images (affects pulsar) commit

Small improvements

• Some matrix conversions are now traceable with torch.jit. commit
• Fixes to compiled code to make windows builds happier commit commit
• A new set of tests, test_camera_pixels, illustrates the precise mapping of pixels for cameras. commit
• Numerical improvements in IOU calculation commit and commit
• New option on TexturesUV to choose sampling_mode. commit
• The plotly visualization will show face colors from a TexturesAtlas with K=1 commit

Internal

• Benchmarks have been moved to tests/benchmarks commit
• Spelling commit
• Some more type annotations for Pointclouds commit
• Test fixes commit
• Special implementation of eigenvalue calculation for 3x3 matrices which can be faster and more reliable than native PyTorch. Currently not used commit

Version 0.6.0

This release contains several new optimized operations related to point clouds.

New features

• Farthest point sampling here, here and here
• Ball query operation. commit
• Sample_pdf importance sampling operation, implemented with a CUDA kernel. Previously in Python in the NeRF project. here and here
• Fast accurate calculation of Intersection over union for 3D boxes. See note. here, here and here
• Subsample method for Pointclouds commit
• Adding point clouds to volumes is now implemented in C++/CUDA, which means it can always operate inplace here, here and here

Breaking changes

• This commit removes _xy_grid from the saved state of the GridRaySampler module, if the PyTorch version is 1.6.0 or higher. This means that you will get errors if you try to load a model saved with such a module by an older version of PyTorch3D (unless strict==False). Similarly the NeRF project’s HarmonicEmbedding no longer stores _frequencies.
• PyTorch 1.5.0 or above is now required.

Bug fixes

• Fix duplicate arguments errors when cameras are specified in certain places commit
• Fix to join_scene for meshes using TexturesUV, which picked the wrong verts_uvs in certain cases commit
• Fix to edge cases when creating TexturesAtlas. commit
• Points to volumes fix when the grid_sizes are not specified. commit

Small improvements

• Making the rasterizer deterministic if there are ties between faces commit
• The function so3_log_map is now torchscriptable commit
• The GridRaySampler change means it can be reused at different image sizes. commit
• More documentation in the renderer, with RasterizationSettings and PointRasterizationSettings being dataclasses commit and here
• Ability to save colors as 8bit (i.e. uint8) when writing data to PLY files commit

Internal

• Coarse rasterization code has been reorganized here, here and here

Version 0.5.0

This release includes several very significant bug fixes and performance improvements as well as some new features for rendering, dataloading and visualization. Please read the breaking changes carefully and update your code accordingly for this new PyTorch3D version.

Breaking changes

• This commit changes the cameras objects and will affect you if (a) you use a cameras object with non-square images, (b) you call the transform_points_screen method, (c) you initialise a cameras object in screen space, i.e. you have been specifying image_size. See here for more details on the changes and how to update your code.
• The functions random_rotations, random_rotation and random_quaternions no longer have a potentially confusing requires_grad parameter. | commit
• The call pytorch3d.loss.mesh_laplacian_smoothing.laplacian_cot(meshes) should now be pytorch3d.ops.cot_laplacian(meshes.verts_packed(), meshes.faces_packed()) | commit

New deprecations

• The function so3_exponential_map deprecated in favor of new function so3_exp_map | commit

New features

• Cameras can be defined and used regardless of coordinate system conventions. They can be defined in NDC or screen space and are converted appropriately to interface with the PyTorch3D renderers according to their conventions | commit
• The standard mesh laplacian calculation has been added and now all three laplacians (standard, cot, norm) live in pytorch3d.ops.laplacian_matrices | commit
• RayBundles can be viewed in plotly_vis | commit
• Support for the OFF file format for loading meshes | commit
• Experimental support for loading some glTF files | commit
• PLY format now supports mesh vertex colors, i.e. TexturesVertex | commit
• Saving a mesh to an OBJ file can include TexturesUV data | commit
• User can now specify vertex normals explicitly on a Meshes object | commit
• Pointcloud normals and mesh vertex normals can now be saved to and loaded from PLY files
• New rotate_on_spot function to relatively rotate a camera position | commit
• New AmbientLights object for ambient-only lighting | commit
• Updated the alpha channel in the hard_rgb_blend function to return the mask of the pixels which have overlapping mesh faces | commit
• Camera conversions from and to OpenCV cameras
• SE3 exponential and logarithm maps | commit
• TensorProperties classes (e.g. Pointclouds and Cameras) now have a cuda() function. | commit

Internal-facing new features

• New linearly extrapolated acos function | commit
• New custom 3x3 determinant calculation | commit
• New function Meshes.has_verts_normals | commit
• Features for enhanced type safety

Bug fixes

• Fix to cotcurv method of the mesh_laplacian_smoothing | commit
  When a Transform3D is constructed without specifying the device but with tensor data input (e.g. a rotation matrix), it will now default to the device of the tensor. | commit
• Fix to frustrum culling when z_clip_value=None | commit
• Fix to mesh rendering when frustrum culling culls everything | commit
• Mesh renderer stability with very small faces | commit
• Transform3d.to() now works to change dtype without changing device | commit
• Multiple issues with shape broadcasting in lighting, shading and blending | commit
• Rendering a nonsquare image when the H and W are not integer multiples commit
• Fix to pulsar non-square image rendering | commit
• Pulsar cameras now more reliably match PyTorch3D ones | commit
• Pulsar gradient now calculated when no other gradient is needed | commit
• TensorProperties classes (e.g. Pointclouds and Cameras) will always remember their device if the default cuda device changes. | commit
• Now matrix_to_quaternion is correct for rotations by 180˚ | commit
• In tutorials: updates for changes to PyTorch | commit (avoiding gradients) and numpy | commit. Load tutorial utils correctly on Colab | commit
• Fix to subsampling multiple large pointclouds for plotly visualization | commit
• Fix bug when join_batch is used to join meshes with TexturesVertex | commit
• PLY files with uint8 colors are now scaled correctly | commit
• Fix save_ply with noncontiguous faces | commit
• Fix to ShapeNet dataloader: works when not loading textures, install will not miss the json files | commit
• Calling extend() with a user’s subclass of a Textures object respects the subclass (#618) | commit
• TexturesUV.join_scene works with any number of channels | commit
• Documentation updates for Volume renderer and RayBundle

Small improvements including speedups

• Improved so3 functions to make gradient computation stable | commit
• Mesh normal consistency speedup | commit
• Add error checks when texture objects are inconsistent with their meshes | commit
• Speedup checking density bounds in volume renderer | [commit](https://github.com/facebookresearch/pytorch3d/comm...

v0.4.0

Changelog

The key new features in this release is support for Implicit/Volume Rendering. This includes several methods for sampling camera rays and marching along the rays in order to render their color. We further introduce support for voxel grids. To this end, we implemented a new Volumes structure and methods for converting between Pointclouds and Volumes. The rendering of implicit surfaces as well as voxel grids has been showcased in two new tutorial jupyter notebooks.

We are also introducing a new projects folder with an implementation of NeRF. We plan to add more examples of papers which can be implemented using components from PyTorch3D.

Key features

• Volumes Datastructure | commit
• Raysamplers: GridRaysampler, MonteCarloRaysampler, NDCGridRaysampler; RayBundle datastructure | commit
  Raymarchers: AbsorptionOnlyRaymarcher, EmissionAbsorptionRaymarcher | commit
• Implicit/Volume Renderer | commit
• Pointclouds to Volumes conversion | commit

Projects

• Reimplementation of NeRF | commit

Additional new features

• Taubin Smoothing for Meshes | commit
• Non Square Image Rasterization for Meshes | commit
• Non Square Image Rasterization for Pointclouds | commit
• Naive PyTorch version of Marching Cubes | commit
• New Pluggable Data Loading API | commit
  • Mesh formats | commit
  • Pointcloud formats | commit

New Tutorials

• Fit Textured Volume | commit
• Fit Neural Radiance Field | commit

Small Updates

• Change io functions to use iopath | commit
• Read heterogenous nonlist PLY properties as arrays | commit
• Update the MeshRasterizer to automatically infer the camera type and set the perspective_correct setting for correcting barycentric coordinates after rasterization | commit

Bug Fixes

• Rasterization of mesh faces partially behind the image plane
  • Full fix which clips meshes at specified z value prior to rasterization instead of only culling. | commit1, commit2
  • Introduced two new rasterization settings (z_clip_value, cull_to_frustum) | commit
• Check for verts/faces in Meshes to be on the same device | commit
• Fix for index error with Texture Atlas sampling | commit

Builds

• For Linux, instead of uploading wheels to PyPI which will only work with one particular version of PyTorch and CUDA, we now provide a range of built wheels on S3. Refer to INSTALL.md for instructions on how to download and install.

v0.3.0

Changelog

The key new feature in this release is support for Pulsar as an alternative backend for point cloud rendering. Pulsar is a highly tuned backend with better scaling performance than the default backend but with less flexibility: it does not have support for custom rasterizers and compositors. It can be used for scenes with millions of spheres and up to 4K resolution and supports opacity optimization.

• CUDA/C++ implementation | commit
• PyTorch interface | commit
• Examples and Demos | commit

Additional new features

• Plotly functions for visualization | commit
  • Support for rendering batches of meshes/cameras/pointclouds
  • Support for viewing plotly plots from PyTorch3D Cameras point-of-view
  • plot_scene function
• Extend sample_points_from_meshes to support sampling textures | commit
• corresponding_cameras_alignment function that estimates a similarity transformation between two sets of cameras | commit
• Support for variable size point radius in the pointcloud rasterizer | commit
• save_ply now by default saves to binary instead of ascii. An option makes the previous functionality available | commit
• Functions to convert to and from axis-angle representation of rotations | commit
• Visualization functions for the TexturesUV class | commit

New tutorials

• Rendering DensePose Data with PyTorch3D | commit1, commit2
• Updates to Pointcloud and Mesh rendering tutorials | commit

Small updates

• Fixes to support multigpu rendering | commit
• Support texture atlas loading for .obj files with meshes which only have material properties | commit
• Add support for changing the background color in Pointcloud compositors | commit
• Add support for returning the fragments from the output of the MeshRenderer | commit
• Support for axis angle representation of rotations | commit

Bug Fixes

• Fixed corner case in look_at_view_transform | commit
• Fixed softmax_rgb_blend with mesh is outside zfar | commit
• Fixed corner case in matrix_to_quaternion function | commit

Misc

The introduction of Pulsar adds the NVIDIA CUB library as a build-time dependency.

v0.2.5

Changelog

New features

• Data loaders for common 3D datasets
  • ShapeNetCore dataloader | commit
  • R2N2 dataloader | commit
• New texturing API
  -Separate classes: TexturesVertex, TexturesUV, TexturesAtlas | commit
  • Existing Textures class is now deprecated and will be removed in the next release.
• Cameras API refactor
  • Renaming and restructure to have consistency across all classes | commit
  • Cameras have been renamed as follows:
    OpenGLPerspectiveCameras -> FoVPerspectiveCameras OpenGLOrthographicCameras -> FoVOrthographicCameras
SfMPerspectiveCameras -> PerspectiveCameras
SfMOrthographicCameras -> OrthographicCameras
  • All cameras now output projected values in NDC with the option to provide params in screen coordinates and convert them to NDC.
  • Refer to the new note cameras.md for more detailed information.
• Barycentric clipping in CUDA
  • Move barycentric clipping from PyTorch to CUDA for increased efficiency. Now available as a rasterization setting clip_barycentric_coords. | commit
• One new representation for rotations
  • Conversions to and from Zhou et al 6d rotation represention | commit
• Customizable Background color
  • Option added to HardPhongShader, HardGouraudShader, and HardFlatShader | commit
• Joining several meshes to render a single scene
  • New join_meshes_as_scene function which also supports joining textures | commit1, commit2
• CUDA op for interpolating face attributes
  • Functionality which was in python moved to cuda
    26d2cc2
• Gather scatter on CPU | commit
• C++/CUDA implementations of sigmoid/softmax blending functions | commit 1
• C++ implementations for point-to-mesh distance functions | commit
• detach method for Meshes, Pointclouds and Textures | commit
• Support for multiple align_modes in the Cubify operator | commit
• Texture maps (i.e. the TexturesUV class) now has align_corners and padding_mode customizable, and the default has changed to align_corners=True. | commit

New tutorials:

• Data loading with ShapeNetCore and R2N2 dataloaders | commit
• Fitting a textured mesh from multiview images | commit

Small updates

• Compatibility with PyTorch 1.6
• Flag to make sorting optional in KNN
  806ca36
• update_padded method on meshes
  • Other optimizations use this | commit

Bug Fixes:

• Temporary fix for rendering from inside a surface results in uninterpretable images | [issue]commit
  • This fix culls all face which are partially behind the image plane
• Scaling Pointclouds by a scalar now works | commit
• SO3 log map fix for singularity at PI | commit
• Join mismatched texture maps on CUDA | commit
• Flat shading fix | commit
• Efficiency PnP bug fix | commit
• look_at_transform now gives the correct location of the camera when the object at location is not at (0,0,0) | commit
• Rendering a batch of meshes with texture maps now correct for all members of the batch #283 | commit
• Extending meshes with texture maps where the verts and the verts_uvs have different lengths is now correct commit

Breaking Changes

• Unused parameter composite_params removed from compostor classes | commit
• Textures has now moved: (prev) from pytorch3d.structures import Textures -> (new) from pytorch3d.renderer import TexturesUV, TexturesVertex, TexturesAtlas. In addition due to the new texturing API, the generic Textures class will be deprecated in the next release (it is still currently available to be imported from pytorch3d.renderer)

v0.2.0

Changelog

New Features

• Pointclouds Datastructure | 767d68a
  • Support for batches of pointclouds with additional helper methods e.g. get_bounding_boxes
• Pointclouds rendering | 5359977
  • C++, CUDA and Python implementations of Pointcloud rendering with heterogeneous pointclouds. Exposed via Rasterizer, Renderer and Compositor Pytorch Classes with 3 types of Compositors available. A jupyter notebook tutorial has also been added.
• Umeyama - estimate a rigid motion between two sets of corresponding points. | e5b1d6d
• Weighted Umeyama | e37085d
• Iterative Closest Point Algorithm | 8abbe22
• Efficient PnP algorithm to Fit 2D to 3D correspondences under perspective assumption | 04d8bf6
• Pointcloud normal estimation | 365945b
  • Supports batches of pointclouds
  • Also available on the Pointclouds datastructure as a method estimate_normals
• Point to Mesh distances | 487d4d6
  • Python, C++ and Cuda implementations of Point to Triangle Face, Face to point algorithms including array versions
• K nearest neighbors
  • Multiple KNN methods for pointclouds based on the input parameters | 870290d
  • KNN with heterogeneous pointclouds (i.e different numbers of points) | 01b5f7b
  • Autograd wrapper for KNN including backward pass for distances | b2b0c5a
    • knn_points function to return the neighbor points and distances
    • knn_gather function to allow gathering of additional features based on the knn indices

Updates to existing Operators

• Chamfer loss support for heterogeneous pointclouds | 790eb8c
  • Support for chamfer loss for two batches of pointclouds where each pointcloud in the batch can have different numbers of points
• Vert align for pointcloud objects | f25af96
• Cameras
  • unproject_points function to convert from screen to world coordinates | 365945b
  • look_at_transform update to enable specifying the eye (camera center) directly | 2480723
• Transforms3D update to allow init of Transforms3D class with a custom matrix | 90dc7a0
• Mesh Rendering update to enable back face culling in rasterization - this is available as a cull_backfaces boolean setting in raster_settings | 4bf3059
• Mesh loading - update to load_obj to support loading textures as per face textures [following the approach from SoftRasterizer]. There is a new boolean argument called 'create_texture_atlasfor theload_obj` function to enable this | c9267ab
• join_meshes_as_batch method to create a batch of meshes from a list of Meshes objects | b64fe51

Bug Fixes

• nan check in sample points from meshes | 6c48ff6
• Texturing function reshape/view fixes | 5d3cc35
• SfMPerspectiveCameras projection matrix - add the principal points after the perspective divide | 677b0bd
• Enable cuda kernels to be laucnhed on any GPU (not just the default) | c3d636d

Breaking changes

• Nearest neighbors implementation has been entirely removed - use KNN instead. | 3794f67
• There are some changes to the interface of chamfer_distance to cope with heterogenous point clouds. | 790eb8c
• The function join_meshes has been renamed to join_meshes_as_batch | b64fe51

v0.1.1

Changelog

New features:

• load_textures boolean parameter for load_obj function | 659ad34
• Single function to load mesh and texture data from OBJs. | 8fe65d5
• Cpu implementations for
  • Nearest neighbor | e290f87
  • Face Areas Normals | e290f87
  • Packed to padded tensor | e290f87
• Mesh Rendering
  • Flat Shading for meshes | ff19c64
  • Barycentric clipping before z buffer and texture interpolation | ff19c64
• Support for building with windows | 9e21659

Bug fixes:

• Several documentation, installation and correctness fixes including:
  expsumlog in soft blending replaced with torch.prod which makes soft blending backward pass stable | ba11c0b
• Fix matrix convention for rotations in Tranforms3D | 8301163
• Rendering flipping - the y axis flip in the blending functions has been removed and the rasterization step has been updated to ensure the directions of the axes are correct in the rendered image. The documentation for the renderer has been updated with the convention for world and camera coordinates. | 15c72be

Breaking changes

• The spelling of /Gourad/ has been fixed to /Gouraud/. | 9ca5489
• Shaders have been renamed to make clear if they are Hard or Soft (probabilistic) | f0dc651

v0.1.0 Initial Release

v0.1.0 Initial release