Add RenderInstanceHelper for doing visual-only (non-physics) scenes #2539
Conversation
eundersander
added
the
do not merge
facebook-github-bot
added
the
CLA Signed
| return Mn::Image2D{Mn::PixelFormat::RGBA8Unorm, size, std::move(outputData)}; | ||
| } | ||
|
|
||
| void SimTest::checkPinholeCameraSemanticObservation( |
There was a problem hiding this comment.
I should find some code re-use with checkPinholeCameraRGBAObservation.
| py::class_<RenderInstanceHelper, RenderInstanceHelper::ptr>( | ||
| m, "RenderInstanceHelper") | ||
| .def(py::init<sim::Simulator&, bool>(), py::arg("sim"), | ||
| py::arg("use_xyzw_orientations"), "todo") |
There was a problem hiding this comment.
I'll replace the todos with documentation before merging.
| #include "esp/sim/Simulator.h" | ||
| #include "esp/sim/SimulatorConfiguration.h" | ||
|
|
||
| namespace py = pybind11; | ||
| using py::literals::operator""_a; | ||
|
|
||
| namespace { |
There was a problem hiding this comment.
I should probably move this to a pybind_utils.cpp inside our bindings library.
| .def("get_num_instances", &RenderInstanceHelper::GetNumInstances, "todo") | ||
| .def( | ||
| "set_world_poses", | ||
| [](RenderInstanceHelper& self, py::array_t<float> positions, |
There was a problem hiding this comment.
Why all this complexity in the binding code?
- I want to allow users to directly pass their numpy arrays into C++, with no memory copy.
- Pybind/numpy C++ code pretty much must be directly in our bindings library (which uses
pybind11_add_module), not other parts of Habitat-sim.
| namespace esp { | ||
| namespace sim { | ||
|
|
||
| RenderInstanceHelper::RenderInstanceHelper(Simulator& sim, |
There was a problem hiding this comment.
I'll add standard method documentation before merging.
| namespace sim { | ||
|
|
||
| RenderInstanceHelper::RenderInstanceHelper(Simulator& sim, | ||
| bool useXYZWOrientations) { |
There was a problem hiding this comment.
Give users the choice of quat convention and force them to declare it.
| namespace esp { | ||
| namespace sim { | ||
|
|
||
| class RenderInstanceHelper { |
There was a problem hiding this comment.
I'll add standard class documentation before merging.
| @@ -245,11 +266,89 @@ void SimTest::reset() { | |||
| CORRADE_COMPARE(pathfinder, simulator->getPathFinder()); | |||
| } | |||
|
|
|||
| Mn::Image2D convertR32uiToRgba8Unorm(const Mn::ImageView2D& inputImage) { | |||
There was a problem hiding this comment.
Maybe I can find a better home for this helper function.
Motivation and Context
DO NOT MERGE: I will make sure the v0.3.3 release is out the door before merging.
RenderInstanceHelper lets user python code add render asset instances to the Habitat visual scene and pose them with an efficient numpy batch interface (
RenderInstanceHelper.set_world_poses).The overall interface is intentionally minimal, so as to simplify porting the backend to other renderers like the experimental batch renderer.
In terms of specifying the render asset instance, we currently only offer 1. asset filepath (e.g. a GLB file) and 2. a semantic ID. We can expand this in the future, e.g. scale and material overrides. I'm avoiding using
AssetInfoandRenderAssetInstanceCreationInfoat the python level because I don't think all the options should be exposed to python users.How Has This Been Tested
Added a test to SimTest.cpp. Also tested locally in an example HITL app that lives outside Habitat-sim.
Types of changes
Checklist