Expose some EnTT functionality to Python

[arthurmoreno]

Hello everyone!

I'm working on a game that uses EnTT, and I initially started development in Python. Midway through, I decided to create a C++ core and gradually migrate some components to C++. The integration between Python and C++ is working quite well overall, but at times it feels tight and not very flexible.

Now that I'm developing and experimenting with more mechanics and features beyond the basics like rendering and physics, I'm aiming to create a more generic exposure of the EnTT registry. This approach would allow me to prototype ideas more quickly on the Python side—which is much faster for me—and later decide whether to migrate certain parts to C++ for performance reasons.

Here how I am doing now:

#ifndef PYREGISTRY_HPP
#define PYREGISTRY_HPP

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>

#include <entt/entt.hpp>
#include <string>
#include <unordered_map>
#include <vector>

#include "components/MovingComponent.hpp"
#include "components/PhysicsComponents.hpp"
#include "components/ItemsComponents.hpp"

namespace py = pybind11;

class PyRegistry {
   public:
    // Constructor that accepts a reference to an existing registry
    PyRegistry(entt::registry& reg) : registry(reg) {}

    // Delete copy constructor and copy assignment operator
    PyRegistry(const PyRegistry&) = delete;
    PyRegistry& operator=(const PyRegistry&) = delete;

    // Delete move constructor and move assignment operator
    PyRegistry(PyRegistry&&) = delete;
    PyRegistry& operator=(PyRegistry&&) = delete;

    // Entity management
    entt::entity create_entity() { return registry.create(); }

    void destroy_entity(entt::entity entity) { registry.destroy(entity); }

    // View entities with specific components
    py::list view(const std::vector<std::string>& component_names) {
        bool has_position = false, has_velocity = false, has_moving = false;

        for (const auto& name : component_names) {
            if (name == "Position") has_position = true;
            if (name == "Velocity") has_velocity = true;
            if (name == "MovingComponent") has_moving = true;
        }

        // Create a view based on the requested components
        if (has_position && has_velocity && has_moving) {
            auto view = registry.view<Position, Velocity, MovingComponent>();
            return entities_to_pylist(view);
        } else if (has_position && has_velocity) {
            auto view = registry.view<Position, Velocity>();
            return entities_to_pylist(view);
        } else if (has_position && has_moving) {
            auto view = registry.view<Position, MovingComponent>();
            return entities_to_pylist(view);
        } else if (has_velocity && has_moving) {
            auto view = registry.view<Velocity, MovingComponent>();
            return entities_to_pylist(view);
        } else if (has_position) {
            auto view = registry.view<Position>();
            return entities_to_pylist(view);
        } else if (has_velocity) {
            auto view = registry.view<Velocity>();
            return entities_to_pylist(view);
        } else if (has_moving) {
            auto view = registry.view<MovingComponent>();
            return entities_to_pylist(view);
        } else {
            // If no valid components specified, return empty list
            return py::list();
        }
    }

    // Get a component from an entity
    template <typename Component>
    Component* get(entt::entity entity) {
        if (registry.all_of<Component>(entity)) {
            return &registry.get<Component>(entity);
        }
        return nullptr;
    }

    // Set a component on an entity
    template <typename Component>
    void set(entt::entity entity, const Component& component) {
        registry.emplace_or_replace<Component>(entity, component);
    }

    // Get a component from an entity
    py::object get_component(uint32_t entity_id, const std::string& component_name) {
        entt::entity entity = entt::entity{entity_id};
        if (component_name == "Position") {
            Position* comp = get<Position>(entity);
            if (comp) return py::cast(*comp);
        }
        if (component_name == "Velocity") {
            Velocity* comp = get<Velocity>(entity);
            if (comp) return py::cast(*comp);
        }
        if (component_name == "MovingComponent") {
            MovingComponent* comp = get<MovingComponent>(entity);
            if (comp) return py::cast(*comp);
        }
        if (component_name == "Inventory") {
            Inventory* comp = get<Inventory>(entity);
            if (comp) return py::cast(*comp);
        }
        return py::none();
    }

    // Set a component on an entity
    void set_component(uint32_t entity_id, const std::string& component_name,
                       py::object component_obj) {
        entt::entity entity = entt::entity{entity_id};
        if (component_name == "Position") {
            Position comp = component_obj.cast<Position>();
            set<Position>(entity, comp);
        }
        if (component_name == "Velocity") {
            Velocity comp = component_obj.cast<Velocity>();
            set<Velocity>(entity, comp);
        }
        if (component_name == "MovingComponent") {
            MovingComponent comp = component_obj.cast<MovingComponent>();
            set<MovingComponent>(entity, comp);
        }
        if (component_name == "Inventory") {
            Inventory comp = component_obj.cast<Inventory>();
            set<Inventory>(entity, comp);
        }
    }

   private:
    entt::registry& registry;

    // Helper to convert entities to a Python list
    template <typename View>
    py::list entities_to_pylist(View view) {
        py::list entity_list;
        for (auto entity : view) {
            entity_list.append(static_cast<uint32_t>(entity));
        }
        return entity_list;
    }
};

#endif  // PYREGISTRY_HPP

I'm looking for advice on how to achieve a more flexible and generic integration between EnTT and Python, specifically regarding exposing the registry for prototyping. If anyone has experience with this or has tackled similar challenges, I'd love to hear your thoughts!

Thanks in advance, and I'm excited to discuss this with you all!

[skypjack]

Just throwing a random idea at you here.
If you designed a general purpose type, like a map of any types as it happens with some js engines, you could create multiple storage of the same type on the C++ side and use the name as a key with the registry.
This way you don't have to hardcode the C++ type and modify the PyRegistry when you add a new component to experiment with.
You sacrifice a bit of performance for sure with this runtime components but you can have much faster iterations and turn them into actual C++ types when you know that a runtime type deserves it.
Does it click?

[arthurmoreno]

The issue I guess with this approach is that I would not be able to interact with certain components in other systems. For example, right now the "PhysicsEngine" in game is entirely on a C++ side. And it runs everytime I call the game.update (which is invoked by python, but has all the logic in C++)

Just some other high level systems/events, like consuming item, attack, etc that are being coded in Python (before they become some sort of bottleneck for performance)

Since I'm relatively new to C++, I'm unsure about the complexity of implementing this approach in my current setup. Maybe with the Runtime reflection system or any other runtime facility that entt might have... If there is no other way, that's fine.

Also, could you point me those js engines that you are mentioning?

[skypjack]

The issue I guess with this approach is that I would not be able to interact with certain components in other systems.

I don't see why this should happen. You can freely mix actual C++ types with, let's say, fake types. As long as you know how to work with them, everything goes smooth.

Also, could you point me those js engines that you are mentioning?

I mean, literally every JS engine out there uses surrogate C++ types to map types in the JS land. V8 does it too, if you want the big one. 🤷‍♂️

As for the implementation, I'm not suggesting this but bear with me for a moment to have a grasp of what I mean.
Imagine an entt::dense_map<entt::id_type, entt::any>. Here you have your type already. You can map type members in the scripting land by (hashed) name and use the any type to store their values. Then, you can use this type with the registry to create the storage you need by (hashed) name, again:

using my_type = entt::dense_map<entt::id_type, entt::any>;
registry.emplace<my_type>(entt::hashed_string::value(component_name)).emplace(args);
registry.emplace<my_type>(entt::hashed_string::value(other_component_name)).emplace(args);

Finally, you can also construct view with them as you like:

entt::basic_view view{registry.storage<my_type>(entt::hashed_string::value(component_name)), registry.storage<my_type>(entt::hashed_string::value(other_name))};
for(auto [entt, comp, other_comp]: view) { ... }

Where comp and other_comp are of type my_type ofc, but you know how to work with them and what scrupting component they refer to, so you're done already.
Did I get the point across better this time? 🙂

[arthurmoreno]

Honestly I didn't see the solution yet. But at this point I guess is my lack of knowledge on both C++ and on EnTT (architecture and data structures). I will keep learning and will come back to this topic later, as it is not the main focus on my development right now.

Thank you so much for the ideas in this mean time, really appreciated.