Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor optional model properties #263

Open
wants to merge 29 commits into
base: main
Choose a base branch
from

Conversation

xiyuoh
Copy link
Member

@xiyuoh xiyuoh commented Jan 14, 2025

This PR targets #254 to refactor the current way of storing optional model description/instance data to file. The following changes aim to address the issues brought up in the ticket:

  • A unified struct Mobility to represent mobile robots and accommodates mobility types other than the provided DifferentialDrive. Mobility is now an optional ModelProperty for model descriptions, and users can opt to select DifferentialDrive or add their own plugins to introduce other kinds of mobility to the editor. Mobility data (kind and config), along with any other optional ModelProperty we may introduce for model descriptions, are stored to file under OptionalModelData, which is a serde_json::Value.
  • Task is no longer an enum that only supports default tasks (GoToPlace and WaitFor) in the site editor. It is similar to Mobility where users can create their own task kinds and add them to the editor using plugins. When creating a new Task, any registered Task type can be selected from the editor, and their own configuration widget will appear accordingly. Tasks are also stored to file under OptionalModelData as a serde_json::Value. To enable adding tasks to a model instance, users will need to toggle the Robot property button for its model description.
  • Updated the way we store ModelPropertyData - it initially used TypeId to identify each ModelProperty, but it is not compatible with serde_json. I've since changed it to ComponentId instead.

Mobility configuration:

image

Task configuration:

image

Some considerations as I was working on the refactor:

  • I still chose to store both Mobility and Tasks within the renamed OptionalModelData component (hopefully the naming is more suitable than OptionalModelProperties) instead of creating separate components for model descriptions/instances. Making the switching from a HashMap to serde_json::Value should help with increasing configurability for either case. Mobility is added for model descriptions and Tasks for model instances, like before.
  • I intentionally didn't add Mobility or Tasks to the Model Description/Instance bundles respectively. To make sure that we don't have an unnecessary component required for non-mobile and non-robot models.
  • I updated MobileRobotMarker to RobotMarker and retained it as a separate ModelProperty from Mobility. This is for future use cases where we may support non-mobile robots that are also capable of performing tasks. The RobotMarker property is used to allow task creation for model instances affiliated to the selected description.
  • I didn't add in Task schedule / integration with scenarios as this PR is getting a little bloated already, happy to refine this refactor and bring in task schedules after.

xiyuoh added 11 commits January 9, 2025 10:07
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Insert RobotMarker if instance has Tasks

Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
@xiyuoh xiyuoh requested a review from mxgrey January 14, 2025 06:24
Copy link
Collaborator

@mxgrey mxgrey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is moving things in a really good direction with the extensibility it gives. But we need to make sure to find the right balance between flexibility and explicitness.

I've left some inline comments to recommend changes that I think will help lean the balance back towards more explicitness.

rmf_site_format/src/task.rs Outdated Show resolved Hide resolved
rmf_site_format/src/model.rs Outdated Show resolved Hide resolved
@@ -163,7 +119,8 @@ pub struct ModelInstance<T: RefTrait> {
pub marker: ModelMarker,
#[serde(skip)]
pub instance_marker: InstanceMarker,
pub optional_properties: OptionalModelProperties<T>,
#[serde(default, skip_serializing_if = "is_default")]
pub optional_data: OptionalModelData,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think shoving Tasks into this struct makes much sense for the long-term vision here.

  1. It implies that users are expected to assign specific tasks to specific robots in a specific order, but Open-RMF should have a task dispatching system.
  2. Tasks would be a good thing to be accounted for in scenarios.

Apologies for complicating the PR, but I would recommend the following:

We change Task to be

pub enum Task {
    Dispatch(DispatchTaskRequest),
    Direct(RobotTaskRequest),
}

DispatchTaskRequest can be based on the dispatch_task_request schema and RobotTaskRequest can be based on the robot_task_request schema.

Each would internally contain a request: TaskRequest.

In the Site struct we would have

pub struct Site {
    ...
    pub tasks: BTreeMap<u32, Task>,
}

and then in Scenarios:

pub struct Scenarios<T> {
    ...
    pub added_tasks: Vec<T>,
    pub removed_tasks: Vec<T>,
}

Admittedly this will complicate things for the experimental mapf extension since we'll need to introduce a task dispatcher before we can generate the motion plans, but I think this will make the most sense for the site editor in the long-term. We can follow-up this PR with a very rudimentary prototype task dispatcher that just assigns tasks to suitable robot instances in a round-robin.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just noticed in your PR description

I didn't add in Task schedule / integration with scenarios as this PR is getting a little bloated already, happy to refine this refactor and bring in task schedules after.

If you want to save this change for a follow-up PR that should be okay.

rmf_site_editor/src/interaction/model.rs Outdated Show resolved Hide resolved
rmf_site_format/src/mobility.rs Outdated Show resolved Hide resolved
xiyuoh and others added 6 commits January 17, 2025 11:13
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Michael X. Grey <greyxmike@gmail.com>
Signed-off-by: Michael X. Grey <grey@openrobotics.org>
Signed-off-by: Michael X. Grey <grey@openrobotics.org>
over

Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Fix Vec2 and Mobility not updating correctly

Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
@xiyuoh
Copy link
Member Author

xiyuoh commented Jan 23, 2025

Thanks @mxgrey for the comments! After another round of iteration here are the main changes with the suggestions:

  • RobotMarker + OptionalModelData combined as Robot struct with properties. Robot is also its own site data instead of being a field under Model Descriptions
  • Collision separated out from Mobility as its own Robot property
  • Tasks no longer stored as OptionalModelData, and instead are stored as their own site data as well
  • Removed OptionalModelData entirely, brought in changes from Make the inner bundle of Parented transparent #264 for a flattened hierarchy of Parented bundles. serde(transparent) no longer necessary.

@xiyuoh xiyuoh requested a review from mxgrey January 23, 2025 02:26
Copy link
Collaborator

@mxgrey mxgrey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This implementation is looking a lot more clean, very nice!

I just left a few more small nitpicks that should be easy to get through, and then a few longer term remarks to think about for a future PR.

Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
* Make robot property kinds components

Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>

* Add plugin system to handle deserializing and updating components

Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>

* Rename back to RobotPropertyData

Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>

* Review comments

Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>

* Do not use Component in rmf_site_format

Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>

* Move RobotProperty and RobotPropertyKind to rmf_site_editor

Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>

* Use type alias for model property query

Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>

* Amend struct name and add property/kinds registration structs

Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>

* Remove unused insert/remove fn

Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>

* Insert default kind value when selecting kind

Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>

* Amend save robot properties

Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>

* Add diagnostic for missing RobotPropertyKind

Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>

* Use Deref/DerefMut

Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>

---------

Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
@xiyuoh xiyuoh force-pushed the xiyu/optional_model_properties branch from e1ee9ea to 96e745e Compare February 14, 2025 02:19
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
@xiyuoh
Copy link
Member Author

xiyuoh commented Feb 14, 2025

Hi @mxgrey ! I've added the RecallPlugin implementation in 0e1a5c3 since changes are pretty minimal. This approach makes it mandatory to implement the RecallPlugin for any RobotProperties added using InspectRobotPropertyPlugin. Another thing to note is that Recall does not need to be implemented for any RobotPropertyKinds, as changes are already tracked via the Property's serialized values, and recalled Kind components will be inserted automatically through update_robot_property_kind_components.

Signed-off-by: Michael X. Grey <greyxmike@gmail.com>
@mxgrey
Copy link
Collaborator

mxgrey commented Feb 14, 2025

Wow thanks for already incorporating the recall plugin!

While playing around with the recall feature I noticed some odd stuttering while toggling the property on/off. I'm not sure if it was being caused by race conditions with the recall system or simply by egui resizing the widgets, but I think 2bcdfa3 reduced the severity of the stuttering. But even if that change has no effect, it's not bad to give a way to order systems around the recall updates.

Another thing to note is that Recall does not need to be implemented for any RobotPropertyKinds, as changes are already tracked via the Property's serialized values

The catch here is that we won't be able to remember previous values for each kind as we switch between the kinds within one Property. In fact when I played with it just now, re-selecting Differential Drive from the drop-down menu when it was already selected will reset the current values in the config to their default values, which I think is really not what we want.

I don't mind fixing that in a follow-up PR, but I do think this is something we should change eventually.

@xiyuoh
Copy link
Member Author

xiyuoh commented Feb 14, 2025

I see! Thanks for testing and catching that, I'll follow up on implementing RecallPlugin for Property Kinds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Review
Development

Successfully merging this pull request may close these issues.

2 participants