Skip to content

Dataprep Operations

Voulz edited this page Oct 25, 2021 · 6 revisions

Be aware:

The Dataprep Operations included in this plugin were made for my workflow bringing SketchUp files into Unreal. They should run properly on other files imported from other applications but I did not get an opportunity to test this. If you encounter any issue or if you have any feature request, please let me know via the Issues page.

Table of Content

Sample File Setup

To demonstrate the given Dataprep Operations, I made a simple Sketchup Model available here:

ArchVis_Dataprep_SketchupScene

Here is its hierarchy:

ArchVis_Dataprep_SketchupOutliner

Additional Screenshots

ArchVis_Dataprep_SketchupScene2

ArchVis_Dataprep_SketchupScene3

ArchVis_Dataprep_SketchupScene4

ArchVis_Dataprep_SketchupScene5

To recap, we have:

  • a component called Laura set with the option Always face camera;
  • a component called Rectangle + Cubes + Cylinder, containing a component called Cubes, two instances of a component called Cylinders (with different rotations and scales), and the direct geometry of a rectangle (not in a group so not displayed in the hierarchy);
  • the component Cubes contains the direct geometry of 2 cubes (not in groups);
  • the component Cylinders contains 2 instances of the same component Cylinder and a group called Square, each only containing geometry.

Here is how this file gets imported via Dataprep:

ArchVis_Dataprep_DPOutliner

Important to note:

  • there is no difference between the import of group or component as the concept is not present in Unreal Engine;
  • every group/component creates an empty Actor with a child StaticMeshActor containing its direct geometry;
  • the group Cubes, even though it only had direct geometry, also gets separated into an empty Actor and two StaticMeshActor, one for each separate "pack" of connected geometry.

Dataprep Operations

Branch Merge

The operation Branch Merge is located in the Operations > On Actor section. It collects the geometry from selected Actors and merge them into single mesh.

Before:

ArchVis_Dataprep_BranchMerge8.jpg

After:

ArchVis_Dataprep_BranchMerge9.jpg

The way this operation works is fundamentally different from the default merge operation. The node generates internally a hierarchy of the selected actors, separating them in multiple branches. Each branch has one root and all the descendants are part of the branch; in the image below for example, there are 2 branches.

The merge operation will merge each of these branches individually.

ArchVis_Dataprep_BranchMerge10_.jpg

Pivot Point Alignment

Creating branches before merging allows more thoughtful options for the location and alignment of the Pivot Point of the Merged Actor.

Branch Root Actor

The default is Branch Root Actor which gives the Pivot Point of each Merged Actor the location and orientation of the root of their respective branch. This option doesn't work with Merge Root Siblings turned on, in which case it internally falls back on the Context Origin option.

ArchVis_Dataprep_BranchMerge5.jpg

Context Origin

The second option is Context Origin where the Pivot Point of each Merged Actor is aligned with the Parent of the root of their respective branch.

ArchVis_Dataprep_BranchMerge6.jpg

World Origin

The last option is World Origin where the Pivot Point of each Merged Actor is simply aligned with the World Origin.

ArchVis_Dataprep_BranchMerge7.jpg

Merge Root Siblings

Original setup, 8 actors are matching the label Cylinder. Internally, the node sees 4 branches, each with an empty Actor which is the root, and attached to it a StaticMeshActor:

ArchVis_Dataprep_BranchMerge1_.jpg

Without Merge Root Siblings, each branch is merged individually (here, 4 new Actors are created):

ArchVis_Dataprep_BranchMerge2_.jpg

With Merge Root Siblings, if the root of some branches are siblings, they are merged together (only 2 new Actors are created):

ArchVis_Dataprep_BranchMerge3_.jpg

Create Debug Axis

As this node as multiple Pivot Location options, turning this option on is a quick way to assess the location and orientation of the Pivot Point after merging without creating a Debug Actor Location node.

ArchVis_Dataprep_BranchMerge4.jpg

Build Adjacency

The operation Build Adjacency is located in the Operations > On Mesh section. It allows the creation of adjacency mesh in the imported meshes.

ArchVis_Dataprep_BuildAdjacency1.jpg

When we try to apply a material with Tesselation and Crack Free Displacement (which is the case with Megascans Surfaces with Displacement turned on for example)

ArchVis_Dataprep_BuildAdjacency4_.jpg

we need the mesh to have a adjacency information built-in else it will not work:

ArchVis_Dataprep_BuildAdjacency3.jpg

ArchVis_Dataprep_BuildAdjacency2.jpg

Re-importing the same file with the Build Adjacency node fixes the issue:

ArchVis_Dataprep_BuildAdjacency5.jpg

Explode

The operation Explode is located in the Operations > On Actor Hierarchy section. It move the children of the selected empty Actor to its own parent, optionally removing the actor itself.

In this example, we selected the empty Actors labelled Cylinder and we exploded them, bringing their children StaticMeshActor Cylinder directly to their parent Cylinders. The empty Actors Cylinder were deleted as the option Delete Exploded Actor was checked.

ArchVis_Dataprep_Explode3.jpg

If we use the Explode node without any selection, it flattens the Hierarchy:

ArchVis_Dataprep_Explode1.jpg

and if we uncheck Delete Exploded Actor, the hierarchy is also flattened but the empty group Actors are kept:

ArchVis_Dataprep_Explode2.jpg

Face Camera

The operation Face Camera is located in the Operations > Behavior section. It makes the selected actor face the camera. It is intended to be used for SketchUp Components with the option 'Always face camera' turned on, so might not work as expected in other cases.

The node requires the name of the tag that defines which component needs to have this feature turned on.

ArchVis_Dataprep_FaceCamera1_.jpg

It should also be noted that even though you might want to have the behavior visible in the Dataprep window after Executing the Recipe, it only works in the Editor Viewport (after commiting or reimporting) and in Game. Here are 2 screenshots from the editor window:

ArchVis_Dataprep_FaceCamera2.jpg

ArchVis_Dataprep_FaceCamera3.jpg

If the actor stopped following the camera and you do not know why, make sure that the Realtime viewport is turned on else it will not work.

ArchVis_Dataprep_FaceCamera4_.jpg

Material Name

The filer Material Name is located in the Select By > String Array section. It select StaticMeshes (not actors) that are using a Material which name matches the filter. It is useful if you import a file and you want to find which StaticMeshes use a certain material.

ArchVis_Dataprep_MaterialName.jpg

Replace Camera

The operation Replace Camera is located in the Operations > On Actor section. It replaces any CineCameraActor with an ArchVisCineCameraActor.

Before:

ArchVis_Dataprep_ReplaceCamera1_.jpg

After:

ArchVis_Dataprep_ReplaceCamera2_.jpg

Be aware:

This node currently doesn't handle Views that were set with Parallel Projection (Ortho) or Two-Point Perspectives in Sketchup as the information about the view is not transfered in the Datasmith file.

Simplify Hierarchy

The operation Simplify Hierarchy is located in the Operations > On Actor Hierarchy section. It deals with the fact that groups were imported into an empty Actor and a separate StaticMeshActor holding its geometry, and this operation replaces the empty Actor with the StaticMeshActor in the Hierarchy.

Before:

ArchVis_Dataprep_SimplifyHierarchy1.jpg

After:

ArchVis_Dataprep_SimplifyHierarchy4.jpg

Note that in the Hierarchy:

  • the group StaticMeshActor Rectangle + Cubes + Cylinder ends up replacing the empty Actor of the same name.
  • because the option Merge Multiple Meshes was checked and because the group Cubes has two meshes to represent its content, they got merged into one new mesh that replaced the existing empty Actor.

For the operation to know which StaticMeshActor belongs to which empty Actor, it looks at the value of the Tag given into the option Tag Name. We can see that SketchUp groups get exported with a Tag called SU.GUID. which matches between the empty group Actor and the StaticMeshActor. If you end up using a file not brought from Sketchup, you will have to find the matching tag and input it into the option Tag Name.

ArchVis_Dataprep_SimplifyHierarchy1_.jpg

ArchVis_Dataprep_SimplifyHierarchy2_.jpg

Debug Operations

All the Debug Operations are located in the Operations > Debug section. These were made to help me debug the other custom operations on this Plugin. They shouldn't be useful to most of you but I left them as they might be to some of you.

The creation of these operations is due to the way Dataprep works: we cannot draw Debug Geometry via Dataprep without changing the Engine Code.

Debug Actor Location

Display an Axis at each selected Actor Location.

ArchVis_Dataprep_DebugActorLocation1.jpg

By default, the operation creates a hidden component (with the property IsVisualizationComponent set to true), but we can generate a separate Actor by setting the option to true:

ArchVis_Dataprep_DebugActorLocation2.jpg

Debug Actor Bounding Box

Display the BoundingBox of each selected Actor. It was created to provide ways to choose custom Pivot Locations when merging but I discontinuated it for now.

ArchVis_Dataprep_DebugActorBoundingBox1.jpg

With the option Use Accurate BoundingBox, the BoundingBox is recomputed from the exact vertices location instead of the easily available BoundingBox. This is therefore more accurate especially for rotated meshes, but this is a slower operation.

ArchVis_Dataprep_DebugActorBoundingBox2.jpg

With the option Use Aligned BoundingBox, the BoundingBox will be locally aligned instead of the default World Aligned one. Works best with Use Accurate BoundingBox also turned on.

ArchVis_Dataprep_DebugActorBoundingBox3.jpg

With the option Include Children, the BoundingBox will contain the selected Actor and all its Attached Children.

ArchVis_Dataprep_DebugActorBoundingBox4.jpg

Delete Debug Data

This Operation deletes any data that was created via the other Debug Operations (including the components and the DPDebugActors). Best to use at the end of the Dataprep recipe to not have to disable each operation individually while ensuring none will be comitted.

Before:

ArchVis_Dataprep_DebugDeleteData1.jpg

After:

ArchVis_Dataprep_DebugDeleteData2.jpg

Custom Operations

Most of the above operations inherit from the base class UDataprepArchVisEditingOperation (located in ArchVisTools\Source\ArchVisToolsEditor\Public\Dataprep) and you can too! This will give you access to the above functions and combine them directly in Blueprint or C++.

To do so in Blueprints, in the Dataprep Editor, click Add New > Create New Operator, save it and open it.

ArchVis_Dataprep_CustomOperation1_.jpg

In the Class Settings, change the Parent Class to DataprepArchVisEditingOperation.

ArchVis_Dataprep_CustomOperation2_.jpg

You should now have access to about 30 functions used internally by the above operations.

ArchVis_Dataprep_CustomOperation3_.jpg