title | summary | category |
---|---|---|
DM Portal Overview |
Learn the features of DM Portal and how to use it. |
reference |
Data Migration (DM) provides a variety of features, including table routing, black & white table lists, and binlog event filter. However, these features also increase the complexity of using DM, especially when users are modifying DM task configurations.
To address this problem, DM provides a simple web application, DM Portal. DM Portal enables users to visually configure the required replication tasks, and generates a task.yaml
file that can be directly executed by DM.
This sections describes the features of DM Portal.
DM Portal supports three replication modes:
- Full replication
- Incremental replication
- All (full + incremental)
DM Portal supports configuring table routing, and merging sharded schemas and tables in DM.
DM Portal supports configuring the binlog event filter in schemas and tables.
DM Portal supports generating configuration files and downloading these files to your local computer. Meanwhile, it automatically creates a file in the /tmp/
directory on the dm-portal server.
Currently, DM Portal's visualized pages cover most DM configuration scenarios, but with the following restrictions:
-
The SQL pattern of binlog event filter is not supported.
-
The editing feature does not support parsing the
task.yaml
file created by the user. The user can only edit thetask.yaml
file generated by the page. -
The editing feature does not support modifying the instance configuration. If the user need to adjust the instance configuration, the
task.yaml
file has to be regenerated. -
The upstream instance configuration on the page can only be used to obtain the upstream table schema. The related upstream instance information still needs to be configured in DM-worker.
This section describes how to deploy DM Portal in two ways: using binary or DM Ansible.
Download DM Portal at dm-portal-latest-linux-amd64.tar.gz. To start DM Portal, run the ./dm-portal
command.
- If you run DM Portal locally, visit
127.0.0.1:8280
in your browser. - If you run DM Portal on a server, configure a proxy on the server.
To deploy DM Portal using DM Ansible, refer to Deploy Data Migration Using DM-Ansible for details.
This section describes how to use DM Portal.
This feature is used to create a task.yaml
file.
Access the dm-portal page, and click Create New Rule.
This feature is used to fill in the task name and select a task type.
Create New Sync Rule is already selected.
- Fill in the task name.
- Choose a task type.
This feature is used to configure the upstream and downstream instance information, including Host, Port, Username, and Password.
Task Name and Task Type are already filled in.
Note:
If you choose Incremental or All in Task Type, you need to configure binlog-file and binlog-pos when configuring the upstream instance information.
- Fill in the upstream instance information.
- Fill in the downstream instance information.
- Click Next.
This feature is used to filter the upstream binlog. You can choose the DDL or DML that needs to be filtered. The filter configured on the database is automatically inherited by tables in that database.
- The upstream and downstream instance information is configured.
- The connection is verified.
Note:
- The binlog filter configuration can only be modified in the upstream instance. Once the database or table is moved to the downstream instance, the configuration cannot be modified.
- The binlog filter configured on the database is automatically inherited by tables in that database.
-
Select the databases or tables that need to be configured.
-
Click the Edit button, and select the binlog types to be filtered.
This feature is used to perform the following operations:
- Select the databases and tables that need to be synced, modify their names, and merge databases and tables
- Revert the last operation
- Reset all configurations of table routing
After the task configuration is completed, DM Portal generates the corresponding task.yaml
file.
The required binlog filter rules are configured.
Note:
- Batch operation is not supported when you merge databases and tables. You can only drag them one by one.
- You can only drag tables when you merge databases and tables. You cannot drag databases.
-
Select the databases and tables that need to be synced from Upstream Instance.
-
Click the Move button and move the selected databases and tables to Downstream Instance.
-
Right click the databases and tables to rename them.
-
Select the required table to perform the following operation:
-
Click Go Back to undo the last operation.
-
Click Reset to clear the downstream instance.
-
Click Finish & Download. DM Portal automatically downloads the
task.yaml
file to the local computer, and creates atask.yaml
configuration file in the/tmp/
directory on the DM Portal server.