Skip to content

Commit

Permalink
Rename format module names as amoro-format-xxx
Browse files Browse the repository at this point in the history
  • Loading branch information
majin.nathan committed Sep 4, 2024
1 parent ef274b1 commit 5be4553
Show file tree
Hide file tree
Showing 950 changed files with 159 additions and 159 deletions.
8 changes: 4 additions & 4 deletions .github/labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,13 +54,13 @@
- "amoro-optimizer/**/*"

"module:mixed-spark":
- "amoro-mixed-format/amoro-mixed-format-spark/**/*"
- "amoro-format-mixed/amoro-format-mixed-spark/**/*"

"module:mixed-flink":
- "amoro-mixed-format/amoro-mixed-format-flink/**/*"
- "amoro-format-mixed/amoro-format-mixed-flink/**/*"

"module:mixed-hive":
- "amoro-mixed-format/amoro-mixed-format-hive/**/*"
- "amoro-format-mixed/amoro-format-mixed-hive/**/*"

"module:mixed-trino":
- "amoro-mixed-format/amoro-mixed-format-trino/**/*"
- "amoro-format-mixed/amoro-format-mixed-trino/**/*"
6 changes: 3 additions & 3 deletions .github/workflows/core-hadoop2-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ on:
- ".github/workflows/**"
- "amoro-ams/**"
- "amoro-core/**"
- "amoro-mixed-format/amoro-mixed-format-flink/**"
- "amoro-mixed-format/amoro-mixed-format-hive/**"
- "amoro-mixed-format/amoro-mixed-format-spark/**"
- "amoro-format-mixed/amoro-format-mixed-flink/**"
- "amoro-format-mixed/amoro-format-mixed-hive/**"
- "amoro-format-mixed/amoro-format-mixed-spark/**"
- "pom.xml"
- "tools/maven/**"

Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/core-hadoop3-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ on:
- ".github/workflows/**"
- "amoro-ams/**"
- "amoro-core/**"
- "amoro-mixed-format/amoro-mixed-format-flink/**"
- "amoro-mixed-format/amoro-mixed-format-hive/**"
- "amoro-mixed-format/amoro-mixed-format-spark/**"
- "amoro-format-mixed/amoro-format-mixed-flink/**"
- "amoro-format-mixed/amoro-format-mixed-hive/**"
- "amoro-format-mixed/amoro-format-mixed-spark/**"
- "pom.xml"
- "tools/maven/**"

Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/trino-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ on:
- ".github/workflows/**"
- "amoro-ams/**"
- "amoro-core/**"
- "amoro-mixed-format/amoro-mixed-format-hive/**"
- "amoro-mixed-format/amoro-mixed-format-trino/**"
- "amoro-format-mixed/amoro-format-mixed-hive/**"
- "amoro-format-mixed/amoro-format-mixed-trino/**"
- "pom.xml"
- "tools/maven/**"

Expand All @@ -42,7 +42,7 @@ jobs:
run: mvn validate -P format-mixed-format-trino

- name: Build trino module with Maven
run: mvn clean install -pl 'amoro-mixed-format/amoro-mixed-format-trino' -am -B -P hadoop2,format-mixed-format-trino,build-mixed-format-trino
run: mvn clean install -pl 'amoro-format-mixed/amoro-format-mixed-trino' -am -B -P hadoop2,format-mixed-format-trino,build-mixed-format-trino

- name: Code coverage
uses: codecov/codecov-action@v3
Expand Down
22 changes: 11 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,29 +99,29 @@ Amoro contains modules as below:
- `amoro-ams` is amoro management service module
- `amoro-web` is the dashboard frontend for ams
- `amoro-optimizer` provides default optimizer implementation
- `amoro-iceberg-format` contains integration of Apache Iceberg format
- `amoro-hudi-format` contains integration of Apache Hudi format
- `amoro-paimon-format` contains integration of Apache Paimon format
- `amoro-mixed-format` provides Mixed format implementation
- `amoro-mixed-format-hive` integrates with Apache Hive and implements Mixed Hive format
- `amoro-mixed-format-flink` provides Flink connectors for Mixed format tables (use amoro-flink-runtime for a shaded version)
- `amoro-mixed-format-spark` provides Spark connectors for Mixed format tables (use amoro-spark-runtime for a shaded version)
- `amoro-mixed-format-trino` provides Trino connectors for Mixed format tables
- `amoro-format-iceberg` contains integration of Apache Iceberg format
- `amoro-format-hudi` contains integration of Apache Hudi format
- `amoro-format-paimon` contains integration of Apache Paimon format
- `amoro-format-mixed` provides Mixed format implementation
- `amoro-format-mixed-hive` integrates with Apache Hive and implements Mixed Hive format
- `amoro-format-mixed-flink` provides Flink connectors for Mixed format tables (use amoro-flink-runtime for a shaded version)
- `amoro-format-mixed-spark` provides Spark connectors for Mixed format tables (use amoro-spark-runtime for a shaded version)
- `amoro-format-mixed-trino` provides Trino connectors for Mixed format tables


## Building

Amoro is built using Maven with JDK 8 and JDK 17(only for `amoro-mixed-format/amoro-mixed-format-trino` module).
Amoro is built using Maven with JDK 8 and JDK 17(only for `amoro-format-mixed/amoro-format-mixed-trino` module).

* Build all modules without `amoro-mixed-format-trino`: `mvn clean package`
* Build all modules without `amoro-format-mixed-trino`: `mvn clean package`
* Build and skip tests: `mvn clean package -DskipTests`
* Build and skip dashboard: `mvn clean package -Pskip-dashboard-build`
* Build and disable disk storage, RocksDB will NOT be introduced to avoid memory overflow: `mvn clean package -DskipTests -Pno-extented-disk-storage`
* Build with hadoop 2.x(the default is 3.x) dependencies: `mvn clean package -DskipTests -Phadoop2`
* Specify Flink version for Flink optimizer(the default is 1.20.0): `mvn clean package -DskipTests -Dflink-optimizer.flink-version=1.20.0`
* If the version of Flink is below 1.15.0, you also need to add the `-Pflink-optimizer-pre-1.15` parameter: `mvn clean package -DskipTests -Pflink-optimizer-pre-1.15 -Dflink-optimizer.flink-version=1.14.6`
* Specify Spark version for Spark optimizer(the default is 3.3.3): `mvn clean package -DskipTests -Dspark-optimizer.spark-version=3.3.3`
* Build `amoro-mixed-format-trino` module under JDK 17: `mvn clean package -DskipTests -Pformat-mixed-format-trino,build-mixed-format-trino -pl 'amoro-mixed-format/amoro-mixed-format-trino' -am`.
* Build `amoro-format-mixed-trino` module under JDK 17: `mvn clean package -DskipTests -Pformat-mixed-format-trino,build-mixed-format-trino -pl 'amoro-format-mixed/amoro-format-mixed-trino' -am`.
* Build all modules: `mvn clean package -DskipTests -Ptoolchain,build-mixed-format-trino`, besides you need config `toolchains.xml` in `${user.home}/.m2/` dir with content below.
* Build a distribution package with all formats integrated: `mvn clean package -Psupport-all-formats`
* Build a distribution package with Apache Paimon format: `mvn clean package -Psupport-paimon-format`
Expand Down
22 changes: 11 additions & 11 deletions amoro-ams/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -41,12 +41,12 @@
<dependencies>
<dependency>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-iceberg-format</artifactId>
<artifactId>amoro-format-iceberg</artifactId>
</dependency>

<dependency>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-mixed-format-hive</artifactId>
<artifactId>amoro-mixed-hive</artifactId>
<exclusions>
<exclusion>
<groupId>org.eclipse.jetty.orbit</groupId>
Expand Down Expand Up @@ -325,7 +325,7 @@
<!-- runtime dependencies -->
<dependency>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-mixed-format-spark-${terminal.spark.major.version}</artifactId>
<artifactId>amoro-format-mixed-spark-${terminal.spark.major.version}</artifactId>
<version>${project.version}</version>
<scope>runtime</scope>
</dependency>
Expand All @@ -348,30 +348,30 @@

<dependency>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-iceberg-format</artifactId>
<artifactId>amoro-format-iceberg</artifactId>
<version>${project.version}</version>
<classifier>tests</classifier>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-mixed-format-hive</artifactId>
<artifactId>amoro-mixed-hive</artifactId>
<version>${project.version}</version>
<scope>test</scope>
<type>test-jar</type>
</dependency>

<dependency>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-paimon-format</artifactId>
<artifactId>amoro-format-paimon</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-paimon-format</artifactId>
<artifactId>amoro-format-paimon</artifactId>
<version>${project.version}</version>
<scope>test</scope>
<type>test-jar</type>
Expand Down Expand Up @@ -469,7 +469,7 @@
<dependencies>
<dependency>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-paimon-format</artifactId>
<artifactId>amoro-format-paimon</artifactId>
</dependency>
</dependencies>
</profile>
Expand All @@ -478,7 +478,7 @@
<dependencies>
<dependency>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-hudi-format</artifactId>
<artifactId>amoro-format-hudi</artifactId>
</dependency>
</dependencies>
</profile>
Expand All @@ -487,11 +487,11 @@
<dependencies>
<dependency>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-paimon-format</artifactId>
<artifactId>amoro-format-paimon</artifactId>
</dependency>
<dependency>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-hudi-format</artifactId>
<artifactId>amoro-format-hudi</artifactId>
</dependency>
</dependencies>
</profile>
Expand Down
2 changes: 1 addition & 1 deletion amoro-hudi-format/pom.xml → amoro-format-hudi/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
<relativePath>../pom.xml</relativePath>
</parent>

<artifactId>amoro-hudi-format</artifactId>
<artifactId>amoro-format-hudi</artifactId>
<name>Amoro Hudi Format Integration</name>

<properties>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
<version>0.8-SNAPSHOT</version>
</parent>

<artifactId>amoro-iceberg-format</artifactId>
<artifactId>amoro-format-iceberg</artifactId>
<name>Amoro Project Iceberg Format</name>
<url>https://amoro.apache.org</url>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-mixed-format-flink</artifactId>
<artifactId>amoro-mixed-flink</artifactId>
<version>0.8-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>

<artifactId>amoro-mixed-format-flink-common-format</artifactId>
<artifactId>amoro-format-mixed-flink-common-format</artifactId>
<name>Amoro Project Mixed Format Flink Common Format</name>
<url>https://amoro.apache.org</url>

Expand All @@ -37,7 +37,7 @@
<dependencies>
<dependency>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-iceberg-format</artifactId>
<artifactId>amoro-format-iceberg</artifactId>
<scope>provided</scope>
<exclusions>
<exclusion>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-mixed-format-flink</artifactId>
<artifactId>amoro-mixed-flink</artifactId>
<version>0.8-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>

<artifactId>amoro-mixed-format-flink-common-iceberg-bridge</artifactId>
<artifactId>amoro-format-mixed-flink-common-iceberg-bridge</artifactId>
<name>Amoro Project Mixed Format Flink Iceberg Bridge</name>
<url>https://amoro.apache.org</url>

Expand All @@ -41,7 +41,7 @@
<!--amoro-->
<dependency>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-iceberg-format</artifactId>
<artifactId>amoro-format-iceberg</artifactId>
<exclusions>
<exclusion>
<groupId>org.ow2.asm</groupId>
Expand All @@ -52,7 +52,7 @@

<dependency>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-mixed-format-hive</artifactId>
<artifactId>amoro-mixed-hive</artifactId>
<version>${project.version}</version>
</dependency>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,12 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-mixed-format-flink</artifactId>
<artifactId>amoro-mixed-flink</artifactId>
<version>0.8-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>

<artifactId>amoro-mixed-format-flink-common</artifactId>
<artifactId>amoro-mixed-flink-common</artifactId>
<name>Amoro Project Mixed Format Flink Common</name>
<url>https://amoro.apache.org</url>

Expand All @@ -44,7 +44,7 @@
<!--amoro-->
<dependency>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-iceberg-format</artifactId>
<artifactId>amoro-format-iceberg</artifactId>
<exclusions>
<exclusion>
<groupId>org.ow2.asm</groupId>
Expand All @@ -55,12 +55,12 @@

<dependency>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-mixed-format-hive</artifactId>
<artifactId>amoro-mixed-hive</artifactId>
</dependency>

<dependency>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-mixed-format-flink-common-iceberg-bridge</artifactId>
<artifactId>amoro-format-mixed-flink-common-iceberg-bridge</artifactId>
<version>${project.version}</version>
</dependency>

Expand Down Expand Up @@ -93,7 +93,7 @@

<dependency>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-mixed-format-flink-common-format</artifactId>
<artifactId>amoro-format-mixed-flink-common-format</artifactId>
<version>${project.parent.version}</version>
</dependency>

Expand Down Expand Up @@ -320,31 +320,31 @@

<dependency>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-iceberg-format</artifactId>
<artifactId>amoro-format-iceberg</artifactId>
<version>${project.version}</version>
<type>test-jar</type>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-mixed-format-hive</artifactId>
<artifactId>amoro-mixed-hive</artifactId>
<version>${project.version}</version>
<classifier>tests</classifier>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-paimon-format</artifactId>
<artifactId>amoro-format-paimon</artifactId>
<version>${project.version}</version>
<classifier>tests</classifier>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.apache.amoro</groupId>
<artifactId>amoro-paimon-format</artifactId>
<artifactId>amoro-format-paimon</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>
Expand Down
Loading

0 comments on commit 5be4553

Please sign in to comment.