-
Notifications
You must be signed in to change notification settings - Fork 722
Description
Describe the subtask
Implementation Status
The Hologres JDBC Catalog has been implemented and is located under catalogs-contrib/catalog-jdbc-hologres. The implementation includes the following completed features:
Completed: Core Catalog Infrastructure
- HologresCatalog (
HologresCatalog.java): Main catalog class extendingJdbcCatalog, withshortName()returning"jdbc-hologres". Creates all necessary Hologres-specific converters and operations via factory methods (createExceptionConverter(),createJdbcTypeConverter(),createJdbcDatabaseOperations(),createJdbcTableOperations(),createJdbcColumnDefaultValueConverter()). - HologresCatalogOperations (
HologresCatalogOperations.java): ExtendsJdbcCatalogOperations, handles PostgreSQL JDBC driver deregistration onclose()to avoid memory leaks when the catalog is unloaded from IsolatedClassLoader. - HologresCatalogCapability (
HologresCatalogCapability.java): ImplementsCapability, defines naming rules following PostgreSQL conventions (regex:^[_a-zA-Z\p{L}/][\w\p{L}-$/=]{0,62}$) and reserves system schema names (pg_catalog,information_schema,hologres).
Completed: Schema (Database) Operations
- HologresSchemaOperations (
HologresSchemaOperations.java): ExtendsJdbcDatabaseOperations, implements schema CRUD operations.- Maps Hologres Database to JDBC Catalog, Hologres Schema to JDBC Schema.
- Filters out 10 system schemas:
pg_toast,pg_catalog,information_schema,hologres,hg_internal,hg_recyclebin,hologres_object_table,hologres_sample,hologres_streaming_mv,hologres_statistic. - Supports schema comments via
COMMENT ON SCHEMAsyntax. - Schema existence check via
schemaExists()usingDatabaseMetaData.getSchemas().
Completed: Table Operations
-
HologresTableOperations (
HologresTableOperations.java): ExtendsJdbcTableOperations, implementsRequireDatabaseOperation, provides full table lifecycle management.-
Table Type Support: Lists and loads
TABLE(regular tables and partition child tables) andPARTITIONED TABLE(partition parent tables) types. ExcludesVIEWandFOREIGN TABLEfrom listing. -
CREATE TABLE: Generates complete DDL with:
- Column definitions with data type, nullable constraints, and default values
WITHclause for Hologres-specific properties (orientation, clustering_key, etc.)- HASH distribution via
distribution_keyinWITHclause - PRIMARY KEY constraints
- Table and column comments via
COMMENT ON TABLE/COLUMNstatements - Single quote escaping in comments (e.g.,
'It''s a test')
-
Generated Column Support: Detects
UnparsedExpressiondefault values and generatesGENERATED ALWAYS AS (expr) STOREDsyntax instead ofDEFAULT. Correctly places theSTOREDkeyword before nullable constraints (NOT NULL/NULL). Distinguishes from auto-increment columns (GENERATED BY DEFAULT AS IDENTITY). -
Partition Table Support:
- Logical Partition Tables (Hologres V3.1+): Reads
is_logical_partitioned_tableandlogical_partition_columnsfromhologres.hg_table_properties. Creates viaLOGICAL PARTITION BY LIST(col1[, col2])syntax. Supports 1–2 partition columns. - Physical Partition Tables: Reads partition information from
pg_catalog.pg_partitioned_table(partition strategy and attribute numbers) andpg_catalog.pg_attribute(attribute number to column name resolution). Creates viaPARTITION BY LIST(column)syntax. Supports exactly 1 partition column.
- Logical Partition Tables (Hologres V3.1+): Reads
-
ALTER TABLE: Supports the following change types:
UpdateComment: Update table comment with Gravitino ID preservationAddColumn: Add column with type and comment only. Setting NOT NULL constraint, default value, and auto-increment are not supported via ALTER TABLE (throwsIllegalArgumentException).RenameColumn: Rename column viaRENAME COLUMNUpdateColumnDefaultValue: ThrowsIllegalArgumentException(Hologres does not support altering column default value via ALTER TABLE)UpdateColumnType: ThrowsIllegalArgumentException(Hologres does not support altering column type via ALTER TABLE)UpdateColumnComment: Update column comment viaCOMMENT ON COLUMNDeleteColumn: Drop column withIF EXISTSsupportUpdateColumnNullability: ThrowsIllegalArgumentException(Hologres does not support altering column nullability via ALTER TABLE)AddIndex: ThrowsIllegalArgumentException(Hologres does not support adding index via ALTER TABLE)DeleteIndex: ThrowsIllegalArgumentException(Hologres does not support deleting index via ALTER TABLE)UpdateColumnAutoIncrement: ThrowsIllegalArgumentException(Hologres does not support altering auto-increment via ALTER TABLE)UpdateColumnPosition: ThrowsIllegalArgumentException(Hologres does not support column reordering)SetProperty/RemoveProperty: ThrowsIllegalArgumentException(not yet supported)
-
Table Properties: Reads from
hologres.hg_table_propertiessystem table. Handles JDBC-to-DDL property key conversion (e.g.,binlog.level→binlog_level,binlog.ttl→binlog_ttl). Filters internal properties viaisUserRelevantProperty()and only exposes user-relevant ones. During CREATE TABLE, filtersdistribution_key,is_logical_partitioned_table, andprimary_keyfrom theWITHclause to avoid duplication. -
Distribution: Reads distribution key from
hologres.hg_table_propertiesand returnsDistributions.hash(). Writes distribution key via theWITHclause. Validates that only HASH distribution strategy is used and at least one column is specified. -
Auto-increment: Not supported. Creating auto-increment columns (e.g.,
GENERATED BY DEFAULT AS IDENTITY) is rejected withIllegalArgumentExceptionin both CREATE TABLE and ALTER TABLE ADD COLUMN. Hologres SERIAL/auto-increment tables that already exist can still be loaded and displayed, but cannot be created through Gravitino.
-
Completed: Type Conversion
-
HologresTypeConverter (
HologresTypeConverter.java): Bidirectional type mapping between Hologres and Gravitino types (185 lines).- Handles
TIMESTAMP/TIMESTAMPTZwithout precision suffix (Hologres does not support precision syntax liketimestamptz(6)). - Supports array types via
_prefixto[]suffix conversion (e.g.,_int4↔int4[]). - Maps unsupported types (
json,jsonb,uuid, etc.) toExternalType. - Array elements must be non-nullable (Hologres limitation).
- Multidimensional arrays are not supported.
- Handles
-
HologresColumnDefaultValueConverter (
HologresColumnDefaultValueConverter.java): Handles PostgreSQL-style default value casting (133 lines).toGravitino(): Parses literal default values with type cast patterns (e.g.,'2024-01-15'::date), handlesNULL::typecast expressions, returnsCURRENT_TIMESTAMPfor the corresponding PostgreSQL value, and falls back toUnparsedExpressionfor complex expressions (e.g., generated column expressions,nextval('seq')).fromGravitino(): Overridden to support convertingUnparsedExpressionback to SQL strings (for generated column round-trip), delegates to super for Literal and DEFAULT_VALUE_NOT_SET.
-
HologresExceptionConverter (
HologresExceptionConverter.java): Maps PostgreSQL SQLSTATE error codes to Gravitino exceptions (85 lines).42P04/42P06→SchemaAlreadyExistsException42P07→TableAlreadyExistsException3D000/3F000→NoSuchSchemaException42P01→NoSuchTableException08xxx→ConnectionFailedException- Password authentication failure →
ConnectionFailedException
Completed: Configuration and Build
- Build:
build.gradle.kts(112 lines) with dependencies oncatalog-jdbc-common,api,common,core, PostgreSQL driver for testing, testcontainers support, Guava, Commons Lang3/Collections4, and Log4j. - Config:
jdbc-hologres.confwith JDBC connection configuration template. - Service Registration:
META-INF/services/org.apache.gravitino.CatalogProviderfor catalog auto-discovery. - Distribution: Build tasks (
copyCatalogLibs,copyCatalogConfig) to package catalog intodistribution/package/catalogs/jdbc-hologres/.
Completed: Unit Tests
5 test classes with comprehensive coverage:
-
TestHologresTypeConverter: Covers all supported type conversions including boolean, integer types, float types, date/time types, decimal, varchar, char, text, binary, array types, external types, and error cases for nullable arrays and multidimensional arrays.
-
TestHologresColumnDefaultValueConverter: Covers
toGravitino()for all literal types (boolean, short, integer, long, float, double, decimal, varchar, text, bpchar, date, time, timestamp), expression fallback toUnparsedExpression,NULL::typehandling,CURRENT_TIMESTAMP, null default value (nullable vs non-nullable), unknown type fallback, andfromGravitino()forUnparsedExpression,Literal, andDEFAULT_VALUE_NOT_SET. -
TestHologresExceptionConverter: Covers all SQLSTATE error code mappings (42P04, 42P06, 42P07, 3D000, 3F000, 42P01, 08xxx), null SQL state handling, and password authentication failure detection.
-
TestHologresCatalogCapability: Covers naming validation patterns (valid names with letters, underscores, digits, hyphens, dollar signs, slashes, Unicode), invalid names (starting with digit, empty, too long), max length validation, and reserved word enforcement for SCHEMA scope.
-
TestHologresTableOperations (830 lines, 35+ test cases): Covers:
- CREATE TABLE SQL generation: Basic, with comments, column comments, distribution, properties, property filtering for
distribution_key/is_logical_partitioned_table/primary_key, physical partition, logical partition (1 and 2 columns), auto-increment rejection, nullable columns, default values, multiple columns, single quote escaping in comments, full-featured table, generated columns (non-nullable and nullable). - RENAME TABLE, DROP TABLE, PURGE TABLE (unsupported exception).
- Partitioning SQL: Physical, logical (single/two columns), rejects non-list transform, rejects multiple columns for physical, rejects >2 columns for logical, rejects empty partitions, rejects multiple transforms, rejects nested field names.
- Index field string generation: Single column, multiple columns, rejects nested column.
- Datetime precision calculation: Timestamp, timestamptz, time, timetz, non-datetime type, negative scale.
Note: ALTER TABLE operations
UpdateColumnDefaultValue,UpdateColumnType,UpdateColumnNullability,AddIndex,DeleteIndex, andUpdateColumnAutoIncrementare all rejected withIllegalArgumentExceptioningenerateAlterTableSql(). CREATE TABLE with indexes and constraints is still supported. - CREATE TABLE SQL generation: Basic, with comments, column comments, distribution, properties, property filtering for