Skip to content

Commit

Permalink
address some comments
Browse files Browse the repository at this point in the history
  • Loading branch information
liangyepianzhou committed Jul 17, 2023
1 parent 886c1bb commit eaddb18
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 12 deletions.
5 changes: 2 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,9 +195,8 @@ similar to the following, and then run `go mod tidy`.
```
github.com/apache/pulsar-client-go/oauth2 v0.0.0-20220630195735-e95cf0633348 // indirect
```
### Schema compatibility with JAVA client
### Schema compatibility with Java client
Due to the differences in programming language features and data representation,
schema incompatibility may arise between different languages.
We have conducted an analysis of the schema compatibility between Java and Go,
and we have provided some feasible solutions.
The analysis of the schema compatibility between Java and Go has been conducted and the corresponding solutions have been listed below.
You can get more details in [schema-compatibility](docs/schema-compatibility.md).
18 changes: 9 additions & 9 deletions docs/schema-compatibility.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
### Description
Pulsar is a high-performance, persistent message middleware that supports multiple programming languages ​​and data models. This article will focus on the comparison between Pulsar's GO Client and JAVA Client in Schema compatibility. We'll focus on four common schema types: Avro, JSON, Proto, and Proto native schema.
Pulsar is a high-performance, persistent message middleware that supports multiple programming languages and data models. This article focuses on the comparison between Pulsar's Go Client and Java Client in schema compatibility. It focuses on four common schema types: Avro, JSON, Proto, and Proto native schema.
#### Avro Schema
Avro schema in Go and Java are compatible, but there are some differences in how the schemas are defined. Go typically uses schema definitions, a string Json, to create schemas, whereas Java often uses class types for schema creation. As a result, Java allows non-primitive fields to be nullable by default, while in Go's schema definition, the nullability of fields needs to be explicitly stated.
Avro schema in Go and Java are compatible, but there are some differences in how the schemas are defined. Go typically uses schema definitions, a string JSON, to create schemas. However, Java often uses class types for schema creation. As a result, Java allows non-primitive fields to be nullable by default, while in Go's schema definition, the nullability of fields needs to be explicitly stated.
GO:
```go
// Compatible with Java schema define
// Compatible with defining a schema in Java
exampleSchemaDefCompatible := NewAvroSchema(`{"fields":
[
{"name":"id","type":"int"},{"default":null,"name":"name","type":["null","string"]}
],
"name":"MyAvro","namespace":"schemaNotFoundTestCase","type":"record"}`, nil)
// Not compatible with Java schema define
// Not compatible with defining a schema in Java
exampleSchemaDefIncompatible := NewAvroSchema(`{"fields":
[
{"name":"id","type":"int"},{"default":null,"name":"name","type":["string"]}
Expand All @@ -35,12 +35,12 @@ Producer<Example> producer = pulsarClient.newProducer(Schema.AVRO(Example.class)
#### JSON Schema
The situation with JSON schema is similar to Avro Schema.
```go
// Compatible with Java schema define
// Compatible with defining a schema in Java
exampleSchemaDefCompatible := "{\"type\":\"record\",\"name\":\"Example\",\"namespace\":\"test\"," +
"\"fields\":[{\"name\":\"ID\",\"type\":\"int\"},{\"name\":\"Name\",\"type\":[\"null\", \"string\"]}]}"

consumerJSCompatible := NewJSONSchema(exampleSchemaDefCompatible, nil)
// Not compatible with Java schema define
// Not compatible with defining a schema in Java
exampleSchemaDefIncompatible := "{\"type\":\"record\",\"name\":\"Example\",\"namespace\":\"test\"," +
"\"fields\":[{\"name\":\"ID\",\"type\":\"int\"},{\"name\":\"Name\",\"type\":\"string\"}]}"

Expand All @@ -59,19 +59,19 @@ message TestMessage {
}
```

Schema define in java,which can be parsed by a Class.
Defining a schema in Java can be parsed by a class.
```json
protoSchemaDef = "{\"type\":\"record\",\"name\":\"TestMessage\",\"namespace\":\"org.apache.pulsar.client.api.schema.proto.Test\",\"fields\":[{\"name\":\"stringField\",\"type\":{\"type\":\"string\",\"avro.java.string\":\"String\"},\"default\":\"\"},{\"name\":\"intField\",\"type\":\"int\",\"default\":0}]}"

```

Schema define in GO, which need to be manually written.
Defining a schema in Go needs to write manually.
```json
protoSchemaDef = "{\"type\":\"record\",\"name\":\"Example\",\"namespace\":\"test\"," +
"\"fields\":[{\"name\":\"num\",\"type\":\"int\"},{\"name\":\"msf\",\"type\":\"string\"}]}"
```
To address the incompatibility between Proto and ProtoNative types, you can follow this approach:
1. In the Java client, parse the message using the Avro Proto library to obtain the Schema Definition.
1. In the Java client, parse the message using the Avro Proto library to obtain the schema definition.
2. Use this obtained schema definition in the Go client to ensure both clients use the same schema definition.
```json
protoSchemaDef = "{\"type\":\"record\",\"name\":\"TestMessage\",\"namespace\":\"org.apache.pulsar.client.api.schema.proto.Test\",\"fields\":[{\"name\":\"stringField\",\"type\":{\"type\":\"string\",\"avro.java.string\":\"String\"},\"default\":\"\"},{\"name\":\"intField\",\"type\":\"int\",\"default\":0}]}"
Expand Down

0 comments on commit eaddb18

Please sign in to comment.