Skip to content

Commit 3f67e32

Browse files
jfallowsjfallows
committed
Merge branch 'release/0.9.97'
2 parents e70a261 + 91dfe54 commit 3f67e32

File tree

178 files changed

+2269
-359
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

178 files changed

+2269
-359
lines changed

‎CHANGELOG.md

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,23 @@
22

33
## [Unreleased](https://github.com/aklivity/zilla/tree/HEAD)
44

5-
[Full Changelog](https://github.com/aklivity/zilla/compare/0.9.95...HEAD)
5+
[Full Changelog](https://github.com/aklivity/zilla/compare/0.9.96...HEAD)
6+
7+
**Implemented enhancements:**
8+
9+
- Support `jwt` guarded identity via custom token claim [\#1276](https://github.com/aklivity/zilla/issues/1276)
10+
- Support `insert into` to seed `kafka` messages via `risingwave` binding [\#1274](https://github.com/aklivity/zilla/issues/1274)
11+
12+
**Merged pull requests:**
13+
14+
- `pgsql` DROP TOPIC command to KafkaDeleteTopicsBeginEx plus catalog unregister subject [\#1280](https://github.com/aklivity/zilla/pull/1280) ([akrambek](https://github.com/akrambek))
15+
- external udf - python support [\#1278](https://github.com/aklivity/zilla/pull/1278) ([ankitk-me](https://github.com/ankitk-me))
16+
- Support jwt guarded identity via custom token claim [\#1277](https://github.com/aklivity/zilla/pull/1277) ([akrambek](https://github.com/akrambek))
17+
- Support insert into to seed kafka messages via risingwave binding [\#1275](https://github.com/aklivity/zilla/pull/1275) ([akrambek](https://github.com/akrambek))
18+
19+
## [0.9.96](https://github.com/aklivity/zilla/tree/0.9.96) (2024-10-01)
20+
21+
[Full Changelog](https://github.com/aklivity/zilla/compare/0.9.95...0.9.96)
622

723
**Implemented enhancements:**
824

‎build/flyweight-maven-plugin/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
<parent>
99
<groupId>io.aklivity.zilla</groupId>
1010
<artifactId>build</artifactId>
11-
<version>0.9.96</version>
11+
<version>0.9.97</version>
1212
<relativePath>../pom.xml</relativePath>
1313
</parent>
1414

‎build/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
<parent>
99
<groupId>io.aklivity.zilla</groupId>
1010
<artifactId>zilla</artifactId>
11-
<version>0.9.96</version>
11+
<version>0.9.97</version>
1212
<relativePath>../pom.xml</relativePath>
1313
</parent>
1414

‎cloud/docker-image/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
<parent>
99
<groupId>io.aklivity.zilla</groupId>
1010
<artifactId>cloud</artifactId>
11-
<version>0.9.96</version>
11+
<version>0.9.97</version>
1212
<relativePath>../pom.xml</relativePath>
1313
</parent>
1414

‎cloud/helm-chart/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
<parent>
99
<groupId>io.aklivity.zilla</groupId>
1010
<artifactId>cloud</artifactId>
11-
<version>0.9.96</version>
11+
<version>0.9.97</version>
1212
<relativePath>../pom.xml</relativePath>
1313
</parent>
1414

‎cloud/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
<parent>
99
<groupId>io.aklivity.zilla</groupId>
1010
<artifactId>zilla</artifactId>
11-
<version>0.9.96</version>
11+
<version>0.9.97</version>
1212
<relativePath>../pom.xml</relativePath>
1313
</parent>
1414

‎conf/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
<parent>
99
<groupId>io.aklivity.zilla</groupId>
1010
<artifactId>zilla</artifactId>
11-
<version>0.9.96</version>
11+
<version>0.9.97</version>
1212
<relativePath>../pom.xml</relativePath>
1313
</parent>
1414

‎incubator/binding-amqp.spec/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
<parent>
99
<groupId>io.aklivity.zilla</groupId>
1010
<artifactId>incubator</artifactId>
11-
<version>0.9.96</version>
11+
<version>0.9.97</version>
1212
<relativePath>../pom.xml</relativePath>
1313
</parent>
1414

‎incubator/binding-amqp/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
<parent>
99
<groupId>io.aklivity.zilla</groupId>
1010
<artifactId>incubator</artifactId>
11-
<version>0.9.96</version>
11+
<version>0.9.97</version>
1212
<relativePath>../pom.xml</relativePath>
1313
</parent>
1414

‎incubator/binding-pgsql-kafka.spec/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
<parent>
99
<groupId>io.aklivity.zilla</groupId>
1010
<artifactId>incubator</artifactId>
11-
<version>0.9.96</version>
11+
<version>0.9.97</version>
1212
<relativePath>../pom.xml</relativePath>
1313
</parent>
1414

‎incubator/binding-pgsql-kafka/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
<parent>
99
<groupId>io.aklivity.zilla</groupId>
1010
<artifactId>incubator</artifactId>
11-
<version>0.9.96</version>
11+
<version>0.9.97</version>
1212
<relativePath>../pom.xml</relativePath>
1313
</parent>
1414

‎incubator/binding-pgsql-kafka/src/main/java/io/aklivity/zilla/runtime/binding/pgsql/kafka/internal/stream/PgsqlKafkaCommandType.java

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@
2222
public enum PgsqlKafkaCommandType
2323
{
2424
CREATE_TOPIC_COMMAND("CREATE TOPIC".getBytes()),
25+
DROP_TOPIC_COMMAND("DROP TOPIC".getBytes()),
2526
UNKNOWN_COMMAND("UNKNOWN".getBytes());
2627

2728
private final byte[] value;

‎incubator/binding-pgsql-kafka/src/main/java/io/aklivity/zilla/runtime/binding/pgsql/kafka/internal/stream/PgsqlKafkaCompletionCommand.java

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
public enum PgsqlKafkaCompletionCommand
1818
{
1919
CREATE_TOPIC_COMMAND("CREATE_TOPIC".getBytes()),
20+
DROP_TOPIC_COMMAND("DROP_TOPIC".getBytes()),
2021
UNKNOWN_COMMAND("UNKNOWN".getBytes());
2122

2223
private final byte[] value;

‎incubator/binding-pgsql-kafka/src/main/java/io/aklivity/zilla/runtime/binding/pgsql/kafka/internal/stream/PgsqlKafkaProxyFactory.java

Lines changed: 117 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616

1717
import static io.aklivity.zilla.runtime.engine.buffer.BufferPool.NO_SLOT;
1818
import static io.aklivity.zilla.runtime.engine.catalog.CatalogHandler.NO_VERSION_ID;
19+
import static java.nio.charset.StandardCharsets.UTF_8;
1920
import static java.util.Objects.requireNonNull;
2021

2122
import java.io.InputStreamReader;
@@ -64,6 +65,7 @@
6465
import net.sf.jsqlparser.parser.CCJSqlParserManager;
6566
import net.sf.jsqlparser.statement.Statement;
6667
import net.sf.jsqlparser.statement.create.table.CreateTable;
68+
import net.sf.jsqlparser.statement.drop.Drop;
6769

6870
public final class PgsqlKafkaProxyFactory implements PgsqlKafkaStreamFactory
6971
{
@@ -143,6 +145,7 @@ public final class PgsqlKafkaProxyFactory implements PgsqlKafkaStreamFactory
143145
Object2ObjectHashMap<PgsqlKafkaCommandType, PgsqlDecoder> pgsqlDecoder =
144146
new Object2ObjectHashMap<>();
145147
pgsqlDecoder.put(PgsqlKafkaCommandType.CREATE_TOPIC_COMMAND, this::decodeCreateTopicCommand);
148+
pgsqlDecoder.put(PgsqlKafkaCommandType.DROP_TOPIC_COMMAND, this::decodeDropTopicCommand);
146149
pgsqlDecoder.put(PgsqlKafkaCommandType.UNKNOWN_COMMAND, this::decodeUnknownCommand);
147150
this.pgsqlDecoder = pgsqlDecoder;
148151
}
@@ -232,6 +235,7 @@ private final class PgsqlProxy
232235
private final String database;
233236
private final PgsqlKafkaBindingConfig binding;
234237
private final KafkaCreateTopicsProxy createTopicsProxy;
238+
private final KafkaDeleteTopicsProxy deleteTopicsProxy;
235239

236240
private final IntArrayQueue queries;
237241

@@ -281,6 +285,7 @@ private PgsqlProxy(
281285
this.queries = new IntArrayQueue();
282286

283287
this.createTopicsProxy = new KafkaCreateTopicsProxy(routedId, resolvedId, this);
288+
this.deleteTopicsProxy = new KafkaDeleteTopicsProxy(routedId, resolvedId, this);
284289
}
285290

286291
private void onAppMessage(
@@ -488,7 +493,7 @@ private void onCommandCompleted(
488493
doAppWindow(traceId, authorization);
489494
}
490495

491-
public void onKafkaCreateTopicsBegin(
496+
public void onKafkaBegin(
492497
long traceId,
493498
long authorization)
494499
{
@@ -1018,7 +1023,81 @@ protected void onKafkaBegin(
10181023

10191024
if (!errorExits)
10201025
{
1021-
delegate.onKafkaCreateTopicsBegin(traceId, authorization);
1026+
delegate.onKafkaBegin(traceId, authorization);
1027+
1028+
doKafkaWindow(traceId, authorization);
1029+
doKafkaEnd(traceId, authorization);
1030+
}
1031+
else
1032+
{
1033+
delegate.cleanup(traceId, authorization);
1034+
}
1035+
}
1036+
}
1037+
1038+
private final class KafkaDeleteTopicsProxy extends KafkaProxy
1039+
{
1040+
private KafkaDeleteTopicsProxy(
1041+
long originId,
1042+
long routedId,
1043+
PgsqlProxy delegate)
1044+
{
1045+
super(originId, routedId, delegate);
1046+
}
1047+
1048+
private void doKafkaBegin(
1049+
long traceId,
1050+
long authorization,
1051+
List<String> topics)
1052+
{
1053+
initialSeq = delegate.initialSeq;
1054+
initialAck = delegate.initialAck;
1055+
initialMax = delegate.initialMax;
1056+
state = PgsqlKafkaState.openingInitial(state);
1057+
1058+
final KafkaBeginExFW kafkaBeginEx =
1059+
kafkaBeginExRW.wrap(extBuffer, 0, extBuffer.capacity())
1060+
.typeId(kafkaTypeId)
1061+
.request(r -> r
1062+
.deleteTopics(c -> c
1063+
.names(ct ->
1064+
topics.forEach(t -> ct.item(i -> i.set(t, UTF_8))))
1065+
.timeout(config.kafkaTopicRequestTimeoutMs())))
1066+
.build();
1067+
1068+
kafka = newKafkaConsumer(this::onKafkaMessage, originId, routedId, initialId, initialSeq, initialAck, initialMax,
1069+
traceId, authorization, 0, kafkaBeginEx);
1070+
}
1071+
1072+
@Override
1073+
protected void onKafkaBegin(
1074+
BeginFW begin)
1075+
{
1076+
final long sequence = begin.sequence();
1077+
final long acknowledge = begin.acknowledge();
1078+
final long traceId = begin.traceId();
1079+
final long authorization = begin.authorization();
1080+
final OctetsFW extension = begin.extension();
1081+
1082+
assert acknowledge <= sequence;
1083+
assert sequence >= replySeq;
1084+
assert acknowledge >= replyAck;
1085+
1086+
replySeq = sequence;
1087+
replyAck = acknowledge;
1088+
state = PgsqlKafkaState.openingReply(state);
1089+
1090+
assert replyAck <= replySeq;
1091+
1092+
final ExtensionFW beginEx = extension.get(extensionRO::tryWrap);
1093+
final KafkaBeginExFW kafkaBeginEx =
1094+
beginEx != null && beginEx.typeId() == kafkaTypeId ? extension.get(kafkaBeginExRO::tryWrap) : null;
1095+
1096+
boolean errorExits = kafkaBeginEx.response().deleteTopics().topics().anyMatch(t -> t.error() != 0);
1097+
1098+
if (!errorExits)
1099+
{
1100+
delegate.onKafkaBegin(traceId, authorization);
10221101

10231102
doKafkaWindow(traceId, authorization);
10241103
doKafkaEnd(traceId, authorization);
@@ -1292,6 +1371,35 @@ else if (server.commandsProcessed == 0)
12921371
}
12931372
}
12941373

1374+
private void decodeDropTopicCommand(
1375+
PgsqlProxy server,
1376+
long traceId,
1377+
long authorization,
1378+
DirectBuffer buffer,
1379+
int offset,
1380+
int length)
1381+
{
1382+
if (server.commandsProcessed == 1)
1383+
{
1384+
server.onCommandCompleted(traceId, authorization, length, PgsqlKafkaCompletionCommand.DROP_TOPIC_COMMAND);
1385+
}
1386+
else if (server.commandsProcessed == 0)
1387+
{
1388+
final Drop drop = (Drop) parseStatement(buffer, offset, length);
1389+
final String topic = drop.getName().getName();
1390+
1391+
final PgsqlKafkaBindingConfig binding = server.binding;
1392+
final String subjectKey = String.format("%s.%s-key", server.database, topic);
1393+
final String subjectValue = String.format("%s.%s-value", server.database, topic);
1394+
1395+
binding.catalog.unregister(subjectKey);
1396+
binding.catalog.unregister(subjectValue);
1397+
1398+
final KafkaDeleteTopicsProxy deleteTopicsProxy = server.deleteTopicsProxy;
1399+
deleteTopicsProxy.doKafkaBegin(traceId, authorization, List.of("%s.%s".formatted(server.database, topic)));
1400+
}
1401+
}
1402+
12951403
private void decodeUnknownCommand(
12961404
PgsqlProxy server,
12971405
long traceId,
@@ -1351,6 +1459,13 @@ private Statement parseStatement(
13511459
sql = sql.replace("CREATE TOPIC", "CREATE TABLE");
13521460
statement = parserManager.parse(new StringReader(sql));
13531461
}
1462+
if (decodeCommandType(buffer, offset, length).
1463+
equals(PgsqlKafkaCommandType.DROP_TOPIC_COMMAND))
1464+
{
1465+
String sql = buffer.getStringWithoutLengthUtf8(offset, length);
1466+
sql = sql.replace("DROP TOPIC", "DROP TABLE");
1467+
statement = parserManager.parse(new StringReader(sql));
1468+
}
13541469
else
13551470
{
13561471
inputStream.wrap(buffer, offset, length);

‎incubator/binding-pgsql-kafka/src/test/java/io/aklivity/zilla/runtime/binding/pgsql/kafka/internal/stream/ProxyIT.java

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,4 +58,15 @@ public void shouldCreateTopic() throws Exception
5858
{
5959
k3po.finish();
6060
}
61+
62+
@Test
63+
@Configuration("proxy.yaml")
64+
@Specification({
65+
"${pgsql}/drop.topic/client",
66+
"${kafka}/drop.topic/server"
67+
})
68+
public void shouldDropTopic() throws Exception
69+
{
70+
k3po.finish();
71+
}
6172
}

‎incubator/binding-pgsql.spec/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
<parent>
99
<groupId>io.aklivity.zilla</groupId>
1010
<artifactId>incubator</artifactId>
11-
<version>0.9.96</version>
11+
<version>0.9.97</version>
1212
<relativePath>../pom.xml</relativePath>
1313
</parent>
1414

‎incubator/binding-pgsql/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
<parent>
99
<groupId>io.aklivity.zilla</groupId>
1010
<artifactId>incubator</artifactId>
11-
<version>0.9.96</version>
11+
<version>0.9.97</version>
1212
<relativePath>../pom.xml</relativePath>
1313
</parent>
1414

‎incubator/binding-risingwave.spec/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
<parent>
99
<groupId>io.aklivity.zilla</groupId>
1010
<artifactId>incubator</artifactId>
11-
<version>0.9.96</version>
11+
<version>0.9.97</version>
1212
<relativePath>../pom.xml</relativePath>
1313
</parent>
1414

‎incubator/binding-risingwave.spec/src/main/scripts/io/aklivity/zilla/specs/binding/risingwave/config/proxy.function.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,4 +22,6 @@ bindings:
2222
options:
2323
udf:
2424
- server: http://localhost:8815
25+
- server: http://localhost:8816
26+
language: python
2527
exit: app1

‎incubator/binding-risingwave.spec/src/main/scripts/io/aklivity/zilla/specs/binding/risingwave/schema/risingwave.schema.patch.json

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -80,13 +80,16 @@
8080
"title": "Language",
8181
"type": "string",
8282
"default": "java",
83-
"const": "java"
83+
"enum":
84+
[
85+
"java",
86+
"python"
87+
]
8488
}
8589
},
8690
"additionalProperties": false
8791
},
88-
"minItems": 1,
89-
"maxItems": 1
92+
"minItems": 1
9093
}
9194
},
9295
"additionalProperties": false

0 commit comments

Comments
 (0)