Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate apps to Spring Functions Catalog #523

Merged
merged 3 commits into from
Mar 9, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
107 changes: 7 additions & 100 deletions README.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,114 +2,20 @@

In this repository, you will find a collection of components that can meet various data integration use cases and requirements.

The repository's primary focus is to provide a set of standalone Java functions that can be useful in the end-user applications as-is.

Besides, this repository builds on the Java functions to generate standalone Spring Cloud Stream applications that can run against Spring Cloud Stream's RabbitMQ or Apache Kafka binder implementations.
This repository builds on top of the https://github.com/spring-cloud/spring-functions-catalog[Spring Functions Catalog] and generates standalone Spring Cloud Stream applications that can run against Spring Cloud Stream's RabbitMQ or Apache Kafka binder implementations.
It is also possible to extend the generator to bundle the Java functions with the other supported binder implementations.

These applications can run standalone or as part of a data flow, such as the one orchestrated using Spring Cloud Data Flow.

=== Project Structure

The repository includes two major sections - `Functions` and `Applications`.
The former hosts the various Java functions, and the latter is for generating the standalone Spring Cloud Stream applications and hosting their related components.

The following are the various components of this repository.

* https://github.com/spring-cloud/stream-applications/tree/master/functions[Standalone Java Functions]
* https://github.com/spring-cloud/stream-applications/tree/master/applications/stream-applications-core[Common Core Components for Applications]
* https://github.com/spring-cloud/stream-applications/tree/master/applications[Spring Cloud Stream Applications]
* https://github.com/spring-cloud/stream-applications/tree/master/stream-applications-build[Build Parent]
* https://github.com/spring-cloud/stream-applications/tree/master/stream-applications-release-train[Release Train]

=== Reusable Functions

|===
| `java.util.Supplier` | `java.util.Function` | `java.util.Consumer`

|link:functions/supplier/debezium-supplier/README.adoc[Debezium supplier]
|link:functions/function/aggregator-function/README.adoc[Aggregator]
|link:functions/consumer/analytics-consumer/README.adoc[Analytics]

|link:functions/supplier/file-supplier/README.adoc[File]
|link:functions/function/filter-function/README.adoc[Filter]
|link:functions/consumer/cassandra-consumer/README.adoc[Cassandra]

|link:functions/supplier/ftp-supplier/README.adoc[FTP]
|link:functions/function/header-enricher-function/README.adoc[Header-Enricher]
|link:functions/consumer/elasticsearch-consumer/README.adoc[Elasticsearch]

|
|link:functions/function/header-filter-function/README.adoc[Header-Filter]
|link:functions/consumer/file-consumer/README.adoc[File]

|link:functions/supplier/http-supplier/README.adoc[HTTP]
|link:functions/function/http-request-function/README.adoc[HTTP Request]
|link:functions/consumer/ftp-consumer/README.adoc[FTP]

|link:functions/supplier/jdbc-supplier/README.adoc[JDBC]
|link:functions/function/image-recognition-function/README.adoc[Image Recognition(Tensorflow)]
|

|link:functions/supplier/jms-supplier/README.adoc[JMS]
|link:functions/function/object-detection-function/README.adoc[Object Detection(Tensorflow)]
|link:functions/consumer/jdbc-consumer/README.adoc[JDBC]

|link:functions/supplier/mail-supplier/README.adoc[Mail]
|link:functions/function/semantic-segmentation-function/README.adoc[Semantic Segmentation(Tensorflow)]
|link:functions/consumer/log-consumer/README.adoc[Log]

|link:functions/supplier/mongodb-supplier/README.adoc[MongoDB]
|link:functions/function/spel-function/README.adoc[SpEL]
|link:functions/consumer/mongodb-consumer/README.adoc[MongoDB]

|link:functions/supplier/mqtt-supplier/README.adoc[MQTT]

|link:functions/function/splitter-function/README.adoc[Splitter]
|link:functions/consumer/mqtt-consumer/README.adoc[MQTT]

|link:functions/supplier/rabbit-supplier/README.adoc[RabbitMQ]
|link:functions/function/task-launch-request-function/README.adoc[Task Launch Request]
|link:functions/consumer/rabbit-consumer/README.adoc[RabbitMQ]

|link:functions/supplier/s3-supplier/README.adoc[AWS S3]
|
|link:functions/consumer/redis-consumer/README.adoc[Redis]

|link:functions/supplier/sftp-supplier/README.adoc[SFTP]
|
|link:functions/consumer/rsocket-consumer/README.adoc[RSocket]

|link:functions/supplier/syslog-supplier/README.adoc[Syslog]
|
|link:functions/consumer/s3-consumer/README.adoc[AWS S3]

|link:functions/supplier/tcp-supplier/README.adoc[TCP]
|
|link:functions/consumer/sftp-consumer/README.adoc[SFTP]

|link:functions/supplier/time-supplier/README.adoc[Time]
|
|link:functions/consumer/tcp-consumer/README.adoc[TCP]

|link:functions/supplier/twitter-supplier/README.adoc[Twitter]
|link:functions/function/twitter-function/README.adoc[Twitter]
|link:functions/consumer/twitter-consumer/README.adoc[Twitter]

|link:functions/supplier/websocket-supplier/README.adoc[Websocket]
|
|link:functions/consumer/websocket-consumer/README.adoc[Websocket]

|
|
|link:functions/consumer/wavefront-consumer/README.adoc[Wavefront]

|link:functions/supplier/xmpp-supplier/README.adoc[XMPP]
|
|link:functions/consumer/xmpp-consumer/README.adoc[XMPP]

|===

=== Reusable Spring Cloud Stream Applications

|===
Expand Down Expand Up @@ -139,19 +45,19 @@ The following are the various components of this repository.
|

|link:applications/source/jms-source/README.adoc[JMS]
|link:applications/processor/image-recognition-processor/README.adoc[Image Recognition(Tensorflow)]
|
|link:applications/sink/jdbc-sink/README.adoc[JDBC]

|link:applications/source/load-generator-source/README.adoc[Load-Generator]
|link:applications/processor/object-detection-processor/README.adoc[Object Detection(Tensorflow)]
|
|link:applications/sink/log-sink/README.adoc[Log]

|link:applications/source/mail-source/README.adoc[Mail]
|
|link:applications/sink/mongodb-sink/README.adoc[MongoDB]

|link:applications/source/mongodb-source/README.adoc[MongoDB]
|link:applications/processor/semantic-segmentation-processor/README.adoc[Semantic Segmentation(Tensorflow)]
|
|link:applications/sink/mqtt-sink/README.adoc[MQTT]

|link:applications/source/mqtt-source/README.adoc[MQTT]
Expand Down Expand Up @@ -180,7 +86,7 @@ The following are the various components of this repository.

|link:applications/source/time-source/README.adoc[Time]
|
|link:applications/sink/tasklauncher-sink/README.adoc[Task Launcher]
|

|link:applications/source/twitter-message-source/README.adoc[Twitter Message]
|
Expand Down Expand Up @@ -253,7 +159,8 @@ You can then build the desired apps.
./build-app.sh . applications/sink/log-sink
....

NOTE: In order to disable metrics by default there needs to be application properties configured like in `default-application.properties`. The `build-app.sh` script will copy default-application.properties into src/main/resources if no application.properties,yml,yaml or json is present.
NOTE: In order to disable metrics by default there needs to be application properties configured like in `default-application.properties`.
The `build-app.sh` script will copy default-application.properties into src/main/resources if no application.properties,yml,yaml or json is present.

=== Additional Resources

Expand Down
12 changes: 6 additions & 6 deletions applications/processor/aggregator-processor/README.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@ Properties grouped by prefix:

=== aggregator

$$aggregation$$:: $$SpEL expression for aggregation strategy. Default is collection of payloads.$$ *($$Expression$$, default: `$$<none>$$`)*
$$correlation$$:: $$SpEL expression for correlation key. Default to correlationId header.$$ *($$Expression$$, default: `$$<none>$$`)*
$$group-timeout$$:: $$SpEL expression for timeout to expiring uncompleted groups.$$ *($$Expression$$, default: `$$<none>$$`)*
$$message-store-entity$$:: $$Persistence message store entity: table prefix in RDBMS, collection name in MongoDb, etc.$$ *($$String$$, default: `$$<none>$$`)*
$$message-store-type$$:: $$Message store type.$$ *($$String$$, default: `$$<none>$$`)*
$$release$$:: $$SpEL expression for release strategy. Default is based on the sequenceSize header.$$ *($$Expression$$, default: `$$<none>$$`)*
$$aggregation$$:: $$<documentation missing>$$ *($$Expression$$, default: `$$<none>$$`)*
artembilan marked this conversation as resolved.
Show resolved Hide resolved
$$correlation$$:: $$<documentation missing>$$ *($$Expression$$, default: `$$<none>$$`)*
$$group-timeout$$:: $$<documentation missing>$$ *($$Expression$$, default: `$$<none>$$`)*
$$message-store-entity$$:: $$<documentation missing>$$ *($$String$$, default: `$$<none>$$`)*
$$message-store-type$$:: $$<documentation missing>$$ *($$String$$, default: `$$<none>$$`)*
$$release$$:: $$<documentation missing>$$ *($$Expression$$, default: `$$<none>$$`)*

=== spring.data.mongodb

Expand Down
6 changes: 3 additions & 3 deletions applications/processor/aggregator-processor/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
<dependencies>
<dependency>
<groupId>org.springframework.cloud.fn</groupId>
<artifactId>aggregator-function</artifactId>
<artifactId>spring-aggregator-function</artifactId>
</dependency>
</dependencies>

Expand All @@ -35,13 +35,13 @@
<name>aggregator</name>
<type>processor</type>
<version>${project.version}</version>
<configClass>org.springframework.cloud.fn.aggregator.AggregatorFunctionConfiguration.class</configClass>
<configClass>AUTOCONFIGURATION</configClass>
<functionDefinition>jsonBytesToMap|aggregatorFunction</functionDefinition>
<maven>
<dependencies>
<dependency>
<groupId>org.springframework.cloud.fn</groupId>
<artifactId>aggregator-function</artifactId>
<artifactId>spring-aggregator-function</artifactId>
</dependency>
</dependencies>
</maven>
Expand Down
6 changes: 3 additions & 3 deletions applications/processor/filter-processor/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
<dependencies>
<dependency>
<groupId>org.springframework.cloud.fn</groupId>
<artifactId>filter-function</artifactId>
<artifactId>spring-filter-function</artifactId>
</dependency>
</dependencies>

Expand All @@ -43,13 +43,13 @@
<name>filter</name>
<type>processor</type>
<version>${project.version}</version>
<configClass>org.springframework.cloud.fn.filter.FilterFunctionConfiguration.class</configClass>
<configClass>AUTOCONFIGURATION</configClass>
<functionDefinition>byteArrayTextToString|filterFunction</functionDefinition>
<maven>
<dependencies>
<dependency>
<groupId>org.springframework.cloud.fn</groupId>
<artifactId>filter-function</artifactId>
<artifactId>spring-filter-function</artifactId>
</dependency>
</dependencies>
</maven>
Expand Down
2 changes: 1 addition & 1 deletion applications/processor/groovy-processor/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@
</dependency>
<dependency>
<groupId>org.springframework.cloud.fn</groupId>
<artifactId>payload-converter-function</artifactId>
<artifactId>spring-payload-converter-function</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
Expand Down
6 changes: 3 additions & 3 deletions applications/processor/header-enricher-processor/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
<dependencies>
<dependency>
<groupId>org.springframework.cloud.fn</groupId>
<artifactId>header-enricher-function</artifactId>
<artifactId>spring-header-enricher-function</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.integration</groupId>
Expand Down Expand Up @@ -48,13 +48,13 @@
<name>header-enricher</name>
<type>processor</type>
<version>${project.version}</version>
<configClass>org.springframework.cloud.fn.header.enricher.HeaderEnricherFunctionConfiguration.class</configClass>
<configClass>AUTOCONFIGURATION</configClass>
<functionDefinition>headerEnricherFunction</functionDefinition>
<maven>
<dependencies>
<dependency>
<groupId>org.springframework.cloud.fn</groupId>
<artifactId>header-enricher-function</artifactId>
<artifactId>spring-header-enricher-function</artifactId>
</dependency>
</dependencies>
</maven>
Expand Down
6 changes: 3 additions & 3 deletions applications/processor/header-filter-processor/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
<dependencies>
<dependency>
<groupId>org.springframework.cloud.fn</groupId>
<artifactId>header-filter-function</artifactId>
<artifactId>spring-header-filter-function</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.integration</groupId>
Expand Down Expand Up @@ -53,13 +53,13 @@
<name>header-filter</name>
<type>processor</type>
<version>${project.version}</version>
<configClass>org.springframework.cloud.fn.header.filter.HeaderFilterFunctionConfiguration.class</configClass>
<configClass>AUTOCONFIGURATION</configClass>
<functionDefinition>headerFilterFunction</functionDefinition>
<maven>
<dependencies>
<dependency>
<groupId>org.springframework.cloud.fn</groupId>
<artifactId>header-filter-function</artifactId>
<artifactId>spring-header-filter-function</artifactId>
</dependency>
</dependencies>
</maven>
Expand Down
2 changes: 1 addition & 1 deletion applications/processor/http-request-processor/README.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ $$body-expression$$:: $$A SpEL expression to derive the request body from the in
$$expected-response-type$$:: $$The type used to interpret the response.$$ *($$Class<?>$$, default: `$$<none>$$`)*
$$headers-expression$$:: $$A SpEL expression used to derive the http headers map to use.$$ *($$Expression$$, default: `$$<none>$$`)*
$$http-method-expression$$:: $$A SpEL expression to derive the request method from the incoming message.$$ *($$Expression$$, default: `$$<none>$$`)*
$$reply-expression$$:: $$A SpEL expression used to compute the final result, applied against the whole http {@link org.springframework.http.ResponseEntity}.$$ *($$Expression$$, default: `$$<none>$$`)*
$$reply-expression$$:: $$A SpEL expression used to compute the final result, applied against the whole http {@link ResponseEntity}.$$ *($$Expression$$, default: `$$<none>$$`)*
$$timeout$$:: $$Request timeout in milliseconds.$$ *($$Long$$, default: `$$30000$$`)*
$$url-expression$$:: $$A SpEL expression against incoming message to determine the URL to use.$$ *($$Expression$$, default: `$$<none>$$`)*

Expand Down
7 changes: 3 additions & 4 deletions applications/processor/http-request-processor/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
</dependency>
<dependency>
<groupId>org.springframework.cloud.fn</groupId>
<artifactId>http-request-function</artifactId>
<artifactId>spring-http-request-function</artifactId>
</dependency>
<dependency>
<groupId>com.squareup.okhttp3</groupId>
Expand Down Expand Up @@ -56,15 +56,14 @@
<name>http-request</name>
<type>processor</type>
<version>${project.version}</version>
<configClass>org.springframework.cloud.fn.http.request.HttpRequestFunctionConfiguration.class
</configClass>
<configClass>AUTOCONFIGURATION</configClass>
<functionDefinition>httpRequestFunction</functionDefinition>

<maven>
<dependencies>
<dependency>
<groupId>org.springframework.cloud.fn</groupId>
<artifactId>http-request-function</artifactId>
<artifactId>spring-http-request-function</artifactId>
</dependency>
</dependencies>
</maven>
Expand Down
56 changes: 0 additions & 56 deletions applications/processor/image-recognition-processor/README.adoc

This file was deleted.

Loading
Loading