Graylog SDK Spring Boot Starter

Graylog SDK Spring Boot Starter to communicate with the Graylog REST API

Getting Started

Graylog SDK Spring Boot Starter is available at the Central Maven Repository.

Maven

<dependency>
  <groupId>com.joonsang.graylog</groupId>
  <artifactId>graylog-sdk-spring-boot-starter</artifactId>
  <version>2.0.5</version>
</dependency>

Gradle

implementation group: 'com.joonsang.graylog', name: 'graylog-sdk-spring-boot-starter', version: '2.0.5'

Troubleshooting

Exception occurred: java.lang.NoSuchMethodError: 'okhttp3.RequestBody okhttp3.RequestBody.create(java.lang.String, okhttp3.MediaType)'
- There is an OkHttp3 version under 4.x somewhere in your project dependencies.
- Add OkHttp3 4.x in your dependency like below.
  - implementation group: 'com.squareup.okhttp3', name: 'okhttp', version: '4.x.x'
- #3

Graylog API Credentials

First create user access token on your Graylog web interface for accessing Graylog APIs.

Refer Graylog documentation: Creating and using Access Token

Graylog 3.2 Search

Usage

Configure environment variables for Graylog SDK with application.properties:

# Graylog API Settings
graylog.sdk.api.scheme=http
graylog.sdk.api.host=localhost
graylog.sdk.api.port=9000
graylog.sdk.api.credentials=base64({graylog_access_token}:token)
graylog.sdk.api.timeout=60000

or application.yml:

# Graylog API Settings
graylog:
  sdk:
    api:
      scheme: http
      host: localhost
      port: 9000
      credentials: base64({graylog_access_token}:token)
      timeout: 60000

Then inject GraylogSearch bean in your project:

private final GraylogSearch graylogSearch;

public YourClassName(GraylogSearch graylogSearch) {
    this.graylogSearch = graylogSearch;
}

Request Graylog REST APIs

1. Search

1.1. Messages

Message search with time range.

Create POJO class specifying your Graylog message fields.

public class YourMessageObject {

    private String message;
    
    private String source;

    private String timestamp;

    // getters and setters
}

Graylog SDK will return the list of message object as you specified.

String from = "2020-07-30T00:00:00Z";
String to = "2020-07-31T00:00:00Z";

Timerange timerange = Timerange.builder()
    .type(TimeRangeType.absolute)
    .from(from)
    .to(to)
    .build();

SortConfig sort = SortConfig.builder()
    .field("timestamp")
    .order(SortConfigOrder.DESC)
    .build();

int pageSize = 20;
int pageNo = 1;

@SuppressWarnings("unchecked")
Page<YourMessageObject> messages = (Page<YourMessageObject>) graylogSearch.getMessages(
    List.of("graylog_stream_id"),
    timerange,
    "source:example.org",
    pageSize,
    pageNo,
    sort,
    YourMessageObject.class
);

1.2. Statistics

Statistics for a query using a time range.

String from = "2020-07-30T00:00:00Z";
String to = "2020-07-31T00:00:00Z";

Timerange timerange = Timerange.builder()
    .type(TimeRangeType.absolute)
    .from(from)
    .to(to)
    .build();

List<Series> seriesList = List.of(
    Series.builder().type(SeriesType.avg).field("process_time").build(),
    Series.builder().type(SeriesType.count).field("process_time").build(),
    Series.builder().type(SeriesType.min).field("process_time").build(),
    Series.builder().type(SeriesType.max).field("process_time").build(),
    Series.builder().type(SeriesType.percentile).percentile(95.0f).field("process_time").build(),
    Series.builder().type(SeriesType.percentile).percentile(99.0f).field("process_time").build(),
    Series.builder().type(SeriesType.count).build()
);

List<Statistics> statistics = graylogSearch.getStatistics(
    List.of("graylog_stream_id"),
    timerange,
    "source:example.org",
    seriesList
);

1.3. Histogram

Datetime histogram of a query using a time range.

String from = "2020-07-30T00:00:00Z";
String to = "2020-07-31T00:00:00Z";

Timerange timerange = Timerange.builder()
    .type(TimeRangeType.absolute)
    .from(from)
    .to(to)
    .build();

Interval interval = Interval.builder()
    .type(IntervalType.timeunit)
    .timeunit(IntervalTimeunit.get(IntervalTimeunit.Unit.minutes, 1))
    .build();

List<Series> seriesList = List.of(
    Series.builder().type(SeriesType.count).build(),
    Series.builder().type(SeriesType.avg).field("process_time").build()
);

List<SearchTypePivot> columnGroups = List.of(
    SearchTypePivot.builder().type(SearchTypePivotType.values).field("field_name").limit(5).build()
);

Histogram histogram = graylogSearch.getHistogram(
    List.of("graylog_stream_id"),
    timerange,
    interval,
    "source:example.org",
    seriesList,
    columnGroups
);

1.4. Terms

Differences between Quick Values and Graylog 3.2 Data Table

Stacking fields on Legacy Terms (Quick Values) means intersecting stacked fields by field like an INTERSECTION

Adding fields on row groups of Terms (Data Table) means combining fields by order, like a UNION

Most common field terms of a query using a time range.

String from = "2020-07-30T00:00:00Z";
String to = "2020-07-31T00:00:00Z";

Timerange timerange = Timerange.builder()
    .type(TimeRangeType.absolute)
    .from(from)
    .to(to)
    .build();

List<Series> seriesList = List.of(
    Series.builder().type(SeriesType.count).build()
);

List<SearchTypePivot> rowGroups = List.of(
    SearchTypePivot.builder().type(SearchTypePivotType.values).field("field_name_1").limit(10).build(),
    SearchTypePivot.builder().type(SearchTypePivotType.values).field("field_name_2").build()
);

List<SearchTypePivot> columnGroups = List.of(
    SearchTypePivot.builder().type(SearchTypePivotType.values).field("field_name_3").limit(5).build()
);

SortConfig sort = SortConfig.builder()
    .type(SortConfigType.series)
    .field("count()")
    .direction(SortConfigDirection.Descending)
    .build();

graylogSearch.getTerms(
    List.of("graylog_stream_id"),
    timerange,
    "source:example.org",
    seriesList,
    rowGroups,
    columnGroups,
    sort
);

1.5. Raw

Search with a search spec builder, returns raw response message from Graylog.

SearchSpec searchSpec = SearchSpec.builder()
    .query(
        Query.builder()
            .filter(
                Filter.builder()
                    .filters(
                        List.of(
                            SearchFilter.builder()
                                .id("graylog_stream_id")
                                .build()
                        )
                    )
                    .build()
            )
            .query(
                SearchQuery.builder()
                    .queryString("source:example.org")
                    .build()
            )
            .timerange(
                Timerange.builder()
                    .type(TimeRangeType.relative)
                    .range(300)
                    .build()
            )
            .searchType(
                SearchType.builder()
                    .name("chart")
                    .series(
                        List.of(
                            Series.builder()
                                .type(SeriesType.count)
                                .build()
                        )
                    )
                    .rollup(true)
                    .rowGroups(
                        List.of(
                            SearchTypePivot.builder()
                                .type(SearchTypePivotType.values)
                                .field("source")
                                .limit(5)
                                .build()
                        )
                    )
                    .columnGroups(List.of())
                    .sort(List.of())
                    .type(SearchTypeType.pivot)
                    .build()
            )
            .build()
    )
    .build();

String result = graylogSearch.raw(searchSpec);

Graylog 3.2 Search Spec Builder

1. Outline of Search Spec

With a search spec builder, it will generate required IDs automatically if not specified

Each search has a search ID, which is made of Object ID.
Each query has a query ID, which is made of UUID.
Each search type has a search type ID, which is made of UUID.

SearchSpec.builder() /* search ID */
    .query(
        Query.builder() /* query ID */
            .filter(... your filter ...)
            .query(... your search query ...)
            .timerange(... your timerange ...)
            .searchType( /* search type ID */
                ... your search type ...
            )
            .build()
    )
    .parameter(... your parameter if needed ...)
    .build();

2. Search Specifications

2.1. Search Spec

Data implementation of Graylog search UI as below

Search spec contains search ID, queries, and parameters.

Generate search spec with specific search ID:

SearchSpec.builder()
    .id("your object id")
    ...
    .build();

Generate search spec with new search ID:

SearchSpec.builder()
    ...
    .build();

2.2. Query

Query contains query ID, filter, search query, time range, and search types.

Generate query with specific query ID:

Query.builder()
    .id("your uuid")
    ...
    .build();

Generate query with new query ID:

Query.builder()
    ...
    .build();

2.3. Filter

Graylog search filter. Equivalent to stream select on Graylog search UI.

Filter.builder()
    .filters(
        List.of(
            SearchFilter.builder()
                .id("your graylog stream id")
                .build()
        )
    )
    .build();

2.4. Timerange

Time range of search. Equivalent to time range select on Graylog search UI.

Relative time range:

Timerange.builder()
    .type(TimeRangeType.relative)
    .range(300)
    .build();

Absolute time range:

Requires ISO 8601 format

Timerange timerange = Timerange.builder()
    .type(TimeRangeType.absolute)
    .from("2020-07-30T00:00:00Z")
    .to("2020-07-31T00:00:00Z")
    .build();

Keyword time range:

Timerange timerange = Timerange.builder()
    .type(TimeRangeType.keyword)
    .keyword("Last five minutes")
    .build();

2.5. Search Query

Graylog search query. Equivalent to search query text field on Graylog search UI.

SearchQuery.builder()
    .queryString("your graylog search query")
    .build();

2.6. Search Type

Data implementation of Graylog widget aggregation UI as below

Generate query with specific search type ID:

SearchType.builder()
    .id("your uuid")
    ...
    .build();

Generate query with new search type ID:

SearchType.builder()
    ...
    .build();

Full sample:

SearchType.builder()
    .name("chart") /* name of search type */
    .series( /* metrics */
        List.of(
            Series.builder()
                .type(SeriesType.count)
                .build()
        )
    )
    .rollup(true)
    .rowGroups( /* rows */
        List.of(
            SearchTypePivot.builder()
                .type(SearchTypePivotType.values)
                .field("source")
                .limit(15)
                .build()
        )
    )
    .columnGroups(List.of()) /* columns */
    .sort(List.of()) /* sorting */
    .type(SearchTypeType.pivot) /* pivot or messages */
    .build()

2.6.1. Series

Equivalent to METRICS on Graylog widget aggregation UI.

count():

Series.builder()
    .type(SeriesType.count)
    .build();

avg(field_name):

Series.builder()
    .type(SeriesType.avg)
    .field("process_time")
    .build();

percentile(field_name, 95):

Series.builder()
    .type(SeriesType.percentile)
    .percentile(95.0f)
    .field("process_time")
    .build();

2.6.2. Row Groups

Equivalent to ROWS on Graylog widget aggregation UI.

SearchTypePivot.builder()
    .type(SearchTypePivotType.values) /* values or time */
    .field("source")
    .limit(15)
    .build()

2.6.3. Column Groups

Equivalent to COLUMNS on Graylog widget aggregation UI.

SearchTypePivot.builder()
    .type(SearchTypePivotType.values) /* values or time */
    .field("source")
    .limit(15)
    .build()

2.6.4. Sort

Equivalent to SORTING and DIRECTION on Graylog widget aggregation UI.

message sort:

SortConfig.builder()
    .field("timestamp")
    .order(SortConfigOrder.DESC)
    .build();

pivot sort:

SortConfig.builder()
    .type(SortConfigType.series)
    .field("count()")
    .order(SortConfigDirection.Descending)
    .build();

Legacy Graylog Search

Legacy search APIs will be no longer available from Graylog 4.0

Usage

Configure environment variables for Graylog SDK with application.properties:

# Graylog API Settings
graylog.sdk.api.scheme=http
graylog.sdk.api.host=localhost
graylog.sdk.api.port=9000
graylog.sdk.api.credentials=base64({graylog_access_token}:token)
graylog.sdk.legacy.timezone=US/Eastern

or application.yml:

# Graylog API Settings
graylog:
  sdk:
    api:
      scheme: http
      host: localhost
      port: 9000
      credentials: base64({graylog_access_token}:token)
    legacy:
      timezone: US/Eastern

Then inject LegacyGraylogSearch bean in your project:

private final LegacyGraylogSearch legacyGraylogSearch;

public YourClassName(LegacyGraylogSearch legacyGraylogSearch) {
    this.legacyGraylogSearch = legacyGraylogSearch;
}

Request Graylog REST APIs

Old APIs in Search section were moved to Legacy/Search section from Graylog 3.2

Currently, only Legacy/Search/Absolute APIs are supported.

1. Legacy Search Absolute

1.1. Messages

Message search with absolute time range.

Create POJO class specifying your Graylog message fields.

public class YourMessageObject {

    private String message;
    
    private String source;

    private String timestamp;

    // getters and setters
}

Graylog SDK will return the list of message object as you specified.

LocalDateTime from = LocalDateTime.parse("2019-11-04 00:00:00", DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"));
LocalDateTime to = LocalDateTime.parse("2019-11-05 00:00:00", DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"));

@SuppressWarnings("unchecked")
List<YourMessageObject> messages = (List<YourMessageObject>) legacyGraylogSearch.getMessages(
    "graylog_stream_id",
    from,
    to,
    "request_id:AQZ4mfVGVqWKD38XZU7aVG",
    YourMessageObject.class
);

If you need messages with paging, try like below.

LocalDateTime from = LocalDateTime.parse("2019-11-04 00:00:00", DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"));
LocalDateTime to = LocalDateTime.parse("2019-11-05 00:00:00", DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"));
int pageSize = 20;
int pageNo = 1;

@SuppressWarnings("unchecked")
Page<YourMessageObject> pagedMessages = (Page<YourMessageObject>) legacyGraylogSearch.getMessages(
    "graylog_stream_id",
    from,
    to,
    "request_id:AQZ4mfVGVqWKD38XZU7aVG",
    pageSize,
    pageNo,
    YourMessageObject.class
);

1.2. Statistics

Field statistics for a query using an absolute time range.

LocalDateTime from = LocalDateTime.parse("2019-11-04 00:00:00", DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"));
LocalDateTime to = LocalDateTime.parse("2019-11-05 00:00:00", DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"));

Statistics statistics = legacyGraylogSearch.getStatistics(
    "graylog_stream_id",
    "field_name",
    from,
    to,
    "process_time:[0 TO 500]"
);

1.3. Histogram

Datetime histogram of a query using an absolute time range.

LocalDateTime from = LocalDateTime.parse("2019-11-04 00:00:00", DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"));
LocalDateTime to = LocalDateTime.parse("2019-11-05 00:00:00", DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"));

Histogram histogram = legacyGraylogSearch.getHistogram(
    "graylog_stream_id",
    TimeUnit.HOUR,
    from,
    to,
    "process_time:[0 TO 500]"
);

1.4. Field Histogram

Field value histogram of a query using an absolute time range.

LocalDateTime from = LocalDateTime.parse("2019-11-04 00:00:00", DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"));
LocalDateTime to = LocalDateTime.parse("2019-11-05 00:00:00", DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"));

FieldHistogram fieldHistogram = legacyGraylogSearch.getFieldHistogram(
    "graylog_stream_id",
    "field_name",
    TimeUnit.HOUR,
    from,
    to,
    "source:example.org"
);

1.5. Terms

Most common field terms of a query using an absolute time range.

LocalDateTime from = LocalDateTime.parse("2019-11-04 00:00:00", DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"));
LocalDateTime to = LocalDateTime.parse("2019-11-05 00:00:00", DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"));

Terms terms = legacyGraylogSearch.getTerms(
    "graylog_stream_id",
    "field_name",
    "field_name_to_stack",
    5,
    from,
    to,
    false,
    false,
    "source:example.org"
);

Code Examples

graylog-sdk-spring-boot-samples in this repository contains the project that show you sample API implementations using Graylog SDK with Spring Boot.
- Check out Graylog Query Builder if you are looking for query builder for Graylog Search Query.

Migration Guide From 1.2.x

You'll find a guide to upgrade from 1.2.x to 2.x here.

Name		Name	Last commit message	Last commit date
Latest commit History 205 Commits
.github/workflows		.github/workflows
gradle/wrapper		gradle/wrapper
graylog-sdk-spring-boot-samples		graylog-sdk-spring-boot-samples
graylog-sdk-spring-boot-starter		graylog-sdk-spring-boot-starter
images		images
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
build.gradle		build.gradle
gradlew		gradlew
gradlew.bat		gradlew.bat
settings.gradle		settings.gradle

License

debugrammer/graylog-sdk-spring-boot

Folders and files

Latest commit

History

Repository files navigation

Graylog SDK Spring Boot Starter

Getting Started

Troubleshooting

Graylog API Credentials

Graylog 3.2 Search

Usage

Request Graylog REST APIs

1. Search

1.1. Messages

1.2. Statistics

1.3. Histogram

1.4. Terms

1.5. Raw

Graylog 3.2 Search Spec Builder

1. Outline of Search Spec

2. Search Specifications

2.1. Search Spec

2.2. Query

2.3. Filter

2.4. Timerange

2.5. Search Query

2.6. Search Type

2.6.1. Series

2.6.2. Row Groups

2.6.3. Column Groups

2.6.4. Sort

Legacy Graylog Search

Usage

Request Graylog REST APIs

1. Legacy Search Absolute

1.1. Messages

1.2. Statistics

1.3. Histogram

1.4. Field Histogram

1.5. Terms

Code Examples

Migration Guide From 1.2.x

About

Topics

Resources

License

Stars

Watchers

Forks

Releases 22

Packages 0

Languages

Packages