feat(EM-43): Set up centralized logging with ELK stack

[devin-ai-integration]

feat(EM-43): Set up centralized logging with ELK stack

Summary

Adds centralized logging infrastructure and a shared logging library for the FTGO microservices platform:

• infrastructure/logging/: Docker Compose for full ELK stack (Elasticsearch 8.13.0, Logstash, Kibana, Filebeat), Logstash pipeline config with JSON parsing and trace ID extraction, Elasticsearch index template + ILM policy (30d retention), Kibana dashboard NDJSON exports
• libs/ftgo-logging/: Spring Boot auto-configuration library providing structured JSON logging via a custom FtgoJsonEncoder, Logstash TCP appender, request ID filter (X-Request-ID), and MDC-based trace/span ID correlation with libs/ftgo-tracing/
• docs/logging/: Architecture documentation, configuration reference, and troubleshooting guide

No existing files were modified. The root settings.gradle is untouched.

Review & Testing Checklist for Human

• Java code has not been compiled or tested locally. The repo's Gradle wrapper (4.10.2/JDK 8) cannot build any Java 17 libs — this is a pre-existing issue affecting all libs. Verify the code compiles in CI or a JDK 17 environment. Pay close attention to FtgoJsonEncoder, LogbackJsonConfiguration, and LogstashAppenderConfiguration for compilation issues.
• LogbackJsonConfiguration.configure() programmatically detaches and replaces the root logger's console appender at @PostConstruct time. This could cause log loss during startup or interfere with other logging configurations. Verify ordering doesn't conflict with Spring Boot's own Logback initialization.
• Two JSON encoding approaches are provided — the programmatic FtgoJsonEncoder (custom hand-rolled encoder) AND the declarative logback-ftgo.xml (using LogstashEncoder from logstash-logback-encoder). Confirm this is intentional and that usage guidance is clear enough, or consolidate to one approach.
• Kibana dashboard NDJSON is hand-crafted, not exported from a running Kibana 8.13.0 instance. Verify it imports cleanly: curl -X POST "http://localhost:5601/api/saved_objects/_import" -H "kbn-xsrf: true" --form file=@ftgo-service-logs-dashboard.ndjson
• End-to-end test: Spin up the ELK stack via docker compose up -d, configure a service with ftgo.logging.logstash.enabled=true, send a traced request, and confirm logs appear in Kibana with traceId/spanId fields populated.

Notes

• The RequestIdFilter is registered as a plain bean; Spring Boot will auto-register it but ordering relative to tracing filters is not explicitly set.
• Logstash pipeline expects JSON logs with traceId/spanId in MDC; this relies on Spring Boot's built-in MDC propagation from Micrometer tracing (should work with ftgo-tracing on the classpath).
• ILM policy deletes logs after 30 days; adjust if needed for compliance.

Link to Devin run: https://app.devin.ai/sessions/7de731f3ee814f21b7b8818a5a8d208b
Requested by: @abj453demo

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring