[TT-2539] added access/transaction logs

[LLe27]

FR Jira Ticket

https://tyktech.atlassian.net/browse/TT-2539

Description

• Added the TYK_GW_ACCESSLOGS_ENABLED Gateway config option
• The Tyk Gateway will determine to print access logs to STDOUT for both success and error handling situations
  • If the TYK_GW_ACCESSLOGS_ENABLED is set to true then the Gateway will print access logs to STDOUT
  • If the TYK_GW_ACCESSLOGS_ENABLED is set to false then the Gateway will not print access logs to STDOUT

Note that this feature is off by default and that the AccessLog struct only contains the more common elements. Below are some examples of an access log

time="Sep 04 08:04:18" level=info APIID=c062396cb62d4e9a5ee37adaf85b9e4c APIKey=00000000 ClientIP=127.0.0.1 ClientRemoteAddr="127.0.0.1:53506" Host="localhost:8080" Method=GET OrgID=66d07f00247d80811d5199c3 Proto=HTTP/1.1 RequestURI=/httpbin/get StatusCode=200 TotalLatency=381 UpstreamAddress="http://httpbin.org/get" UpstreamLatency=381 UpstreamPath=/get UpstreamURI=/get UserAgent=curl/8.1.2 prefix=access-log
time="Sep 04 08:08:20" level=info APIID=c062396cb62d4e9a5ee37adaf85b9e4c APIKey=00000000 ClientIP=127.0.0.1 ClientRemoteAddr="127.0.0.1:53566" Host="localhost:8080" Method=GET OrgID=66d07f00247d80811d5199c3 Proto=HTTP/1.1 RequestURI=/httpbin/get StatusCode=401 TotalLatency=0 UpstreamAddress=":///httpbin/get" UpstreamLatency=0 UpstreamPath=/httpbin/get UpstreamURI=/httpbin/get UserAgent=curl/8.1.2 prefix=access-log

Related Issue

Motivation and Context

Today the Tyk Gateway does not print access logs for success API calls but instead only for error API calls. Providing access logs for both scenarios within the Tyk Gateway is extremely valuable especially if you are monitoring logs, capturing analytics or even debugging. Providing the option to turn on or off the Tyk Gateway access logs will provide clients more insights in for API calls in regards to success and error situations.

How This Has Been Tested

• Manual testing
• Unit testing
• Performance testing/benchmarks

Screenshots (if appropriate)

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Refactoring or add test (improvements in base code or adds test coverage to functionality)

Checklist

• I ensured that the documentation is up to date
• I explained why this PR updates go.mod in detail with reasoning why it's required
• I would like a code coverage CI quality gate exception and have explained why

[github-actions]

API Changes

--- prev.txt	2024-09-12 08:45:12.627540046 +0000
+++ current.txt	2024-09-12 08:45:09.227540563 +0000
@@ -5188,6 +5188,12 @@
 
 TYPES
 
+type AccessLogsConfig struct {
+	// Enable the transaction logs. Default: false
+	Enabled bool `json:"enabled"`
+}
+    AccessLogsConfig defines the type of transactions logs printed to stdout
+
 type AnalyticsConfigConfig struct {
 	// Set empty for a Self-Managed installation or `rpc` for multi-cloud.
 	Type string `json:"type"`
@@ -5637,6 +5643,10 @@
 	// If not set or left empty, it will default to `standard`.
 	LogFormat string `json:"log_format"`
 
+	// You can configure the transaction logs to be turned on
+	// If not set or left empty, it will default to 'false'
+	AccessLogs AccessLogsConfig `json:"access_logs"`
+
 	// Section for configuring OpenTracing support
 	// Deprecated: use OpenTelemetry instead.
 	Tracer Tracer `json:"tracing"`
@@ -11316,6 +11326,12 @@
 CONSTANTS
 
 const (
+	HashSha256    = crypto.HashSha256
+	HashMurmur32  = crypto.HashMurmur32
+	HashMurmur64  = crypto.HashMurmur64
+	HashMurmur128 = crypto.HashMurmur128
+)
+const (
 	// DefaultConn is the default connection type. Not analytics and Not cache.
 	DefaultConn = "default"
 	// CacheConn is the cache connection type
@@ -11323,26 +11339,26 @@
 	// AnalyticsConn is the analytics connection type
 	AnalyticsConn = "analytics"
 )
-const B64JSONPrefix = "ey"
-    `{"` in base64
-
-const MongoBsonIdLength = 24
 
 VARIABLES
 
 var (
+	HashStr = crypto.HashStr
+	HashKey = crypto.HashKey
+)
+var (
+	GenerateToken = crypto.GenerateToken
+	TokenHashAlgo = crypto.TokenHashAlgo
+	TokenID       = crypto.TokenID
+	TokenOrg      = crypto.TokenOrg
+)
+var (
 	// ErrRedisIsDown is returned when we can't communicate with redis
 	ErrRedisIsDown = errors.New("storage: Redis is either down or was not configured")
 
 	// ErrStorageConn is returned when we can't get a connection from the ConnectionHandler
 	ErrStorageConn = fmt.Errorf("Error trying to get singleton instance: %w", ErrRedisIsDown)
 )
-var (
-	HashSha256    = "sha256"
-	HashMurmur32  = "murmur32"
-	HashMurmur64  = "murmur64"
-	HashMurmur128 = "murmur128"
-)
 var ErrKeyNotFound = errors.New("key not found")
     ErrKeyNotFound is a standard error for when a key is not found in the
     storage engine
@@ -11351,19 +11367,9 @@
 
 FUNCTIONS
 
-func GenerateToken(orgID, keyID, hashAlgorithm string) (string, error)
-    If hashing algorithm is empty, use legacy key generation
-
-func HashKey(in string, hashKey bool) string
-func HashStr(in string, withAlg ...string) string
 func NewConnector(connType string, conf config.Config) (model.Connector, error)
     NewConnector creates a new storage connection.
 
-func TokenHashAlgo(token string) string
-func TokenID(token string) (id string, err error)
-    TODO: add checks
-
-func TokenOrg(token string) string
 
 TYPES
 

[github-actions]

💥 CI tests failed 🙈

git-state

all ok

Please look at the run or in the Checks tab.

[jeffy-mathew]

this change is unrelated, right?

[titpetric]

looks like it needs a revert, yes

[sonarcloud]

Quality Gate failed

Failed conditions
0.0% Coverage on New Code (required ≥ 80%)
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarCloud

Catch issues before they fail your Quality Gate with our IDE extension SonarLint