From a554d36174797f4fa39d3e180ccad9d799bd93ed Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hans=20H=C3=BCbner?= Date: Mon, 22 Apr 2024 14:27:16 +0200 Subject: [PATCH 1/5] feat: add EmmyLuaDebugger hook The change provides a hook in the init_worker phase to start the EmmyLua debugger. The debugger is enabled by setting the `KONG_EMMY_DEBUGGER` to the absolute pathname of the EmmyLuaDebugger shared library (e.g. /usr/local/bin/emmy_core.dylib). The host and port that the debugger interface listens on can be defined by setting the `KONG_EMMY_DEBUGGER_HOST` and `KONG_EMMY_DEBUGGER_PORT` environment variables, respectively. If the environment variable `KONG_EMMY_DEBUGGER_WAIT` is set, the debugger hook waits for the IDE to connect at the beginning of the worker startup process to allow debugging of startup issues. Kong should be started with KONG_NGINX_WORKER_PROCESSES set to 1 to enable only one worker process. Debugging multiple worker processes is not supported. This feature is based on work by @mehuled, published in https://dev.to/mehuled/breakpoint-debugging-with-kong-plugin-development-75a --- DEVELOPER.md | 77 +++++++++++++++++ .../unreleased/kong/feat-emmy-debugger.yml | 3 + kong-3.7.0-0.rockspec | 1 + kong/init.lua | 4 + kong/tools/emmy_debugger.lua | 82 +++++++++++++++++++ 5 files changed, 167 insertions(+) create mode 100644 changelog/unreleased/kong/feat-emmy-debugger.yml create mode 100644 kong/tools/emmy_debugger.lua diff --git a/DEVELOPER.md b/DEVELOPER.md index a3d97b85b856..e797f934faa9 100644 --- a/DEVELOPER.md +++ b/DEVELOPER.md @@ -438,6 +438,83 @@ how to access (or create) a development container with a well-defined tool and r - See [How to create a GitHub codespace](https://docs.github.com/en/codespaces/developing-in-codespaces/creating-a-codespace#creating-a-codespace). - See [How to create a VSCode development container](https://code.visualstudio.com/docs/remote/containers#_quick-start-try-a-development-container). +## Debugging Kong Gateway with EmmyLua and IntelliJ IDEA/VSCode + +[EmmyLua](https://emmylua.github.io/) is a plugin for IntelliJ IDEA and VSCode that provides Lua language +support. It comes with debugger support that makes it possible to set breakpoints in Lua code +and inspect variables. Kong Gateway can be debugged using EmmyLua by following these steps: + +### Install the IDE + +#### IntelliJ IDEA + +Download and install IntelliJ IDEA from [here](https://www.jetbrains.com/idea/download/). Note +that IntelliJ is a commercial product and requires a paid license after the trial period. + +#### VSCode + +Download and install MS Visual Studio Code from [here](https://code.visualstudio.com/download). + +### Install EmmyLua + +#### IntelliJ IDEA + +Go to the `Settings`->`Plugins`->`Marketplace` and search for `EmmyLua`. +Install the plugin. + +#### VSCode + +Go to the `Settings`->`Extensions` and search for `EmmyLua`. +Install the plugin (publisher is `Tangzx`). + +### Download and install the EmmyLua debugging server + +The [EmmyLuaDebugger](https://github.com/EmmyLua/EmmyLuaDebugger) is a standalone C++ program +that runs on the same machine as Kong Gateway and that mediates between the IDE's +debugger and the Lua code running in Kong Gateway. It can be downloaded from +[GitHub](https://github.com/EmmyLua/EmmyLuaDebugger/releases). The release +ZIP file contains a single share library named emmy_core.so (Linux) or emmy_core.dylib (macOS). +Place this file in a directory that is convenient for you and remember the path. + +Depending on your Linux version, you may need to compile +[EmmyLuaDebugger](https://github.com/EmmyLua/EmmyLuaDebugger) on your +own system as the release binaries published on GitHub assume a pretty +recent version of GLIBC to be present. + +### Start Kong Gateway with the EmmyLua debugger + +To enable the EmmyLua debugger, the `KONG_EMMY_DEBUGGER` environment variable must be set to +the absolute path of the debugger shared library file when Kong Gateway is started. It is +also advisable to start Kong Gateway with only one worker process, as debugging multiple worker +processes is not supported. For example: + +```shell +KONG_EMMY_DEBUGGER=/path/to/emmy_core.so KONG_NGINX_WORKER_PROCESSES=1 kong start +``` + +### Create debugger configuration + +#### IntelliJ IDEA + +Go to `Run`->`Edit Configurations` and click the `+` button to add a new +configuration. Select `Emmy Debugger(NEW)` as the configuration type. Enter a descriptive +name for the configuration, e.g. "Kong Gateway Debug". Click `OK` to save the configuration. + +#### VSCode + +Go to `Run`->`Add Configuration` and choose `EmmyLua New Debugger`. Enter a descriptive name +for the configuration, e.g. "Kong Gateway Debug". Save `launch.json`. + +### Start the EmmyLua debugger + +To connect the EmmyLua debugger to Kong Gateway, click the `Run`->`Debug` menu item in IntelliJ +(`Run`->`Start Debugging` in VSCode) and select the configuration that you've just created. You +will notice that the restart and stop buttons on the top of your IDE will change to solid green +and red colors. You can now set breakpoints in your Lua code and start debugging. Try setting +a breakpoint in the global `access` function that is defined `runloop/handler.lua` and send +a proxy request to the Gateway. The debugger should stop at the breakpoint and you can +inspect the variables in the request context. + ## What's next - Refer to the [Kong Gateway Docs](https://docs.konghq.com/gateway/) for more information. diff --git a/changelog/unreleased/kong/feat-emmy-debugger.yml b/changelog/unreleased/kong/feat-emmy-debugger.yml new file mode 100644 index 000000000000..cba62bc50a74 --- /dev/null +++ b/changelog/unreleased/kong/feat-emmy-debugger.yml @@ -0,0 +1,3 @@ +message: | + Added support for debugging with EmmyLuaDebugger +type: feature diff --git a/kong-3.7.0-0.rockspec b/kong-3.7.0-0.rockspec index 35a94a8afcac..84f72ad4c951 100644 --- a/kong-3.7.0-0.rockspec +++ b/kong-3.7.0-0.rockspec @@ -193,6 +193,7 @@ build = { ["kong.tools.ip"] = "kong/tools/ip.lua", ["kong.tools.http"] = "kong/tools/http.lua", ["kong.tools.cjson"] = "kong/tools/cjson.lua", + ["kong.tools.emmy_debugger"] = "kong/tools/emmy_debugger.lua", ["kong.tools.redis.schema"] = "kong/tools/redis/schema.lua", ["kong.runloop.handler"] = "kong/runloop/handler.lua", diff --git a/kong/init.lua b/kong/init.lua index 93f8b7089ca1..0a16660a1962 100644 --- a/kong/init.lua +++ b/kong/init.lua @@ -89,6 +89,7 @@ local process = require "ngx.process" local tablepool = require "tablepool" local table_new = require "table.new" local utils = require "kong.tools.utils" +local emmy_debugger = require "kong.tools.emmy_debugger" local get_ctx_table = require("resty.core.ctx").get_ctx_table local admin_gui = require "kong.admin_gui" local wasm = require "kong.runloop.wasm" @@ -793,6 +794,9 @@ end function Kong.init_worker() + + emmy_debugger.init() + local ctx = ngx.ctx ctx.KONG_PHASE = PHASES.init_worker diff --git a/kong/tools/emmy_debugger.lua b/kong/tools/emmy_debugger.lua new file mode 100644 index 000000000000..1795ceb1774a --- /dev/null +++ b/kong/tools/emmy_debugger.lua @@ -0,0 +1,82 @@ +local pl_path = require "pl.path" +local utils = require "kong.tools.utils" + +local debugger = os.getenv("KONG_EMMY_DEBUGGER") +local emmy_debugger_host = os.getenv("KONG_EMMY_DEBUGGER_HOST") or "localhost" +local emmy_debugger_port = os.getenv("KONG_EMMY_DEBUGGER_PORT") or 9966 +local emmy_debugger_wait = os.getenv("KONG_EMMY_DEBUGGER_WAIT") +local emmy_debugger_source_path = utils.split(os.getenv("KONG_EMMY_DEBUGGER_SOURCE_PATH") or "", ":") + +local function find_source(path) + if pl_path.exists(path) then + return path + end + + if path:match("^=") then + -- code is executing from .conf file, don't attempt to map + return path + end + + for _, source_path in ipairs(emmy_debugger_source_path) do + local full_path = pl_path.join(source_path, path) + if pl_path.exists(full_path) then + return full_path + end + end + + ngx.log(ngx.ERR, "source file " .. path .. " not found in KONG_EMMY_DEBUGGER_SOURCE_PATH") + + return path +end + +local function init() + if not debugger then + return + end + + if not pl_path.isabs(debugger) then + ngx.log(ngx.ERR, "KONG_EMMY_DEBUGGER (" .. debugger .. ") must be an absolute path") + return + end + if not pl_path.exists(debugger) then + ngx.log(ngx.ERR, "KONG_EMMY_DEBUGGER (" .. debugger .. ") file not found") + return + end + local ext = pl_path.extension(debugger) + if ext ~= ".so" and ext ~= ".dylib" then + ngx.log(ngx.ERR, "KONG_EMMY_DEBUGGER (" .. debugger .. ") must be a .so (Linux) or .dylib (macOS) file") + return + end + if ngx.worker.id() ~= 0 then + ngx.log(ngx.ERR, "KONG_EMMY_DEBUGGER is only supported in the first worker process, suggest setting KONG_NGINX_WORKER_PROCESSES to 1") + return + end + + ngx.log(ngx.NOTICE, "loading EmmyLua debugger " .. debugger) + + _G.emmy = { + fixPath = find_source + } + + local name = pl_path.basename(debugger):sub(1, -#ext - 1) + + local save_cpath = package.cpath + package.cpath = pl_path.dirname(debugger) .. '/?' .. ext + local dbg = require(name) + package.cpath = save_cpath + + dbg.tcpListen(emmy_debugger_host, emmy_debugger_port) + + ngx.log(ngx.NOTICE, "EmmyLua debugger loaded, listening on port ", emmy_debugger_port) + + if emmy_debugger_wait then + -- Wait for IDE connection + ngx.log(ngx.NOTICE, "waiting for IDE to connect") + dbg.waitIDE() + ngx.log(ngx.NOTICE, "IDE connected") + end +end + +return { + init = init +} From 60d468209d3a640633f3312a3f87caf16c3434a5 Mon Sep 17 00:00:00 2001 From: chronolaw Date: Tue, 30 Apr 2024 09:17:59 +0800 Subject: [PATCH 2/5] docs(changelog): add changelog entry for #12814 --- .../kong/feat-hybrid-sync-mixed-route-policy.yml | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 changelog/unreleased/kong/feat-hybrid-sync-mixed-route-policy.yml diff --git a/changelog/unreleased/kong/feat-hybrid-sync-mixed-route-policy.yml b/changelog/unreleased/kong/feat-hybrid-sync-mixed-route-policy.yml new file mode 100644 index 000000000000..4d469c5e9790 --- /dev/null +++ b/changelog/unreleased/kong/feat-hybrid-sync-mixed-route-policy.yml @@ -0,0 +1,7 @@ +message: | + When CP runs with `expressions` flavor: + - if mixed config is detected and a lower DP is attached to the CP, no config will be sent at all + - if the expression is invalid on CP, no config will be sent at all + - if the expression is invalid on lower DP, it will be sent to the DP and DP validation will catch this and communicate back to the CP (this could result in partial config application) +type: feature +scope: Core From 05ed50109ca77455cb304b6e8ef58e90585139d5 Mon Sep 17 00:00:00 2001 From: chronolaw Date: Sun, 28 Apr 2024 09:44:30 +0800 Subject: [PATCH 3/5] chore(conf): update the description of router_flavor --- kong.conf.default | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kong.conf.default b/kong.conf.default index cd26ae391c1b..fac253492aee 100644 --- a/kong.conf.default +++ b/kong.conf.default @@ -1659,7 +1659,7 @@ # is not visible. # - `expressions`: the DSL based expression router engine # will be used under the hood. Traditional router - # config interface is not visible and you must write + # config interface is still visible, and you could also write # Router Expression manually and provide them in the # `expression` field on the `Route` object. # - `traditional`: the pre-3.0 Router engine will be From 410298b001ab7f04d079f4a9666ba23d5a542593 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hans=20H=C3=BCbner?= Date: Tue, 30 Apr 2024 10:50:29 +0200 Subject: [PATCH 4/5] fix(emmy-lua-debugger): add support status warning for feature --- changelog/unreleased/kong/feat-emmy-debugger.yml | 3 ++- kong/tools/emmy_debugger.lua | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/changelog/unreleased/kong/feat-emmy-debugger.yml b/changelog/unreleased/kong/feat-emmy-debugger.yml index cba62bc50a74..d43352d4e53f 100644 --- a/changelog/unreleased/kong/feat-emmy-debugger.yml +++ b/changelog/unreleased/kong/feat-emmy-debugger.yml @@ -1,3 +1,4 @@ message: | - Added support for debugging with EmmyLuaDebugger + Added support for debugging with EmmyLuaDebugger. This feature is a + tech preview and not officially supported by Kong Inc. for now. type: feature diff --git a/kong/tools/emmy_debugger.lua b/kong/tools/emmy_debugger.lua index 1795ceb1774a..9053a041f16c 100644 --- a/kong/tools/emmy_debugger.lua +++ b/kong/tools/emmy_debugger.lua @@ -53,6 +53,7 @@ local function init() end ngx.log(ngx.NOTICE, "loading EmmyLua debugger " .. debugger) + ngx.log(ngx.WARN, "The EmmyLua integration for Kong is a feature solely for your convenience during development. Kong assumes no liability as a result of using the integration and does not endorse it’s usage. Issues related to usage of EmmyLua integration should be directed to the respective project instead.") _G.emmy = { fixPath = find_source From f5a7d4ee5f8b6fd99caeed02f92700db5bfe66ee Mon Sep 17 00:00:00 2001 From: Thijs Schreijer Date: Thu, 2 May 2024 06:39:08 +0200 Subject: [PATCH 5/5] chore(makefile): add target 'test-logs' to show test logs (#12926) --- Makefile | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/Makefile b/Makefile index abeac75ec637..851c1b4655d7 100644 --- a/Makefile +++ b/Makefile @@ -39,7 +39,7 @@ endif .PHONY: install dev \ lint test test-integration test-plugins test-all \ pdk-phase-check functional-tests \ - fix-windows release wasm-test-filters + fix-windows release wasm-test-filters test-logs ROOT_DIR:=$(shell dirname $(realpath $(lastword $(MAKEFILE_LIST)))) KONG_SOURCE_LOCATION ?= $(ROOT_DIR) @@ -159,6 +159,9 @@ ifndef test_spec endif @$(VENV) $(TEST_CMD) $(test_spec) +test-logs: + tail -F servroot/logs/error.log + pdk-phase-checks: dev rm -f t/phase_checks.stats rm -f t/phase_checks.report @@ -198,3 +201,4 @@ install-legacy: @luarocks make OPENSSL_DIR=$(OPENSSL_DIR) CRYPTO_DIR=$(OPENSSL_DIR) YAML_DIR=$(YAML_DIR) dev-legacy: remove install-legacy dependencies +