Skip to content

Commit

Permalink
fix(pluginservers): restart all instances of a plugin
Browse files Browse the repository at this point in the history
When the plugin -- which acts as a plugin server -- dies for some
reason, all entities associated with it must be invalidated, not only
the current.

If two instances A and B of an external plugin P exist (e.g., if applied
to different route objects), they receive a unique ID [1] defined by the
the plugin server when the instance is started. If A and B are started
sequentially and are the first instances, they get IDs 0 and 1, respectively.
This ID is cached on the Kong side by the plugin server manager [2]. If
plugin P dies, Kong plugin server manager will detect this condition and
restart it.

As it was, the code would remove whichever instance first
detected the condition and leave the other in `running_instances`; once
the plugin server was up, the removed instance would be restarted, in
some cases with the same Id already in use by the stale instance that
should have been removed from `running_instances`. Subsequent requests
may then run the wrong instance of the external plugin. This change
fixes the issue by ensuring that both instances A and B are removed
from the `running_instances` table,

[1] https://github.com/Kong/go-pdk/blob/4deec45a5bfe421a0ceebcbf7931d395250949b0/server/instance.go#L47
[2] `kong/runloop/plugin_servers/process.lua`
  • Loading branch information
gszr committed Jul 27, 2023
1 parent d5ef304 commit 32528da
Show file tree
Hide file tree
Showing 2 changed files with 19 additions and 1 deletion.
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,11 @@
- Make `kong vault get` CLI command work in dbless mode by injecting the necessary directives into the kong cli nginx.conf.
[#11127](https://github.com/Kong/kong/pull/11127)
[#11291](https://github.com/Kong/kong/pull/11291)
- Fix an issue where a crashing Go plugin server process would cause subsequent
requests proxied through Kong to execute Go plugins with inconsistent configurations.
The issue only affects scenarios where the same Go plugin is applied to different Route
or Service entities.
[#11306](https://github.com/Kong/kong/pull/11306)

#### Admin API

Expand Down
15 changes: 14 additions & 1 deletion kong/runloop/plugin_servers/init.lua
Original file line number Diff line number Diff line change
Expand Up @@ -270,7 +270,20 @@ end
--- reset_instance: removes an instance from the table.
function reset_instance(plugin_name, conf)
local key = type(conf) == "table" and kong.plugin.get_id() or plugin_name
running_instances[key] = nil
local current_instance = running_instances[key]

--
-- the same plugin (which acts as a plugin server) is shared among
-- instances of the plugin; for example, the same plugin can be applied
-- to many routes
-- `reset_instance` is called when (but not only) the plugin server died;
-- in such case, all associated instances must be removed, not only the current
--
for k, instance in pairs(running_instances) do
if instance.rpc == current_instance.rpc then
running_instances[k] = nil
end
end
end


Expand Down

1 comment on commit 32528da

@khcp-gha-bot
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bazel Build

Docker image available kong/kong:32528da4f09d26cfc6e19ffe78025d4038c69d51
Artifacts available https://github.com/Kong/kong/actions/runs/5684843169

Please sign in to comment.