Skip to content

Commit

Permalink
Add ability to explicitly create estimate function
Browse files Browse the repository at this point in the history
This adds an `Oban.Met.Migration` module and an option to disable
auto-migrating in the reporter.

Closes #4
  • Loading branch information
sorentwo committed Dec 19, 2024
1 parent dcceebf commit ea64120
Show file tree
Hide file tree
Showing 4 changed files with 146 additions and 6 deletions.
29 changes: 29 additions & 0 deletions guides/introduction/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,32 @@ declares an override to set the limit to 200k:
```elixir
{Oban.Met, reporter: [estimate_limit: 200_000]}
```

## Explicit Migrations

Met will create the necessary estimate function automatically when possible. The migration isn't
necessary under normal circumstances, but is provided to avoid permission issues or allow full
control over database changes.

```bash
mix ecto.gen.migration add_oban_met
```

Open the generated migration and delegate the `up/0` and `down/0` functions to
`Oban.Met.Migration`:

```elixir
defmodule MyApp.Repo.Migrations.AddObanMet do
use Ecto.Migration

def up, do: Oban.Met.Migration.up()
def down, do: Oban.Met.Migration.down()
end
```

Then, after disabling auto-start, configure the reporter not to auto-migrate if you run the
explicit migration:

```elixir
{Oban.Met, reporter: [auto_migrate: false]}
```
94 changes: 94 additions & 0 deletions lib/oban/met/migration.ex
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
defmodule Oban.Met.Migration do
@moduledoc """
Migrations that add estimate functionality.
> #### Migrations Are Not Required {: .tip}
>
> Met will create the necessary estimate function automatically when possible. This migration
> isn't necessary under normal circumstances, but is provided to avoid permission issues or
> allow full control over database changes.
>
> See the section on [Explicit Migrations](installation.html) for more information.
## Usage
To use migrations in your application you'll need to generate an `Ecto.Migration` that wraps
calls to `Oban.Met.Migration`:
```bash
mix ecto.gen.migration add_oban_met
```
Open the generated migration and delegate the `up/0` and `down/0` functions to
`Oban.Met.Migration`:
```elixir
defmodule MyApp.Repo.Migrations.AddObanMet do
use Ecto.Migration
def up, do: Oban.Met.Migration.up()
def down, do: Oban.Met.Migration.down()
end
```
This will run all of the necessary migrations for your database.
"""

use Ecto.Migration

@doc """
Run the `up` migration.
## Example
Run all migrations up to the current version:
Oban.Met.Migration.up()
Run migrations in an alternate prefix:
Oban.Met.Migration.up(prefix: "payments")
"""
def up(opts \\ []) when is_list(opts) do
prefix = Keyword.get(opts, :prefix, "public")

execute """
CREATE OR REPLACE FUNCTION #{prefix}.oban_count_estimate(state text, queue text)
RETURNS integer AS $func$
DECLARE
plan jsonb;
BEGIN
EXECUTE 'EXPLAIN (FORMAT JSON)
SELECT id
FROM #{prefix}.oban_jobs
WHERE state = $1::#{prefix}.oban_job_state
AND queue = $2'
INTO plan
USING state, queue;
RETURN plan->0->'Plan'->'Plan Rows';
END;
$func$
LANGUAGE plpgsql
"""
end

@doc """
Run the `down` migration.
## Example
Run all migrations up to the current version:
Oban.Met.Migration.down()
Run migrations in an alternate prefix:
Oban.Met.Migration.down(prefix: "payments")
"""
def down(opts \\ []) when is_list(opts) do
prefix = Keyword.get(opts, :prefix, "public")

execute "DROP FUNCTION IF EXISTS #{prefix}.oban_count_estimate(text, text)"
end
end
11 changes: 6 additions & 5 deletions lib/oban/met/reporter.ex
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ defmodule Oban.Met.Reporter do
:name,
:queue_timer,
:check_timer,
auto_migrate: true,
checks: @empty_states,
check_counter: 0,
check_interval: :timer.seconds(1),
Expand Down Expand Up @@ -116,17 +117,17 @@ defmodule Oban.Met.Reporter do
# An `EXPLAIN` can only be executed as the top level of a query, or through an SQL function's
# EXECUTE as we're doing here. A named function helps the performance because it is prepared,
# and we have to support distributed databases that don't allow DO/END functions.
defp create_estimate_function(%{conf: conf, function_created?: false} = state) do
defp create_estimate_function(%{auto_migrate: true, function_created?: false} = state) do
query = """
CREATE OR REPLACE FUNCTION #{conf.prefix}.oban_count_estimate(state text, queue text)
CREATE OR REPLACE FUNCTION #{state.conf.prefix}.oban_count_estimate(state text, queue text)
RETURNS integer AS $func$
DECLARE
plan jsonb;
BEGIN
EXECUTE 'EXPLAIN (FORMAT JSON)
SELECT id
FROM #{conf.prefix}.oban_jobs
WHERE state = $1::#{conf.prefix}.oban_job_state
FROM #{state.conf.prefix}.oban_jobs
WHERE state = $1::#{state.conf.prefix}.oban_job_state
AND queue = $2'
INTO plan
USING state, queue;
Expand All @@ -136,7 +137,7 @@ defmodule Oban.Met.Reporter do
LANGUAGE plpgsql
"""

Repo.query!(conf, query, [])
Repo.query!(state.conf, query, [])

%{state | function_created?: true}
end
Expand Down
18 changes: 17 additions & 1 deletion test/oban/met/reporter_test.exs
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,23 @@ defmodule Oban.Met.ReporterTest do
Notifier.listen(conf.name, [:metrics])
send(pid, :checkpoint)

refute_received {:notification, :metrics, _}
refute_receive {:notification, :metrics, _}, 20
end

@tag oban_opts: [peer: Oban.Peers.Isolated, testing: :disabled]
test "skipping estimate function creation without auto_migrate", %{conf: conf} do
pid = start_supervised!({Reporter, conf: conf, name: @name, auto_migrate: false})

Notifier.listen(conf.name, [:metrics])
send(pid, :checkpoint)

assert_receive {:notification, :metrics, _}

assert %{rows: [[false]]} =
Repo.query!(
"SELECT EXISTS(SELECT * FROM pg_proc WHERE proname = 'oban_count_estimate')",
[]
)
end
end

Expand Down

0 comments on commit ea64120

Please sign in to comment.