Skip to content

Commit

Permalink
Extract Ham out of Hammox
Browse files Browse the repository at this point in the history
  • Loading branch information
@edgurgel
edgurgel committed Aug 18, 2024
1 parent 8f070dd commit 9d98de7
Show file tree
Hide file tree
Showing 37 changed files with 1,034 additions and 1,833 deletions.
54 changes: 44 additions & 10 deletions .github/workflows/ci.yml
View file
Original file line number Diff line number Diff line change
@@ -1,20 +1,54 @@
on: push
name: CI

on: [push, pull_request]

jobs:
format:
name: Format & credo
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Install OTP and Elixir
uses: erlef/setup-beam@v1
with:
otp-version: 26.x
elixir-version: 1.16.x

- name: Install dependencies
run: mix deps.get

- name: Compile with --warnings-as-errors
run: mix compile --warnings-as-errors

- name: Run "mix format"
run: mix format --check-formatted

test:
name: Test (Elixir ${{matrix.elixir}} | Erlang/OTP ${{matrix.otp}})
runs-on: ubuntu-latest
name: OTP ${{matrix.otp}} / Elixir ${{matrix.elixir}}
strategy:
fail-fast: false
matrix:
otp: ['25.0']
elixir: ['1.13.4']
include:
- otp: 26.x
elixir: 1.16.x
coverage: true
- otp: 27.x
elixir: 1.17.x
env:
MIX_ENV: test
steps:
- uses: actions/checkout@v2
- uses: erlef/setup-beam@v1
- uses: actions/checkout@v4

- name: Install OTP and Elixir
uses: erlef/setup-beam@v1
with:
otp-version: ${{matrix.otp}}
elixir-version: ${{matrix.elixir}}
- run: mix deps.get
- run: mix format --check-formatted
- run: mix credo --strict
- run: mix test

- name: Install dependencies
run: mix deps.get --only test

- name: Run tests
run: mix test --trace
2 changes: 1 addition & 1 deletion .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ erl_crash.dump
*.ez

# Ignore package tarball (built via "mix hex.build").
hammox-*.tar
ham-*.tar

# Ignore ElixirLS cache
.elixir_ls
1 change: 1 addition & 0 deletions LICENSE
View file
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,7 @@
identification within third-party archives.

Copyright 2019 Michał Szewczak
Copyright 2024 Eduardo Gurgel

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand Down
246 changes: 58 additions & 188 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,246 +1,116 @@
# Hammox
# Ham

[![CI](https://github.com/msz/hammox/actions/workflows/ci.yml/badge.svg)](https://github.com/msz/hammox/actions/workflows/ci.yml)
[![Module Version](https://img.shields.io/hexpm/v/hammox.svg)](https://hex.pm/packages/hammox)
[![Hex Docs](https://img.shields.io/badge/hex-docs-lightgreen.svg)](https://hexdocs.pm/hammox/)
[![Total Download](https://img.shields.io/hexpm/dt/hammox.svg)](https://hex.pm/packages/hammox)
[![License](https://img.shields.io/hexpm/l/hammox.svg)](https://github.com/msz/hammox/blob/master/LICENSE)
[![Last Updated](https://img.shields.io/github/last-commit/msz/hammox.svg)](https://github.com/msz/hammox/commits/master)
[![CI](https://github.com/edgurgel/ham/actions/workflows/ci.yml/badge.svg)](https://github.com/edgurgel/ham/actions/workflows/ci.yml)
[![Module Version](https://img.shields.io/hexpm/v/edgurgel.svg)](https://hex.pm/packages/ham)
[![Hex Docs](https://img.shields.io/badge/hex-docs-lightgreen.svg)](https://hexdocs.pm/ham)
[![Total Download](https://img.shields.io/hexpm/dt/ham.svg)](https://hex.pm/packages/ham)
[![License](https://img.shields.io/hexpm/l/ham.svg)](https://github.com/edgurgel/ham/blob/main/LICENSE)
[![Last Updated](https://img.shields.io/github/last-commit/edgurgel/ham.svg)](https://github.com/edgurgel/ham/commits/main)

Hammox is a library for rigorous unit testing using mocks, explicit
behaviours and contract tests. You can use it to ensure both your mocks and
implementations fulfill the same contract.
Ham is a library to validate function arguments and return values against their typespecs.
It was originally extracted out from [Ham](https://github.com/msz/hammox/) but
without the Mox integration. Hammox - Mox = Ham!
Thanks [@msz](https://github.com/msz) for creating Hammox!

It takes the excellent [Mox](https://github.com/plataformatec/mox) library
and pushes its philosophy to its limits, providing automatic contract tests
based on behaviour typespecs while maintaining full compatibility with code
already using Mox.

Hammox aims to catch as many contract bugs as possible while providing useful
deep stacktraces so they can be easily tracked down and fixed.
The main reason why I wanted to extract it out is so that it can be used as part of Mimic or other testing libraries.

## Installation

If you are currently using [Mox](https://github.com/plataformatec/mox),
delete it from your list of dependencies in `mix.exs`. Then add `:hammox`:
Add `:ham`:

```elixir
def deps do
[
{:hammox, "~> 0.7", only: :test}
{:ham, "~> 0.1", only: :test}
]
end
```

## Starting from scratch

Read ["Mocks and explicit contracts"](http://blog.plataformatec.com.br/2015/10/mocks-and-explicit-contracts/)
by José Valim. Then proceed to the [Mox documentation](https://hexdocs.pm/mox/Mox.html).
Once you are comfortable with Mox, switch to using Hammox.

## Migrating from Mox

Replace all occurrences of `Mox` with `Hammox`. Nothing more is required; all
your mock calls in test are now ensured to conform to the behaviour typespec.
## Using Ham

## Example
One can simply let Ham apply a module function using `Ham.apply/2` (function or macro) and it will validate argument and return value:

### Typical mock setup

Let's say we have a database which can get us user data. We have a module,
`RealDatabase` (not shown), which implements the following behaviour:
```elixir
defmodule Database do
@callback get_users() :: [binary()]
end
```
We use this client in a `Stats` module which can aggregate data about users:
```elixir
defmodule Stats do
def count_users(database \\ RealDatabase) do
length(database.get_users())
end
end
```
And we create a unit test for it:
```elixir
defmodule StatsTest do
use ExUnit.Case, async: true

test "count_users/0 returns correct user count" do
assert 2 == Stats.count_users()
end
end
iex> Ham.apply(URI, :decode, ["https%3A%2F%2Felixir-lang.org"])
"https://elixir-lang.org"
iex> Ham.apply(URI, :char_reserved?, ["a"])
** (Ham.TypeMatchError) 1st argument value "a" does not match 1st parameter's type byte().
Value "a" does not match type 0..255.
```
For this test to work, we would have to start a real instance of the database
and provision it with two users. This is of course unnecessary brittleness —
in a unit test, we only want to test that our Stats code provides correct
results given specific inputs. To simplify, we will create a mocked Database
using Mox and use it in the test:

```elixir
defmodule StatsTest do
use ExUnit.Case, async: true
import Mox

test "count_users/0 returns correct user count" do
defmock(DatabaseMock, for: Database)
expect(DatabaseMock, :get_users, fn ->
["joe", "jim"]
end)

assert 2 == Stats.count_users(DatabaseMock)
end
end
iex> Ham.apply(URI.decode("https%3A%2F%2Felixir-lang.org"))
"https://elixir-lang.org"
iex> Ham.apply(URI.char_reserved?("a"))
** (Ham.TypeMatchError) 1st argument value "a" does not match 1st parameter's type byte().
Value "a" does not match type 0..255.
```
The test now passes as expected.

### The contract breaks
Another way is to pass the args and return value without Ham executing anything:

Imagine that some time later we want to add error flagging for our database
client. We change `RealDatabase` and the corresponding behaviour, `Database`,
to return an ok/error tuple instead of a raw value:
```elixir
defmodule Database do
@callback get_users() :: {:ok, [binary()]} | {:error, term()}
end
```

However, The `Stats.count_users/0` test *will still pass*, even though the
function will break when the real database client is used! This is because
the mock is now invalid — it no longer implements the given behaviour, and
therefore breaks the contract. Even though Mox is supposed to create mocks
following explicit contracts, it does not take typespecs into account.

This is where Hammox comes in. Simply replace all occurrences of Mox with
Hammox (for example, `import Mox` becomes `import Hammox`, etc) and you
will now get this when trying to run the test:

```none
** (Hammox.TypeMatchError)
Returned value ["joe", "jim"] does not match type {:ok, [binary()]} | {:error, term()}.
iex> Ham.validate(URI, :char_reserved?, ["a"], true)
{:error,
%Ham.TypeMatchError{
reasons: [
{:arg_type_mismatch, 0, "a", {:type, {324, 24}, :byte, []}},
{:type_mismatch, "a", {:type, 0, :range, [{:integer, 0, 0}, {:integer, 0, 255}]}}
]
}}

iex> Ham.validate!(URI, :char_reserved?, ["a"], true)
** (Ham.TypeMatchError) 1st argument value "a" does not match 1st parameter's type byte().
Value "a" does not match type 0..255.
```
Now the consistency between the mock and its behaviour is enforced.

### Completing the triangle
Both `apply` and `validate` accept `behaviours` as an option to declare that the module
implements certain behaviours.
Hammox automatically checks mocks with behaviours, but what about the real
implementations? The real goal is to keep all units implementing a given
behaviour in sync.

You can decorate any function with Hammox checks by using `Hammox.protect/2`.
It will return an anonymous function which you can use in place of the
original module function. An example test:
For example let's implement a module that does a poor job of implementing `Access`

```elixir
defmodule RealDatabaseTest do
use ExUnit.Case, async: true
defmodule CustomAccess do
@behaviour Access

test "get_users/0 returns list of users" do
get_users_0 = Hammox.protect({RealDatabase, :get_users, 0}, Database)
assert {:ok, ["real-jim", "real-joe"]} == get_users_0.()
end
def fetch(_data, _key), do: :poor
def get_and_update(_data, _key, _function), do: :poor
def pop(data, key), do: :poor
end
```

It's a good idea to put setup logic like this in a `setup_all` hook and then
access the protected functions using the test context:

```elixir
defmodule RealDatabaseTest do
use ExUnit.Case, async: true

setup_all do
%{get_users_0: Hammox.protect({RealDatabase, :get_users, 0}, Database)}
end

test "get_users/0 returns list of users", %{get_users_0: get_users_0} do
assert {:ok, ["real-jim", "real-joe"]} == get_users_0.()
end
end
```

Hammox also provides a `setup_all` friendly version of `Hammox.protect` which
leverages this pattern. Simply pass both the implementation module and the
behaviour module and you will get a map of all callbacks defined by the
behaviour as decorated implementation functions.

```elixir
defmodule RealDatabaseTest do
use ExUnit.Case, async: true

setup_all do
Hammox.protect(RealDatabase, Database)
end

test "get_users/0 returns list of users", %{get_users_0: get_users_0} do
assert {:ok, ["real-jim", "real-joe"]} == get_users_0.()
end
end
```

Alternatively, if you're up for trading explicitness for some macro magic,
you can use `use Hammox.Protect` to locally define protected versions of
functions you're testing, as if you `import`ed the module:

```elixir
defmodule RealDatabaseTest do
use ExUnit.Case, async: true
use Hammox.Protect, module: RealDatabase, behaviour: Database

test "get_users/0 returns list of users" do
assert {:ok, ["real-jim", "real-joe"]} == get_users()
end
end
Ham.apply(CustomAccess.fetch([], "key"), behaviours: [Access])
** (Ham.TypeMatchError) Returned value :poor does not match type {:ok, Access.value()} | :error.
Value :wrong does not match type {:ok, Access.value()} | :error.
```

## Why use Hammox for my application code when I have Dialyzer?

Dialyzer cannot detect Mox style mocks not conforming to typespec.
## Why use Ham for my application code when I have Dialyzer?

The main aim of Hammox is to enforce consistency between behaviours, mocks
and implementations. This is best achieved when both mocks and
implementations are subjected to the exact same checks.

Dialyzer is a static analysis tool; Hammox is a dynamic contract test
provider. They operate differently and one can catch some bugs when the other
doesn't. While it is true that Hammox would be redundant given a strong,
strict, TypeScript-like type system for Elixir, Dialyzer is far from providing
that sort of coverage.
Dialyzer is a powerful static analysis tool that can uncover serious problems.
But during tests dialyzer is not as useful. Ham can be used to validate function arguments
and return values during tests even if they are randomly generated or loaded from files.

## Protocol types

A `t()` type defined on a protocol is taken by Hammox to mean "a struct
A `t()` type defined on a protocol is taken by Ham to mean "a struct
implementing the given protocol". Therefore, trying to pass `:atom` for an
`Enumerable.t()` will produce an error, even though the type is defined as
`term()`:

```none
** (Hammox.TypeMatchError)
** (Ham.TypeMatchError)
Returned value :atom does not match type Enumerable.t().
Value :atom does not implement the Enumerable protocol.
```

## Disable protection for specific mocks

Hammox also includes Mox as a dependency. This means that if you would like
to disable Hammox protection for a specific mock, you can simply use vanilla
Mox for that specific instance. They will interoperate without problems.

## Limitations
- For anonymous function types in typespecs, only the arity is checked.
Parameter types and return types are not checked.

## Telemetry

Hammox now includes telemetry events! See [Telemetry Guide](https://hexdocs.pm/hammox/Telemetry.html) for more information.

## License

Copyright 2019 Michał Szewczak

Copyright 2024 Eduardo Gurgel

Licensed under the Apache License, Version 2.0 (the "License"); you may not
use this file except in compliance with the License. You may obtain a copy of
the License at
Expand Down
Loading

0 comments on commit 9d98de7

Please sign in to comment.