test-case-guide.adoc

Test Case Guide

Table of Contents

• Write a good test
• JUnit 5 extensions
  • ServiceRegistryExtension
  • DomainModelExtension
  • SessionFactoryExtension
  • DialectFilterExtension
  • ExpectedExceptionExtension
  • FailureExpectedExtension
  • LoggingInspectionsExtension and MessageKeyInspectionExtension
  • EntityManagerExtension
• JUnit 4
  • Test templates
  • Annotations

This is meant as a guide for writing test cases to be attached to bug reports in the Hibernate Jira. Really most of the information here works just as well when asking for help on community help channels (forums, IRC, HipChat, etc).

Write a good test

There are a number of tenants that make up a good test case as opposed to a poor one. In fact there are a few guides for this across the web including (MCVE) and (SSCCE). These guides all assert the same ideas albeit using different terms. Given the ubiquity of StackOverflow and the fact that the MCVE guidelines were written specifically for StackOverflow, we will use those terms here as we assume most developers have seen them before:

• (M)inimal - Provide just the minimal information needed. If second level caching is irrelevant to the bug report then the test should not use second level caching. If entity inheritance is irrelevant then do not use it in the test. If your application uses Spring Data, remove Spring Data from the test.

• (C)omplete - Provide all information needed to reproduce the problem. If a bug only occurs when using bytecode enhancement, then the test should include bytecode enhancement. In other words the test should be self-contained.

• (V)erifiable - The test should actually reproduce the problem being reported.

JUnit 5 extensions

JUnit 5 offers better support for integration, compared to JUnit 4, via extensions. Hibernate builds on those concepts in its hibernate-testing module allowing set up of test fixtures using annotations. The following sections describe the Hibernate extensions.

Note	The extensions exist in the org.hibernate.testing.orm.junit package, as opposed to the older org.hibernate.testing.junit4 package used with JUnit 4.

ServiceRegistryExtension

Manages a ServiceRegistry as part of the test lifecycle. 2 in fact, depending on the annotation(s) used.

@BootstrapServiceRegistry

configures Hibernate’s bootstrap BootstrapServiceRegistry which manages class-loading, etc. @BootstrapServiceRegistry is used to provide Java services and Hibernate Integrator implementations for the test.

@ServiceRegistry

configures Hibernate’s standard StandardServiceRegistry. @ServiceRegistry is used to provide settings, contributors, services, etc.

Also exposes ServiceRegistryScope via JUnit 5 ParameterResolver. ServiceRegistryScope allows access to the managed ServiceRegistry from tests and callbacks.

@BootstrapServiceRegistry(
    javaServices=@JavaServices(
        role=TypeContributions.class,
        impls=CustomTypeContributions.class
    ),
    ...
)
@ServiceRegistry(
    settings=@Setting(
            name="hibernate.show_sql",
            value="true"
    ),
    services=@Service(
            role=ConnectionProvider.class,
            impl=CustomConnectionProvider.class
    ),
    ...
)
class TheTest {
    @Test void testIt(ServiceRegistryScope scope) {
        StandardServiceRegistry reg = scope.getRegistry();
        ...
    }
}

DomainModelExtension

Manages the domain model for the test as part of its lifecycle.

@DomainModel

defines the sources of the domain model used in the test - type contributions, managed classes, XML mappings, etc.

If available, this extension uses the ServiceRegistry instances available from ServiceRegistryExtension.

Exposes DomainModelScope via JUnit5 ParameterResolver, allowing access to details about the domain model from the org.hibernate.mapping "boot model".

@DomainModel(
    standardDomainModels=StandardDomainModel.ANIMAL,
    annotatedClasses={Entity1.class, Entity2.class},
    xmlMappings="resource/path/to/my-mapping.xml",
    ...
)
class TheTest {
    @Test void testIt(DomainModelScope scope) {
        MetadataImplementor meta = scope.getDomainModel();
        ...

        PersistentClass entityMapping = scope.getEntityBinding(Entity1.class);
        ...

        scope.withHierarchy(Entity1.class, (entityMapping) -> {
            ...
        }
    }
}

SessionFactoryExtension

Manages a Hibernate SessionFactory as part of the test lifecycle.

@SessionFactory

is used to configure the runtime aspects of the SessionFactory fixture.

If available, uses the ServiceRegistry instances available from ServiceRegistryExtension as well as the domain model defined by DomainModelExtension.

Exposes SessionFactoryScope via JUnit5 ParameterResolver.

@SessionFactory(
    generateStatistics=true,
    exportSchema=true,
    useCollectingStatementInspector=true,
    ...
)
class TheTest {
    @Test void testIt(SessionFactoryScope scope) {
        SQLStatementInspector sqlCollector = scope.getCollectingStatementInspector();
        sqlCollector.clear();

        scope.inTransaction( (session) -> {
            ...
            assertThat(sqlCollector.getSqlQueries()).isEmpty();
        } );

        Entity1 e = scope.fromTransaction( (session) -> {
            Entity1 it = session.find(Entity1.class, id);
            ...
            return it;
        } );
    }
}

DialectFilterExtension

Allows filtering tests based on Dialect used. Implemented as a JUnit ExecutionCondition which is used to dynamically determine whether a test should be run. Used in conjunction with:

@RequiresDialect

says to only run this test for the given Dialect(s).

@SkipForDialect

says to skip this test for the given Dialect(s).

ExpectedExceptionExtension

Used with @ExpectedException to allow testing that an excepted exception occurs as the "success" condition.

@DomainModel(...)
@SessionFactory(...)
class TheTest {
    @Test
    @ExpectedException(UnknownEntityTypeException.class)
    void testIt(SessionFactoryScope) {
        scope.inTransaction( (session) -> {
            // Should fail as MyEmbeddable is not an entity
            session.find(MyEmbeddable.class, 1);
        } );
    }
}

FailureExpectedExtension

Used with @FailureExpected to indicate that a test is (currently) expected to fail. You might use this, e.g., for a test that is the reproducer for a bug report before working on it. It basically just flips the success/failure condition. In fact, a test marked with @FailureExpected will be marked a failure if it succeeds.

@Test
@JiraKey("HHH-123456789")
@FailureExpected
void bugReproducer(...) {...}

LoggingInspectionsExtension and MessageKeyInspectionExtension

Both are used for testing log messages.

@LoggingInspections

used to watch more than one "message key".

MessageKeyInspection

used to watch a single "message key".

EntityManagerExtension

Used in conjunction with @Jpa to build tests with an EntityManagerFactory fixture.

Since Hibernate’s SessionFactory is a EntityManagerFactory, @BootstrapServiceRegistry, @ServiceRegistry, @DomainModel and @SessionFactory can also be used to perform tests with a (SessionFactory as a) EntityManagerFactory fixture.

The distinction with @Jpa is that EntityManagerExtension uses the JPA-defined bootstrap APIs. How the SessionFactory is built is the difference.

JUnit 4

Historically, Hibernate used JUnit 4 for its test suite. Since the release of JUnit 5, we’ve moved to using the testing approach outlined in JUnit 5 extensions. However, many existing tests still use the legacy JUnit 4 based infrastructure (boilerplate) based on "test templates".

Test templates

The Hibernate team maintains a set of "test templates" intended to help developers write tests. These test templates are maintained in GitHub @ hibernate-test-case-templates

• If you want to use the Hibernate native API, you should follow the instructions from this article.

• If you want to use JPA, you should use the JPA templates that were detailed in this article.

Note	the test templates are generally not a good starting point for problems building the SessionFactory/EntityManager. In JUnit terms they manage the SessionFactory/EntityManager as set-up and teardown constructs._

Annotations

When using "test templates" you can annotate a single test or a whole test class with one of the following annotations:

• FailureExpected - allows to skip a single test or all tests of a class, because test failures are expected. The test will actually run, but not lead to an error report. In fact if a test is marked with @FailureExpected and the test actually succeeds, an error occurs. As a parameter to this annotation a jira key is required.

• NotImplementedYet - test classes or methods annotated with @NotImplementedYet will run but not fail if the feature(s) that are being tested are not implemented yet for the current version. Optionally, a message and a version that is expected to have the feature already implemented can be provided as parameters.

• RequiresDialect - tests methods/classes annotated with @RequiresDialect will only run if the current Dialect matches the one specified as annotation parameter. You can also specify a comment and/or jira key explaining why this test requires a certain dialect

• RequiresDialectFeature - tests methods/classes annotated with @RequiresDialectFeature will only run if the current Dialect offers the specified feature. Examples for this features are SupportsSequences, SupportsExpectedLobUsagePattern or SupportsIdentityColumns. You can add more features if you need to. Have a look at DialectChecks.

• SkipForDialect - tests methods/classes annotated with @SkipForDialect will not run if the current Dialect matches the one specified as annotation parameter. You can also specify a comment and/or jira key explaining why this test has to be skipped for the Dialect.