@@ -44,17 +44,17 @@ runner {
4444
4545See the <<parallel_execution.adoc#parallel-execution,Parallel Execution>> section for a detailed description.
4646
47- === Random Test Order Configuration
47+ === Test Order Configuration
4848
4949[source,groovy]
5050----
5151runner {
52- randomizeSpecRunOrder true // randomize specification run order
53- randomizeFeatureRunOrder true // randomize feature run order within each specification
52+ orderer new RandomSpecOrderer() // randomize specification run order
5453}
5554----
5655
57- See the <<#_randomize_run_order>> section for more details.
56+ Instead of the default run order, you can configure Spock to execute specifications and/or features e.g. in random,
57+ alphabetical or manually assigned, annotation-based order. See the <<#_run_order>> section for more details.
5858
5959== Built-In Extensions
6060
@@ -721,21 +721,111 @@ runner {
721721}
722722----
723723
724- === Randomize Run Order
724+ === Run Order
725725
726726Ideally, automated tests in general and Spock specifications in particular should be independent of each other. The same
727- applies to feature methods within a specification. One heuristic way to gain confidence that this is indeed the case, is
728- to randomize the order in which specifications and/or features within each specification are executed .
727+ applies to feature methods within a specification. Therefore, you should not rely on any specific order of execution
728+ (run order) .
729729
730- By default, Spock executes both specifications and features in deterministic order, even though users should not rely on
731- any particular run order. You can explicitly activate run order randomization separately for specifications and features
732- or combine both randomization modes, using the <<spock-configuration-file,Spock Configuration File>>:
730+ Nevertheless, you have options to influence the run order, using the <<spock-configuration-file>> and a set of built-in
731+ orderers derived from super class `SpecOrderer`. Please check the Javadocs for package
732+ `org.spockframework.runtime.extension.builtin.orderer` for more details. You can also write your own `SpecOrderer`, if
733+ none of the built-in ones satisfies your needs.
734+
735+ Please note that `@Stepwise` always trumps any run order you might have configured, i.e. `@Stepwise` "wins" against
736+ `SpecOrderer`.
737+
738+ ==== Random Run Order
739+
740+ One helpful way to heuristically increase your confidence that your tests are indeed independent of each other, is to
741+ explicitly say goodbye to deterministic run order by randomizing it.
742+
743+ [source,groovy]
744+ ----
745+ import org.spockframework.runtime.extension.builtin.orderer.RandomSpecOrderer
746+
747+ runner {
748+ orderer new RandomSpecOrderer(
749+ true, // Randomize overall specification run order
750+ true, // Randomize the run order of feature methods within specifications
751+ System.currentTimeMillis() // Set a fixed value, if you want repeatable pseudo-random numbers.
752+ // This might be helpful for reproducing issues when debugging your tests.
753+ )
754+ }
755+ ----
756+
757+ ==== Alphabetical Run Order
758+
759+ Less useful than random run order, but available anyway, is a way to execute specifications and/or features
760+ alphabetically, based on their display names and a simple `String.compareTo(String)` (no fancy locale-based collation).
761+ The default sorting direction is ascending, optionally you can also sort elements in descending order.
733762
734763[source,groovy]
735764----
765+ import org.spockframework.runtime.extension.builtin.orderer.AlphabeticalSpecOrderer
766+
736767runner {
737- randomizeSpecRunOrder true // randomize specification run order
738- randomizeFeatureRunOrder true // randomize feature run order within each specification
768+ orderer new AlphabeticalSpecOrderer(
769+ true, // Run specifications in alphabetical order by display name
770+ true, // Run feature methods within specifications in alphabetical order by display name
771+ false // Sort in ascending order (use 'true' for descending order)
772+ )
773+ }
774+ ----
775+
776+ ==== Annotation-Based Run Order
777+
778+ If you want to basically retain Spock's default run order for most or at least some of your specifications and/or
779+ feature methods, but modify it for particular specs/features, or take it to the extreme and manually assign run orders
780+ everywhere, use the `@Order(int)` annotation in combination with the annotation-based orderer:
781+
782+ [source,groovy]
783+ ----
784+ import org.spockframework.runtime.extension.builtin.orderer.AnnotatationBasedSpecOrderer
785+
786+ runner {
787+ orderer new AnnotatationBasedSpecOrderer()
788+ }
789+ ----
790+
791+ Please note, that `@Order` annotations have no effect whatsoever, if `AnnotatationBasedSpecOrderer` is not configured
792+ as the active orderer. E.g., you cannot expect to be able to use random ordering in combination with manually assigning
793+ run orders via annotations for some exceptions. Annotation-based ordering must be explicitly activated and is only
794+ available as a modification of Spock's default run order.
795+
796+ Using `@Order`, the basic idea is to assume unannotated specifications and features to all carry an implicit `@Order(0)`
797+ annotation. If you wish to run some specs/features before others, assign them a lower (negative) run order. If you want
798+ to run them after the default-ordered elements, assign them a higher (positive) order number:
799+
800+ [source,groovy]
801+ ----
802+ @Order(1) // Execute after default-ordered specs
803+ class FirstSpec extends Specification {
804+ // Execute features in order 'three', 'one', 'two' in ascending order of assigned @Order values
805+ @Order(2) def one() { expect: true }
806+ @Order(3) def two() { expect: true }
807+ @Order(1) def three() { expect: true }
808+ }
809+
810+ @Order(-1) // Execute before default-ordered specs
811+ class SecondSpec extends Specification {
812+ def foo() { expect: true } // Default order
813+ @Order(99) def bar() { expect: true } // Execute after default-ordered features
814+ @Order(-5) def zot() { expect: true } // Execute before default-ordered features
815+ }
816+
817+ // Default order
818+ class ThirdSpec extends Specification {
819+ def "some feature"() { expect: true } // Default order
820+ @Order(1) def "another feature"() { expect: true } // Execute after default-ordered features
821+ def "one more feature"() { expect: true } // Default order
822+ }
823+
824+ // Default order
825+ class FourthSpec extends Specification {
826+ def 'feature X'() { expect: true } // Default order
827+ def 'feature M'() { expect: true } // Default order
828+ @Order(-1) def 'feature D'() { expect: true } // Execute before default-ordered features
739829}
740830----
741831
0 commit comments