Trait providing an apply method to which alert messages about a running suite of tests can be reported.

An Alerter is essentially used to wrap a Reporter and provide easy ways to send alert messages to that Reporter via an AlertProvided event. Alerter contains an apply method that takes a string and an optional payload object of type Any. The Alerter will forward the passed alert message string to the Reporter as the message parameter, and the optional payload object as the payload parameter, of an AlertProvided event.

For insight into the differences between Alerter, Notifier, and Informer, see the main documentation for trait Alerting.

Trait that contains the alert method, which can be used to send an alert to the reporter.

One difference between alert and the info method of Informer is that info messages provided during a test are recorded and sent as part of test completion event, whereas alert messages are sent right away as AlertProvided messages. For long-running tests, alert allows you to send "alert notifications" to the reporter right away, so users can be made aware of potential problems being experienced by long-running tests. By contrast, info messages will only be seen by the user after the test has completed, and are more geared towards specification (such as Given/When/Then messages) than notification.

The difference between alert and the update method of Updating is that alert is intended to be used for warnings or notifications of potential problems, whereas update is just for status updates. In string reporters for which ANSI color is enabled, update notifications are shown in green and alert notifications in yellow.

Trait providing an implicit conversion that allows clues to be placed after a block of code.

You can use the withClue construct provided by Assertions, which is extended by every style trait in ScalaTest, to add extra information to reports of failed or canceled tests. The withClue from Assertions places the "clue string" at the front, both in the code and in the resulting message:

withClue("This is a prepended clue;") {
  1 + 1 should equal (3)
}

The above expression will yield the failure message:

This is a prepended clue; 2 did not equal 3

If you mix in this trait, or import its members via its companion object, you can alternatively place the clue string at the end, like this:

{ 1 + 1 should equal (3) } withClue "now the clue comes after"

The above expression will yield the failure message:

2 did not equal 3 now the clue comes after

If no space is already present, either at the beginning of the clue string or at the end of the current message, a space will be placed between the two, unless the clue string starts with one of the punctuation characters: comma (,), period (.), or semicolon (;). For example, the failure message in the above example includes an extra space inserted between 3 and now.

By contrast this code, which has a clue string starting with comma:

{ 1 + 1 should equal (3) } withClue ", now the clue comes after"

Will yield a failure message with no extra inserted space:

2 did not equal 3, now the clue comes after

The withClue method will only append the clue string to the detail message of exception types that mix in the ModifiableMessage trait. See the documentation for ModifiableMessage for more information.

Note: the reason this functionality is not provided by Assertions directly, like the prepended withClue construct, is because appended clues require an implicit conversion. ScalaTest only gives you one implicit conversion by default in any test class to minimize the potential for conflicts with other implicit conversions you may be using. All other implicit conversions, including the one provided by this trait, you must explicitly invite into your code through inheritance or an import.

Arguments bundle passed to four of ScalaTest's lifecycle methods: run, runNestedSuites, runTests, and runTest.

The signatures of these methods, defined in trait Suite, are:

def run(testName: Option[String], args: Args)
def runNestedSuites(args: Args)
def runTests(testName: Option[String], args: Args)
def runTest(testName: String, args: Args)

The purpose of bundling these arguments into an Args object instead of passing them in individually is to make the signature of these four lifecycle methods easier to read, write, and remember, as well as to make the methods more pleasant to override in user code.

Trait that contains ScalaTest's basic assertion methods.

You can use the assertions provided by this trait in any ScalaTest Suite, because Suite mixes in this trait. This trait is designed to be used independently of anything else in ScalaTest, though, so you can mix it into anything. (You can alternatively import the methods defined in this trait. For details, see the documentation for the Assertions companion object.

In any Scala program, you can write assertions by invoking assert and passing in a Boolean expression, such as:

val left = 2
val right = 1
assert(left == right)

If the passed expression is true, assert will return normally. If false, Scala's assert will complete abruptly with an AssertionError. This behavior is provided by the assert method defined in object Predef, whose members are implicitly imported into every Scala source file. This Assertions trait defines another assert method that hides the one in Predef. It behaves the same, except that if false is passed it throws TestFailedException instead of AssertionError. Why? Because unlike AssertionError, TestFailedException carries information about exactly which item in the stack trace represents the line of test code that failed, which can help users more quickly find an offending line of code in a failing test. In addition, ScalaTest's assert provides better error messages than Scala's assert.

If you pass the previous Boolean expression, left == right to assert in a ScalaTest test, a failure will be reported that, because assert is implemented as a macro, includes reporting the left and right values. For example, given the same code as above but using ScalaTest assertions:

import org.scalatest.Assertions._
val left = 2
val right = 1
assert(left == right)

The detail message in the thrown TestFailedException from this assert will be: "2 did not equal 1".

ScalaTest's assert macro works by recognizing patterns in the AST of the expression passed to assert and, for a finite set of common expressions, giving an error message that an equivalent ScalaTest matcher expression would give. Here are some examples, where a is 1, b is 2, c is 3, d is 4, xs is List(a, b, c), and num is 1.0:

assert(a == b || c >= d)
// Error message: 1 did not equal 2, and 3 was not greater than or equal to 4

assert(xs.exists(_ == 4))
// Error message: List(1, 2, 3) did not contain 4

assert("hello".startsWith("h") && "goodbye".endsWith("y"))
// Error message: "hello" started with "h", but "goodbye" did not end with "y"

assert(num.isInstanceOf[Int])
// Error message: 1.0 was not instance of scala.Int

assert(Some(2).isEmpty)
// Error message: Some(2) was not empty

For expressions that are not recognized, the macro currently prints out a string representation of the (desugared) AST and adds "was false". Here are some examples of error messages for unrecognized expressions:

assert(None.isDefined)
// Error message: scala.None.isDefined was false

assert(xs.exists(i => i > 10))
// Error message: xs.exists(((i: Int) => i.>(10))) was false

You can augment the standard error message by providing a String as a second argument to assert, like this:

val attempted = 2
assert(attempted == 1, "Execution was attempted " + attempted + " times instead of 1 time")

Using this form of assert, the failure report will be more specific to your problem domain, thereby helping you debug the problem. This Assertions trait also mixes in the TripleEquals, which gives you a === operator that allows you to customize Equality, perform equality checks with numeric Tolerance, and enforce type constraints at compile time with sibling traits TypeCheckedTripleEquals and ConversionCheckedTripleEquals.

Expected results

Although the assert macro provides a natural, readable extension to Scala's assert mechanism that provides good error messages, as the operands become lengthy, the code becomes less readable. In addition, the error messages generated for == and === comparisons don't distinguish between actual and expected values. The operands are just called left and right, because if one were named expected and the other actual, it would be difficult for people to remember which was which. To help with these limitations of assertions, Suite includes a method called assertResult that can be used as an alternative to assert. To use assertResult, you place the expected value in parentheses after assertResult, followed by curly braces containing code that should result in the expected value. For example:

val a = 5
val b = 2
assertResult(2) {
  a - b
}

In this case, the expected value is 2, and the code being tested is a - b. This assertion will fail, and the detail message in the TestFailedException will read, "Expected 2, but got 3."

Forcing failures

If you just need the test to fail, you can write:

fail()

Or, if you want the test to fail with a message, write:

fail("I've got a bad feeling about this")

Achieving success

In async style tests, you must end your test body with either Future[Assertion] or Assertion. ScalaTest's assertions (including matcher expressions) have result type Assertion, so ending with an assertion will satisfy the compiler. If a test body or function body passed to Future.map does not end with type Assertion, however, you can fix the type error by placing succeed at the end of the test or function body:

succeed // Has type Assertion

Expected exceptions

Sometimes you need to test whether a method throws an expected exception under certain circumstances, such as when invalid arguments are passed to the method. You can do this in the JUnit 3 style, like this:

val s = "hi"
try {
  s.charAt(-1)
  fail()
}
catch {
  case _: IndexOutOfBoundsException => // Expected, so continue
}

If charAt throws IndexOutOfBoundsException as expected, control will transfer to the catch case, which does nothing. If, however, charAt fails to throw an exception, the next statement, fail(), will be run. The fail method always completes abruptly with a TestFailedException, thereby signaling a failed test.

To make this common use case easier to express and read, ScalaTest provides two methods: assertThrows and intercept. Here's how you use assertThrows:

val s = "hi"
assertThrows[IndexOutOfBoundsException] { // Result type: Assertion
  s.charAt(-1)
}

This code behaves much like the previous example. If charAt throws an instance of IndexOutOfBoundsException, assertThrows will return Succeeded. But if charAt completes normally, or throws a different exception, assertThrows will complete abruptly with a TestFailedException.

The intercept method behaves the same as assertThrows, except that instead of returning Succeeded, intercept returns the caught exception so that you can inspect it further if you wish. For example, you may need to ensure that data contained inside the exception have expected values. Here's an example:

val s = "hi"
val caught =
  intercept[IndexOutOfBoundsException] { // Result type: IndexOutOfBoundsException
    s.charAt(-1)
  }
assert(caught.getMessage.indexOf("-1") != -1)

Checking that a snippet of code does or does not compile

Often when creating libraries you may wish to ensure that certain arrangements of code that represent potential “user errors” do not compile, so that your library is more error resistant. ScalaTest's Assertions trait includes the following syntax for that purpose:

assertDoesNotCompile("val a: String = 1")

If you want to ensure that a snippet of code does not compile because of a type error (as opposed to a syntax error), use:

assertTypeError("val a: String = 1")

Note that the assertTypeError call will only succeed if the given snippet of code does not compile because of a type error. A syntax error will still result on a thrown TestFailedException.

If you want to state that a snippet of code does compile, you can make that more obvious with:

assertCompiles("val a: Int = 1")

Although the previous three constructs are implemented with macros that determine at compile time whether the snippet of code represented by the string does or does not compile, errors are reported as test failures at runtime.

Assumptions

Trait Assertions also provides methods that allow you to cancel a test. You would cancel a test if a resource required by the test was unavailable. For example, if a test requires an external database to be online, and it isn't, the test could be canceled to indicate it was unable to run because of the missing database. Such a test assumes a database is available, and you can use the assume method to indicate this at the beginning of the test, like this:

assume(database.isAvailable)

For each overloaded assert method, trait Assertions provides an overloaded assume method with an identical signature and behavior, except the assume methods throw TestCanceledException whereas the assert methods throw TestFailedException. As with assert, assume hides a Scala method in Predef that performs a similar function, but throws AssertionError. And just as you can with assert, you will get an error message extracted by a macro from the AST passed to assume, and can optionally provide a clue string to augment this error message. Here are some examples:

assume(database.isAvailable, "The database was down again")
assume(database.getAllUsers.count === 9)

Forcing cancelations

For each overloaded fail method, there's a corresponding cancel method with an identical signature and behavior, except the cancel methods throw TestCanceledException whereas the fail methods throw TestFailedException. Thus if you just need to cancel a test, you can write:

cancel()

If you want to cancel the test with a message, just place the message in the parentheses:

cancel("Can't run the test because no internet connection was found")

Getting a clue

If you want more information that is provided by default by the methods if this trait, you can supply a "clue" string in one of several ways. The extra information (or "clues") you provide will be included in the detail message of the thrown exception. Both assert and assertResult provide a way for a clue to be included directly, intercept does not. Here's an example of clues provided directly in assert:

assert(1 + 1 === 3, "this is a clue")

and in assertResult:

assertResult(3, "this is a clue") { 1 + 1 }

The exceptions thrown by the previous two statements will include the clue string, "this is a clue", in the exception's detail message. To get the same clue in the detail message of an exception thrown by a failed intercept call requires using withClue:

withClue("this is a clue") {
  intercept[IndexOutOfBoundsException] {
    "hi".charAt(-1)
  }
}

The withClue method will only prepend the clue string to the detail message of exception types that mix in the ModifiableMessage trait. See the documentation for ModifiableMessage for more information. If you wish to place a clue string after a block of code, see the documentation for AppendedClues.

Note: ScalaTest's assertTypeError construct is in part inspired by the illTyped macro of shapeless.

The base trait of ScalaTest's asynchronous testing styles, which defines a withFixture lifecycle method that accepts as its parameter a test function that returns a FutureOutcome.

The withFixture method add by this trait has the following signature and implementation:

def withFixture(test: NoArgAsyncTest): FutureOutcome = {
  test()
}

This trait enables testing of asynchronous code without blocking. Instead of returning Outcome like TestSuite's withFixture, this trait's withFixture method returns a FutureOutcome. Similarly, the apply method of test function interface, NoArgAsyncTest, returns FutureOutcome:

// In trait NoArgAsyncTest:
def apply(): FutureOutcome

The withFixture method supports async testing, because when the test function returns, the test body has not necessarily finished execution.

The recommended way to ensure cleanup is performed after a test body finishes execution is to use a complete-lastly clause, syntax that is defined in trait CompleteLastly, which this trait extends. Using cleanup-lastly will ensure that cleanup will occur whether FutureOutcome-producing code completes abruptly by throwing an exception, or returns normally yielding a FutureOutcome. In the latter case, complete-lastly will register the cleanup code to execute asynchronously when the FutureOutcome completes.

The withFixture method is designed to be stacked, and to enable this, you should always call the super implementation of withFixture, and let it invoke the test function rather than invoking the test function directly. In other words, instead of writing “test()”, you should write “super.withFixture(test)”. Thus, the recommended structure of a withFixture implementation that performs cleanup looks like this:

// Your implementation
override def withFixture(test: NoArgAsyncTest) = {
  // Perform setup here
  complete {
    super.withFixture(test) // Invoke the test function
  } lastly {
    // Perform cleanup here
  }
}

If you have no cleanup to perform, you can write withFixture like this instead:

// Your implementation
override def withFixture(test: NoArgAsyncTest) = {
  // Perform setup here
  super.withFixture(test) // Invoke the test function
}

The test function and withFixture method returns a FutureOutcome, a ScalaTest class that wraps a Scala Future[Outcome] and offers methods more specific to asynchronous test outcomes. In a Scala Future, any exception results in a scala.util.Failure. In a FutureOutcome, a thrown TestPendingException always results in a Pending, a thrown TestCanceledException always results in a Canceled, and any other exception, so long as it isn't suite-aborting, results in a Failed. This is true of the asynchronous test code itself that's represented by the FutureOutcome and any transformation or callback registered on the FutureOutcome in withFixture.

If you want to perform an action only for certain outcomes, you'll need to register code performing that action on the FutureOutcome using one of FutureOutcome's callback registration methods:

• onSucceededThen - executed if the Outcome is a Succeeded.
• onFailedThen - executed if the Outcome is a Failed.
• onCanceledThen - executed if the Outcome is a Canceled.
• onPendingThen - executed if the Outcome is a Pending.
• onOutcomeThen - executed on any Outcome (i.e., no suite-aborting exception is thrown).
• onAbortedThen - executed if a suite-aborting exception is thrown.
• onCompletedThen - executed whether the result is an Outcome or a thrown suite-aborting exception.

For example, if you want to perform an action if a test fails, you'd register the callback using onFailedThen, like this:

// Your implementation
override def withFixture(test: NoArgAsyncTest) = {

  // Perform setup here

  val futureOutcome = super.withFixture(test) // Invoke the test function

  futureOutcome onFailedThen { ex =>
    // perform action that you want to occur
    // only if a test fails here
  }
}

Note that all callback registration methods, such as onFailedThen used in the previous example, return a new FutureOutcome that won't complete until the the original FutureOutcome and the callback has completed. If the callback throws an exception, the resulting FutureOutcome will represent that exception. For example, if a FutureOutcome results in Failed, but a callback registered on that FutureOutcome with onFailedThen throws TestPendingException, the result of the FutureOutcome returned by onFailedThen will be Pending.

Lastly, if you want to change the outcome in some way in withFixture, you'll need to use the change method of FutureOutcome, like this:

// Your implementation
override def withFixture(test: NoArgAsyncTest) = {

  // Perform setup here

  val futureOutcome = super.withFixture(test) // Invoke the test function

  futureOutcome change { outcome =>
    // transform the outcome into a new outcome here
  }
}

Trait defining abstract "lifecycle" methods that are implemented in AsyncTestSuite and can be overridden in stackable modification traits.

The main use case for this trait is to override withFixture in a mixin trait. Here's an example:

trait Builder extends AsyncTestSuiteMixin { this: AsyncTestSuite =>

  final val builder = new ThreadSafeStringBuilder

  abstract override def withFixture(test: NoArgAsyncTest) = {
    builder.append("ScalaTest is ")
    complete {
      super.withFixture(test) // To be stackable, must call super.withFixture
    } lastly {
      builder.clear()
    }
  }
}

Trait that can be mixed into suites that need code executed before and after running each test.

A test fixture is composed of the objects and other artifacts (files, sockets, database connections, etc.) tests use to do their work. When multiple tests need to work with the same fixtures, it is important to try and avoid duplicating the fixture code across those tests. The more code duplication you have in your tests, the greater drag the tests will have on refactoring the actual production code. Trait BeforeAndAfter offers one way to eliminate such code duplication: a before clause that will register code to be run before each test, and an after clause that will register code to be run after.

Here's an example:

package org.scalatest.examples.flatspec.beforeandafter

import org.scalatest._
import collection.mutable.ListBuffer

class ExampleSpec extends FlatSpec with BeforeAndAfter {

  val builder = new StringBuilder
  val buffer = new ListBuffer[String]

  before {
    builder.append("ScalaTest is ")
  }

  after {
    builder.clear()
    buffer.clear()
  }

  "Testing" should "be easy" in {
    builder.append("easy!")
    assert(builder.toString === "ScalaTest is easy!")
    assert(buffer.isEmpty)
    buffer += "sweet"
  }

  it should "be fun" in {
    builder.append("fun!")
    assert(builder.toString === "ScalaTest is fun!")
    assert(buffer.isEmpty)
  }
}

The before and after methods can each only be called once per Suite, and cannot be invoked after run has been invoked. If either of the registered before or after functions complete abruptly with an exception, it will be reported as an aborted suite and no more tests will be attempted in that suite.

Note that the only way before and after code can communicate with test code is via some side-effecting mechanism, commonly by reassigning instance vars or by changing the state of mutable objects held from instance vals (as in this example). If using instance vars or mutable objects held from instance vals you wouldn't be able to run tests in parallel in the same instance of the test class unless you synchronized access to the shared, mutable state. This is why ScalaTest's ParallelTestExecution trait extends OneInstancePerTest. By running each test in its own instance of the class, each test has its own copy of the instance variables, so you don't need to synchronize. Were you to mix ParallelTestExecution into the ExampleSuite above, the tests would run in parallel just fine without any synchronization needed on the mutable StringBuilder and ListBuffer[String] objects.

Although BeforeAndAfter provides a minimal-boilerplate way to execute code before and after tests, it isn't designed to enable stackable traits, because the order of execution would be non-obvious. If you want to factor out before and after code that is common to multiple test suites, you should use trait BeforeAndAfterEach instead.

The advantage this trait has over BeforeAndAfterEach is that its syntax is more concise. The main disadvantage is that it is not stackable, whereas BeforeAndAfterEach is. I.e., you can write several traits that extend BeforeAndAfterEach and provide beforeEach methods that include a call to super.beforeEach, and mix them together in various combinations. By contrast, only one call to the before registration function is allowed in a suite or spec that mixes in BeforeAndAfter. In addition, BeforeAndAfterEach allows you to access the config map and test name via the TestData passed to its beforeEach and afterEach methods, whereas BeforeAndAfter gives you no access to the config map.

Stackable trait that can be mixed into suites that need methods invoked before and after executing the suite.

This trait allows code to be executed before and/or after all the tests and nested suites of a suite are run. This trait overrides run and calls the beforeAll method, then calls super.run. After the super.run invocation completes, whether it returns normally or completes abruptly with an exception, this trait's run method will invoke afterAll.

Trait BeforeAndAfterAll defines beforeAll and afterAll methods that take no parameters. This trait's implementation of these methods do nothing.

For example, the following ExampleSpec mixes in BeforeAndAfterAll and in beforeAll, creates and writes to a temp file. Each test class, ExampleSpec and all its nested suites--OneSpec, TwoSpec, RedSpec, and BlueSpec--tests that the file exists. After all of the nested suites have executed, afterAll is invoked, which deletes the file. (Note: if you're unfamiliar with the withFixture(OneArgTest) approach to shared fixtures, check out the documentation for trait fixture.FlatSpec.)

package org.scalatest.examples.beforeandafterall

import org.scalatest._
import java.io._

trait TempFileExistsSpec extends fixture.FlatSpecLike {

  protected val tempFileName = "tmp.txt"

  type FixtureParam = File
  override def withFixture(test: OneArgTest) = {
    val file = new File(tempFileName)
    withFixture(test.toNoArgTest(file)) // loan the fixture to the test
  }

  "The temp file" should ("exist in " + suiteName) in { file =>
    assert(file.exists)
  }
}

class OneSpec extends TempFileExistsSpec
class TwoSpec extends TempFileExistsSpec
class RedSpec extends TempFileExistsSpec
class BlueSpec extends TempFileExistsSpec

class ExampleSpec extends Suites(
  new OneSpec,
  new TwoSpec,
  new RedSpec,
  new BlueSpec
) with TempFileExistsSpec with BeforeAndAfterAll {

  // Set up the temp file needed by the test, taking
  // a file name from the config map
  override def beforeAll() {
    val writer = new FileWriter(tempFileName)
    try writer.write("Hello, suite of tests!")
    finally writer.close()
  }

  // Delete the temp file
  override def afterAll() {
    val file = new File(tempFileName)
    file.delete()
  }
}

If you do supply a mapping for "tempFileName" in the config map, you'll see that the temp file is available to all the tests:

scala> org.scalatest.run(new ExampleSpec)
ExampleSpec:
OneSpec:
The temp file
- should exist in OneSpec
TwoSpec:
The temp file
- should exist in TwoSpec
RedSpec:
The temp file
- should exist in RedSpec
BlueSpec:
The temp file
- should exist in BlueSpec
The temp file
- should exist in ExampleSpec

Note: this trait uses the Status result of Suite's "run" methods to ensure that the code in afterAll is executed after all the tests and nested suites are executed even if a Distributor is passed.

Note that it is not guaranteed that afterAll is invoked from the same thread as beforeAll, so if there's any shared state between beforeAll and afterAll you'll need to make sure they are synchronized correctly.

Trait that can be mixed into suites that need methods that make use of the config map invoked before and/or after executing the suite.

This trait allows code to be executed before and/or after all the tests and nested suites of a suite are run. This trait overrides run and calls the beforeAll(ConfigMap) method, then calls super.run. After the super.run invocation completes, whether it returns normally or completes abruptly with an exception, this trait's run method will invoke afterAll(ConfigMap).

Note that this trait differs from BeforeAndAfterAll in that it gives the beforeAll and afterAll code access to the config map. If you don't need the config map, use BeforeAndAfterAll instead.

Trait BeforeAndAfterAllConfigMap defines beforeAll and afterAll methods that take a configMap. This trait's implemention of each method does nothing.

For example, the following ExampleSpec mixes in BeforeAndAfterAllConfigMap and in beforeAll, creates and writes to a temp file, taking the name of the temp file from the configMap. This same configMap is then passed to the run methods of the nested suites, OneSpec, TwoSpec, RedSpec, and BlueSpec, so those suites can access the filename and, therefore, the file's contents. After all of the nested suites have executed, afterAll is invoked, which again grabs the file name from the configMap and deletes the file. Each of these five test classes extend trait TempFileExistsSpec, which defines a test that ensures the temp file exists. (Note: if you're unfamiliar with the withFixture(OneArgTest) approach to shared fixtures, check out the documentation for trait fixture.FlatSpec.)

package org.scalatest.examples.beforeandafterallconfigmap

import org.scalatest._
import java.io._

trait TempFileExistsSpec extends fixture.FlatSpec {

  type FixtureParam = File
  override def withFixture(test: OneArgTest) = {
    val fileName = test.configMap.getRequired[String]("tempFileName")
    val file = new File(fileName)
    withFixture(test.toNoArgTest(file)) // loan the fixture to the test
  }

  "The temp file" should ("exist in " + suiteName) in { file =>
    assert(file.exists)
  }
}

class OneSpec extends TempFileExistsSpec
class TwoSpec extends TempFileExistsSpec
class RedSpec extends TempFileExistsSpec
class BlueSpec extends TempFileExistsSpec

class ExampleSpec extends Suites(
  new OneSpec,
  new TwoSpec,
  new RedSpec,
  new BlueSpec
) with TempFileExistsSpec with BeforeAndAfterAllConfigMap {

  private val tempFileName = "tempFileName"

  // Set up the temp file needed by the test, taking
  // a file name from the config map
  override def beforeAll(cm: ConfigMap) {
    assume(
      cm.isDefinedAt(tempFileName),
      "must place a temp file name in the config map under the key: " + tempFileName
    )
    val fileName = cm.getRequired[String](tempFileName)
    val writer = new FileWriter(fileName)
    try writer.write("Hello, suite of tests!")
    finally writer.close()
  }

  // Delete the temp file
  override def afterAll(cm: ConfigMap) {
    val fileName = cm.getRequired[String]("tempFileName")
    val file = new File(fileName)
    file.delete()
  }
}

Running the above class in the interpreter will give an error if you don't supply a mapping for "tempFileName" in the config map:

scala> org.scalatest.run(new ExampleSpec)
ExampleSpec:
Exception encountered when invoking run on a suite. *** ABORTED ***
  Exception encountered when invoking run on a suite. (:30)
*** RUN ABORTED ***
  An exception or error caused a run to abort: must place a temp file name in the config map under the key: tempFileName (:30)

If you do supply a mapping for "tempFileName" in the config map, you'll see that the temp file is available to all the tests:

scala> (new ExampleSpec).execute(configMap = ConfigMap("tempFileName" -> "tmp.txt"))
ExampleSpec:
OneSpec:
The temp file
- should exist in OneSpec
TwoSpec:
The temp file
- should exist in TwoSpec
RedSpec:
The temp file
- should exist in RedSpec
BlueSpec:
The temp file
- should exist in BlueSpec
The temp file
- should exist in ExampleSpec

Note: As of 2.0.M5, this trait uses the newly added Status result of Suite's "run" methods to ensure that the code in afterAll is executed after all the tests and nested suites are executed even if a Distributor is passed.

Note that it is not guaranteed that afterAll is invoked from the same thread as beforeAll, so if there's any shared state between beforeAll and afterAll you'll need to make sure they are synchronized correctly.

Stackable trait that can be mixed into suites that need code executed before and/or after running each test.

A test fixture is composed of the objects and other artifacts (files, sockets, database connections, etc.) tests use to do their work. When multiple tests need to work with the same fixtures, it is important to try and avoid duplicating the fixture code across those tests. The more code duplication you have in your tests, the greater drag the tests will have on refactoring the actual production code, and the slower your compile will likely be. Trait BeforeAndAfterEach offers one way to eliminate such code duplication: a beforeEach method that will be run before each test (like JUnit's setUp), and an afterEach method that will be run after (like JUnit's tearDown).

Here's an example:

package org.scalatest.examples.flatspec.composingbeforeandaftereach

import org.scalatest._
import collection.mutable.ListBuffer

trait Builder extends BeforeAndAfterEach { this: Suite =>

  val builder = new StringBuilder

  override def beforeEach() {
    builder.append("ScalaTest is ")
    super.beforeEach() // To be stackable, must call super.beforeEach
  }

  override def afterEach() {
    try {
      super.afterEach() // To be stackable, must call super.afterEach
    }
    finally {
      builder.clear()
    }
  }
}

trait Buffer extends BeforeAndAfterEach { this: Suite =>

  val buffer = new ListBuffer[String]

  override def afterEach() {
    try {
      super.afterEach() // To be stackable, must call super.afterEach
    }
    finally {
      buffer.clear()
    }
  }
}

class ExampleSpec extends FlatSpec with Builder with Buffer {

  "Testing" should "be easy" in {
    builder.append("easy!")
    assert(builder.toString === "ScalaTest is easy!")
    assert(buffer.isEmpty)
    buffer += "sweet"
  }

  it should "be fun" in {
    builder.append("fun!")
    assert(builder.toString === "ScalaTest is fun!")
    assert(buffer.isEmpty)
    buffer += "clear"
  }
}

To get the same ordering as withFixture, place your super.beforeEach call at the end of each beforeEach method, and the super.afterEach call at the beginning of each afterEach method, as shown in the previous example. It is a good idea to invoke super.afterEach in a try block and perform cleanup in a finally clause, as shown in the previous example, because this ensures the cleanup code is performed even if super.afterEach throws an exception.

The main advantage of BeforeAndAfterEach over BeforeAndAfter is that BeforeAndAfterEach. enables trait stacking. The main disadvantage of BeforeAndAfterEach compared to BeforeAndAfter is that BeforeAndAfterEach requires more boilerplate. If you don't need trait stacking, use BeforeAndAfter instead of BeforeAndAfterEach. If you want to make use of test data (the test name, config map, etc.) in your beforeEach or afterEach method, use trait BeforeAndAfterEachTestData instead.

Stackable trait that can be mixed into suites that need code that makes use of test data (test name, tags, config map, etc.) executed before and/or after running each test.

A test fixture is composed of the objects and other artifacts (files, sockets, database connections, etc.) tests use to do their work. When multiple tests need to work with the same fixtures, it is important to try and avoid duplicating the fixture code across those tests. The more code duplication you have in your tests, the greater drag the tests will have on refactoring the actual production code. Trait BeforeAndAfterEachTestData offers one way to eliminate such code duplication: a beforeEach(TestData) method that will be run before each test (like JUnit's setUp), and an afterEach(TestData) method that will be run after (like JUnit's tearDown).

Here's an example:

package org.scalatest.examples.flatspec.composingbeforeandaftereachtestdata

import org.scalatest._
import collection.mutable.ListBuffer

trait Builder extends BeforeAndAfterEachTestData { this: Suite =>

  val builder = new StringBuilder

  override def beforeEach(td: TestData) {
    builder.append(td.name)
    super.beforeEach(td) // To be stackable, must call super.beforeEach(TestData)
  }

  override def afterEach(td: TestData) {
    try {
      super.afterEach(td) // To be stackable, must call super.afterEach(TestData)
    }
    finally {
      builder.clear()
    }
  }
}

trait Buffer extends BeforeAndAfterEachTestData { this: Suite =>

  val buffer = new ListBuffer[String]

  override def afterEach(td: TestData) {
    try {
      super.afterEach(td) // To be stackable, must call super.afterEach(TestData)
    }
    finally {
      buffer.clear()
    }
  }
}

class ExampleSpec extends FlatSpec with Builder with Buffer {

  "Testing" should "be easy" in {
    builder.append("!")
    assert(builder.toString === "Testing should be easy!")
    assert(buffer.isEmpty)
    buffer += "sweet"
  }

  it should "be fun" in {
    builder.append("!")
    assert(builder.toString === "Testing should be fun!")
    assert(buffer.isEmpty)
    buffer += "clear"
  }
}

To get the same ordering as withFixture, place your super.beforeEach(TestData) call at the end of each beforeEach(TestData) method, and the super.afterEach(TestData) call at the beginning of each afterEach(TestData) method, as shown in the previous example. It is a good idea to invoke super.afterEach(TestData) in a try block and perform cleanup in a finally clause, as shown in the previous example, because this ensures the cleanup code is performed even if super.afterEach(TestData) throws an exception.

Besides enabling trait stacking, the other main advantage of BeforeAndAfterEachTestData over BeforeAndAfter is that BeforeAndAfterEachTestData allows you to make use of test data (such as the test name and config map) in your before and/or after code, whereas BeforeAndAfter does not.

The main disadvantage of BeforeAndAfterEachTestData compared to BeforeAndAfter and BeforeAndAfterEach is that BeforeAndAfterEachTestData requires more boilerplate. If you don't need trait stacking or access to the test data, use BeforeAndAfter instead of BeforeAndAfterEachTestData. If you need trait stacking, but not access to the TestData, use BeforeAndAfterEach instead.

Trait that when mixed into a TestSuite cancels any remaining tests in that TestSuite instance after a test fails.

The intended use case for this trait is if you have a suite of long-running tests that are related such that if one fails, you aren't interested in running the others, you can use this trait to simply cancel any remaining tests, so you need not wait long for them to complete.

Note that this trait only cancels tests in the same TestSuite instance, because it uses a private volatile instance variable as a flag to indicate whether or not a test has failed. If you are running each test in its own instance, therefore, it would not cancel the remaining tests, because they would not see the same flag. For this reason, this trait contains a final implementation of a method defined in OneInstancePerTest, to prevent it from being mixed into any class that also mixes in OneInstancePerTest, including by mixing in ParallelTestExecution or a path trait.

Trait providing class Checkpoint, which enables multiple assertions to be performed within a test, with any failures accumulated and reported together at the end of the test.

Because ScalaTest uses exceptions to signal failed assertions, normally execution of a test will stop as soon as the first failed assertion is encountered. Trait Checkpoints provides an option when you want to continue executing the remainder of the test body, or part of it, even if an assertion has already failed in that test.

To use a Checkpoint (once you've mixed in or imported the members of trait Checkpoints), you first need to create one, like this:

val cp = new Checkpoint

Then give the Checkpoint assertions to execute by passing them (via a by-name parameter) to its apply method, like this:

val (x, y) = (1, 2)
cp { x should be < 0 }
cp { y should be > 9 }

Both of the above assertions will fail, but it won't be reported yet. The Checkpoint will execute them right away, each time its apply method is invoked. But it will catch the TestFailedExceptions and save them, only reporting them later when reportAll is invoked. Thus, at the end of the test, you must call reportAll, like this:

cp.reportAll()

This reportAll invocation will complete abruptly with a TestFailedException whose message includes the message, source file, and line number of each of the checkpointed assertions that previously failed. For example:

1 was not less than 0 (in Checkpoint) at ExampleSpec.scala:12
2 was not greater than 9 (in Checkpoint) at ExampleSpec.scala:13

Make sure you invoke reportAll before the test completes, otherwise any failures that were detected by the Checkpoint will not be reported.

Note that a Checkpoint will catch and record for later reporting (via reportAll) exceptions that mix in StackDepth except for TestCanceledException, TestRegistrationClosedException, NotAllowedException, and DuplicateTestNameException. If a block of code passed to a Checkpoint's apply method completes abruptly with any of the StackDepth exceptions in the previous list, or any non-StackDepth exception, that invocation of the apply method will complete abruptly with the same exception immediately. Unless you put reportAll in a finally clause and handle this case, such an unexpected exception will cause you to lose any information about assertions that failed earlier in the test and were recorded by the Checkpoint.

Trait that provides a complete-lastly construct, which ensures cleanup code in lastly is executed whether the code passed to complete completes abruptly with an exception or successfully results in a Future, FutureOutcome, or other type with an implicit Futuristic instance.

This trait is mixed into ScalaTest's async testing styles, to make it easy to ensure cleanup code will execute whether code that produces a "futuristic" value (any type F for which a Futuristic[F] instance is implicitly available). ScalaTest provides implicit Futuristic instances for Future[T] for any type T and FutureOutcome.

If the future-producing code passed to complete throws an exception, the cleanup code passed to lastly will be executed immediately, and the same exception will be rethrown, unless the code passed to lastly also completes abruptly with an exception. In that case, complete-lastly will complete abruptly with the exception thrown by the code passed to lastly (this mimics the behavior of finally).

Otherwise, if the code passed to complete successfully returns a Future (or other "futuristic" type), complete-lastly will register the cleanup code to be performed once the future completes and return a new future that will complete once the original future completes and the subsequent cleanup code has completed execution. The future returned by complete-lastly will have the same result as the original future passed to complete, unless the cleanup code throws an exception. If the cleanup code passed to lastly throws an exception, the future returned by lastly will fail with that exception.

The complete-lastly syntax is intended to be used to ensure cleanup code is executed in async testing styles like try-finally is used in traditional testing styles. Here's an example of complete-lastly used in withFixture in an async testing style:

// Your implementation
override def withFixture(test: NoArgAsyncTest) = {

  // Perform setup here

  complete {
    super.withFixture(test) // Invoke the test function
  } lastly {
    // Perform cleanup here
  }
}

A map of configuration data.

A ConfigMap can be populated from the Runner command line via -D arguments. Runner passes it to many methods where you can use it to configure your test runs. For example, Runner passed the ConfigMap to:

• the apply method of Reporters via RunStarting events
• the run method of Suite
• the runNestedSuites method of Suite
• the runTests method of Suite
• the runTest method of Suite
• the withFixture(NoArgTest) method of Suite
• the withFixture(OneArgTest) method of fixture.Suite
• the beforeEach(TestData) method of BeforeAndAfterEachTestData
• the afterEach(TestData) method of BeforeAndAfterEachTestData

In addition to accessing the ConfigMap in overriden implementations of the above methods, you can also transform and pass along a modified ConfigMap.

A ConfigMap maps string keys to values of any type, i.e., it is a Map[String, Any]. To get a configuration value in a variable of the actual type of that value, therefore, you'll need to perform an unsafe cast. If this cast fails, you'll get an exception, which so long as the ConfigMap is used only in tests, will result in either a failed or canceled test or aborted suite. To give such exceptions nice stack depths and error messages, and to eliminate the need for using asInstanceOf in your test code, ConfigMap provides three methods for accessing values at expected types.

The getRequired method returns the value bound to a key cast to a specified type, or throws TestCanceledException if either the key is not bound or is bound to an incompatible type. Here's an example:

val tempFileName: String = configMap.getRequired[String]("tempFileName")

The getOptional method returns the value bound to a key cast to a specified type, wrapped in a Some, returns None if the key is not bound, or throws TestCanceledException if the key exists but is bound to an incompatible type. Here's an example:

val tempFileName: Option[String] = configMap.getOptional[String]("tempFileName")

The getWithDefault method returns the value bound to a key cast to a specified type, returns a specified default value if the key is not bound, or throws TestCanceledException if the key exists but is either not bound or is bound to an incompatible type. Here's an example:

val tempFileName: String = configMap.getWithDefault[String]("tempFileName", "tmp.txt")

Wrapper Suite that passes an instance of the config map to the constructor of the wrapped Suite when run is invoked.

Each time run is invoked on an instance of ConfigMapWrapperSuite, this suite will create a new instance of the suite to wrap, passing to the constructor the config map passed to run. This way, if the same ConfigMapWrapperSuite instance is run multiple times, each time with a different config map, an instance of the wrapped suite will be created for each config map. In addition to being passed to the wrapped suite's constructor, the config map passed to the ConfigMapWrapperSuite's run method will also be passed to the run method of the newly created wrapped suite instance.

The config map is accessible inside a Suite in many ways. It is passed to run, runNestedSuites, runTests, and runTest. It is also passed to withFixture, accessible via a method on NoArgTest and OneArgTest. It is passed to an overloaded forms of the beforeEach and afterEach methods of trait BeforeAndAfterEach, as well as overloaded forms of the beforeAll and afterAll methods of trait BeforeAndAfterAll. Tests themselves can have information taken from the config map, or the entire config map, through various means. The config map may be passed into the test via a ConfigMapFixture, for example. Class ConfigMapWrapperSuite represents one more way to get at the config map inside a suite of test: ConfigMapWrapperSuite will pass the config map to the constructor of your suite class, bringing it easily into scope for tests and helper methods alike.

Having the config map passed to the suite constructor might be more convenient in some cases, but in the case of the org.scalatest.path traits, it is necessary if a test needs information from a config map. The reason is that in a path trait, the test code is executed eagerly, before run is invoked. The results of the tests are registered when the tests are executed, and those results are merely reported once run is invoked. Thus by the time run has been invoked, it is too late to get the config map to the tests, which have already been executed. Using a ConfigMapWrapperSuite solves that problem. By passing the config map to the constructor, it is available early enough for the running tests to use it. Here's an example:

import org.scalatest._

@WrapWith(classOf[ConfigMapWrapperSuite])
class ExampleSpec(configMap: ConfigMap) extends path.FunSpec {

  describe("A widget database") {
    it("should contain consistent values") {
      val dbName = configMap("WidgetDbName") // Can access config map
      // ...
    }
  }
}

A trait that represent an expression recorded by DiagrammedExprMacro, which includes the following members:

• a boolean value
• an anchor that records the position of this expression
• anchor values of this expression (including sub-expressions)

DiagrammedExpr is used by code generated from DiagrammedAssertionsMacro, it needs to be public so that the generated code can be compiled. It is expected that ScalaTest users would ever need to use DiagrammedExpr directly.

A sorter for the events of a run's distributed suites.

This trait is used, for example, when -PS is passed to Runner, to sort the events of distributed suites such that each suite's events are propagated together, with a timeout if an event takes too long.

A sorter for the events of a suite's distributed tests.

This trait is used, for example, by ParallelTestExecution to sort the events of tests back into sequential order, with a timeout if an event takes too long.

Trait whose instances facilitate parallel execution of Suites. An optional Distributor is passed to the run method of Suite. If a Distributor is indeed passed, trait Suite's implementation of run will populate that Distributor with its nested Suites (by passing them to the Distributor's apply method) rather than executing the nested Suites directly. It is then up to another thread or process to execute those Suites.

If you have a set of nested Suites that must be executed sequentially, you can mix in trait SequentialNestedSuiteExecution, which overrides runNestedSuites and calls super's runNestedSuites implementation, passing in None for the Distributor.

Implementations of this trait must be thread safe.

Annotation used to indicate that an otherwise discoverable test class should not be discovered.

Note: This is actually an annotation defined in Java, not a Scala trait. It must be defined in Java instead of Scala so it will be accessible at runtime. It has been inserted into Scaladoc by pretending it is a trait.

ScalaTest will discover any class that either extends Suite and has a public, no-arg constructor, or is annotated with a valid WrapWith annotation. If you wish to prevent a class from being discovered, simply annotate it with DoNotDiscover, like this:

import org.scalatest._

@DoNotDiscover
class SetSpec extends FlatSpec {

  "An empty Set" should "have size 0" in {
    assert(Set.empty.size === 0)
  }

  it should "produce NoSuchElementException when head is invoked" in {
    intercept[NoSuchElementException] {
      Set.empty.head
    }
  }
}

ScalaTest will run classes annotated with DoNotDiscover if asked to explicitly, it just won't discover them.

Note that because reflection is not supported on Scala.js, this annotation will only work on the JVM, not on Scala.js.

Trait to which markup text tests can be reported.

Note: Documenter will be described in more detail in a future 2.0 milestone release. As of this release you can't see its effects yet.

Dynamic tags for a run.

Instances of this class are passed to the Filter constructor to support running selected suites and tests via dynamic tagging. For example, dynamic tags can be used to rerun tests that failed previously, or tests selected via a wildcard from Runner or the Scala interpreter.

Trait that provides an implicit conversion that adds left.value and right.value methods to Either, which will return the selected value of the Either if defined, or throw TestFailedException if not.

This construct allows you to express in one statement that an Either should be left or right and that its value should meet some expectation. Here's are some examples:

either1.right.value should be > 9
either2.left.value should be ("Muchas problemas")

Or, using assertions instead of matcher expressions:

assert(either1.right.value > 9)
assert(either2.left.value === "Muchas problemas")

Were you to simply invoke right.get or left.get on the Either, if the Either wasn't defined as expected (e.g., it was a Left when you expected a Right), it would throw a NoSuchElementException:

val either: Either[String, Int] = Left("Muchas problemas")

either.right.get should be > 9 // either.right.get throws NoSuchElementException

The NoSuchElementException would cause the test to fail, but without providing a stack depth pointing to the failing line of test code. This stack depth, provided by TestFailedException (and a few other ScalaTest exceptions), makes it quicker for users to navigate to the cause of the failure. Without EitherValues, to get a stack depth exception you would need to make two statements, like this:

val either: Either[String, Int] = Left("Muchas problemas")

either should be ('right) // throws TestFailedException
either.right.get should be > 9

The EitherValues trait allows you to state that more concisely:

val either: Either[String, Int] = Left("Muchas problemas")

either.right.value should be > 9 // either.right.value throws TestFailedException

A case class implementation of java.util.Map.Entry to make it easier to test Java Maps with ScalaTest Matchers.

In Java, java.util.Map is not a subtype of java.util.Collection, and does not actually define an element type. You can ask a Java Map for an “entry set” via the entrySet method, which will return the Map's key/value pairs wrapped in a set of java.util.Map.Entry, but a Map is not actually a collection of Entry. To make Java Maps easier to work with, however, ScalaTest matchers allows you to treat a Java Map as a collection of Entry, and defines this convenience implementation of java.util.Map.Entry. Here's how you use it:

javaMap should contain (Entry(2, 3))
javaMap should contain oneOf (Entry(2, 3), Entry(3, 4))

Superclass for the two outcomes of running a test that contain an exception: Failed and Canceled.

This class provides a toOption method that returns a Some wrapping the contained exception, and an isExceptional field with the value true. It's companion object provides an extractor that enables patterns that match a test that either failed or canceled, as in:

outcome match {
  case Exceptional(ex) => // handle failed or canceled case
  case _ => // handle succeeded, pending, or omitted case
}

Outcome for a test that failed, containing an exception describing the cause of the failure.

Note: the difference between this Failed class and the similarly named FailedStatus object is that an instance of this class indicates one test failed, whereas the FailedStatus object indicates either one or more tests failed and/or one or more suites aborted during a run. Both are used as the result type of Suite lifecycle methods, but Failed is a possible result of withFixture, whereas FailedStatus is a possible result of run, runNestedSuites, runTests, or runTest. In short, Failed is always just about one test, whereas FailedStatus could be about something larger: multiple tests or an entire suite.

Filter whose apply method determines which of the passed tests to run and ignore based on tags to include and exclude passed as as class parameters.

This class handles the org.scalatest.Ignore tag specially, in that its apply method indicates which tests should be ignored based on whether they are tagged with org.scalatest.Ignore. If "org.scalatest.Ignore" is not passed in the tagsToExclude set, it will be implicitly added. However, if the tagsToInclude option is defined, and the contained set does not include "org.scalatest.Ignore", then only those tests that are both tagged with org.scalatest.Ignore and at least one of the tags in the tagsToInclude set will be included in the result of apply and marked as ignored (so long as the test is not also marked with a tag other than org.scalatest.Ignore that is a member of the tagsToExclude set. For example, if SlowAsMolasses is a member of the tagsToInclude set and a test is tagged with both org.scalatest.Ignore and SlowAsMolasses, and SlowAsMolasses appears in the tagsToExclude set, the SlowAsMolasses tag will "overpower" the org.scalatest.Ignore tag, and the test will be filtered out entirely rather than being ignored.

Annotation used to mark a trait or class as defining a testing style that has a org.scalatest.finders.Finder implementation, which IDEs and other tools can use to discover tests and scopes.

Note: This is actually an annotation defined in Java, not a Scala trait. It must be defined in Java instead of Scala so it will be accessible at runtime. It has been inserted into Scaladoc by pretending it is a trait.

This annotation is used to enable different styles of testing, including both native ScalaTest styles and custom user-created styles, to have rich IDE support. The "Finder API" is released separately from ScalaTest proper, because it is only used by tools such as IDEs.

The base trait of ScalaTest's "fixture" async testing styles, which enable you to pass fixture objects into tests.

This trait provides a final override of withFixture(OneArgTest), declared in supertrait FixtureSuite, because the withFixture(OneArgTest) lifecycle method assumes synchronous testing. Here is its signature:

def withFixture(test: OneArgTest): Outcome

The test function interface, OneArgTest, offers an apply method that takes a FixtureParam and returns Outcome:

// In trait OneArgTest:
def apply(fixture: FixtureParam): Outcome

Because the result of a test is an Outcome, when the test function returns, the test body must have determined an outcome already. It will already be one of Succeeded, Failed, Canceled, or Pending. This is also true when withFixture(OneArgTest) returns: because the result type of withFixture(OneArgTest) is Outcome, the test body has by definition has already finished execution.

This trait overrides and makes abstract the runTest method. Subtraits must must implement this method to call withFixture(OneArgAsyncTest) instead of withFixture(OneArgTest), where withFixture(OneArgAsyncTest) is a new method declared in this trait with the following signature and implementation:

def withFixture(test: OneArgAsyncTest): FutureOutcome = {
  test()
}

Instead of returning Outcome like withFixture, the withFixture method returns a FutureOutcome. Similarly, the apply method of test function interface, OneArgAsyncTest, returns FutureOutcome:

// In trait OneArgAsyncTest:
def apply(fixture: FixtureParam): FutureOutcome

The withFixture method supports async testing, because when the test function returns, the test body has not necessarily finished execution.

The recommended way to ensure cleanup is performed after a test body finishes execution is to use the complete-lastly syntax, defined in supertrait org.scalatest.CompleteLastly, which will ensure that cleanup will occur whether future-producing code completes abruptly by throwing an exception, or returns normally yielding a future. In the latter case, complete-lastly will register the cleanup code to execute asynchronously when the future completes.

To enable the stacking of traits that define withFixture(NoArgAsyncTest), it is a good idea to let withFixture(NoArgAsyncTest) invoke the test function instead of invoking the test function directly. To do so, you'll need to convert the OneArgAsyncTest to a NoArgAsyncTest. You can do that by passing the fixture object to the toNoArgAsyncTest method of OneArgAsyncTest. In other words, instead of writing “test(theFixture)”, you'd delegate responsibility for invoking the test function to the withFixture(NoArgAsyncTest) method of the same instance by writing:

withFixture(test.toNoArgAsyncTest(theFixture))

Thus, the recommended structure of a withFixture implementation that performs cleanup looks like this:

// Your implementation
override def withFixture(test: OneArgAsyncTest) = {

  // Perform setup here
  val theFixture = ...

  complete {
    withFixture(test.toNoArgAsyncTest(theFixture)) // Invoke the test function
  } lastly {
    // Perform cleanup here
  }
}

If you have no cleanup to perform, you can write withFixture like this instead:

// Your implementation
override def withFixture(test: OneArgAsyncTest) = {

  // Perform setup here
  val theFixture = ...

  withFixture(test.toNoArgAsyncTest(theFixture)) // Invoke the test function
}

If you want to perform an action only for certain outcomes, you'll need to register code performing that action as a callback on the Future using one of Future registration methods: onComplete, onSuccess, or onFailure. Note that if a test fails, that will be treated as a scala.util.Success(org.scalatest.Failure). So if you want to perform an action if a test fails, for example, you'd register the callaback using onSuccess, like this:

// Your implementation
override def withFixture(test: OneArgAsyncTest) = {

  // Perform setup here
  val theFixture = ...

  val futureOutcome =
      withFixture(test.toNoArgAsyncTest(theFixture)) // Invoke the test function

  futureOutcome onFailedThen { _ =>
    // perform action that you want to occur
    // only if a test fails here
  }
}

Lastly, if you want to transform the outcome in some way in withFixture, you'll need to use either the map or transform methods of Future, like this:

// Your implementation
override def withFixture(test: OneArgAsyncTest) = {

  // Perform setup here
  val theFixture = ...

  val futureOutcome =
      withFixture(test.toNoArgAsyncTest(theFixture)) // Invoke the test function

  futureOutcome change { outcome =>
    // transform the outcome into a new outcome here
  }
}

Note that a NoArgAsyncTest's apply method will only return a Failure if the test completes abruptly with an exception (such as OutOfMemoryError) that should cause the suite to abort rather than the test to fail. Thus usually you would use map to transform future outcomes, not transform, so that such suite-aborting exceptions pass through unchanged. The suite will abort asynchronously with any exception returned in a Failure.

Marker trait for fixture-context objects, that enables them to be used in testing styles that require type Assertion

A fixture-context object is a way to share fixtures between different tests that is most useful when different tests need different combinations of fixture objects. The fixture-context object technique is only appropriate if you don't need to clean up the fixtures after using them.

To use this technique, you define instance variables intialized with fixture objects in traits and/or classes, then in each test instantiate an object that contains just the fixture objects needed by the test. Traits allow you to mix together just the fixture objects needed by each test, whereas classes allow you to pass data in via a constructor to configure the fixture objects. Here's an example FlatSpec in which fixture objects are partitioned into two traits and each test just mixes together the traits it needs:

package org.scalatest.examples.flatspec.fixturecontext

import collection.mutable.ListBuffer
import org.scalatest.FlatSpec
import org.scalatest.FixtureContext

class ExampleSpec extends FlatSpec {

  trait Builder extends FixtureContext {
    val builder = new StringBuilder("ScalaTest is ")
  }

  trait Buffer extends FixtureContext {
    val buffer = ListBuffer("ScalaTest", "is")
  }

  // This test needs the StringBuilder fixture
  "Testing" should "be productive" in new Builder {
    builder.append("productive!")
    assert(builder.toString === "ScalaTest is productive!")
  }

  // This test needs the ListBuffer[String] fixture
  "Test code" should "be readable" in new Buffer {
    buffer += ("readable!")
    assert(buffer === List("ScalaTest", "is", "readable!"))
  }

  // This test needs both the StringBuilder and ListBuffer
  it should "be clear and concise" in new Builder with Buffer {
    builder.append("clear!")
    buffer += ("concise!")
    assert(builder.toString === "ScalaTest is clear!")
    assert(buffer === List("ScalaTest", "is", "concise!"))
  }
}

Extending FixtureContext, which extends trait org.scalatest.compatible.Assertion makes it more convenient to use fixture-context objects in styles, such as async styles, that require test bodies to have type Assertion.

The base trait for ScalaTest's synchronous testing styles that accept a fixture object passed into tests. This trait defines a withFixture lifecycle method that takes as its parameter a test function that accepts a fixture object and returns an Outcome.

The abstract withFixture method add by this trait has the following signature:

def withFixture(test: OneArgTest): Outcome

The apply method of test function interface, OneArgTest, also returns Outcome:

// In trait OneArgTest:
def apply(fixture: FixtureParam): Outcome

Because the result of a test is an Outcome, when the test function returns, the test body must have determined an outcome already. It will already be one of Succeeded, Failed, Canceled, or Pending. This is also true when withFixture(OneArgTest) returns: because the result type of withFixture(OneArgTest) is Outcome, the test has by definition already finished execution.

The recommended way to ensure cleanup is performed after a test body finishes execution is to use a try-finally clause. Using try-finally will ensure that cleanup will occur whether the test function completes abruptly by throwing a suite-aborting exception, or returns normally yielding an Outcome. Note that the only situation in which a test function will complete abruptly with an exception is if the test body throws a suite-aborting exception. Any other exception will be caught and reported as either a Failed, Canceled, or Pending.

To enable the stacking of traits that define withFixture(NoArgTest), it is a good idea to let withFixture(NoArgTest) invoke the test function instead of invoking the test function directly. To do so, you'll need to convert the OneArgTest to a NoArgTest. You can do that by passing the fixture object to the toNoArgTest method of OneArgTest. In other words, instead of writing “test(theFixture)”, you'd delegate responsibility for invoking the test function to the withFixture(NoArgTest) method of the same instance by writing:

withFixture(test.toNoArgTest(theFixture))

The withFixture method is designed to be stacked, and to enable this, you should always call the super implementation of withFixture, and let it invoke the test function rather than invoking the test function directly. In other words, instead of writing “test(...)”, you should write “super.withFixture(test)”. Thus, the recommended structure of a withFixture implementation that performs cleanup looks like this:

// Your implementation
type FixtureParam = String

override def withFixture(test: OneArgTest) = {
  // Perform setup here
  val theFixture = "hello"
  try {
    withFixture(test.toNoArgTest(theFixture)) // Invoke the test function
  } finally {
    // Perform cleanup here
  }
}

If you have no cleanup to perform, you can write withFixture like this instead:

// Your implementation
type FixtureParam = String

override def withFixture(test: NoArgTest) = {
  // Perform setup here
  val theFixture = "hello"
  withFixture(test.toNoArgTest(theFixture)) // Invoke the test function
}

If you want to perform an action only for certain outcomes, you can use a pattern match. For example, if you want to perform an action if a test fails, you'd match on Failed, like this:

// Your implementation
type FixtureParam = String

override def withFixture(test: NoArgTest) = {

  // Perform setup here
  val theFixture = "hello"

  val outcome = withFixture(test.toNoArgTest(theFixture)) // Invoke the test function

  outcome match {
    case failed: Failed =>
      // perform action that you want to occur
      // only if a test fails here
      failed
    case other => other
  }
}

If you want to change the outcome in some way in withFixture, you can also use a pattern match. For example, if a particular exception intermittently causes a test to fail, and can transform those failures into cancelations, like this:

// Your implementation
type FixtureParam = String

override def withFixture(test: NoArgTest) = {

  val theFixture = "hello"

  withFixture(test.toNoArgTest(theFixture)) match {
    case Failed(ex: ParticularException) =>
      Canceled("Muting flicker", ex)
    case other => other
  }
}

Wrapper class for Future[Outcome] that presents a more convenient API for manipulation in withFixture methods in async styles.

This type serves as the result type of both test functions and withFixture methods in ScalaTest's async styles. A Future[Outcome] is not used as this result type for two reasons. First, Outcome treats exceptions specially, and as a result methods on Future would usually not yield the desired Future[Outcome] result. Only run-aborting exceptions should result in a failed Future[Outcome]. Any other thrown exception other than TestCanceledException or TestPendingException should result in a successfulFuture containing a org.scalatest.Failed. A thrown TestCanceledException should result in a successful Future containing an org.scalatest.Canceled; A thrown TestPendingException should result in a successful Future containing a org.scalatest.Pending. If manipulating a Future[Outcome] directly, by contrast, any thrown exception would result in a failed Future.

Additionally, to be consistent with corresponding transformations in traditional testing styles, methods registering callbacks should return a new future outcome that doesn't complete until both the original future outcome has completed and the subsequent callback has completed execution. Additionally, if the callback itself throws an exception, that exception should determine the result of the future outcome returned by the callback registration method. This behavior is rather inconvenient to obtain on the current Future API, so FutureOutcome provides well-named methods that have this behavior.

Lastly, the FutureOutcome is intended to help prevent confusion by eliminating the need to work with types like scala.util.Success(org.scalatest.Failed). For this purpose a org.scalactic.Or is used instead of a scala.util.Try to describe results of FutureOutcome.

A FutureOutcome represents a computation that can result in an Outcome or an "abort." An abort means that a run-aborting exception occurred during the computation. Any other, non-run-aborting exception will be represented as an non-Succeeded Outcome: one of Failed, Canceled, or Pending.

The methods of FutureOutcome include the following callback registration methods:

• onSucceededThen - registers a callback to be executed if the future outcome is Succeeded.
• onFailedThen - registers a callback to be executed if the future outcome is Failed.
• onCanceledThen - registers a callback to be executed if the future outcome is Canceled.
• onPendingThen - registers a callback to be executed if the future outcome is Pending.
• onOutcomeThen - registers a callback to be executed if the future outcome is actually an Outcome and not an abort.
• onAbortedThen - registers a callback to be executed if the future outcome aborts.
• onCompletedThen - registers a callback to be executed upon completion no matter how the future outcome completes.

The callback methods listed previously can be used to perform a side effect once a FutureOutcome completes. To change an Outcome into a different Outcome asynchronously, use the change registration method, which takes a function from Outcome to Outcome. The other methods on FutureOutcome, isCompleted and value, allow you to poll a FutureOutcome. None of the methods on FutureOutcome block. Lastly, because an implicit Futuristic instance is provided for FutureOutcome, you can use complete-lastly syntax with FutureOutcome.

Trait that contains methods named given, when, then, and and, which take a string message and implicit Informer, and forward the message to the informer.

Here's an example:

package org.scalatest.examples.flatspec.info

import collection.mutable
import org.scalatest._

class SetSpec extends FlatSpec with GivenWhenThen {

  "A mutable Set" should "allow an element to be added" in {
    Given("an empty mutable Set")
    val set = mutable.Set.empty[String]

    When("an element is added")
    set += "clarity"

    Then("the Set should have size 1")
    assert(set.size === 1)

    And("the Set should contain the added element")
    assert(set.contains("clarity"))

    info("That's all folks!")
  }
}

If you run this SetSpec from the interpreter, you will see the following output:

scala> org.scalatest.run(new SetSpec)
A mutable Set
- should allow an element to be added
  + Given an empty mutable Set
  + When an element is added
  + Then the Set should have size 1
  + And the Set should contain the added element
  + That's all folks! 

Annotation used to tag a test, or suite of tests, as ignored.

Note: This is actually an annotation defined in Java, not a Scala trait. It must be defined in Java instead of Scala so it will be accessible at runtime. It has been inserted into Scaladoc by pretending it is a trait.

If you wish to temporarily ignore an entire suite of tests, you can annotate the test class with @Ignore, like this:

package org.scalatest.examples.flatspec.ignoreall

import org.scalatest._

@Ignore
class SetSpec extends FlatSpec {

  "An empty Set" should "have size 0" in {
    assert(Set.empty.size === 0)
  }

  it should "produce NoSuchElementException when head is invoked" in {
    intercept[NoSuchElementException] {
      Set.empty.head
    }
  }
}

When you mark a test class with a tag annotation, ScalaTest will mark each test defined in that class with that tag. Thus, marking the SetSpec in the above example with the @Ignore tag annotation means that both tests in the class will be ignored. If you run the above SetSpec in the Scala interpreter, you'll see:

scala> org.scalatest.run(new SetSpec)
An empty Set
- should have size 0 !!! IGNORED !!!
- should produce NoSuchElementException when head is invoked !!! IGNORED !!!

Note that marking a test class as ignored won't prevent it from being discovered by ScalaTest. Ignored classes will be discovered and run, and all their tests will be reported as ignored. This is intended to keep the ignored class somewhat visible, to encourage the developers to eventually fix and un-ignore it. If you want to prevent a class from being discovered at all, use the DoNotDiscover annotation instead.

Another use case for @Ignore is to mark test methods as ignored in traits Spec and fixture.Spec. Here's an example:

package org.scalatest.examples.spec.ignore

import org.scalatest._

class SetSpec extends RefSpec {

  @Ignore def `an empty Set should have size 0` {
    assert(Set.empty.size === 0)
  }

  def `invoking head on an empty Set should produce NoSuchElementException` {
    intercept[NoSuchElementException] {
      Set.empty.head
    }
  }
}

If you run this version of SetSpec in the Scala interpreter, you'll see that it runs only the second test and reports that the first test was ignored:

scala> org.scalatest.run(new SetSpec)
SetSpec:
- an empty Set should have size 0 !!! IGNORED !!!
- invoking head on an empty Set should produce NoSuchElementException

Trait to which custom information about a running suite of tests can be reported.

An Informer is essentially used to wrap a Reporter and provide easy ways to send custom information to that Reporter via an InfoProvided event. Informer contains an apply method that takes a string and an optional payload object of type Any. The Informer will forward the passed message string to the Reporter as the message parameter, and the optional payload object as the payload parameter, of an InfoProvided event.

Here's an example in which the Informer is used both directly via info method of trait FlatSpec and indirectly via the methods of trait GivenWhenThen:

package org.scalatest.examples.flatspec.info

import collection.mutable
import org.scalatest._

class SetSpec extends FlatSpec with GivenWhenThen {

  "A mutable Set" should "allow an element to be added" in {
    given("an empty mutable Set")
    val set = mutable.Set.empty[String]

    when("an element is added")
    set += "clarity"

    then("the Set should have size 1")
    assert(set.size === 1)

    and("the Set should contain the added element")
    assert(set.contains("clarity"))

    info("That's all folks!")
  }
}

If you run this SetSpec from the interpreter, you will see the following output:

scala> org.scalatest.run(new SetSpec)
A mutable Set
- should allow an element to be added
  + Given an empty mutable Set
  + When an element is added
  + Then the Set should have size 1
  + And the Set should contain the added element
  + That's all folks! 

Trait containing the inside construct, which allows you to make statements about nested object graphs using pattern matching.

For example, given the following case classes:

case class Address(street: String, city: String, state: String, zip: String)
case class Name(first: String, middle: String, last: String)
case class Record(name: Name, address: Address, age: Int)

You could write:

inside (rec) { case Record(name, address, age) =>
  inside (name) { case Name(first, middle, last) =>
    first should be ("Sally")
    middle should be ("Ann")
    last should be ("Jones")
  }
  inside (address) { case Address(street, city, state, zip) =>
    street should startWith ("25")
    city should endWith ("Angeles")
    state should equal ("CA")
    zip should be ("12345")
  }
  age should be < 99
}

If an assertion fails, the error message will include the toString of each value passed to inside clauses enclosing the failed assertion. For example, if rec in the previous expression was defined like this:

val rec = Record(
  Name("Sally", "Anna", "Jones"),
  Address("25 Main St", "Los Angeles", "CA", "12345"),
  38
)

The error message will read:

"Ann[a]" was not equal to "Ann[]", inside Name(Sally,Anna,Jones),
inside Record(Name(Sally,Anna,Jones),Address(25 Main St,Los Angeles,CA,12345),38)

Provides nestable inspector methods (or just inspectors) that enable assertions to be made about collections.

For example, the forAll method enables you to state that something should be true about all elements of a collection, such as that all elements should be positive:

scala> import org.scalatest._
import org.scalatest._

scala> import Assertions._
import Assertions._

scala> import Inspectors._
import Inspectors._

scala> val xs = List(1, 2, 3, 4, 5)
xs: List[Int] = List(1, 2, 3, 4, 5)

scala> forAll (xs) { x => assert(x > 0) }

Or, with matchers:

scala> import Matchers._
import Matchers._

scala> forAll (xs) { x => x should be > 0 }

To make assertions about nested collections, you can nest the inspector method invocations. For example, given the following list of lists of Int:

scala> val yss =
     |   List(
     |     List(1, 2, 3),
     |     List(1, 2, 3),
     |     List(1, 2, 3)
     |   )
yss: List[List[Int]] = List(List(1, 2, 3), List(1, 2, 3), List(1, 2, 3))

You can assert that all Int elements in all nested lists are positive by nesting two forAll method invocations, like this:

scala> forAll (yss) { ys =>
     |   forAll (ys) { y => y should be > 0 }
     | }

The full list of inspector methods are:

• forAll - succeeds if the assertion holds true for every element
• forAtLeast - succeeds if the assertion holds true for at least the specified number of elements
• forAtMost - succeeds if the assertion holds true for at most the specified number of elements
• forBetween - succeeds if the assertion holds true for between the specified minimum and maximum number of elements, inclusive
• forEvery - same as forAll, but lists all failing elements if it fails (whereas forAll just reports the first failing element)
• forExactly - succeeds if the assertion holds true for exactly the specified number of elements

The error messages produced by inspector methods are designed to make sense no matter how deeply you nest the method invocations. Here's an example of a nested inspection that fails and the resulting error message:

scala> forAll (yss) { ys =>
     |   forAll (ys) { y => y should be < 2 }
     | }
org.scalatest.exceptions.TestFailedException: forAll failed, because:
  at index 0, forAll failed, because:
    at index 1, 2 was not less than 2 (<console>:20)
  in List(1, 2, 3) (<console>:20)
in List(List(1, 2, 3), List(1, 2, 3), List(1, 2, 3))
     at org.scalatest.InspectorsHelper$.forAll(Inspectors.scala:146)
     ...

One way the error message is designed to help you understand the error is by using indentation that mimics the indentation of the source code (optimistically assuming the source will be nicely indented). The error message above indicates the outer forAll failed because its initial List (i.e., at index 0) failed the assertion, which was that all elements of that initial List[Int] at index 0 should be less than 2. This assertion failed because index 1 of that inner list contained the value 2, which was indeed “not less than 2.” The error message for the inner list is an indented line inside the error message for the outer list. The actual contents of each list are displayed at the end in inspector error messages, also indented appropriately. The actual contents are placed at the end so that for very large collections, the contents will not drown out and make it difficult to find the messages that describe actual causes of the failure.

The forAll and forEvery methods are similar in that both succeed only if the assertion holds for all elements of the collection. They differ in that forAll will only report the first element encountered that failed the assertion, but forEvery will report all elements that fail the assertion. The tradeoff is that while forEvery gives more information, it may take longer to run because it must inspect every element of the collection. The forAll method can simply stop inspecting once it encounters the first failing element. Here's an example that shows the difference in the forAll and forEvery error messages:

scala> forAll (xs) { x => x should be < 3 }
org.scalatest.exceptions.TestFailedException: forAll failed, because:
  at index 2, 3 was not less than 3 (<console>:18)
in List(1, 2, 3, 4, 5)
     at org.scalatest.InspectorsHelper$.forAll(Inspectors.scala:146)
     ...

scala> forEvery (xs) { x => x should be < 3 }
org.scalatest.exceptions.TestFailedException: forEvery failed, because:
  at index 2, 3 was not less than 3 (<console>:18),
  at index 3, 4 was not less than 3 (<console>:18),
  at index 4, 5 was not less than 3 (<console>:18)
in List(1, 2, 3, 4, 5)
     at org.scalatest.InspectorsHelper$.forEvery(Inspectors.scala:226)
     ...

Note that if you're using matchers, you can alternatively use inspector shorthands for writing non-nested inspections. Here's an example:

scala> all (xs) should be > 3
org.scalatest.exceptions.TestFailedException: 'all' inspection failed, because:
  at index 0, 1 was not greater than 3
in List(1, 2, 3, 4, 5)
     at org.scalatest.InspectorsHelper$.forAll(Inspectors.scala:146)

You can use Inspectors on any scala.collection.GenTraversable, java.util.Collection, java.util.Map (with Entry), Array, or String. Here are some examples:

scala> import org.scalatest._
import org.scalatest._

scala> import Inspectors._
import Inspectors._

scala> import Matchers._
import Matchers._

scala> forAll (Array(1, 2, 3)) { e => e should be < 5 }

scala> import collection.JavaConverters._
import collection.JavaConverters._

scala> val js = List(1, 2, 3).asJava
js: java.util.List[Int] = [1, 2, 3]

scala> forAll (js) { j => j should be < 5 }

scala> val jmap = Map("a" -> 1, "b" -> 2).asJava
jmap: java.util.Map[String,Int] = {a=1, b=2}

scala> forAtLeast(1, jmap) { e => e shouldBe Entry("b", 2) }

scala> forAtLeast(2, "hello, world!") { c => c shouldBe 'o' }

Trait that provides an implicit conversion that adds to collection types a loneElement method, which will return the value of the lone element if the collection does indeed contain one and only one element, or throw TestFailedException if not.

This construct allows you to express in one statement that a collection should contain one and only one element and that the element value should meet some expectation. Here's an example:

set.loneElement should be > 9

Or, using an assertion instead of a matcher expression:

assert(set.loneElement > 9)

The loneElement syntax can be used with any collection type C for which an implicit Collecting[C] is available. ScalaTest provides implicit Collecting instances for scala.collection.GenTraversable, Array, and java.util.Collection. You can enable the loneElement syntax on other collection types by defining an implicit Collecting instances for those types.

If you want to use loneElement with a java.util.Map, first transform it to a set of entries with entrySet, and if helpful, use ScalaTest's Entry class:

scala> import org.scalatest._
import org.scalatest._

scala> import LoneElement._
import LoneElement._

scala> import Matchers._
import Matchers._

scala> val jmap = new java.util.HashMap[String, Int]
jmap: java.util.HashMap[String,Int] = {}

scala> jmap.put("one", 1)
res0: Int = 0

scala> jmap.entrySet.loneElement should be (Entry("one", 1))

Trait that can be mixed into a Suite to disable the implicit conversions provided by default in trait Assertions, which trait Suite extends.

This trait can be used to quickly solve a problem in which ScalaTest's default implicit conversion is clashing with those of some other library you need to use in your tests. After mixing in this trait, like this:

class MySuite extends FunSuite with NonImplicitAssertions {
  // ... 
}

You can write tests using assert (without triple equals), assertResult, assertThrows, intercept, assertCompiles, assertDoesNotCompile, and assertTypeError:

assert(a < 7)

assertResult(2) { 1 + 1 }

assertThrows[IndexOutOfBoundsException] {
  "hi".charAt(-1)
}

val caught =
  intercept[IndexOutOfBoundsException] {
    "hi".charAt(-1)
  }

assertDoesNotCompile("val a: String = 1")

assertTypeError("val a: String = 1")

assertCompiles("val a: Int = 1")

Trait providing an apply method to which status updates about a running suite of tests can be reported.

An Notifier is essentially used to wrap a Reporter and provide easy ways to send status updates to that Reporter via an NoteProvided event. Notifier contains an apply method that takes a string and an optional payload object of type Any. The Notifier will forward the passed alert message string to the Reporter as the message parameter, and the optional payload object as the payload parameter, of an NoteProvided event.

For insight into the differences between Notifier, Alerter, and Informer, see the main documentation for trait Notifying.

Trait that contains the note method, which can be used to send a status notification to the reporter.

The difference between note and the info method of Informer is that info messages provided during a test are recorded and sent as part of test completion event, whereas note messages are sent right away as NoteProvided messages. For long-running tests, note allows you to send "status notifications" to the reporter right away, so users can track the progress of the long-running tests. By contrast, info messages will only be seen by the user after the test has completed, and are more geared towards specification (such as Given/When/Then messages) than notification.

The difference between note and the alert method of Alerting is that alert is intended to be used for warnings or notifications of potential problems, whereas note is just for status notifications. In string reporters for which ANSI color is enabled, note notifications are shown in green and alert notifications in yellow.

Trait that facilitates a style of testing in which each test is run in its own instance of the suite class to isolate each test from the side effects of the other tests in the suite.

If you mix this trait into a Suite, you can initialize shared reassignable fixture variables as well as shared mutable fixture objects in the constructor of the class. Because each test will run in its own instance of the class, each test will get a fresh copy of the instance variables. This is the approach to test isolation taken, for example, by the JUnit framework. OneInstancePerTest can, therefore, be handy when porting JUnit tests to ScalaTest.

Here's an example of OneInstancePerTest being used in a FunSuite:

import org.scalatest.FunSuite
import org.scalatest.OneInstancePerTest
import collection.mutable.ListBuffer

class MySuite extends FunSuite with OneInstancePerTest {

  val builder = new StringBuilder("ScalaTest is ")
  val buffer = new ListBuffer[String]

  test("easy") {
    builder.append("easy!")
    assert(builder.toString === "ScalaTest is easy!")
    assert(buffer.isEmpty)
    buffer += "sweet"
  }

  test("fun") {
    builder.append("fun!")
    assert(builder.toString === "ScalaTest is fun!")
    assert(buffer.isEmpty)
  }
}

OneInstancePerTest is supertrait to ParallelTestExecution, in which running each test in its own instance is intended to make it easier to write suites of tests that run in parallel (by reducing the likelihood of concurrency bugs in those suites.) OneInstancePerTest is also supertrait to the path traits, path.FunSpec and path.FreeSpec, to make it obvious these traits run each test in a new, isolated instance.

For the details on how OneInstancePerTest works, see the documentation for methods runTests and runTest, which this trait overrides.

Trait that provides an implicit conversion that adds a value method to Option, which will return the value of the option if it is defined, or throw TestFailedException if not.

This construct allows you to express in one statement that an option should be defined and that its value should meet some expectation. Here's an example:

opt.value should be > 9

Or, using an assertion instead of a matcher expression:

assert(opt.value > 9)

Were you to simply invoke get on the Option, if the option wasn't defined, it would throw a NoSuchElementException:

val opt: Option[Int] = None

opt.get should be > 9 // opt.get throws NoSuchElementException

The NoSuchElementException would cause the test to fail, but without providing a stack depth pointing to the failing line of test code. This stack depth, provided by TestFailedException (and a few other ScalaTest exceptions), makes it quicker for users to navigate to the cause of the failure. Without OptionValues, to get a stack depth exception you would need to make two statements, like this:

val opt: Option[Int] = None

opt should be ('defined) // throws TestFailedException
opt.get should be > 9

The OptionValues trait allows you to state that more concisely:

val opt: Option[Int] = None

opt.value should be > 9 // opt.value throws TestFailedException

Superclass for the possible outcomes of running a test.

Outcome is the result type of the withFixture methods of traits Suite and FixtureSuite, as well as their NoArgTest and OneArgTest function types. The four possible outcomes are:

• Succeeded - indicates a test succeeded
• Failed - indicates a test failed and contains an exception describing the failure
• Canceled - indicates a test was canceled and contains an exception describing the cancelation
• Pending - indicates a test was pending

Note that "ignored" does not appear as a type of Outcome, because tests are marked as ignored on the outside and skipped over as the suite executes. So an ignored test never runs, and therefore never has an outcome. By contrast, a test is determined to be pending by running the test and observing the actual outcome. If the test body completes abruptly with a TestPendingException, then the outcome was that the test was pending.

Trait that contains the outcomeOf method, which executes a passed code block and transforms the outcome into an Outcome, using the same mechanism used by ScalaTest to produce an Outcome when executing a test.

For an example of outcomeOf in action, see the documentation for class TableFor2.

Trait that causes that the tests of any suite it is mixed into to be run in parallel if a Distributor is passed to runTests.

ScalaTest's normal approach for running suites of tests in parallel is to run different suites in parallel, but the tests of any one suite sequentially. This approach should provide sufficient distribution of the work load in most cases, but some suites may encapsulate multiple long-running tests. Such suites may dominate the execution time of the run. If so, mixing in this trait into just those suites will allow their long-running tests to run in parallel with each other, thereby helping to reduce the total time required to run an entire run.

To make it easier for users to write tests that run in parallel, this trait runs each test in its own instance of the class. Running each test in its own instance enables tests to use the same instance vars and mutable objects referenced from instance variables without needing to synchronize. Although ScalaTest provides functional approaches to factoring out common test code that can help avoid such issues, running each test in its own instance is an insurance policy that makes running tests in parallel easier and less error prone.

For the details on how ParallelTestExecution works, see the documentation for methods run, runTests, and runTest, which this trait overrides.

Note: This trait's implementation of runTest is final, to ensure that behavior related to individual tests are executed by the same thread that executes the actual test. This means, for example, that you won't be allowed to write ...with ParallelTestExecution with BeforeAndAfter. Instead, you'd need to put ParallelTestExecution last, as in: with BeforeAndAfter with ParallelTestExecution. For more details, see the documentation for the runTest method.

Trait that provides an implicit conversion that adds a valueAt method to PartialFunction, which will return the value (result) of the function applied to the argument passed to valueAt, or throw TestFailedException if the partial function is not defined at the argument.

This construct allows you to express in one statement that a partial function should be defined for a particular input, and that its result value should meet some expectation. Here's an example:

pf.valueAt("IV") should equal (4)

Or, using an assertion instead of a matcher expression:

assert(pf.valueAt("IV") === 4)

Were you to simply invoke apply on the PartialFunction, passing in an input value, if the partial function wasn't defined at that input, it would throw some exception, but likely not one that provides a stack depth:

// Note: a Map[K, V] is a PartialFunction[K, V]
val pf: PartialFunction[String, Int] = Map("I" -> 1, "II" -> 2, "III" -> 3, "IV" -> 4)

pf("V") should equal (5) // pf("V") throws NoSuchElementException

The NoSuchElementException thrown in this situation would cause the test to fail, but without providing a stack depth pointing to the failing line of test code. This stack depth, provided by TestFailedException (and a few other ScalaTest exceptions), makes it quicker for users to navigate to the cause of the failure. Without PartialFunctionValues, to get a stack depth exception you would need to make two statements, like this:

val pf: PartialFunction[String, Int] = Map("I" -> 1, "II" -> 2, "III" -> 3, "IV" -> 4)

pf.isDefinedAt("V") should be (true) // throws TestFailedException
pf("V") should equal (5)

The PartialFunctionValues trait allows you to state that more concisely:

val pf: PartialFunction[String, Int] = Map("I" -> 1, "II" -> 2, "III" -> 3, "IV" -> 4)

pf.valueAt("V") should equal (5) // pf.valueAt("V") throws TestFailedException

Trait facilitating the inclusion of a payload in a thrown ScalaTest exception.

This trait includes a withPayload construct that enables a payload object (or modified payload object) to be included as the payload of a thrown exception.

Many ScalaTest events include an optional "payload" field that can be used to pass information to a custom reporter. This trait facilitates such customization, by making it easy to insert a payload into a thrown exception, such as a TestFailedException. The thrown exception must mix in Payload. ScalaTest looks for trait Payload and fires any payloads it finds in the relevant ScalaTest event stimulated by the exception, such as a TestFailed event stimulated by a TestFailedException. Here's an example in which a GUI snapshot is included as a payload when a test fails:

withPayload(generateGUISnapshot()) {
  1 + 1 should === (3)
}

Trait mixed into the result type of the pending statement of trait Assertions, which always throws TestPendingException.

This type is used primarily to facilitate the is (pending) syntax of traits FlatSpec, WordSpec, and FLatSpec as well the is (pending) or (pending) syntax of sibling traits in the org.scalatest.fixture package. Because the pending method in Assertions always completes abruptly with an exception, its type would be inferred to be Nothing, which is a relatively common type. To make sure syntax like is (pending) only works with method pending, it is helpful to have a specially named "Nothing" type.

Trait that facilitates the testing of private methods.

To test a private method, mix in trait PrivateMethodTester and create a PrivateMethod object, like this:

val decorateToStringValue = PrivateMethod[String]('decorateToStringValue)

The type parameter on PrivateMethod, in this case String, is the result type of the private method you wish to invoke. The symbol passed to the PrivateMethod.apply factory method, in this case 'decorateToStringValue, is the name of the private method to invoke. To test the private method, use the invokePrivate operator, like this:

targetObject invokePrivate decorateToStringValue(1)

Here, targetObject is a variable or singleton object name referring to the object whose private method you want to test. You pass the arguments to the private method in the parentheses after the PrivateMethod object. The result type of an invokePrivate operation will be the type parameter of the PrivateMethod object, thus you need not cast the result to use it. In other words, after creating a PrivateMethod object, the syntax to invoke the private method looks like a regular method invocation, but with the dot (.) replaced by invokePrivate. The private method is invoked dynamically via reflection, so if you have a typo in the method name symbol, specify the wrong result type, or pass invalid parameters, the invokePrivate operation will compile, but throw an exception at runtime.

One limitation to be aware of is that you can't use PrivateMethodTester to test a private method declared in a trait, because the class the trait gets mixed into will not declare that private method. Only the class generated to hold method implementations for the trait will have that private method. If you want to test a private method declared in a trait, and that method does not use any state of that trait, you can move the private method to a companion object for the trait and test it using PrivateMethodTester that way. If the private trait method you want to test uses the trait's state, your best options are to test it indirectly via a non-private trait method that calls the private method, or make the private method package access and test it directly via regular static method invocations.

Also, if you want to use PrivateMethodTester to invoke a parameterless private method, you'll need to use empty parens. Instead of:

targetObject invokePrivate privateParameterlessMethod

You'll need to write:

targetObject invokePrivate privateParameterlessMethod()

Trait that causes tests to be run in pseudo-random order.

Although the tests are run in pseudo-random order, events will be fired in the “normal” order for the Suite that mixes in this trait, as determined by runTests.

The purpose of this trait is to reduce the likelihood of unintentional order dependencies between tests in the same test class.

Offers two methods for transforming futures when exceptions are expected.

This trait offers two methods for testing for expected exceptions in the context of futures: recoverToSucceededIf and recoverToExceptionIf. Because this trait is mixed into trait AsyncTestSuite, both of its methods are available by default in any async-style suite.

If you just want to ensure that a future fails with a particular exception type, and do not need to inspect the exception further, use recoverToSucceededIf:

recoverToSucceededIf[IllegalStateException] { // Result type: Future[Assertion]
  emptyStackActor ? Peek
}

The recoverToSucceededIf method performs a job similar to assertThrows, except in the context of a future. It transforms a Future of any type into a Future[Assertion] that succeeds only if the original future fails with the specified exception. Here's an example in the REPL:

scala> import org.scalatest.RecoverMethods._
import org.scalatest.RecoverMethods._

scala> import scala.concurrent.Future
import scala.concurrent.Future

scala> import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.ExecutionContext.Implicits.global

scala> recoverToSucceededIf[IllegalStateException] {
     |   Future { throw new IllegalStateException }
     | }
res0: scala.concurrent.Future[org.scalatest.Assertion] = ...

scala> res0.value
res1: Option[scala.util.Try[org.scalatest.Assertion]] = Some(Success(Succeeded))

Otherwise it fails with an error message similar to those given by assertThrows:

scala> recoverToSucceededIf[IllegalStateException] {
     |   Future { throw new RuntimeException }
     | }
res2: scala.concurrent.Future[org.scalatest.Assertion] = ...

scala> res2.value
res3: Option[scala.util.Try[org.scalatest.Assertion]] =
    Some(Failure(org.scalatest.exceptions.TestFailedException: Expected exception
      java.lang.IllegalStateException to be thrown, but java.lang.RuntimeException
      was thrown))

scala> recoverToSucceededIf[IllegalStateException] {
     |   Future { 42 }
     | }
res4: scala.concurrent.Future[org.scalatest.Assertion] = ...

scala> res4.value
res5: Option[scala.util.Try[org.scalatest.Assertion]] =
    Some(Failure(org.scalatest.exceptions.TestFailedException: Expected exception
      java.lang.IllegalStateException to be thrown, but no exception was thrown))

The recoverToExceptionIf method differs from the recoverToSucceededIf in its behavior when the assertion succeeds: recoverToSucceededIf yields a Future[Assertion], whereas recoverToExceptionIf yields a Future[T], where T is the expected exception type.

recoverToExceptionIf[IllegalStateException] { // Result type: Future[IllegalStateException]
  emptyStackActor ? Peek
}

In other words, recoverToExpectionIf is to intercept as recovertToSucceededIf is to assertThrows. The first one allows you to perform further assertions on the expected exception. The second one gives you a result type that will satisfy the type checker at the end of the test body. Here's an example showing recoverToExceptionIf in the REPL:

scala> val futureEx =
     |   recoverToExceptionIf[IllegalStateException] {
     |     Future { throw new IllegalStateException("hello") }
     |   }
futureEx: scala.concurrent.Future[IllegalStateException] = ...

scala> futureEx.value
res6: Option[scala.util.Try[IllegalStateException]] =
    Some(Success(java.lang.IllegalStateException: hello))

scala> futureEx map { ex => assert(ex.getMessage == "world") }
res7: scala.concurrent.Future[org.scalatest.Assertion] = ...

scala> res7.value
res8: Option[scala.util.Try[org.scalatest.Assertion]] =
    Some(Failure(org.scalatest.exceptions.TestFailedException: "[hello]" did not equal "[world]"))

Trait whose instances collect the results of a running suite of tests and presents those results in some way to the user. Instances of this trait can be called "report functions" or "reporters."

Reporters receive test results via fifteen events. Each event is fired to pass a particular kind of information to the reporter. The events are:

• DiscoveryStarting
• DiscoveryCompleted
• RunStarting
• RunStopped
• RunAborted
• RunCompleted
• ScopeOpened
• ScopeClosed
• ScopePending
• TestStarting
• TestSucceeded
• TestFailed
• TestCanceled
• TestIgnored
• TestPending
• SuiteStarting
• SuiteCompleted
• SuiteAborted
• InfoProvided
• MarkupProvided
• AlertProvided
• NoteProvided

Reporters may be implemented such that they only present some of the reported events to the user. For example, you could define a reporter class that does nothing in response to SuiteStarting events. Such a class would always ignore SuiteStarting events.

The term test as used in the TestStarting, TestSucceeded, and TestFailed event names is defined abstractly to enable a wide range of test implementations. ScalaTest's style traits (subclasse of trait Suite) fire TestStarting to indicate they are about to invoke one of their tests, TestSucceeded to indicate a test returned normally, and TestFailed to indicate a test completed abruptly with an exception. Although the execution of a Suite subclass's tests will likely be a common event reported via the TestStarting, TestSucceeded, and TestFailed events, because of the abstract definition of “test” used by the the event classes, these events are not limited to this use. Information about any conceptual test may be reported via the TestStarting, TestSucceeded, and TestFailed events.

Likewise, the term suite as used in the SuiteStarting, SuiteAborted, and SuiteCompleted event names is defined abstractly to enable a wide range of suite implementations. Object Runner fires SuiteStarting to indicate it is about to invoke run on a Suite, SuiteCompleted to indicate a Suite's run method returned normally, and SuiteAborted to indicate a Suite's run method completed abruptly with an exception. Similarly, class Suite fires SuiteStarting to indicate it is about to invoke run on a nested Suite, SuiteCompleted to indicate a nested Suite's run method returned normally, and SuiteAborted to indicate a nested Suite's run method completed abruptly with an exception. Although the execution of a Suite's run method will likely be a common event reported via the SuiteStarting, SuiteAborted, and SuiteCompleted events, because of the abstract definition of "suite" used by the event classes, these events are not limited to this use. Information about any conceptual suite may be reported via the SuiteStarting, SuiteAborted, and SuiteCompleted events.

Extensibility

You can create classes that extend Reporter to report test results in custom ways, and to report custom information passed as an event "payload." Reporter classes can handle events in any manner, including doing nothing.

Subtrait of Reporter that contains a dispose method for releasing any finite, non-memory resources, such as file handles, held by the Reporter. Runner will invoke dispose on any ResourcefulReporter when it no longer needs the Reporter.

Provides methods that can be used in withFixture implementations to retry tests in various scenarios.

Trait Retries is intended to help you deal with “flickers”—tests that usually pass, but occasionally fail. The best way to deal with such tests is to fix them so they always pass. Sometimes, however, this is not practical. In such cases, flickers can waste your time by forcing you to investigate test failures that turn out to be flickers. Or worse, like the boy who cried wolf, the flickers may train you an your colleagues to not pay attention to failures such that you don't notice real problems, at least not in a timely manner.

Trait Retries offers methods that will retry a failed and/or canceled test once, on the same thread, with or without a delay. These methods take a block that results in Outcome, and are intended to be used in withFixture methods. You should be very selective about which tests you retry, retrying those for which you have good evidence to conclude they are flickers. Thus it is recommended you only retry tests that are tagged with Retryable, and only tag tests as such once they have flickered consistently for a while, and only after you invested a reasonable effort into fixing them properly.

Here's an example showing how you might use Retries:

package org.scalatest.examples.tagobjects.retryable

import org.scalatest._
import tagobjects.Retryable

class SetSpec extends FlatSpec with Retries {

  override def withFixture(test: NoArgTest) = {
    if (isRetryable(test))
      withRetry { super.withFixture(test) }
    else
      super.withFixture(test)
  }

  "An empty Set" should "have size 0" taggedAs(Retryable) in {
    assert(Set.empty.size === 0)
  }
}

A Suite class mixing in SequentialNestedSuiteExecution that takes zero to many Suites, which will be returned from its nestedSuites method.

For example, you can define a suite that always executes a list of nested suites like this:

class StepsSuite extends Sequential(
  new Step1Suite,
  new Step2Suite,
  new Step3Suite,
  new Step4Suite,
  new Step5Suite
)

When StepsSuite is executed, it will execute its nested suites in the passed order: Step1Suite, Step2Suite, Step3Suite, Step4Suite, and Step5Suite. Because Sequential extends SequentialNestedSuiteExecution, the distributor passed to runNestedSuites will always be None. So not only will the suites passed to the constructor be executed sequentially, any tests and nested suites of the passed suites will also be executed sequentually.

The difference between Sequential and Stepwise is that although Stepwise executes its own nested suites sequentially, it passes whatever distributor was passed to it to those nested suites. Thus the nested suites could run their own nested suites and tests in parallel if that distributor is defined. By contrast, Sequential always passes None for the distributor to the nested suites, so any and every test and nested suite contained within the nested suites passed to the Sequential construtor will be executed sequentially.

Trait that causes the nested suites of any suite it is mixed into to be run sequentially even if a Distributor is passed to runNestedSuites. This trait overrides the runNestedSuites method and fowards every parameter passed to it to a superclass invocation of runNestedSuites, except it always passes None for the Distributor. Mix in this trait into any suite whose nested suites need to be run sequentially even with the rest of the run is being executed concurrently.

Trait that causes StackDepth exceptions thrown by a running test (such as TestFailedExceptions) to have the exception's stack trace severed at the stack depth. Because the stack depth indicates the exact line of code that caused the exception to be thrown, the severed stack trace will show that offending line of code on top. This can make the line of test code that discovered a problem to be more easily found in IDEs and tools that don't make use of ScalaTest's StackDepth exceptions directly.

Trait whose instances provide a run method and configuration fields that implement the ScalaTest shell: its DSL for the Scala interpreter.

The main command of the ScalaTest shell is run, which you can use to run a suite of tests. The shell also provides several commands for configuring a call to run:

• color (the default) - display results in color (green for success; red for failure; yellow for warning; blue for statistics)
• nocolor - display results without color
• durations - display durations of (i.e., how long it took to run) tests and suites
• nodurations (the default) - do not display durations of tests and suites
• shortstacks - display short (i.e., truncated to show just the most useful portion) stack traces for all exceptions
• fullstacks - display full stack trackes for all exceptions
• nostacks (the default) - display no stack trace for StackDepth exceptions and a short stack trace for non-StackDepth exceptions
• stats - display statistics before and after the run, such as expected test count before the run and tests succeeded, failed, pending, etc., counts after the run
• nostats (the default) not display statistics before or after the run

The default configuration is color, nodurations, nostacks, and nostats.

All of these commands are fields of trait org.scalatest.Shell. Each configuration command is a field that refers to another Shell instance with every configuration parameter the same except for the one you've asked to change. For example, durations provides a Shell instance that has every parameter configured the same way, except with durations enabled. When you invoke run on that, you will get a run with durations enabled and every other configuration parameter at its default value.

The other useful "command" to know about, though not technically part of the shell, is the apply factory method in the Suites singleton object. This allows you to easily create composite suites out of nested suites, which you can then pass to run. This will be demonstrated later in this documentation.

Using the ScalaTest shell

The package object of the org.scalatest package, although it does not extend Shell, declares all the same members as Shell. Its run method runs with all the Shell configuration parameters set to their default values. A good way to use the ScalaTest shell, therefore, is to import the members of package org.scalatest:

scala> import org.scalatest._
import org.scalatest._

One thing importing org.scalatest._ allows you to do is access any of ScalaTest's classes and traits by shorter names, for example:

scala> class ArithmeticSuite extends FunSuite with matchers.Matchers {
     |   test("addition works") {
     |     1 + 1 should equal (2)
     |   }
     |   ignore("subtraction works") {
     |     1 - 1 should equal (0)
     |   }
     |   test("multiplication works") {
     |     1 * 1 should equal (2)
     |   }
     |   test("division works") (pending)
     | }
defined class ArithmeticSuite

But importing org.scalatest._ also brings into scope the commands of the Shell, so you can, for example, invoke run without qualification:

scala> run(new ArithmeticSuite)
ArithmeticSuite:
- addition works
- subtraction works !!! IGNORED !!!
- multiplication works *** FAILED ***
  1 did not equal 2 (:16)
- division works (pending)

Configuring a single run

To configure a single run, you can prefix run by one or more configuration commands, separated by dots. For example, to enable durations during a single run, you would write:

scala> durations.run(new ArithmeticSuite)
ArithmeticSuite:
- addition works (102 milliseconds)
- subtraction works !!! IGNORED !!!
- multiplication works *** FAILED *** (36 milliseconds)
  1 did not equal 2 (:16)
- division works (pending)

To enable statistics during a single run, you would write:

scala> stats.run(new ArithmeticSuite)
Run starting. Expected test count is: 3
ArithmeticSuite:
- addition works
- subtraction works !!! IGNORED !!!
- multiplication works *** FAILED ***
  1 did not equal 2 (:16)
- division works (pending)
Run completed in 386 milliseconds.
Total number of tests run: 2
Suites: completed 1, aborted 0
Tests: succeeded 1, failed 1, ignored 1, pending 1
*** 1 TEST FAILED ***

And to enable both durations and statistics during a single run, you could write:

scala> durations.stats.run(new ArithmeticSuite)
Run starting. Expected test count is: 3
ArithmeticSuite:
- addition works (102 milliseconds)
- subtraction works !!! IGNORED !!!
- multiplication works *** FAILED (36 milliseconds)***
  1 did not equal 2 (:16)
- division works (pending)
Run completed in 386 milliseconds.
Total number of tests run: 2
Suites: completed 1, aborted 0
Tests: succeeded 1, failed 1, ignored 1, pending 1
*** 1 TEST FAILED ***

The order doesn't matter when you are chaining multiple configuration commands. You'll get the same result whether you write durations.stats.run or stats.durations.run.

To disable color, use nocolor:

scala> nocolor.run(new ArithmeticSuite)
ArithmeticSuite:
- addition works
- subtraction works !!! IGNORED !!!
- multiplication works *** FAILED ***
  1 did not equal 2 (:16)
- division works (pending)

To enable short stack traces during a single run, use shortstacks:

scala> shortstacks.run(new ArithmeticSuite)
ArithmeticSuite:
- addition works (101 milliseconds)
- subtraction works !!! IGNORED !!!
- multiplication works *** FAILED *** (33 milliseconds)
  1 did not equal 2 (:16)
  org.scalatest.exceptions.TestFailedException:
  ...
  at line2$object$$iw$$iw$$iw$$iw$ArithmeticSuite$$anonfun$3.apply$mcV$sp(:16)
  at line2$object$$iw$$iw$$iw$$iw$ArithmeticSuite$$anonfun$3.apply(:16)
  at line2$object$$iw$$iw$$iw$$iw$ArithmeticSuite$$anonfun$3.apply(:16)
  at org.scalatest.FunSuite$$anon$1.apply(FunSuite.scala:992)
  at org.scalatest.Suite$class.withFixture(Suite.scala:1661)
  at line2$object$$iw$$iw$$iw$$iw$ArithmeticSuite.withFixture(:8)
  at org.scalatest.FunSuite$class.invokeWithFixture$1(FunSuite.scala:989)
  ...
- division works (pending)

Changing the default configuration

If you want to change the default for multiple runs, you can import the members of your favorite Shell configuration. For example, if you always like to run with durations and statistics enabled, you could write:

scala> import stats.durations._
import stats.durations._

Now anytime you run statistics and durations will, by default, be enabled:

scala> run(new ArithmeticSuite)
Run starting. Expected test count is: 3
ArithmeticSuite:
- addition works (9 milliseconds)
- subtraction works !!! IGNORED !!!
- multiplication works *** FAILED *** (10 milliseconds)
  1 did not equal 2 (:18)
- division works (pending)
Run completed in 56 milliseconds.
Total number of tests run: 2
Suites: completed 1, aborted 0
Tests: succeeded 1, failed 1, ignored 1, pending 1
*** 1 TEST FAILED ***

Running multiple suites

If you want to run multiple suites, you can use the factory method in the Suites singleton object. If you wrap a comma-separated list of suite instances inside Suites(...), for example, you'll get a suite instance that contains no tests, but whose nested suites includes the suite instances you placed between the parentheses. You can place Suites inside Suites to any level of depth, creating a tree of suites to pass to run. Here's a (contrived) example in which ArithmeticSuite is executed four times:

scala> run(Suites(new ArithmeticSuite, new ArithmeticSuite, Suites(new ArithmeticSuite, new ArithmeticSuite)))
Run starting. Expected test count is: 12
Suites:
ArithmeticSuite:
- addition works (0 milliseconds)
- subtraction works !!! IGNORED !!!
- multiplication works *** FAILED *** (1 millisecond)
  1 did not equal 2 (:16)
- division works (pending)
ArithmeticSuite:
- addition works (1 millisecond)
- subtraction works !!! IGNORED !!!
- multiplication works *** FAILED *** (0 milliseconds)
  1 did not equal 2 (:16)
- division works (pending)
Suites:
ArithmeticSuite:
- addition works (0 milliseconds)
- subtraction works !!! IGNORED !!!
- multiplication works *** FAILED *** (0 milliseconds)
  1 did not equal 2 (:16)
- division works (pending)
ArithmeticSuite:
- addition works (0 milliseconds)
- subtraction works !!! IGNORED !!!
- multiplication works *** FAILED *** (0 milliseconds)
  1 did not equal 2 (:16)
- division works (pending)
Run completed in 144 milliseconds.
Total number of tests run: 8
Suites: completed 6, aborted 0
Tests: succeeded 4, failed 4, ignored 4, pending 4
*** 4 TESTS FAILED ***

Running a single test

The run command also allows you to specify the name of a test to run and/or a config map. You can run a particular test in a suite, for example, by specifying the test name after the suite instance in your call to run, like this:

scala> run(new ArithmeticSuite, "addition works")
ArithmeticSuite:
- addition works

Status implementation that can change its state over time.

A StatefulStatus begins its life in a successful state, and will remain successful unless setFailed is called. Once setFailed is called, the status will remain at failed. The setFailed method can be called multiple times (even though invoking it once is sufficient to permanently set the status to failed), but only up until setCompleted has been called. After setCompleted has been called, any invocation of setFailed will be greeted with an IllegalStateException.

Instances of this class are thread safe.

The result status of running a test or a suite, which is used to support parallel and asynchronous execution of tests.

This trait is the result type of the "run" lifecycle methods of trait Suite: run, runNestedSuites, runTests, and runTest. It can be used to determine whether a test or suite has completed, and if so, whether it succeeded, and if not, whether an exception was thrown that was not yet reported via a ScalaTest event. A Status is like a domain-specific Future[Boolean], where:

• an activity in which no test failed and no suite aborted is represented by Success(true)
• an activity during which at least one test failed or one suite aborted, but all exceptions that occured were reported by a ScalaTest events (such as TestFailedException) is represented by Success(false)
• an activity during which at least one test failed or one suite aborted and at least one exception occurred that was not reported via a ScalaTest event is represented by Failure(unreportedException)

Note that pending and canceled tests will not cause a Status to fail. Only failed tests and aborted suites will cause a Status to fail.

One use case of Status is to ensure that "after" code (such as an afterEach or afterAll method) does not execute until after the relevant entity (one test, one suite, or all of a suite's tests or nested suites) has completed. Another use case is to implement the default behavior of asynchronous styles, in which subsequent each test does not begin execution until after the previous test has completed.

A Suite class that takes zero to many Suites, which will be returned from its nestedSuites method and executed in “stepwise” fashion by its runNestedSuites method.

For example, you can define a suite that always executes a list of nested suites like this:

class StepsSuite extends Stepwise(
  new Step1Suite,
  new Step2Suite,
  new Step3Suite,
  new Step4Suite,
  new Step5Suite
)

When StepsSuite is executed, regardless of whether a Distributor is passed, it will execute its nested suites sequentially in the passed order: Step1Suite, Step2Suite, Step3Suite, Step4Suite, and Step5Suite.

The difference between Stepwise and Sequential is that although Stepwise executes its own nested suites sequentially, it passes whatever distributor was passed to it to those nested suites. Thus the nested suites could run their own nested suites and tests in parallel if that distributor is defined. By contrast, Sequential always passes None for the distributor to the nested suites, so any and every test and nested suite contained within the nested suites passed to the Sequential construtor will be executed sequentially.

Trait that causes the nested suites of any suite it is mixed into to be run sequentially even if a Distributor is passed to runNestedSuites. This trait overrides the runNestedSuites method and fowards every parameter passed to it to a superclass invocation of runNestedSuites, except it always passes None for the Distributor. Mix in this trait into any suite whose nested suites need to be run sequentially even with the rest of the run is being executed concurrently.

Trait whose instances can accept a stop request and indicate whether a stop has already been requested.

This is passed in to the run method of Suite, so that running suites of tests can be requested to stop early.

Trait providing a streamlined method that returns a Uniformity[T] instance for any subtype of scala.xml.NodeSeq that will normalize the XML by removing empty text nodes and trimming non-empty text nodes.

Here's an example of some unnormalized XML:

<summer>
  <day></day>
  <night>
    with lots of stars
  </night>
</summer>

The Uniformity returned by this trait's streamlined method would transform the above XML to:

<summer><day></day><night>with lots of stars</night></summer>

The streamlined method can be used with the Explicitly DSL, like this:

xmlElem should equal (
  <summer>
    <day></day>
    <night>
      with lots of stars
    </night>
  </summer>
) (after being streamlined[Elem])

The goal of this trait is to provide a normalization for XML that makes it easier to test XML objects for equality. White space is significant in XML, and is taken into account by the default equality for XML, accessed by invoking the == method on an XML NodeSeq. Here's an example:

scala> val xmlElem = <summer><day></day><night>with lots of stars</night></summer>
xmlElem: scala.xml.Elem = <summer><day></day><night>with lots of stars</night></summer>

scala> xmlElem == <summer>
     |   <day></day>
     |   <night>
     |     with lots of stars
     |   </night>
     | </summer>
res1: Boolean = false

The above equality comparison produces false because of whitespace differences in the XML. When such whitespace differences are unimportant to the actual application, it can make it easier to write readable test code if you can compare XML for equality without taking into account empty text nodes, or leading and trailing whitespace in nonempty text nodes. The streamlined method of this trait provides a Uniformity instance that does just that:

scala> import org.scalactic._
import org.scalactic._

scala> import Explicitly._
import Explicitly._

scala> import TripleEquals._
import TripleEquals._

scala> import org.scalatest.StreamlinedXml._
import StreamlinedXml._

scala> import scala.xml.Elem
import scala.xml.Elem *

scala> (xmlElem === <summer>
     |   <day></day>
     |   <night>
     |     with lots of stars
     |   </night>
     | </summer>) (after being streamlined[Elem])
res9: Boolean = true

Trait providing an implicit Equality[T] for subtypes of scala.xml.NodeSeq that before testing for equality, will normalize left and right sides by removing empty XML text nodes and trimming non-empty text nodes.

Here's an example of some unnormalized XML:

<summer>
  <day></day>
  <night>
    with lots of stars
  </night>
</summer>

Prior to testing it for equality, the implicit Equality[T] provided by this trait would transform the above XML to:

<summer><day></day><night>with lots of stars</night></summer>

The goal of this trait is to provide an implicit Equality for XML that makes it easier to write tests involving XML. White space is significant in XML, and is taken into account by the default equality for XML, accessed by invoking the == method on an XML NodeSeq. Here's an example:

scala> val xmlElem = <summer><day></day><night>with lots of stars</night></summer>
xmlElem: scala.xml.Elem = <summer><day></day><night>with lots of stars</night></summer>

scala> import org.scalatest.Assertions._
import org.scalatest.Assertions._

scala> assert(xmlElem === <summer>
     |   <day></day>
     |     <night>
     |       with lots of stars
     |     </night>
     |   </summer>)
org.scalatest.exceptions.TestFailedException: <summer><day></day><night>with lots of stars</night></summer> did not equal <summer>
  <day></day>
    <night>
      with lots of stars
    </night>
  </summer>
  at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:500)
  at org.scalatest.Assertions$.newAssertionFailedException(Assertions.scala:1538)
  at org.scalatest.Assertions$AssertionsHelper.macroAssert(Assertions.scala:466)
  ... 53 elided

The above assertion fails because of whitespace differences in the XML. When such whitespace differences are unimportant to the actual application, it can make it easier to write readable test code if you can compare XML for equality without taking into account empty text nodes, or leading and trailing whitespace in nonempty text nodes. This trait provides an Equality[T] instance that does just that:

scala> import org.scalatest.StreamlinedXmlEquality._
import org.scalatest.StreamlinedXmlEquality._

scala> assert(xmlElem === <summer>
     |   <day></day>
     |   <night>
     |     with lots of stars
     |   </night>
     | </summer>)

Subtrait of NormMethods that provides an implicit Uniformity[T] for subtypes of scala.xml.NodeSeq that enables you to streamline XML by invoking .norm on it.

Here's an example:

scala> <good><day>sunshine</day></good> == <good>
     |   <day>
     |     sunshine
     |   </day>
     | </good>
res1: Boolean = false

scala> import org.scalactic._
import org.scalactic._

scala> import TripleEquals._
import TripleEquals._

scala> import org.scalatest.StreamlinedXmlNormMethods._
import org.scalatest.StreamlinedXmlNormMethods._

scala> <good><day>sunshine</day></good> === <good>
     |   <day>
     |     sunshine
     |   </day>
     | </good>.norm
res2: Boolean = true

A suite of tests. A Suite instance encapsulates a conceptual suite (i.e., a collection) of tests.

This trait provides an interface composed of "lifecycle methods" that allow suites of tests to be run. Its implementation enables a default way of writing and executing tests. Subtraits and subclasses can override Suite's lifecycle methods to enable other ways of writing and executing tests.

Nested suites

A Suite can refer to a collection of other Suites, which are called nested Suites. Those nested Suites can in turn have their own nested Suites, and so on. Large test suites can be organized, therefore, as a tree of nested Suites. This trait's run method, in addition to invoking its test methods, invokes run on each of its nested Suites.

A List of a Suite's nested Suites can be obtained by invoking its nestedSuites method. If you wish to create a Suite that serves as a container for nested Suites, whether or not it has test methods of its own, simply override nestedSuites to return a List of the nested Suites. Because this is a common use case, ScalaTest provides a convenience Suites class, which takes a variable number of nested Suites as constructor parameters. Here's an example:

package org.scalatest.examples.suite.nested

import org.scalatest._

class ASuite extends FunSuite {
  test("A should have ASCII value 41 hex") {
    assert('A' === 0x41)
  }
  test("a should have ASCII value 61 hex") {
    assert('a' === 0x61)
  }
}
class BSuite extends FunSuite {
  test("B should have ASCII value 42 hex") {
    assert('B' === 0x42)
  }
  test("b should have ASCII value 62 hex") {
    assert('b' === 0x62)
  }
}
class CSuite extends FunSuite {
  test("C should have ASCII value 43 hex") {
    assert('C' === 0x43)
  }
  test("c should have ASCII value 63 hex") {
    assert('c' === 0x63)
  }
}

class ASCIISuite extends Suites(
  new ASuite,
  new BSuite,
  new CSuite
)

If you now run ASCIISuite:

scala> org.scalatest.run(new ASCIISuite)

You will see reports printed to the standard output that indicate the nested suites—ASuite, BSuite, and CSuite—were run:

ASCIISuite:
ASuite:
- A should have ASCII value 41 hex
- a should have ASCII value 61 hex
BSuite:
- B should have ASCII value 42 hex
- b should have ASCII value 62 hex
CSuite:
- C should have ASCII value 43 hex
- c should have ASCII value 63 hex

Note that Runner can discover Suites automatically, so you need not necessarily define nested Suites explicitly. See the documentation for Runner for more information.

The config map

In some cases you may need to pass information to a suite of tests. For example, perhaps a suite of tests needs to grab information from a file, and you want to be able to specify a different filename during different runs. You can accomplish this in ScalaTest by passing the filename in a config map of key-value pairs, which is passed to run as a ConfigMap. The values in the config map are called "config objects," because they can be used to configure suites, reporters, and tests.

You can specify a string config object is via the ScalaTest Runner, either via the command line or ScalaTest's ant task. (See the documentation for Runner for information on how to specify config objects on the command line.) The config map is passed to run, runNestedSuites, runTests, and runTest, so one way to access it in your suite is to override one of those methods. If you need to use the config map inside your tests, you can access it from the NoArgTest passed to withFixture, or the OneArgTest passed to withFixture in the traits in the org.scalatest.fixture package. (See the documentation for FixtureSuite for instructions on how to access the config map in tests.)

Executing suites in parallel

The run method takes as one of its parameters an optional Distributor. If a Distributor is passed in, this trait's implementation of run puts its nested Suites into the distributor rather than executing them directly. The caller of run is responsible for ensuring that some entity runs the Suites placed into the distributor. The -P command line parameter to Runner, for example, will cause Suites put into the Distributor to be run in parallel via a pool of threads. If you wish to execute the tests themselves in parallel, mix in ParallelTestExecution.

"Run-aborting" exceptions

The Javadoc documentation for java.lang.Error states:

An Error is a subclass of Throwable that indicates serious problems that a reasonable application should not try to catch. Most such errors are abnormal conditions.

Because Errors are used to denote serious errors, trait Suite and its subtypes in the ScalaTest API do not always treat a test that completes abruptly with an Error as a test failure, but sometimes as an indication that serious problems have arisen that should cause the run to abort. For example, if a test completes abruptly with an OutOfMemoryError, it will not be reported as a test failure, but will instead cause the run to abort. Because not everyone uses Errors only to represent serious problems, however, ScalaTest only behaves this way for the following run-aborting exception types (and their subclasses):

• java.lang.annotation.AnnotationFormatError
• java.awt.AWTError
• java.nio.charset.CoderMalfunctionError
• javax.xml.parsers.FactoryConfigurationError
• java.lang.LinkageError
• java.lang.ThreadDeath
• javax.xml.transform.TransformerFactoryConfigurationError
• java.lang.VirtualMachineError

The previous list includes all Errors that exist as part of Java 1.5 API, excluding java.lang.AssertionError. ScalaTest does treat a thrown AssertionError as an indication of a test failure. In addition, any other Error that is not an instance of a type mentioned in the previous list will be caught by the Suite traits in the ScalaTest API and reported as the cause of a test failure.

Although trait Suite and all its subtypes in the ScalaTest API consistently behave this way with regard to Errors, this behavior is not required by the contract of Suite. Subclasses and subtraits that you define, for example, may treat all Errors as test failures, or indicate errors in some other way that has nothing to do with exceptions.

Extensibility

Trait Suite provides default implementations of its methods that should be sufficient for most applications, but many methods can be overridden when desired. Here's a summary of the methods that are intended to be overridden:

• run - override this method to define custom ways to run suites of tests.
• runNestedSuites - override this method to define custom ways to run nested suites.
• runTests - override this method to define custom ways to run a suite's tests.
• runTest - override this method to define custom ways to run a single named test.
• testNames - override this method to specify the Suite's test names in a custom way.
• tags - override this method to specify the Suite's test tags in a custom way.
• nestedSuites - override this method to specify the Suite's nested Suites in a custom way.
• suiteName - override this method to specify the Suite's name in a custom way.
• expectedTestCount - override this method to count this Suite's expected tests in a custom way.

For example, this trait's implementation of testNames performs reflection to discover methods starting with test, and places these in a Set whose iterator returns the names in alphabetical order. If you wish to run tests in a different order in a particular Suite, perhaps because a test named testAlpha can only succeed after a test named testBeta has run, you can override testNames so that it returns a Set whose iterator returns testBeta before testAlpha. (This trait's implementation of run will invoke tests in the order they come out of the testNames Set iterator.)

Alternatively, you may not like starting your test methods with test, and prefer using @Test annotations in the style of Java's JUnit 4 or TestNG. If so, you can override testNames to discover tests using either of these two APIs @Test annotations, or one of your own invention. (This is in fact how org.scalatest.junit.JUnitSuite and org.scalatest.testng.TestNGSuite work.)

Moreover, test in ScalaTest does not necessarily mean test method. A test can be anything that can be given a name, that starts and either succeeds or fails, and can be ignored. In org.scalatest.FunSuite, for example, tests are represented as function values. This approach might look foreign to JUnit users, but may feel more natural to programmers with a functional programming background. To facilitate this style of writing tests, FunSuite overrides testNames, runTest, and run such that you can define tests as function values.

You can also model existing JUnit 3, JUnit 4, or TestNG tests as suites of tests, thereby incorporating tests written in Java into a ScalaTest suite. The "wrapper" classes in packages org.scalatest.junit and org.scalatest.testng exist to make this easy. No matter what legacy tests you may have, it is likely you can create or use an existing Suite subclass that allows you to model those tests as ScalaTest suites and tests and incorporate them into a ScalaTest suite. You can then write new tests in Scala and continue supporting older tests in Java.

Trait defining abstract "lifecycle" methods that are implemented in Suite and can be overridden in stackable modification traits.

The main purpose of SuiteMixin is to differentiate core Suite style traits, such as Spec, FunSuite, and FunSpec from stackable modification traits for Suites such as BeforeAndAfterEach, OneInstancePerTest, and SequentialNestedSuiteExecution. Because these stackable traits extend SuiteMixin instead of Suite, you can't define a suite by simply extending one of the stackable traits:

class MySuite extends BeforeAndAfterEach // Won't compile

Instead, you need to extend a core Suite trait and mix the stackable BeforeAndAfterEach trait into that, like this:

class MySuite extends FunSuite with BeforeAndAfterEach // Compiles fine

A Suite class that takes zero to many Suites in its constructor, which will be returned from its nestedSuites method.

For example, you can define a suite that always executes a list of nested suites like this:

class StepsSuite extends Suites(
  new Step1Suite,
  new Step2Suite,
  new Step3Suite,
  new Step4Suite,
  new Step5Suite
)

If StepsSuite is executed sequentially, it will execute its nested suites in the passed order: Step1Suite, Step2Suite, Step3Suite, Step4Suite, and Step5Suite. If StepsSuite is executed in parallel, the nested suites will be executed concurrently.

Class whose subclasses can be used to tag tests in style traits in which tests are defined as functions.

ScalaTest has two ways to tag tests: annotations and instances of this Tag class. To tag a test method or an entire test class, you use a tag annotation, whereas to tag a test function, you use a Tag object. Though not required, it is usually a good idea to define both an annotation and a corresponding Tag object for each conceptual tag you want, so you can tag anything: test functions, test classes, and test methods. The name of the conceptual tag is the fully qualified name of the annotation interface, so you must pass this name to the Tag constructor.

For example, imagine you want to tag integration tests that use the actual database, and are, therefore, generally slower. You could create a tag annotation and object called DbTest. To give them both the same simple name, you can declare them in different packages. The tag annotation must be written in Java, not Scala, because annotations written in Scala are not accessible at runtime. Here's an example:

package com.mycompany.myproject.testing.tags;

import java.lang.annotation.*;
import org.scalatest.TagAnnotation

@TagAnnotation
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD, ElementType.TYPE})
public @interface DbTest {}

Given this annotation's fully qualified name is com.mycompany.myproject.testing.tags.DbTest the corresponding Tag object decaration must have that name passed to its constructor, like this:

package com.mycompany.myproject.testing.tagobjects

object DbTest extends Tag("com.mycompany.myproject.testing.tags.DbTest")

Given these definitions, you could tag a test function as a DbTest in, for example, a FlatSpec like this:

import org.scalatest.FlatSpec
import com.mycompany.myproject.testing.tagobjects.DbTest

class ExampleSpec extends FlatSpec {

  "Integration tests" can "sometimes be slow" taggedAs(DbTest) in {
    Thread.sleep(1000)
  }
}

You could tag a test method as a DbTest in, for example, a Suite like this:

import org.scalatest.Suite
import com.mycompany.myproject.testing.tags.DbTest

class ExampleSuite extends Suite {

  @DbTest
  def `integration tests can sometimes be slow` {
    Thread.sleep(1000)
  }
}

And you could tag all the tests in an entire test class by annotating the class, like this:

import org.scalatest.FlatSpec
import com.mycompany.myproject.testing.tags.DbTest

@DBTest
class ExampleSpec extends FlatSpec {

  "Integration tests" can "sometimes be slow" in {
    Thread.sleep(1000)
  }

  they should "likely sometimes be excluded " in {
    Thread.sleep(1000)
  }
}

In the previous example, both tests will be tagged as DBTests even though the tests are not tagged as such individually.

When you run ScalaTest and want to either include or exclude DbTests, you'd give the fully qualified name of the tag annotation (which is also the name passed to the corresponding Tag constructor) to Runner. For example, here's how you'd exclude DbTests on the Runner command line:

-l com.mycompany.myproject.testing.tags.DbTest

For examples of tagging in other style traits, see the "Tagging tests" section in the documentation for the trait:

• Tagging FeatureSpec tests
• Tagging FlatSpec tests
• Tagging FreeSpec tests
• Tagging FunSpec tests
• Tagging FunSuite tests
• Tagging PropSpec tests
• Tagging Spec tests
• Tagging WordSpec tests

Annotation used to annotate annotation interfaces that define tags for ScalaTest tests.

Note: This is actually an annotation defined in Java, not a Scala trait. It must be defined in Java instead of Scala so it will be accessible at runtime. It has been inserted into Scaladoc by pretending it is a trait.

ScalaTest will only consider annotations that are themselves annotated with TagAnnotation as tag annotations, to avoid accidentally interpreting arbitrary annotations as tags. You use TagAnnotation, therefore, when you define a tag annotation (which you must do in Java). Here's an example:

package com.mycompany.myproject.testing.tags;

import java.lang.annotation.*;
import org.scalatest.TagAnnotation *

@TagAnnotation
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD, ElementType.TYPE})
public @interface DbTest {}

For more information, see the documentation for class Tag.

A bundle of information about the current test.

A TestData object is passed to the withFixture methods of traits Suite and FixtureSuite (both NoArgTest and OneArgTest extend TestData) and to the beforeEach and afterEach methods of trait BeforeAndAfterEach. This enables fixtures and tests to make use of the test name and configuration objects in the config map.

In ScalaTest's event model, a test may be surrounded by “scopes.” Each test and scope is associated with string of text. A test's name is a concatenation of the text of any surrounding scopes followed by the text provided with the test itself, after each text element has been trimmed and one space inserted between each component. Here's an example:

package org.scalatest.examples.freespec

import org.scalatest.FreeSpec

class SetSpec extends FreeSpec {

  "A Set" - {
    "when empty" - {
      "should have size 0" in {
        assert(Set.empty.size === 0)
      }

      "should produce NoSuchElementException when head is invoked" in {
        assertThrows[NoSuchElementException] {
          Set.empty.head
        }
      }
    }
  }
}

The above FreeSpec contains two tests, both nested inside the same two scopes. The outermost scope names the subject, A Set. The nested scope qualifies the subject with when empty. Inside that scope are the two tests. The text of the tests are:

• should have size 0
• should produce NoSuchElementException when head is invoked

Therefore, the names of these two tests are:

• A Stack when empty should have size 0
• A Stack when empty should produce NoSuchElementException when head is invoked

The TestData instance for the first test would contain:

• name: "A Stack when empty should have size 0"
• scopes: collection.immutable.IndexedSeq("A Stack", "when empty")
• text: "should have size 0"

The base trait of ScalaTest's synchronous testing styles, which defines a withFixture lifecycle method that accepts as its parameter a test function that returns an Outcome.

The withFixture method add by this trait has the following signature and implementation:

def withFixture(test: NoArgTest): Outcome = {
  test()
}

The apply method of test function interface, NoArgTest, also returns Outcome:

// In trait NoArgTest:
def apply(): Outcome

Because the result of a test is an Outcome, when the test function returns, the test body must have determined an outcome already. It will already be one of Succeeded, Failed, Canceled, or Pending. This is also true when withFixture(NoArgTest) returns: because the result type of withFixture(NoArgTest) is Outcome, the test has by definition already finished execution.

The recommended way to ensure cleanup is performed after a test body finishes execution is to use a try-finally clause. Using try-finally will ensure that cleanup will occur whether the test function completes abruptly by throwing a suite-aborting exception, or returns normally yielding an Outcome. Note that the only situation in which a test function will complete abruptly with an exception is if the test body throws a suite-aborting exception. Any other exception will be caught and reported as either a Failed, Canceled, or Pending.

The withFixture method is designed to be stacked, and to enable this, you should always call the super implementation of withFixture, and let it invoke the test function rather than invoking the test function directly. In other words, instead of writing “test()”, you should write “super.withFixture(test)”. Thus, the recommended structure of a withFixture implementation that performs cleanup looks like this:

// Your implementation
override def withFixture(test: NoArgTest) = {
  // Perform setup here
  try {
    super.withFixture(test) // Invoke the test function
  } finally {
    // Perform cleanup here
  }
}

If you have no cleanup to perform, you can write withFixture like this instead:

// Your implementation
override def withFixture(test: NoArgTest) = {
  // Perform setup here
  super.withFixture(test) // Invoke the test function
}

If you want to perform an action only for certain outcomes, you can use a pattern match. For example, if you want to perform an action if a test fails, you'd match on Failed, like this:

// Your implementation
override def withFixture(test: NoArgTest) = {

  // Perform setup here

  val outcome = super.withFixture(test) // Invoke the test function

  outcome match {
    case failed: Failed =>
      // perform action that you want to occur
      // only if a test fails here
      failed
    case other => other
  }
}

If you want to change the outcome in some way in withFixture, you can also use a pattern match. For example, if a particular exception intermittently causes a test to fail, and can transform those failures into cancelations, like this:

// Your implementation
override def withFixture(test: NoArgTest) = {

  super.withFixture(test) match {
    case Failed(ex: ParticularException) =>
      Canceled("Muting flicker", ex)
    case other => other
  }
}

Trait defining abstract "lifecycle" methods that are implemented in TestSuite and can be overridden in stackable modification traits.

The main use case for this trait is to override withFixture in a mixin trait. Here's an example:

trait Builder extends TestSuiteMixin { this: TestSuite =>

  val builder = new StringBuilder

  abstract override def withFixture(test: NoArgTest) = {
    builder.append("ScalaTest is ")
    try super.withFixture(test) // To be stackable, must call super.withFixture
    finally builder.clear()
  }
}

SuiteMixin trait that overrides run to execute tests before nested suites.

The default behavior of run executes nested suites before running tests declared in this suite. This trait overrides run and executes this suites's tests first, then executes its nested suites.

Class that tracks the progress of a series of Ordinals produced by invoking next and nextNewOldPair on the current Ordinal.

Instances of this class are thread safe. Multiple threads can invoke nextOrdinal and nextTracker concurrently. This facilitates multi-threaded tests that send infoProvided reports concurrently. When using a Dispatcher to execute suites in parallel, the intention is that each Tracker will only be used by one thread. For example, if the optional Dispatcher passed to Suite's implementation of runNestedSuites is defined, that method will obtain a new Tracker by invoking nextTracker for each nested suite it passes to the Dispatcher.

Trait that provides an implicit conversion that adds success and failure methods to scala.util.Try, enabling you to make assertions about the value of a Success or the exception of a Failure.

The success method will return the Try on which it is invoked as a Success if the Try actually is a Success, or throw TestFailedException if not. The failure method will return the Try on which it is invoked as a Failure if the Try actually is a Failure, or throw TestFailedException if not.

This construct allows you to express in one statement that an Try should be either a Success or a Failure and that its value or exception, respectively,should meet some expectation. Here's an example:

try1.success.value should be > 9
try2.failure.exception should have message "/ by zero"

Or, using assertions instead of a matchers:

assert(try1.success.value > 9)
assert(try2.failure.exception.getMessage == "/ by zero")

Were you to simply invoke get on the Try, if the Try wasn't a Success, it would throw the exception contained in the Failure:

val try2 = Try { 1 / 0 }

try2.get should be < 9 // try2.get throws ArithmeticException

The ArithmeticException would cause the test to fail, but without providing a stack depth pointing to the failing line of test code. This stack depth, provided by TestFailedException (and a few other ScalaTest exceptions), makes it quicker for users to navigate to the cause of the failure. Without TryValues, to get a stack depth exception you would need to make two statements, like this:

try2 should be a 'success // throws TestFailedException
try2.get should be < 9

The TryValues trait allows you to state that more concisely:

try2.success.value should be < 9 // throws TestFailedException

Annotation to associate a wrapper suite with a non-Suite class, so it can be run via ScalaTest.

Note: This is actually an annotation defined in Java, not a Scala trait. It must be defined in Java instead of Scala so it will be accessible at runtime. It has been inserted into Scaladoc by pretending it is a trait.

A class will be considered annotated with WrapWith if it is annotated directly or one of its superclasses (but not supertraits) are annotated with WrapWith. The wrapper suite must have a public, one-arg constructor that takes a Class instance whose type parameter is compatible with the class to wrap: i.e., the class being annotated with WrapWith. ScalaTest will load the class to wrap and construct a new instance of the wrapper suite, passing in the Class instance for the class to wrap. Here's an example:

import org.scalacheck.Properties

@WrapWith(classOf[ScalaCheckPropertiesSpec])
class StringSpecification extends Properties("String") {
  // ...
}

The ScalaCheckPropertiesSpec would need to have a public, no-arg constructor that accepts subclasses of org.scalacheck.Properties:

import org.scalacheck.Properties
import org.scalatest.Suite

class ScalaCheckPropertiesSpec(clazz: Class[_ <: Properties]) extends Suite {
  // ...
}

The name org.scalatest.AsyncFeatureSpec has been deprecated and will be removed in a future version of ScalaTest. Please use its new name, org.scalatest.featurespec.AsyncFeatureSpec, instead.

This type has been renamed to support future modularization of ScalaTest.

This can be rewritten automatically with autofix: https://github.com/scalatest/autofix/tree/master/3.1.x.

The name org.scalatest.AsyncFeatureSpecLike has been deprecated and will be removed in a future version of ScalaTest. Please use its new name, org.scalatest.featurespec.AsyncFeatureSpecLike, instead.

This type has been renamed to support future modularization of ScalaTest.

This can be rewritten automatically with autofix: https://github.com/scalatest/autofix/tree/master/3.1.x.

The name org.scalatest.AsyncFlatSpec has been deprecated and will be removed in a future version of ScalaTest. Please use its new name, org.scalatest.flatspec.AsyncFlatSpec, instead.

This type has been renamed to support future modularization of ScalaTest.

This can be rewritten automatically with autofix: https://github.com/scalatest/autofix/tree/master/3.1.x.

The name org.scalatest.AsyncFlatSpecLike has been deprecated and will be removed in a future version of ScalaTest. Please use its new name, org.scalatest.flatspec.AsyncFlatSpecLike, instead.

This type has been renamed to support future modularization of ScalaTest.

This can be rewritten automatically with autofix: https://github.com/scalatest/autofix/tree/master/3.1.x.

The name org.scalatest.AsyncFreeSpec has been deprecated and will be removed in a future version of ScalaTest. Please use its new name, org.scalatest.freespec.AsyncFreeSpec, instead.

This type has been renamed to support future modularization of ScalaTest.

This can be rewritten automatically with autofix: https://github.com/scalatest/autofix/tree/master/3.1.x.

The name org.scalatest.AsyncFreeSpecLike has been deprecated and will be removed in a future version of ScalaTest. Please use its new name, org.scalatest.freespec.AsyncFreeSpecLike, instead.

This type has been renamed to support future modularization of ScalaTest.

This can be rewritten automatically with autofix: https://github.com/scalatest/autofix/tree/master/3.1.x.

The name org.scalatest.AsyncFunSpec has been deprecated and will be removed in a future version of ScalaTest. Please use its new name, org.scalatest.funspec.AsyncFunSpec, instead.

This type has been renamed to support future modularization of ScalaTest.

This can be rewritten automatically with autofix: https://github.com/scalatest/autofix/tree/master/3.1.x.

The name org.scalatest.AsyncFunSpecLike has been deprecated and will be removed in a future version of ScalaTest. Please use its new name, org.scalatest.funspec.AsyncFunSpecLike, instead.

This type has been renamed to support future modularization of ScalaTest.

This can be rewritten automatically with autofix: https://github.com/scalatest/autofix/tree/master/3.1.x.

The name org.scalatest.AsyncFunSuite has been deprecated and will be removed in a future version of ScalaTest. Please use its new name, org.scalatest.funsuite.AsyncFunSuite, instead.

This type has been renamed to support future modularization of ScalaTest.

This can be rewritten automatically with autofix: https://github.com/scalatest/autofix/tree/master/3.1.x.

The name org.scalatest.AsyncFunSuiteLike has been deprecated and will be removed in a future version of ScalaTest. Please use its new name, org.scalatest.funsuite.AsyncFunSuiteLike, instead.

This type has been renamed to support future modularization of ScalaTest.

This can be rewritten automatically with autofix: https://github.com/scalatest/autofix/tree/master/3.1.x.