ScalaTest 1.0
|
|
org/scalatest/Suite.scala
]
trait
Suite
extends
Assertions with
AbstractSuiteSuite
instance encapsulates a conceptual
suite (i.e., a collection) of tests.
This trait provides an interface that allows suites of tests to be run.
Its implementation enables a default way of writing and executing tests. Subtraits and subclasses can
override Suite
's methods to enable other ways of writing and executing tests.
This trait's default approach allows tests to be defined as methods whose name starts with "test
."
This approach is easy to understand, and a good way for Scala beginners to start writing tests.
More advanced Scala programmers may prefer to mix together other Suite
subtraits defined in ScalaTest,
or create their own, to write tests in the way they feel makes them most productive. Here's a quick overview
of some of the options to help you get started:
For JUnit 3 users
If you are using JUnit 3 (version 3.8 or earlier releases) and you want to write JUnit 3 tests in Scala, look at
AssertionsForJUnit
,
ShouldMatchersForJUnit
, and
JUnit3Suite
.
For JUnit 4 users
If you are using JUnit 4 and you want to write JUnit 4 tests in Scala, look at
JUnitSuite
, and
JUnitRunner
. With JUnitRunner
,
you can use any of the traits described here and still run your tests with JUnit 4.
For TestNG users
If you are using TestNG and you want to write TestNG tests in Scala, look at
TestNGSuite
.
For high-level testing
If you want to write tests at a higher level than unit tests, such as integration tests, acceptance tests,
or functional tests, check out FeatureSpec
.
For unit testing
If you prefer a behavior-driven development (BDD) style, in which tests are combined with text that
specifies the behavior being tested, look at
Spec
,
FlatSpec
, and
WordSpec
. Otherwise, if you just want to write tests
and don't want to combine testing with specifying, look at
FunSuite
or read on to learn how to write
tests using this base trait, Suite
.
To use this trait's approach to writing tests, simply create classes that
extend Suite
and define test methods. Test methods have names of the form testX
,
where X
is some unique, hopefully meaningful, string. A test method must be public and
can have any result type, but the most common result type is Unit
. Here's an example:
import org.scalatest.Suite class MySuite extends Suite { def testAddition() { val sum = 1 + 1 assert(sum === 2) assert(sum + 2 === 4) } def testSubtraction() { val diff = 4 - 1 assert(diff === 3) assert(diff - 2 === 1) } }
You can run a Suite
by invoking on it one of four overloaded execute
methods. These methods, which print test results to the
standard output, are intended to serve as a
convenient way to run tests from within the Scala interpreter. For example,
to run MySuite
from within the Scala interpreter, you could write:
scala> (new MySuite).execute()
And you would see:
Test Starting - MySuite: testAddition Test Succeeded - MySuite: testAddition Test Starting - MySuite: testSubtraction Test Succeeded - MySuite: testSubtraction
Or, to run just the testAddition
method, you could write:
scala> (new MySuite).execute("testAddition")
And you would see:
Test Starting - MySuite: testAddition Test Succeeded - MySuite: testAddition
Two other execute
methods that are intended to be run from the interpreter accept a "config" map of key-value
pairs (see Config map, below). Each of these execute
methods invokes a run
method takes seven
parameters. This run
method, which actually executes the suite, will usually be invoked by a test runner, such
as org.scalatest.tools.Runner
or an IDE. See the documentation
for Runner
for more detail.
Assertions and ===
Inside test methods in a Suite
, you can write assertions by invoking assert
and passing in a Boolean
expression,
such as:
val left = 2 val right = 1 assert(left == right)
If the passed expression is true
, assert
will return normally. If false
,
assert
will complete abruptly with a TestFailedException
. This exception is usually not caught
by the test method, which means the test method itself will complete abruptly by throwing the TestFailedException
. Any
test method that completes abruptly with a TestFailedException
or any Exception
is considered a failed
test. A test method that returns normally is considered a successful test.
If you pass a Boolean
expression to assert
, a failed assertion will be reported, but without
reporting the left and right values. You can alternatively encode these values in a String
passed as
a second argument to assert
, as in:
val left = 2 val right = 1 assert(left == right, left + " did not equal " + right)
Using this form of assert
, the failure report will include the left and right values, thereby
helping you debug the problem. However, ScalaTest provides the ===
operator to make this easier.
(The ===
operator is defined in trait Assertions
which trait Suite
extends.)
You use it like this:
val left = 2 val right = 1 assert(left === right)
Because you use ===
here instead of ==
, the failure report will include the left
and right values. For example, the detail message in the thrown TestFailedException
from the assert
shown previously will include, "2 did not equal 1".
From this message you will know that the operand on the left had the value 2, and the operand on the right had the value 1.
If you're familiar with JUnit, you would use ===
in a ScalaTest Suite
where you'd use assertEquals
in a JUnit TestCase
.
The ===
operator is made possible by an implicit conversion from Any
to Equalizer
. If you're curious to understand the mechanics, see the documentation for
Equalizer
and the convertToEqualizer
method.
Expected results
Although===
provides a natural, readable extension to Scala's assert
mechanism,
as the operands become lengthy, the code becomes less readable. In addition, the ===
comparison
doesn't distinguish between actual and expected values. The operands are just called left
and right
,
because if one were named expected
and the other actual
, it would be difficult for people to
remember which was which. To help with these limitations of assertions, Suite
includes a method called expect
that
can be used as an alternative to assert
with ===
. To use expect
, you place
the expected value in parentheses after expect
, followed by curly braces containing code
that should result in the expected value. For example:
val a = 5 val b = 2 expect(2) { a - b }
In this case, the expected value is 2
, and the code being tested is a - b
. This expectation will fail, and
the detail message in the TestFailedException
will read, "Expected 2, but got 3."
Intercepted exceptions
Sometimes you need to test whether a method throws an expected exception under certain circumstances, such as when invalid arguments are passed to the method. You can do this in the JUnit style, like this:
val s = "hi" try { s.charAt(-1) fail() } catch { case _: IndexOutOfBoundsException => // Expected, so continue }
If charAt
throws IndexOutOfBoundsException
as expected, control will transfer
to the catch case, which does nothing. If, however, charAt
fails to throw an exception,
the next statement, fail()
, will be executed. The fail
method always completes abruptly with
a TestFailedException
, thereby signaling a failed test.
To make this common use case easier to express and read, ScalaTest provides an intercept
method. You use it like this:
val s = "hi" intercept[IndexOutOfBoundsException] { s.charAt(-1) }
This code behaves much like the previous example. If charAt
throws an instance of IndexOutOfBoundsException
,
intercept
will return that exception. But if charAt
completes normally, or throws a different
exception, intercept
will complete abruptly with a TestFailedException
. The intercept
method returns the
caught exception so that you can inspect it further if you wish, for example, to ensure that data contained inside
the exception has the expected values. Here's an example:
val s = "hi" val caught = intercept[IndexOutOfBoundsException] { s.charAt(-1) } assert(caught.getMessage === "String index out of range: -1")
Using other assertions
ScalaTest also supports another style of assertions via its matchers DSL. By mixing in
trait ShouldMatchers
, you can
write suites that look like:
import org.scalatest.Suite import org.scalatest.matchers.ShouldMatchers class MySuite extends Suite with ShouldMatchers { def testAddition() { val sum = 1 + 1 sum should equal (2) sum + 2 should equal (4) } def testSubtraction() { val diff = 4 - 1 diff should equal (3) diff - 2 should equal (1) } }
If you prefer the word "must
" to the word "should
," you can alternatively mix in
trait MustMatchers
.
If you are comfortable with assertion mechanisms from other test frameworks, chances
are you can use them with ScalaTest. Any assertion mechanism that indicates a failure with an exception
can be used as is with ScalaTest. For example, to use the assertEquals
methods provided by JUnit or TestNG, simply import them and use them. (You will of course need
to include the relevant JAR file for the framework whose assertions you want to use on either the
classpath or runpath when you run your tests.) Here's an example in which JUnit's assertions are
imported, then used within a ScalaTest suite:
import org.scalatest.Suite import org.junit.Assert._ class MySuite extends Suite { def testAddition() { val sum = 1 + 1 assertEquals(2, sum) assertEquals(4, sum + 2) } def testSubtraction() { val diff = 4 - 1 assertEquals(3, diff) assertEquals(1, diff - 2) } }
Nested suites
A Suite
can refer to a collection of other Suite
s,
which are called nested Suite
s. Those nested Suite
s can in turn have
their own nested Suite
s, and so on. Large test suites can be organized, therefore, as a tree of
nested Suite
s.
This trait's run
method, in addition to invoking its
test methods, invokes run
on each of its nested Suite
s.
A List
of a Suite
's nested Suite
s can be obtained by invoking its
nestedSuites
method. If you wish to create a Suite
that serves as a
container for nested Suite
s, whether or not it has test methods of its own, simply override nestedSuites
to return a List
of the nested Suite
s. Because this is a common use case, ScalaTest provides
a convenience SuperSuite
class, which takes a List
of nested Suite
s as a constructor
parameter. Here's an example:
import org.scalatest.Suite class ASuite extends Suite class BSuite extends Suite class CSuite extends Suite class AlphabetSuite extends SuperSuite( List( new ASuite, new BSuite, new CSuite ) )
If you now run AlphabetSuite
, for example from the interpreter:
scala> (new AlphabetSuite).run()
You will see reports printed to the standard output that indicate nested
suites—ASuite
, BSuite
, and
CSuite
—were run.
Note that Runner
can discover Suite
s automatically, so you need not
necessarily specify SuperSuite
s explicitly. See the documentation
for Runner
for more information.
Shared fixtures
A test fixture is objects or other artifacts (such as files, sockets, database
connections, etc.) used by tests to do their work.
If a fixture is used by only one test method, then the definitions of the fixture objects can
be local to the method, such as the objects assigned to sum
and diff
in the
previous MySuite
examples. If multiple methods need to share an immutable fixture, one approach
is to assign them to instance variables. Here's a (very contrived) example, in which the object assigned
to shared
is used by multiple test methods:
import org.scalatest.Suite class MySuite extends Suite { // Sharing immutable fixture objects via instance variables val shared = 5 def testAddition() { val sum = 2 + 3 assert(sum === shared) } def testSubtraction() { val diff = 7 - 2 assert(diff === shared) } }
In some cases, however, shared mutable fixture objects may be changed by test methods such that
they need to be recreated or reinitialized before each test. Shared resources such
as files or database connections may also need to
be created and initialized before, and cleaned up after, each test. JUnit 3 offers methods setUp
and
tearDown
for this purpose. In ScalaTest, you can use the BeforeAndAfterEach
trait,
which will be described later, to implement an approach similar to JUnit's setUp
and tearDown
, however, this approach usually involves reassigning var
s
between tests. Before going that route, you may wish to consider some approaches that
avoid var
s. One approach is to write one or more create-fixture methods
that return a new instance of a needed object (or a tuple or case class holding new instances of
multiple objects) each time it is called. You can then call a create-fixture method at the beginning of each
test method that needs the fixture, storing the fixture object or objects in local variables. Here's an example:
import org.scalatest.Suite import scala.collection.mutable.ListBuffer class MySuite extends Suite { // create objects needed by tests and return as a tuple def createFixture = ( new StringBuilder("ScalaTest is "), new ListBuffer[String] ) def testEasy() { val (builder, lbuf) = createFixture builder.append("easy!") assert(builder.toString === "ScalaTest is easy!") assert(lbuf.isEmpty) lbuf += "sweet" } def testFun() { val (builder, lbuf) = createFixture builder.append("fun!") assert(builder.toString === "ScalaTest is fun!") assert(lbuf.isEmpty) } }
If different tests in the same Suite
require different fixtures, you can create multiple create-fixture methods and
call the method (or methods) needed by each test at the begining of the test. If every test method requires the same set of
mutable fixture objects, one other approach you can take is make them simply val
s and mix in trait
OneInstancePerTest
. If you mix in OneInstancePerTest
, each test
will be run in its own instance of the Suite
, similar to the way JUnit tests are executed.
Although the create-fixture and OneInstancePerTest
approaches take care of setting up a fixture before each
test, they don't address the problem of cleaning up a fixture after the test completes. In this situation,
one option is to mix in the BeforeAndAfterEach
trait.
BeforeAndAfterEach
's beforeEach
method will be run before, and its afterEach
method after, each test (like JUnit's setUp
and tearDown
methods, respectively).
For example, you could create a temporary file before each test, and delete it afterwords, like this:
import org.scalatest.Suite import org.scalatest.BeforeAndAfterEach import java.io.FileReader import java.io.FileWriter import java.io.File class MySuite extends Suite with BeforeAndAfterEach { private val FileName = "TempFile.txt" private var reader: FileReader = _ // Set up the temp file needed by the test override def beforeEach() { val writer = new FileWriter(FileName) try { writer.write("Hello, test!") } finally { writer.close() } // Create the reader needed by the test reader = new FileReader(FileName) } // Close and delete the temp file override def afterEach() { reader.close() val file = new File(FileName) file.delete() } def testReadingFromTheTempFile() { var builder = new StringBuilder var c = reader.read() while (c != -1) { builder.append(c.toChar) c = reader.read() } assert(builder.toString === "Hello, test!") } def testFirstCharOfTheTempFile() { assert(reader.read() === 'H') } def testWithoutAFixture() { assert(1 + 1 === 2) } }
In this example, the instance variable reader
is a var
, so
it can be reinitialized between tests by the beforeEach
method.
Although the BeforeAndAfterEach
approach should be familiar to the users of most
test other frameworks, ScalaTest provides another alternative that also allows you to perform cleanup
after each test: overriding withFixture(NoArgTest)
.
To execute each test, Suite
's implementation of the runTest
method wraps an invocation
of the appropriate test method in a no-arg function. runTest
passes that test function to the withFixture(NoArgTest)
method, which is responsible for actually running the test by invoking the function. Suite
's
implementation of withFixture(NoArgTest)
simply invokes the function, like this:
// Default implementation protected def withFixture(test: NoArgTest) { test() }
The withFixture(NoArgTest)
method exists so that you can override it and set a fixture up before, and clean it up after, each test.
Thus, the previous temp file example could also be implemented without mixing in BeforeAndAfterEach
, like this:
import org.scalatest.Suite import java.io.FileReader import java.io.FileWriter import java.io.File class MySuite extends Suite { private var reader: FileReader = _ override def withFixture(test: NoArgTest) { val FileName = "TempFile.txt" // Set up the temp file needed by the test val writer = new FileWriter(FileName) try { writer.write("Hello, test!") } finally { writer.close() } // Create the reader needed by the test reader = new FileReader(FileName) try { test() // Invoke the test function } finally { // Close and delete the temp file reader.close() val file = new File(FileName) file.delete() } } def testReadingFromTheTempFile() { var builder = new StringBuilder var c = reader.read() while (c != -1) { builder.append(c.toChar) c = reader.read() } assert(builder.toString === "Hello, test!") } def testFirstCharOfTheTempFile() { assert(reader.read() === 'H') } def testWithoutAFixture() { assert(1 + 1 === 2) } }
If you prefer to keep your test classes immutable, one final variation is to use the
FixtureSuite
trait from the
org.scalatest.fixture
package. Tests in an org.scalatest.fixture.FixtureSuite
can have a fixture
object passed in as a parameter. You must indicate the type of the fixture object
by defining the Fixture
type member and define a withFixture
method that takes a one-arg test function.
(A FixtureSuite
has two overloaded withFixture
methods, therefore, one that takes a OneArgTest
and the other, inherited from Suite
, that takes a NoArgTest
.)
Inside the withFixture(OneArgTest)
method, you create the fixture, pass it into the test function, then perform any
necessary cleanup after the test function returns. Instead of invoking each test directly, a FixtureSuite
will
pass a function that invokes the code of a test to withFixture(OneArgTest)
. Your withFixture(OneArgTest)
method, therefore,
is responsible for actually running the code of the test by invoking the test function.
For example, you could pass the temp file reader fixture to each test that needs it
by overriding the withFixture(OneArgTest)
method of a FixtureSuite
, like this:
import org.scalatest.fixture.FixtureSuite import java.io.FileReader import java.io.FileWriter import java.io.File class MySuite extends FixtureSuite { // No vars needed in this one type FixtureParam = FileReader def withFixture(test: OneArgTest) { val FileName = "TempFile.txt" // Set up the temp file needed by the test val writer = new FileWriter(FileName) try { writer.write("Hello, test!") } finally { writer.close() } // Create the reader needed by the test val reader = new FileReader(FileName) try { // Run the test, passing in the temp file reader test(reader) } finally { // Close and delete the temp file reader.close() val file = new File(FileName) file.delete() } } def testReadingFromTheTempFile(reader: FileReader) { var builder = new StringBuilder var c = reader.read() while (c != -1) { builder.append(c.toChar) c = reader.read() } assert(builder.toString === "Hello, test!") } def testFirstCharOfTheTempFile(reader: FileReader) { assert(reader.read() === 'H') } def testWithoutAFixture() { assert(1 + 1 === 2) } }
It is worth noting that the only difference in the test code between the mutable
BeforeAndAfterEach
approach shown previously and the immutable FixtureSuite
approach shown here is that two of the FixtureSuite
's test methods take a FileReader
as
a parameter. Otherwise the test code is identical. One benefit of the explicit parameter is that, as demonstrated
by the testWithoutAFixture
method, a FixtureSuite
test method need not take the fixture. (Tests that don't take a fixture as a parameter are passed to the withFixture
that takes a NoArgTest
, shown previously.) So you can have some tests that take a fixture, and others that don't.
In this case, the FixtureSuite
provides documentation indicating which
test methods use the fixture and which don't, whereas the BeforeAndAfterEach
approach does not.
If you want to execute code before and after all tests (and nested suites) in a suite, such
as you could do with @BeforeClass
and @AfterClass
annotations in JUnit 4, you can use the beforeAll
and afterAll
methods of BeforeAndAfterAll
. See the documentation for BeforeAndAfterAll
for
an example.
In some cases you may need to pass information to a suite of tests.
For example, perhaps a suite of tests needs to grab information from a file, and you want
to be able to specify a different filename during different runs. You can accomplish this in ScalaTest by passing
the filename in the config map of key-value pairs, which is passed to run
as a Map[String, Any]
.
The values in the config map are called "config objects," because they can be used to configure
suites, reporters, and tests.
You can specify a string config object is via the ScalaTest Runner
, either via the command line
or ScalaTest's ant task.
(See the documentation for Runner for information on how to specify
config objects on the command line.)
The config map is passed to run
, runNestedSuites
, runTests
, and runTest
,
so one way to access it in your suite is to override one of those methods. If you need to use the config map inside your tests, you
can use one of the traits in the org.scalatest.fixture
package. (See the
documentation for FixtureSuite
for instructions on how to access the config map in tests.)
Tagging tests
A Suite
's tests may be classified into groups by tagging them with string names. When executing
a Suite
, groups of tests can optionally be included and/or excluded. In this
trait's implementation, tags are indicated by annotations attached to the test method. To
create a new tag type to use in Suite
s, simply define a new Java annotation that itself is annotated with the org.scalatest.TagAnnotation
annotation.
(Currently, for annotations to be
visible in Scala programs via Java reflection, the annotations themselves must be written in Java.) For example,
to create a tag named SlowAsMolasses
, to use to mark slow tests, you would
write in Java:
import java.lang.annotation.*; import org.scalatest.TagAnnotation @TagAnnotation @Retention(RetentionPolicy.RUNTIME) @Target({ElementType.METHOD, ElementType.TYPE}) public @interface SlowAsMolasses {}
Given this new annotation, you could place a Suite
test method into the SlowAsMolasses
group
(i.e., tag it as being SlowAsMolasses
) like this:
@SlowAsMolasses def testSleeping() = sleep(1000000)
The primary run
method takes a Filter
, whose constructor takes an optional
Set[String]
s called tagsToInclude
and a Set[String]
called
tagsToExclude
. If tagsToInclude
is None
, all tests will be run
except those those belonging to tags listed in the
tagsToExclude
Set
. If tagsToInclude
is defined, only tests
belonging to tags mentioned in the tagsToInclude
set, and not mentioned in tagsToExclude
,
will be run.
Note, the TagAnnotation
annotation was introduced in ScalaTest 1.0, when "groups" were renamed
to "tags." In 1.0 and 1.1, the TagAnnotation
will continue to not be required by an annotation on a Suite
method. Any annotation on a Suite
method will be considered a tag until 1.2, to give users time to add
TagAnnotation
s on any tag annotations they made prior to the 1.0 release. From 1.2 onward, only annotations
themselves annotated by TagAnnotation
will be considered tag annotations.
Ignored tests
Another common use case is that tests must be “temporarily” disabled, with the
good intention of resurrecting the test at a later time. ScalaTest provides an Ignore
annotation for this purpose. You use it like this:
import org.scalatest.Suite import org.scalatest.Ignore class MySuite extends Suite { def testAddition() { val sum = 1 + 1 assert(sum === 2) assert(sum + 2 === 4) } @Ignore def testSubtraction() { val diff = 4 - 1 assert(diff === 3) assert(diff - 2 === 1) } }
If you run this version of MySuite
with:
scala> (new MySuite).run()
It will run only testAddition
and report that testSubtraction
was ignored. You'll see:
Test Starting - MySuite: testAddition Test Succeeded - MySuite: testAddition Test Ignored - MySuite: testSubtraction
Ignore
is implemented as a tag. The Filter
class effectively
adds org.scalatest.Ignore
to the tagsToExclude
Set
if it not already
in the tagsToExclude
set passed to its primary constructor. The only difference between
org.scalatest.Ignore
and the tags you may define and exclude is that ScalaTest reports
ignored tests to the Reporter
. The reason ScalaTest reports ignored tests is as a feeble
attempt to encourage ignored tests to be eventually fixed and added back into the active suite of tests.
Pending tests
A pending test is one that has been given a name but is not yet implemented. The purpose of pending tests is to facilitate a style of testing in which documentation of behavior is sketched out before tests are written to verify that behavior (and often, the before the behavior of the system being tested is itself implemented). Such sketches form a kind of specification of what tests and functionality to implement later.
To support this style of testing, a test can be given a name that specifies one
bit of behavior required by the system being tested. The test can also include some code that
sends more information about the behavior to the reporter when the tests run. At the end of the test,
it can call method pending
, which will cause it to complete abruptly with TestPendingException
.
Because tests in ScalaTest can be designated as pending with TestPendingException
, both the test name and any information
sent to the reporter when running the test can appear in the report of a test run. (In other words,
the code of a pending test is executed just like any other test.) However, because the test completes abruptly
with TestPendingException
, the test will be reported as pending, to indicate
the actual test, and possibly the functionality it is intended to test, has not yet been implemented.
Although pending tests may be used more often in specification-style suites, such as
org.scalatest.Spec
, you can also use it in Suite
, like this:
import org.scalatest.Suite class MySuite extends Suite { def testAddition() { val sum = 1 + 1 assert(sum === 2) assert(sum + 2 === 4) } def testSubtraction() { pending } }
If you run this version of MySuite
with:
scala> (new MySuite).run()
It will run both tests but report that testSubtraction
is pending. You'll see:
Test Starting - MySuite: testAddition Test Succeeded - MySuite: testAddition Test Starting - MySuite: testSubtraction Test Pending - MySuite: testSubtraction
Informers
One of the parameters to the primary run
method is an Reporter
, which
will collect and report information about the running suite of tests.
Information about suites and tests that were run, whether tests succeeded or failed,
and tests that were ignored will be passed to the Reporter
as the suite runs.
Most often the reporting done by default by Suite
's methods will be sufficient, but
occasionally you may wish to provide custom information to the Reporter
from a test method.
For this purpose, you can optionally include an Informer
parameter in a test method, and then
pass the extra information to the Informer
via its apply
method. The Informer
will then pass the information to the Reporter
by sending an InfoProvided
event.
Here's an example:
import org.scalatest._ class MySuite extends Suite { def testAddition(info: Informer) { assert(1 + 1 === 2) info("Addition seems to work") } }If you run this
Suite
from the interpreter, you will see the message
included in the printed report:
scala> (new MySuite).run() Test Starting - MySuite: testAddition(Reporter) Info Provided - MySuite: testAddition(Reporter) Addition seems to work Test Succeeded - MySuite: testAddition(Reporter)
Executing suites in parallel
The primary run
method takes as its last parameter an optional Distributor
. If
a Distributor
is passed in, this trait's implementation of run
puts its nested
Suite
s into the distributor rather than executing them directly. The caller of run
is responsible for ensuring that some entity runs the Suite
s placed into the
distributor. The -c
command line parameter to Runner
, for example, will cause
Suite
s put into the Distributor
to be run in parallel via a pool of threads.
Treatement of java.lang.Error
s
The Javadoc documentation for java.lang.Error
states:
AnError
is a subclass ofThrowable
that indicates serious problems that a reasonable application should not try to catch. Most such errors are abnormal conditions.
Because Error
s are used to denote serious errors, trait Suite
and its subtypes in the ScalaTest API do not always treat a test
that completes abruptly with an Error
as a test failure, but sometimes as an indication that serious problems
have arisen that should cause the run to abort. For example, if a test completes abruptly with an OutOfMemoryError
,
it will not be reported as a test failure, but will instead cause the run to abort. Because not everyone uses Error
s only to represent serious
problems, however, ScalaTest only behaves this way for the following exception types (and their subclasses):
java.lang.annotation.AnnotationFormatError
java.awt.AWTError
java.nio.charset.CoderMalfunctionError
javax.xml.parsers.FactoryConfigurationError
java.lang.LinkageError
java.lang.ThreadDeath
javax.xml.transform.TransformerFactoryConfigurationError
java.lang.VirtualMachineError
The previous list includes all Error
s that exist as part of Java 1.5 API, excluding java.lang.AssertionError
. ScalaTest
does treat a thrown AssertionError
as an indication of a test failure. In addition, any other Error
that is not an instance of a
type mentioned in the previous list will be caught by the Suite
traits in the ScalaTest API and reported as the cause of a test failure.
Although trait Suite
and all its subtypes in the ScalaTest API consistently behave this way with regard to Error
s,
this behavior is not required by the contract of Suite
. Subclasses and subtraits that you define, for example, may treat all
Error
s as test failures, or indicate errors in some other way that has nothing to do with exceptions.
Extensibility
Trait Suite
provides default implementations of its methods that should
be sufficient for most applications, but many methods can be overridden when desired. Here's
a summary of the methods that are intended to be overridden:
run
- override this method to define custom ways to run suites of
tests.runNestedSuites
- override this method to define custom ways to run nested suites.runTests
- override this method to define custom ways to run a suite's tests.runTest
- override this method to define custom ways to run a single named test.testNames
- override this method to specify the Suite
's test names in a custom way.tags
- override this method to specify the Suite
's test tags in a custom way.nestedSuites
- override this method to specify the Suite
's nested Suite
s in a custom way.suiteName
- override this method to specify the Suite
's name in a custom way.expectedTestCount
- override this method to count this Suite
's expected tests in a custom way.
For example, this trait's implementation of testNames
performs reflection to discover methods starting with test
,
and places these in a Set
whose iterator returns the names in alphabetical order. If you wish to run tests in a different
order in a particular Suite
, perhaps because a test named testAlpha
can only succeed after a test named
testBeta
has run, you can override testNames
so that it returns a Set
whose iterator returns
testBeta
before testAlpha
. (This trait's implementation of run
will invoke tests
in the order they come out of the testNames
Set
iterator.)
Alternatively, you may not like starting your test methods with test
, and prefer using @Test
annotations in
the style of Java's JUnit 4 or TestNG. If so, you can override testNames
to discover tests using either of these two APIs
@Test
annotations, or one of your own invention. (This is in fact
how org.scalatest.junit.JUnitSuite
and org.scalatest.testng.TestNGSuite
work.)
Moreover, test in ScalaTest does not necessarily mean test method. A test can be anything that can be given a name,
that starts and either succeeds or fails, and can be ignored. In org.scalatest.FunSuite
, for example, tests are represented
as function values. This
approach might look foreign to JUnit users, but may feel more natural to programmers with a functional programming background.
To facilitate this style of writing tests, FunSuite
overrides testNames
, runTest
, and run
such that you can
define tests as function values.
You can also model existing JUnit 3, JUnit 4, or TestNG tests as suites of tests, thereby incorporating tests written in Java into a ScalaTest suite.
The "wrapper" classes in packages org.scalatest.junit
and org.scalatest.testng
exist to make this easy.
No matter what legacy tests you may have, it is likely you can create or use an existing Suite
subclass that allows you to model those tests
as ScalaTest suites and tests and incorporate them into a ScalaTest suite. You can then write new tests in Scala and continue supporting
older tests in Java.
Method Summary | |
final def
|
execute
: Unit
Executes this
Suite , printing results to the standard output. |
final def
|
execute
(testName : java.lang.String) : Unit
Executes the test specified as
testName in this Suite , printing results to the standard output. |
final def
|
execute
(configMap : scala.collection.immutable.Map[java.lang.String, Any]) : Unit
Executes this
Suite with the specified configMap , printing results to the standard output. |
final def
|
execute
(testName : java.lang.String, configMap : scala.collection.immutable.Map[java.lang.String, Any]) : Unit
Executes the test specified as
testName in this Suite with the specified configMap , printing
results to the standard output. |
def
|
expectedTestCount
(filter : Filter) : Int
The total number of tests that are expected to run when this
Suite 's run method is invoked. |
final def
|
groups
: scala.collection.immutable.Map[java.lang.String, scala.collection.immutable.Set[java.lang.String]]
The
groups methods has been deprecated and will be removed in a future version of ScalaTest.
Please call (and override) tags instead. |
def
|
nestedSuites
: scala.List[Suite]
A
List of this Suite object's nested Suite s. If this Suite contains no nested Suite s,
this method returns an empty List . This trait's implementation of this method returns an empty List . |
def
|
pending
: PendingNothing
Throws
TestPendingException to indicate a test is pending. |
def
|
pendingUntilFixed
(f : => Unit) : Unit
Execute the passed block of code, and if it completes abruptly, throw
TestPendingException , else
throw TestFailedException . |
def
|
run
(testName : scala.Option[java.lang.String], reporter : Reporter, stopper : Stopper, filter : Filter, configMap : scala.collection.immutable.Map[java.lang.String, Any], distributor : scala.Option[Distributor], tracker : Tracker) : Unit
Runs this suite of tests.
|
protected def
|
runNestedSuites
(reporter : Reporter, stopper : Stopper, filter : Filter, configMap : scala.collection.immutable.Map[java.lang.String, Any], distributor : scala.Option[Distributor], tracker : Tracker) : Unit
Run zero to many of this
Suite 's nested Suite s. |
protected def
|
runTest
(testName : java.lang.String, reporter : Reporter, stopper : Stopper, configMap : scala.collection.immutable.Map[java.lang.String, Any], tracker : Tracker) : Unit
Run a test.
|
protected def
|
runTests
(testName : scala.Option[java.lang.String], reporter : Reporter, stopper : Stopper, filter : Filter, configMap : scala.collection.immutable.Map[java.lang.String, Any], distributor : scala.Option[Distributor], tracker : Tracker) : Unit
Run zero to many of this
Suite 's tests. |
def
|
suiteName
: java.lang.String
A user-friendly suite name for this
Suite . |
def
|
tags
: scala.collection.immutable.Map[java.lang.String, scala.collection.immutable.Set[java.lang.String]]
A
Map whose keys are String tag names with which tests in this Suite are marked, and
whose values are the Set of test names marked with each tag. If this Suite contains no tags, this
method returns an empty Map . |
def
|
testNames
: scala.collection.immutable.Set[java.lang.String]
An
Set of test names. If this Suite contains no tests, this method returns an empty Set . |
protected def
|
withFixture
(test : NoArgTest) : Unit
Run the passed test function in the context of a fixture established by this method.
|
Methods inherited from Assertions | |
assert, assert, assert, assert, convertToEqualizer, intercept, expect, expect, fail, fail, fail, fail |
Methods inherited from AnyRef | |
getClass, hashCode, equals, clone, toString, notify, notifyAll, wait, wait, wait, finalize, ==, !=, eq, ne, synchronized |
Methods inherited from Any | |
==, !=, isInstanceOf, asInstanceOf |
Class Summary | |
protected trait
|
NoArgTest
extends () => Unit
A test function taking no arguments, which also provides a test name and config map.
|
Method Details |
def
nestedSuites : scala.List[Suite]
List
of this Suite
object's nested Suite
s. If this Suite
contains no nested Suite
s,
this method returns an empty List
. This trait's implementation of this method returns an empty List
.final
def
execute : Unit
Suite
, printing results to the standard output.
This method implementation calls run
on this Suite
, passing in:
testName
- None
reporter
- a reporter that prints to the standard outputstopper
- a Stopper
whose apply
method always returns false
filter
- a Filter
constructed with None
for tagsToInclude
and Set()
for tagsToExclude
configMap
- an empty Map[String, Any]
distributor
- None
tracker
- a new Tracker
This method serves as a convenient way to execute a Suite
, especially from
within the Scala interpreter.
Note: In ScalaTest, the terms "execute" and "run" basically mean the same thing and
can be used interchangably. The reason this convenience method and its three overloaded forms
aren't named run
is because junit.framework.TestCase
declares a run
method
that takes no arguments but returns a junit.framework.TestResult
. That
run
method would not overload with this method if it were named run
,
because it would have the same parameters but a different return type than the one
defined in TestCase
. To facilitate integration with JUnit 3, therefore,
these convenience "run" methods are named execute
. In particular, this allows trait
org.scalatest.junit.JUnit3Suite
to extend both org.scalatest.Suite
and
junit.framework.TestCase
, which enables the creating of classes that
can be run with either ScalaTest or JUnit 3.
final
def
execute(configMap : scala.collection.immutable.Map[java.lang.String, Any]) : Unit
Suite
with the specified configMap
, printing results to the standard output.
This method implementation calls run
on this Suite
, passing in:
testName
- None
reporter
- a reporter that prints to the standard outputstopper
- a Stopper
whose apply
method always returns false
filter
- a Filter
constructed with None
for tagsToInclude
and Set()
for tagsToExclude
configMap
- the specified configMap
Map[String, Any]
distributor
- None
tracker
- a new Tracker
This method serves as a convenient way to execute a Suite
, passing in some objects via the configMap
, especially from within the Scala interpreter.
Note: In ScalaTest, the terms "execute" and "run" basically mean the same thing and
can be used interchangably. The reason this convenience method and its three overloaded forms
aren't named run
is described the documentation of the overloaded form that
takes no parameters: execute().
configMap -
a Map
of key-value pairs that can be used by the executing Suite
of tests.NullPointerException -
if the passed configMap
parameter is null
.final
def
execute(testName : java.lang.String) : Unit
testName
in this Suite
, printing results to the standard output.
This method implementation calls run
on this Suite
, passing in:
testName
- Some(testName)
reporter
- a reporter that prints to the standard outputstopper
- a Stopper
whose apply
method always returns false
filter
- a Filter
constructed with None
for tagsToInclude
and Set()
for tagsToExclude
configMap
- an empty Map[String, Any]
distributor
- None
tracker
- a new Tracker
This method serves as a convenient way to run a single test, especially from within the Scala interpreter.
Note: In ScalaTest, the terms "execute" and "run" basically mean the same thing and
can be used interchangably. The reason this convenience method and its three overloaded forms
aren't named run
is described the documentation of the overloaded form that
takes no parameters: execute().
testName -
the name of one test to run.NullPointerException -
if the passed testName
parameter is null
.IllegalArgumentException -
if testName
is defined, but no test with the specified test name exists in this Suite
final
def
execute(testName : java.lang.String, configMap : scala.collection.immutable.Map[java.lang.String, Any]) : Unit
testName
in this Suite
with the specified configMap
, printing
results to the standard output.
This method implementation calls run
on this Suite
, passing in:
testName
- Some(testName)
reporter
- a reporter that prints to the standard outputstopper
- a Stopper
whose apply
method always returns false
filter
- a Filter
constructed with None
for tagsToInclude
and Set()
for tagsToExclude
configMap
- the specified configMap
Map[String, Any]
distributor
- None
tracker
- a new Tracker
This method serves as a convenient way to execute a single test, passing in some objects via the configMap
, especially from
within the Scala interpreter.
Note: In ScalaTest, the terms "execute" and "run" basically mean the same thing and
can be used interchangably. The reason this convenience method and its three overloaded forms
aren't named run
is described the documentation of the overloaded form that
takes no parameters: execute().
testName -
the name of one test to run.configMap -
a Map
of key-value pairs that can be used by the executing Suite
of tests.NullPointerException -
if either of the passed testName
or configMap
parameters is null
.IllegalArgumentException -
if testName
is defined, but no test with the specified test name exists in this Suite
def
tags : scala.collection.immutable.Map[java.lang.String, scala.collection.immutable.Set[java.lang.String]]
Map
whose keys are String
tag names with which tests in this Suite
are marked, and
whose values are the Set
of test names marked with each tag. If this Suite
contains no tags, this
method returns an empty Map
.
This trait's implementation of this method uses Java reflection to discover any Java annotations attached to its test methods. The
fully qualified name of each unique annotation that extends TagAnnotation
is considered a tag. This trait's
implementation of this method, therefore, places one key/value pair into to the
Map
for each unique tag annotation name discovered through reflection. The mapped value for each tag name key will contain
the test method name, as provided via the testNames
method.
Subclasses may override this method to define and/or discover tags in a custom manner, but overriding method implementations
should never return an empty Set
as a value. If a tag has no tests, its name should not appear as a key in the
returned Map
.
Note, the TagAnnotation
annotation was introduced in ScalaTest 1.0, when "groups" were renamed
to "tags." In 1.0 and 1.1, the TagAnnotation
will continue to not be required by an annotation on a Suite
method. Any annotation on a Suite
method will be considered a tag until 1.2, to give users time to add
TagAnnotation
s on any tag annotations they made prior to the 1.0 release. From 1.2 onward, only annotations
themselves annotated by TagAnnotation
will be considered tag annotations.
final
def
groups : scala.collection.immutable.Map[java.lang.String, scala.collection.immutable.Set[java.lang.String]]
groups
methods has been deprecated and will be removed in a future version of ScalaTest.
Please call (and override) tags
instead.
def
testNames : scala.collection.immutable.Set[java.lang.String]
Set
of test names. If this Suite
contains no tests, this method returns an empty Set
.
This trait's implementation of this method uses Java reflection to discover all public methods whose name starts with "test"
,
which take either nothing or a single Informer
as parameters. For each discovered test method, it assigns a test name
comprised of just the method name if the method takes no parameters, or the method name plus (Informer)
if the
method takes a Informer
. Here are a few method signatures and the names that this trait's implementation assigns them:
def testCat() {} // test name: "testCat" def testCat(Informer) {} // test name: "testCat(Informer)" def testDog() {} // test name: "testDog" def testDog(Informer) {} // test name: "testDog(Informer)" def test() {} // test name: "test" def test(Informer) {} // test name: "test(Informer)"
This trait's implementation of this method returns an immutable Set
of all such names, excluding the name
testNames
. The iterator obtained by invoking elements
on this
returned Set
will produce the test names in their natural order, as determined by String
's
compareTo
method.
This trait's implementation of runTests
invokes this method
and calls runTest
for each test name in the order they appear in the returned Set
's iterator.
Although this trait's implementation of this method returns a Set
whose iterator produces String
test names in a well-defined order, the contract of this method does not required a defined order. Subclasses are free to
override this method and return test names in an undefined order, or in a defined order that's different from String
's
natural order.
Subclasses may override this method to produce test names in a custom manner. One potential reason to override testNames
is
to run tests in a different order, for example, to ensure that tests that depend on other tests are run after those other tests.
Another potential reason to override is allow tests to be defined in a different manner, such as methods annotated @Test
annotations
(as is done in JUnitSuite
and TestNGSuite
) or test functions registered during construction (as is
done in FunSuite
and Spec
).
This method should set up the fixture needed by the tests of the
current suite, invoke the test function, and if needed, perform any clean
up needed after the test completes. Because the NoArgTest
function
passed to this method takes no parameters, preparing the fixture will require
side effects, such as reassigning instance var
s in this Suite
or initializing
a globally accessible external database. If you want to avoid reassigning instance var
s
you can use FixtureSuite.
This trait's implementation of runTest
invokes this method for each test, passing
in a NoArgTest
whose apply
method will execute the code of the test.
This trait's implementation of this method simply invokes the passed NoArgTest
function.
test -
the no-arg test function to run with a fixtureprotected
def
runTest(testName : java.lang.String, reporter : Reporter, stopper : Stopper, configMap : scala.collection.immutable.Map[java.lang.String, Any], tracker : Tracker) : Unit
This trait's implementation uses Java reflection to invoke on this object the test method identified by the passed testName
.
Implementations of this method are responsible for ensuring a TestStarting
event
is fired to the Reporter
before executing any test, and either TestSucceeded
,
TestFailed
, or TestPending
after executing any nested
Suite
. (If a test is marked with the org.scalatest.Ignore
tag, the
runTests
method is responsible for ensuring a TestIgnored
event is fired and that
this runTest
method is not invoked for that ignored test.)
testName -
the name of one test to run.reporter -
the Reporter
to which results will be reportedstopper -
the Stopper
that will be consulted to determine whether to stop execution early.configMap -
a Map
of key-value pairs that can be used by the executing Suite
of tests.tracker -
a Tracker
tracking Ordinal
s being fired by the current thread.NullPointerException -
if any of testName
, reporter
, stopper
, configMap
or tracker
is null
.IllegalArgumentException -
if testName
is defined, but no test with the specified test name exists in this Suite
protected
def
runTests(testName : scala.Option[java.lang.String], reporter : Reporter, stopper : Stopper, filter : Filter, configMap : scala.collection.immutable.Map[java.lang.String, Any], distributor : scala.Option[Distributor], tracker : Tracker) : Unit
Suite
's tests.
This method takes a testName
parameter that optionally specifies a test to invoke.
If testName
is defined, this trait's implementation of this method
invokes runTest
on this object, passing in:
testName
- the String
value of the testName
Option
passed
to this methodreporter
- the Reporter
passed to this method, or one that wraps and delegates to itstopper
- the Stopper
passed to this method, or one that wraps and delegates to itconfigMap
- the configMap
Map
passed to this method, or one that wraps and delegates to it
This method takes a Filter
, which encapsulates an optional Set
of tag names that should be included
(tagsToInclude
) and a Set
that should be excluded (tagsToExclude
), when deciding which
of this Suite
's tests to run.
If tagsToInclude
is None
, all tests will be run
except those those belonging to tags listed in the tagsToExclude
Set
. If tagsToInclude
is defined, only tests
belonging to tags mentioned in the tagsToInclude
Set
, and not mentioned in the tagsToExclude
Set
will be run. However, if testName
is defined, tagsToInclude
and tagsToExclude
are essentially ignored.
Only if testName
is None
will tagsToInclude
and tagsToExclude
be consulted to
determine which of the tests named in the testNames
Set
should be run. This trait's implementation
behaves this way, and it is part of the general contract of this method, so all overridden forms of this method should behave
this way as well. For more information on test tags, see the main documentation for this trait and for class Filter
.
Note that this means that even if a test is marked as ignored, for example a test method in a Suite
annotated with
org.scalatest.Ignore
, if that test name is passed as testName
to runTest
, it will be invoked
despite the Ignore
annotation.
If testName
is None
, this trait's implementation of this method
invokes testNames
on this Suite
to get a Set
of names of tests to potentially run.
(A testNames
value of None
essentially acts as a wildcard that means all tests in
this Suite
that are selected by tagsToInclude
and tagsToExclude
should be run.)
For each test in the testName
Set
, in the order
they appear in the iterator obtained by invoking the elements
method on the Set
, this trait's implementation
of this method checks whether the test should be run based on the Filter
.
If so, this implementation invokes runTest
, passing in:
testName
- the String
name of the test to run (which will be one of the names in the testNames
Set
)reporter
- the Reporter
passed to this method, or one that wraps and delegates to itstopper
- the Stopper
passed to this method, or one that wraps and delegates to itconfigMap
- the configMap
passed to this method, or one that wraps and delegates to it
If a test is marked with the org.scalatest.Ignore
tag, implementations
of this method are responsible for ensuring a TestIgnored
event is fired for that test
and that runTest
is not called for that test.
testName -
an optional name of one test to run. If None
, all relevant tests should be run. I.e., None
acts like a wildcard that means run all relevant tests in this Suite
.reporter -
the Reporter
to which results will be reportedstopper -
the Stopper
that will be consulted to determine whether to stop execution early.filter -
a Filter
with which to filter tests based on their tagsconfigMap -
a Map
of key-value pairs that can be used by the executing Suite
of tests.distributor -
an optional Distributor
, into which to put nested Suite
s to be run by another entity, such as concurrently by a pool of threads. If None
, nested Suite
s will be run sequentially.tracker -
a Tracker
tracking Ordinal
s being fired by the current thread.NullPointerException -
if any of the passed parameters is null
.IllegalArgumentException -
if testName
is defined, but no test with the specified test name exists in this Suite
def
run(testName : scala.Option[java.lang.String], reporter : Reporter, stopper : Stopper, filter : Filter, configMap : scala.collection.immutable.Map[java.lang.String, Any], distributor : scala.Option[Distributor], tracker : Tracker) : Unit
If testName
is None
, this trait's implementation of this method
calls these two methods on this object in this order:
runNestedSuites(report, stopper, tagsToInclude, tagsToExclude, configMap, distributor)
runTests(testName, report, stopper, tagsToInclude, tagsToExclude, configMap)
If testName
is defined, then this trait's implementation of this method
calls runTests
, but does not call runNestedSuites
. This behavior
is part of the contract of this method. Subclasses that override run
must take
care not to call runNestedSuites
if testName
is defined. (The
OneInstancePerTest
trait depends on this behavior, for example.)
Subclasses and subtraits that override this run
method can implement them without
invoking either the runTests
or runNestedSuites
methods, which
are invoked by this trait's implementation of this method. It is recommended, but not required,
that subclasses and subtraits that override run
in a way that does not
invoke runNestedSuites
also override runNestedSuites
and make it
final. Similarly it is recommended, but not required,
that subclasses and subtraits that override run
in a way that does not
invoke runTests
also override runTests
(and runTest
,
which this trait's implementation of runTests
calls) and make it
final. The implementation of these final methods can either invoke the superclass implementation
of the method, or throw an UnsupportedOperationException
if appropriate. The
reason for this recommendation is that ScalaTest includes several traits that override
these methods to allow behavior to be mixed into a Suite
. For example, trait
BeforeAndAfterEach
overrides runTests
s. In a Suite
subclass that no longer invokes runTests
from run
, the
BeforeAndAfterEach
trait is not applicable. Mixing it in would have no effect.
By making runTests
final in such a Suite
subtrait, you make
the attempt to mix BeforeAndAfterEach
into a subclass of your subtrait
a compiler error. (It would fail to compile with a complaint that BeforeAndAfterEach
is trying to override runTests
, which is a final method in your trait.)
testName -
an optional name of one test to run. If None
, all relevant tests should be run. I.e., None
acts like a wildcard that means run all relevant tests in this Suite
.reporter -
the Reporter
to which results will be reportedstopper -
the Stopper
that will be consulted to determine whether to stop execution early.filter -
a Filter
with which to filter tests based on their tagsconfigMap -
a Map
of key-value pairs that can be used by the executing Suite
of tests.distributor -
an optional Distributor
, into which to put nested Suite
s to be run by another entity, such as concurrently by a pool of threads. If None
, nested Suite
s will be run sequentially.tracker -
a Tracker
tracking Ordinal
s being fired by the current thread.NullPointerException -
if any passed parameter is null
.IllegalArgumentException -
if testName
is defined, but no test with the specified test name exists in this Suite
protected
def
runNestedSuites(reporter : Reporter, stopper : Stopper, filter : Filter, configMap : scala.collection.immutable.Map[java.lang.String, Any], distributor : scala.Option[Distributor], tracker : Tracker) : Unit
Suite
's nested Suite
s.
If the passed distributor
is None
, this trait's
implementation of this method invokes run
on each
nested Suite
in the List
obtained by invoking nestedSuites
.
If a nested Suite
's run
method completes abruptly with an exception, this trait's implementation of this
method reports that the Suite
aborted and attempts to run the
next nested Suite
.
If the passed distributor
is defined, this trait's implementation
puts each nested Suite
into the Distributor
contained in the Some
, in the order in which the
Suite
s appear in the List
returned by nestedSuites
, passing
in a new Tracker
obtained by invoking nextTracker
on the Tracker
passed to this method.
Implementations of this method are responsible for ensuring SuiteStarting
events
are fired to the Reporter
before executing any nested Suite
, and either SuiteCompleted
or SuiteAborted
after executing any nested Suite
.
reporter -
the Reporter
to which results will be reportedstopper -
the Stopper
that will be consulted to determine whether to stop execution early.filter -
a Filter
with which to filter tests based on their tagsconfigMap -
a Map
of key-value pairs that can be used by the executing Suite
of tests.distributor -
an optional Distributor
, into which to put nested Suite
s to be run by another entity, such as concurrently by a pool of threads. If None
, nested Suite
s will be run sequentially.tracker -
a Tracker
tracking Ordinal
s being fired by the current thread.NullPointerException -
if any passed parameter is null
.
def
suiteName : java.lang.String
Suite
.
This trait's
implementation of this method returns the simple name of this object's class. This
trait's implementation of runNestedSuites
calls this method to obtain a
name for Report
s to pass to the suiteStarting
, suiteCompleted
,
and suiteAborted
methods of the Reporter
.
Suite
object's suite name.
def
pending : PendingNothing
TestPendingException
to indicate a test is pending.
A pending test is one that has been given a name but is not yet implemented. The purpose of pending tests is to facilitate a style of testing in which documentation of behavior is sketched out before tests are written to verify that behavior (and often, the before the behavior of the system being tested is itself implemented). Such sketches form a kind of specification of what tests and functionality to implement later.
To support this style of testing, a test can be given a name that specifies one
bit of behavior required by the system being tested. The test can also include some code that
sends more information about the behavior to the reporter when the tests run. At the end of the test,
it can call method pending
, which will cause it to complete abruptly with TestPendingException
.
Because tests in ScalaTest can be designated as pending with TestPendingException
, both the test name and any information
sent to the reporter when running the test can appear in the report of a test run. (In other words,
the code of a pending test is executed just like any other test.) However, because the test completes abruptly
with TestPendingException
, the test will be reported as pending, to indicate
the actual test, and possibly the functionality it is intended to test, has not yet been implemented.
Note: This method always completes abruptly with a TestPendingException
. Thus it always has a side
effect. Methods with side effects are usually invoked with parentheses, as in pending()
. This
method is defined as a parameterless method, in flagrant contradiction to recommended Scala style, because it
forms a kind of DSL for pending tests. It enables tests in suites such as FunSuite
or Spec
to be denoted by placing "(pending)
" after the test name, as in:
test("that style rules are not laws") (pending)
Readers of the code see "pending" in parentheses, which looks like a little note attached to the test name to indicate
it is pending. Whereas "(pending())
looks more like a method call, "(pending)
" lets readers
stay at a higher level, forgetting how it is implemented and just focusing on the intent of the programmer who wrote the code.
TestPendingException
, else
throw TestFailedException
.
This method can be used to temporarily change a failing test into a pending test in such a way that it will
automatically turn back into a failing test once the problem originally causing the test to fail has been fixed.
At that point, you need only remove the pendingUntilFixed
call. In other words, a
pendingUntilFixed
surrounding a block of code that isn't broken is treated as a test failure.
The motivation for this behavior is to encourage people to remove pendingUntilFixed
calls when
there are no longer needed.
This method facilitates a style of testing in which tests are written before the code they test. Sometimes you may
encounter a test failure that requires more functionality than you want to tackle without writing more tests. In this
case you can mark the bit of test code causing the failure with pendingUntilFixed
. You can then write more
tests and functionality that eventually will get your production code to a point where the original test won't fail anymore.
At this point the code block marked with pendingUntilFixed
will no longer throw an exception (because the
problem has been fixed). This will in turn cause pendingUntilFixed
to throw TestFailedException
with a detail message explaining you need to go back and remove the pendingUntilFixed
call as the problem orginally
causing your test code to fail has been fixed.
f -
a block of code, which if it completes abruptly, should trigger a TestPendingException
TestPendingException -
if the passed block of code completes abruptly with an Exception
or AssertionError
Suite
's run
method is invoked.
This trait's implementation of this method returns the sum of:
testNames
List
, minus the number of tests marked as ignored
expectedTestCount
on every nested Suite
contained in
nestedSuites
filter -
a Filter
with which to filter tests to count based on their tags
ScalaTest 1.0
|
|