Class used via an implicit conversion to enable any two objects to be compared with===
in assertions in tests.
A test function taking no arguments, which also provides a test name and config map.
o != arg0
is the same as !(o == (arg0))
.
o != arg0
is the same as !(o == (arg0))
.
the object to compare against this object for dis-equality.
false
if the receiver object is equivalent to the argument; true
otherwise.
o == arg0
is the same as if (o eq null) arg0 eq null else o.equals(arg0)
.
o == arg0
is the same as if (o eq null) arg0 eq null else o.equals(arg0)
.
the object to compare against this object for equality.
true
if the receiver object is equivalent to the argument; false
otherwise.
o == arg0
is the same as o.equals(arg0)
.
o == arg0
is the same as o.equals(arg0)
.
the object to compare against this object for equality.
true
if the receiver object is equivalent to the argument; false
otherwise.
This method is used to cast the receiver object to be of type T0
.
This method is used to cast the receiver object to be of type T0
.
Note that the success of a cast at runtime is modulo Scala's erasure semantics. Therefore the expression1.asInstanceOf[String]
will throw a ClassCastException
at runtime, while the expressionList(1).asInstanceOf[List[String]]
will not. In the latter example, because the type argument is erased as
part of compilation it is not possible to check whether the contents of the list are of the requested typed.
the receiver object.
Assert that an Option[String]
is None
.
Assert that an Option[String]
is None
.
If the condition is None
, this method returns normally.
Else, it throws TestFailedException
with the String
value of the Some
included in the TestFailedException
's
detail message.
This form of assert
is usually called in conjunction with an
implicit conversion to Equalizer
, using a ===
comparison, as in:
assert(a === b)
For more information on how this mechanism works, see the documentation forEqualizer
.
the Option[String]
to assert
Assert that an Option[String]
is None
.
Assert that an Option[String]
is None
.
If the condition is None
, this method returns normally.
Else, it throws TestFailedException
with the String
value of the Some
, as well as theString
obtained by invoking toString
on the
specified message
,
included in the TestFailedException
's detail message.
This form of assert
is usually called in conjunction with an
implicit conversion to Equalizer
, using a ===
comparison, as in:
assert(a === b, "extra info reported if assertion fails")
For more information on how this mechanism works, see the documentation forEqualizer
.
the Option[String]
to assert
An objects whose toString
method returns a message to include in a failure report.
Assert that a boolean condition, described in String
message
, is true.
Assert that a boolean condition, described in String
message
, is true.
If the condition is true
, this method returns normally.
Else, it throws TestFailedException
with theString
obtained by invoking toString
on the
specified message
as the exception's detail message.
the boolean condition to assert
An objects whose toString
method returns a message to include in a failure report.
Assert that a boolean condition is true.
Assert that a boolean condition is true.
If the condition is true
, this method returns normally.
Else, it throws TestFailedException
.
the boolean condition to assert
This method creates and returns a copy of the receiver object.
This method creates and returns a copy of the receiver object.
The default implementation of the clone
method is platform dependent.
a copy of the receiver object.
Implicit conversion from Any
to Equalizer
, used to enable
assertions with ===
comparisons.
Implicit conversion from Any
to Equalizer
, used to enable
assertions with ===
comparisons.
For more information on this mechanism, see the documentation for </code>Equalizer</code>.
Because trait Suite
mixes in Assertions
, this implicit conversion will always be
available by default in ScalaTest Suite
s. This is the only implicit conversion that is in scope by default in every
ScalaTest Suite
. Other implicit conversions offered by ScalaTest, such as those that support the matchers DSL
or invokePrivate
, must be explicitly invited into your test code, either by mixing in a trait or importing the
members of its companion object. The reason ScalaTest requires you to invite in implicit conversions (with the exception of the
implicit conversion for ===
operator) is because if one of ScalaTest's implicit conversions clashes with an
implicit conversion used in the code you are trying to test, your program won't compile. Thus there is a chance that if you
are ever trying to use a library or test some code that also offers an implicit conversion involving a ===
operator,
you could run into the problem of a compiler error due to an ambiguous implicit conversion. If that happens, you can turn off
the implicit conversion offered by this convertToEqualizer
method simply by overriding the method in yourSuite
subclass, but not marking it as implicit:
// In your Suite subclass override def convertToEqualizer(left: Any) = new Equalizer(left)
the object whose type to convert to Equalizer
.
This method is used to test whether the argument (arg0
) is a reference to the
receiver object (this
).
This method is used to test whether the argument (arg0
) is a reference to the
receiver object (this
).
The eq
method implements an [http://en.wikipedia.org/wiki/Equivalence_relation equivalence relation] on
non-null instances of AnyRef
:
* It is reflexive: for any non-null instance x
of type AnyRef
, x.eq(x)
returns true
.
* It is symmetric: for any non-null instances x
and y
of type AnyRef
, x.eq(y)
returns true
if and
only if y.eq(x)
returns true
.
* It is transitive: for any non-null instances x
, y
, and z
of type AnyRef
if x.eq(y)
returns true
and y.eq(z)
returns true
, then x.eq(z)
returns true
.
Additionally, the eq
method has three other properties.
* It is consistent: for any non-null instances x
and y
of type AnyRef
, multiple invocations of
x.eq(y)
consistently returns true
or consistently returns false
.
* For any non-null instance x
of type AnyRef
, x.eq(null)
and null.eq(x)
returns false
.
* null.eq(null)
returns true
.
When overriding the equals
or hashCode
methods, it is important to ensure that their behavior is
consistent with reference equality. Therefore, if two objects are references to each other (o1 eq o2
), they
should be equal to each other (o1 == o2
) and they should hash to the same value (o1.hashCode == o2.hashCode
).
the object to compare against this object for reference equality.
true
if the argument is a reference to the receiver object; false
otherwise.
This method is used to compare the receiver object (this
) with the argument object (arg0
) for equivalence.
This method is used to compare the receiver object (this
) with the argument object (arg0
) for equivalence.
The default implementations of this method is an [http://en.wikipedia.org/wiki/Equivalence_relation equivalence
relation]:
* It is reflexive: for any instance x
of type Any
, x.equals(x)
should return true
.
* It is symmetric: for any instances x
and y
of type Any
, x.equals(y)
should return true
if and
only if y.equals(x)
returns true
.
* It is transitive: for any instances x
, y
, and z
of type AnyRef
if x.equals(y)
returns true
and
y.equals(z)
returns true
, then x.equals(z)
should return true
.
If you override this method, you should verify that your implementation remains an equivalence relation.
Additionally, when overriding this method it is often necessary to override hashCode
to ensure that objects
that are "equal" (o1.equals(o2)
returns true
) hash to the same
scala.Int
(o1.hashCode.equals(o2.hashCode)
).
the object to compare against this object for equality.
true
if the receiver object is equivalent to the argument; false
otherwise.
Executes one or more tests in this Suite
, printing results to the standard output.
Executes one or more tests in this Suite
, printing results to the standard output.
This method invokes run
on itself, passing in values that can be configured via the parameters to this
method, all of which have default values. This behavior is convenient when working with ScalaTest in the Scala interpreter.
Here's a summary of this method's parameters and how you can use them:
The testName
parameter
If you leave testName
at its default value (of null
), this method will pass None
to
the testName
parameter of run
, and as a result all the tests in this suite will be executed. If you
specify a testName
, this method will pass Some(testName)
to run
, and only that test
will be run. Thus to run all tests in a suite from the Scala interpreter, you can write:
scala> (new MySuite).execute()
To run just the test named "my favorite test"
in a suite from the Scala interpreter, you would write:
scala> (new MySuite).execute("my favorite test")
Or:
scala> (new MySuite).execute(testName = "my favorite test")
The configMap
parameter
If you provide a value for the configMap
parameter, this method will pass it to run
. If not, the default value
of an empty Map
will be passed. For more information on how to use a config map to configure your test suites, see
the config map section in the main documentation for this trait. Here's an example in which you configure
a run with the name of an input file:
scala> (new MySuite).execute(configMap = Map("inputFileName" -> "in.txt")
The color
parameter
If you leave the color
parameter unspecified, this method will configure the reporter it passes to run
to print
to the standard output in color (via ansi escape characters). If you don't want color output, specify false for color
, like this:
scala> (new MySuite).execute(color = false)
The durations
parameter
If you leave the durations
parameter unspecified, this method will configure the reporter it passes to run
tonot print durations for tests and suites to the standard output. If you want durations printed, specify true for durations
,
like this:
scala> (new MySuite).execute(durations = true)
The shortstacks
and fullstacks
parameters
If you leave both the shortstacks
and fullstacks
parameters unspecified, this method will configure the reporter
it passes to run
to not print stack traces for failed tests if it has a stack depth that identifies the offending
line of test code. If you prefer a short stack trace (10 to 15 stack frames) to be printed with any test failure, specify true forshortstacks
:
scala> (new MySuite).execute(shortstacks = true)
For full stack traces, set fullstacks
to true:
scala> (new MySuite).execute(fullstacks = true)
If you specify true for both shortstacks
and fullstacks
, you'll get full stack traces.
The stats
parameter
If you leave the stats
parameter unspecified, this method will not fire RunStarting
and either RunCompleted
or RunAborted
events to the reporter it passes to run
.
If you specify true for stats
, this method will fire the run events to the reporter, and the reporter will print the
expected test count before the run, and various statistics after, including the number of suites completed and number of tests that
succeeded, failed, were ignored or marked pending. Here's how you get the stats:
scala> (new MySuite).execute(stats = true)
To summarize, this method will pass to run
:
testName
- None
if this method's testName
parameter is left at its default value of null
, else Some(testName)
.reporter
- a reporter that prints to the standard outputstopper
- a Stopper
whose apply
method always returns false
filter
- a Filter
constructed with None
for tagsToInclude
and Set()
for tagsToExclude
configMap
- the configMap
passed to this methoddistributor
- None
tracker
- a new Tracker
Note: In ScalaTest, the terms "execute" and "run" basically mean the same thing and
can be used interchangably. The reason this method isn't named run
is that it takes advantage of
default arguments, and you can't mix overloaded methods and default arguments in Scala. (If named run
,
this method would have the same name but different arguments than the main run
method that
takes seven arguments. Thus it would overload and couldn't be used with default argument values.)
Design note: This method has two "features" that may seem unidiomatic. First, the default value of testName
is null
.
Normally in Scala the type of testName
would be Option[String]
and the default value would
be None
, as it is in this trait's run
method. The null
value is used here for two reasons. First, in
ScalaTest 1.5, execute
was changed from four overloaded methods to one method with default values, taking advantage of
the default and named parameters feature introduced in Scala 2.8.
To not break existing source code, testName
needed to have type String
, as it did in two of the overloadedexecute
methods prior to 1.5. The other reason is that execute
has always been designed to be called primarily
from an interpeter environment, such as the Scala REPL (Read-Evaluate-Print-Loop). In an interpreter environment, minimizing keystrokes is king.
A String
type with a null
default value lets users type suite.execute("my test name")
rather thansuite.execute(Some("my test name"))
, saving several keystrokes.
The second non-idiomatic feature is that shortstacks
and fullstacks
are all lower case rather than
camel case. This is done to be consistent with the Shell
, which also uses those forms. The reason
lower case is used in the Shell
is to save keystrokes in an interpreter environment. Most Unix commands, for
example, are all lower case, making them easier and quicker to type. In the ScalaTestShell
, methods like shortstacks
, fullstacks
, and nostats
, etc., are
designed to be all lower case so they feel more like shell commands than methods.
the name of one test to run.
a Map
of key-value pairs that can be used by the executing Suite
of tests.
a boolean that configures whether output is printed in color
a boolean that configures whether test and suite durations are printed to the standard output
a boolean that configures whether short stack traces should be printed for test failures
a boolean that configures whether full stack traces should be printed for test failures
a boolean that configures whether test and suite statistics are printed to the standard output
Expect that the value passed as expected
equals the value passed as actual
.
Expect that the value passed as expected
equals the value passed as actual
.
If the actual
value equals the expected
value
(as determined by ==
), expect
returns
normally. Else, expect
throws anTestFailedException
whose detail message includes the expected and actual values.
the expected value
the actual value, which should equal the passed expected
value
Expect that the value passed as expected
equals the value passed as actual
.
Expect that the value passed as expected
equals the value passed as actual
.
If the actual
equals the expected
(as determined by ==
), expect
returns
normally. Else, if actual
is not equal to expected
, expect
throws anTestFailedException
whose detail message includes the expected and actual values, as well as the String
obtained by invoking toString
on the passed message
.
the expected value
An object whose toString
method returns a message to include in a failure report.
the actual value, which should equal the passed expected
value
The total number of tests that are expected to run when this Suite
's run
method is invoked.
The total number of tests that are expected to run when this Suite
's run
method is invoked.
This trait's implementation of this method returns the sum of:
testNames
List
, minus the number of tests marked as ignoredexpectedTestCount
on every nested Suite
contained in
nestedSuites
a Filter
with which to filter tests to count based on their tags
Throws TestFailedException
, with the passedThrowable
cause, to indicate a test failed.
Throws TestFailedException
, with the passedThrowable
cause, to indicate a test failed.
The getMessage
method of the thrown TestFailedException
will return cause.toString()
.
a Throwable
that indicates the cause of the failure.
Throws TestFailedException
, with the passedString
message
as the exception's detail
message and Throwable
cause, to indicate a test failed.
Throws TestFailedException
, with the passedString
message
as the exception's detail
message and Throwable
cause, to indicate a test failed.
A message describing the failure.
A Throwable
that indicates the cause of the failure.
Throws TestFailedException
, with the passedString
message
as the exception's detail
message, to indicate a test failed.
Throws TestFailedException
, with the passedString
message
as the exception's detail
message, to indicate a test failed.
A message describing the failure.
Throws TestFailedException
to indicate a test failed.
Throws TestFailedException
to indicate a test failed.
This method is called by the garbage collector on the receiver object when garbage collection determines that there are no more references to the object.
This method is called by the garbage collector on the receiver object when garbage collection determines that there are no more references to the object.
The details of when and if the finalize
method are invoked, as well as the interaction between finalize
and non-local returns and exceptions, are all platform dependent.
Returns a representation that corresponds to the dynamic class of the receiver object.
Returns a representation that corresponds to the dynamic class of the receiver object.
The nature of the representation is platform dependent.
a representation that corresponds to the dynamic class of the receiver object.
Returns a hash code value for the object.
Returns a hash code value for the object.
The default hashing algorithm is platform dependent.
Note that it is allowed for two objects to have identical hash codes (o1.hashCode.equals(o2.hashCode)
) yet
not be equal (o1.equals(o2)
returns false
). A degenerate implementation could always return 0
.
However, it is required that if two objects are equal (o1.equals(o2)
returns true
) that they have
identical hash codes (o1.hashCode.equals(o2.hashCode)
). Therefore, when overriding this method, be sure
to verify that the behavior is consistent with the equals
method.
the hash code value for the object.
Intercept and return an exception that's expected to be thrown by the passed function value.
Intercept and return an exception that's expected to
be thrown by the passed function value. The thrown exception must be an instance of the
type specified by the type parameter of this method. This method invokes the passed
function. If the function throws an exception that's an instance of the specified type,
this method returns that exception. Else, whether the passed function returns normally
or completes abruptly with a different exception, this method throws TestFailedException
.
Note that the type specified as this method's type parameter may represent any subtype ofAnyRef
, not just Throwable
or one of its subclasses. In
Scala, exceptions can be caught based on traits they implement, so it may at times make sense
to specify a trait that the intercepted exception's class must mix in. If a class instance is
passed for a type that could not possibly be used to catch an exception (such as String
,
for example), this method will complete abruptly with a TestFailedException
.
the function value that should throw the expected exception
an implicit Manifest
representing the type of the specified
type parameter.
the intercepted exception, if it is of the expected type
This method is used to test whether the dynamic type of the receiver object is T0
.
This method is used to test whether the dynamic type of the receiver object is T0
.
Note that the test result of the test is modulo Scala's erasure semantics. Therefore the expression1.isInstanceOf[String]
will return false
, while the expression List(1).isInstanceOf[List[String]]
will
return true
. In the latter example, because the type argument is erased as part of compilation it is not
possible to check whether the contents of the list are of the requested typed.
true
if the receiver object is an instance of erasure of type T0
; false
otherwise.
o.ne(arg0)
is the same as !(o.eq(arg0))
.
o.ne(arg0)
is the same as !(o.eq(arg0))
.
the object to compare against this object for reference dis-equality.
false
if the argument is not a reference to the receiver object; true
otherwise.
A List
of this Suite
object's nested Suite
s. If this Suite
contains no nested Suite
s,
this method returns an empty List
. This trait's implementation of this method returns an empty List
.
A List
of this Suite
object's nested Suite
s. If this Suite
contains no nested Suite
s,
this method returns an empty List
. This trait's implementation of this method returns an empty List
.
Wakes up a single thread that is waiting on the receiver object's monitor.
Wakes up a single thread that is waiting on the receiver object's monitor.
Wakes up all threads that are waiting on the receiver object's monitor.
Wakes up all threads that are waiting on the receiver object's monitor.
Throws TestPendingException
to indicate a test is pending.
Throws TestPendingException
to indicate a test is pending.
A pending test is one that has been given a name but is not yet implemented. The purpose of pending tests is to facilitate a style of testing in which documentation of behavior is sketched out before tests are written to verify that behavior (and often, the before the behavior of the system being tested is itself implemented). Such sketches form a kind of specification of what tests and functionality to implement later.
To support this style of testing, a test can be given a name that specifies one
bit of behavior required by the system being tested. The test can also include some code that
sends more information about the behavior to the reporter when the tests run. At the end of the test,
it can call method pending
, which will cause it to complete abruptly with TestPendingException
.
Because tests in ScalaTest can be designated as pending with TestPendingException
, both the test name and any information
sent to the reporter when running the test can appear in the report of a test run. (In other words,
the code of a pending test is executed just like any other test.) However, because the test completes abruptly
with TestPendingException
, the test will be reported as pending, to indicate
the actual test, and possibly the functionality it is intended to test, has not yet been implemented.
Note: This method always completes abruptly with a TestPendingException
. Thus it always has a side
effect. Methods with side effects are usually invoked with parentheses, as in pending()
. This
method is defined as a parameterless method, in flagrant contradiction to recommended Scala style, because it
forms a kind of DSL for pending tests. It enables tests in suites such as FunSuite
or Spec
to be denoted by placing "(pending)
" after the test name, as in:
test("that style rules are not laws") (pending)
Readers of the code see "pending" in parentheses, which looks like a little note attached to the test name to indicate
it is pending. Whereas "(pending())
looks more like a method call, "(pending)
" lets readers
stay at a higher level, forgetting how it is implemented and just focusing on the intent of the programmer who wrote the code.
Execute the passed block of code, and if it completes abruptly, throw TestPendingException
, else
throw TestFailedException
.
Execute the passed block of code, and if it completes abruptly, throw TestPendingException
, else
throw TestFailedException
.
This method can be used to temporarily change a failing test into a pending test in such a way that it will
automatically turn back into a failing test once the problem originally causing the test to fail has been fixed.
At that point, you need only remove the pendingUntilFixed
call. In other words, apendingUntilFixed
surrounding a block of code that isn't broken is treated as a test failure.
The motivation for this behavior is to encourage people to remove pendingUntilFixed
calls when
there are no longer needed.
This method facilitates a style of testing in which tests are written before the code they test. Sometimes you may
encounter a test failure that requires more functionality than you want to tackle without writing more tests. In this
case you can mark the bit of test code causing the failure with pendingUntilFixed
. You can then write more
tests and functionality that eventually will get your production code to a point where the original test won't fail anymore.
At this point the code block marked with pendingUntilFixed
will no longer throw an exception (because the
problem has been fixed). This will in turn cause pendingUntilFixed
to throw TestFailedException
with a detail message explaining you need to go back and remove the pendingUntilFixed
call as the problem orginally
causing your test code to fail has been fixed.
a block of code, which if it completes abruptly, should trigger a TestPendingException
Runs this suite of tests.
Runs this suite of tests.
If testName
is None
, this trait's implementation of this method
calls these two methods on this object in this order:
runNestedSuites(report, stopper, tagsToInclude, tagsToExclude, configMap, distributor)
runTests(testName, report, stopper, tagsToInclude, tagsToExclude, configMap)
If testName
is defined, then this trait's implementation of this method
calls runTests
, but does not call runNestedSuites
. This behavior
is part of the contract of this method. Subclasses that override run
must take
care not to call runNestedSuites
if testName
is defined. (TheOneInstancePerTest
trait depends on this behavior, for example.)
Subclasses and subtraits that override this run
method can implement them without
invoking either the runTests
or runNestedSuites
methods, which
are invoked by this trait's implementation of this method. It is recommended, but not required,
that subclasses and subtraits that override run
in a way that does not
invoke runNestedSuites
also override runNestedSuites
and make it
final. Similarly it is recommended, but not required,
that subclasses and subtraits that override run
in a way that does not
invoke runTests
also override runTests
(and runTest
,
which this trait's implementation of runTests
calls) and make it
final. The implementation of these final methods can either invoke the superclass implementation
of the method, or throw an UnsupportedOperationException
if appropriate. The
reason for this recommendation is that ScalaTest includes several traits that override
these methods to allow behavior to be mixed into a Suite
. For example, traitBeforeAndAfterEach
overrides runTests
s. In a Suite
subclass that no longer invokes runTests
from run
, theBeforeAndAfterEach
trait is not applicable. Mixing it in would have no effect.
By making runTests
final in such a Suite
subtrait, you make
the attempt to mix BeforeAndAfterEach
into a subclass of your subtrait
a compiler error. (It would fail to compile with a complaint that BeforeAndAfterEach
is trying to override runTests
, which is a final method in your trait.)
an optional name of one test to run. If None
, all relevant tests should be run.
I.e., None
acts like a wildcard that means run all relevant tests in this Suite
.
the Reporter
to which results will be reported
the Stopper
that will be consulted to determine whether to stop execution early.
a Filter
with which to filter tests based on their tags
a Map
of key-value pairs that can be used by the executing Suite
of tests.
an optional Distributor
, into which to put nested Suite
s to be run
by another entity, such as concurrently by a pool of threads. If None
, nested Suite
s will be run sequentially.
a Tracker
tracking Ordinal
s being fired by the current thread.
Run zero to many of this Suite
's nested Suite
s.
Run zero to many of this Suite
's nested Suite
s.
If the passed distributor
is None
, this trait's
implementation of this method invokes run
on each
nested Suite
in the List
obtained by invoking nestedSuites
.
If a nested Suite
's run
method completes abruptly with an exception, this trait's implementation of this
method reports that the Suite
aborted and attempts to run the
next nested Suite
.
If the passed distributor
is defined, this trait's implementation
puts each nested Suite
into the Distributor
contained in the Some
, in the order in which theSuite
s appear in the List
returned by nestedSuites
, passing
in a new Tracker
obtained by invoking nextTracker
on the Tracker
passed to this method.
Implementations of this method are responsible for ensuring SuiteStarting
events
are fired to the Reporter
before executing any nested Suite
, and either SuiteCompleted
or SuiteAborted
after executing any nested Suite
.
the Reporter
to which results will be reported
the Stopper
that will be consulted to determine whether to stop execution early.
a Filter
with which to filter tests based on their tags
a Map
of key-value pairs that can be used by the executing Suite
of tests.
an optional Distributor
, into which to put nested Suite
s to be run
by another entity, such as concurrently by a pool of threads. If None
, nested Suite
s will be run sequentially.
a Tracker
tracking Ordinal
s being fired by the current thread.
Run a test.
Run a test.
This trait's implementation uses Java reflection to invoke on this object the test method identified by the passed testName
.
Implementations of this method are responsible for ensuring a TestStarting
event
is fired to the Reporter
before executing any test, and either TestSucceeded
,TestFailed
, or TestPending
after executing any nestedSuite
. (If a test is marked with the org.scalatest.Ignore
tag, therunTests
method is responsible for ensuring a TestIgnored
event is fired and that
this runTest
method is not invoked for that ignored test.)
the name of one test to run.
the Reporter
to which results will be reported
the Stopper
that will be consulted to determine whether to stop execution early.
a Map
of key-value pairs that can be used by the executing Suite
of tests.
a Tracker
tracking Ordinal
s being fired by the current thread.
Run zero to many of this Suite
's tests.
Run zero to many of this Suite
's tests.
This method takes a testName
parameter that optionally specifies a test to invoke.
If testName
is defined, this trait's implementation of this method
invokes runTest
on this object, passing in:
testName
- the String
value of the testName
Option
passed
to this methodreporter
- the Reporter
passed to this method, or one that wraps and delegates to itstopper
- the Stopper
passed to this method, or one that wraps and delegates to itconfigMap
- the configMap
Map
passed to this method, or one that wraps and delegates to itThis method takes a Filter
, which encapsulates an optional Set
of tag names that should be included
(tagsToInclude
) and a Set
that should be excluded (tagsToExclude
), when deciding which
of this Suite
's tests to run.
If tagsToInclude
is None
, all tests will be run
except those those belonging to tags listed in the tagsToExclude
Set
. If tagsToInclude
is defined, only tests
belonging to tags mentioned in the tagsToInclude
Set
, and not mentioned in the tagsToExclude
Setwill be run. However, if testName
is defined, tagsToInclude
and tagsToExclude
are essentially ignored.
Only if testName
is None
will tagsToInclude
and tagsToExclude
be consulted to
determine which of the tests named in the testNames
Set
should be run. This trait's implementation
behaves this way, and it is part of the general contract of this method, so all overridden forms of this method should behave
this way as well. For more information on test tags, see the main documentation for this trait and for class Filter
.
Note that this means that even if a test is marked as ignored, for example a test method in a Suite
annotated withorg.scalatest.Ignore
, if that test name is passed as testName
to runTest
, it will be invoked
despite the Ignore
annotation.
If testName
is None
, this trait's implementation of this method
invokes testNames
on this Suite
to get a Set
of names of tests to potentially run.
(A testNames
value of None
essentially acts as a wildcard that means all tests in
this Suite
that are selected by tagsToInclude
and tagsToExclude
should be run.)
For each test in the testName
Set
, in the order
they appear in the iterator obtained by invoking the elements
method on the Set
, this trait's implementation
of this method checks whether the test should be run based on the Filter
.
If so, this implementation invokes runTest
, passing in:
testName
- the String
name of the test to run (which will be one of the names in the testNames
Set
)reporter
- the Reporter
passed to this method, or one that wraps and delegates to itstopper
- the Stopper
passed to this method, or one that wraps and delegates to itconfigMap
- the configMap
passed to this method, or one that wraps and delegates to itIf a test is marked with the org.scalatest.Ignore
tag, implementations
of this method are responsible for ensuring a TestIgnored
event is fired for that test
and that runTest
is not called for that test.
an optional name of one test to run. If None
, all relevant tests should be run.
I.e., None
acts like a wildcard that means run all relevant tests in this Suite
.
the Reporter
to which results will be reported
the Stopper
that will be consulted to determine whether to stop execution early.
a Filter
with which to filter tests based on their tags
a Map
of key-value pairs that can be used by the executing Suite
of tests.
an optional Distributor
, into which to put nested Suite
s to be run
by another entity, such as concurrently by a pool of threads. If None
, nested Suite
s will be run sequentially.
a Tracker
tracking Ordinal
s being fired by the current thread.
A user-friendly suite name for this Suite
.
A user-friendly suite name for this Suite
.
This trait's
implementation of this method returns the simple name of this object's class. This
trait's implementation of runNestedSuites
calls this method to obtain a
name for Report
s to pass to the suiteStarting
, suiteCompleted
,
and suiteAborted
methods of the Reporter
.
this Suite
object's suite name.
A Map
whose keys are String
tag names with which tests in this Suite
are marked, and
whose values are the Set
of test names marked with each tag. If this Suite
contains no tags, this
method returns an empty Map
.
A Map
whose keys are String
tag names with which tests in this Suite
are marked, and
whose values are the Set
of test names marked with each tag. If this Suite
contains no tags, this
method returns an empty Map
.
This trait's implementation of this method uses Java reflection to discover any Java annotations attached to its test methods. The
fully qualified name of each unique annotation that extends TagAnnotation
is considered a tag. This trait's
implementation of this method, therefore, places one key/value pair into to theMap
for each unique tag annotation name discovered through reflection. The mapped value for each tag name key will contain
the test method name, as provided via the testNames
method.
Subclasses may override this method to define and/or discover tags in a custom manner, but overriding method implementations
should never return an empty Set
as a value. If a tag has no tests, its name should not appear as a key in the
returned Map
.
An Set
of test names. If this Suite
contains no tests, this method returns an empty Set
.
An Set
of test names. If this Suite
contains no tests, this method returns an empty Set
.
This trait's implementation of this method uses Java reflection to discover all public methods whose name starts with "test"
,
which take either nothing or a single Informer
as parameters. For each discovered test method, it assigns a test name
comprised of just the method name if the method takes no parameters, or the method name plus (Informer)
if the
method takes a Informer
. Here are a few method signatures and the names that this trait's implementation assigns them:
def testCat() {} // test name: "testCat" def testCat(Informer) {} // test name: "testCat(Informer)" def testDog() {} // test name: "testDog" def testDog(Informer) {} // test name: "testDog(Informer)" def test() {} // test name: "test" def test(Informer) {} // test name: "test(Informer)"
This trait's implementation of this method returns an immutable Set
of all such names, excluding the nametestNames
. The iterator obtained by invoking elements
on this
returned Set
will produce the test names in their natural order, as determined by String
'scompareTo
method.
This trait's implementation of runTests
invokes this method
and calls runTest
for each test name in the order they appear in the returned Set
's iterator.
Although this trait's implementation of this method returns a Set
whose iterator produces String
test names in a well-defined order, the contract of this method does not required a defined order. Subclasses are free to
override this method and return test names in an undefined order, or in a defined order that's different from String
's
natural order.
Subclasses may override this method to produce test names in a custom manner. One potential reason to override testNames
is
to run tests in a different order, for example, to ensure that tests that depend on other tests are run after those other tests.
Another potential reason to override is allow tests to be defined in a different manner, such as methods annotated @Test
annotations
(as is done in JUnitSuite
and TestNGSuite
) or test functions registered during construction (as is
done in FunSuite
and Spec
).
Returns a string representation of the object.
Returns a string representation of the object.
The default representation is platform dependent.
a string representation of the object.
Executes the block of code passed as the second parameter, and, if it
completes abruptly with a ModifiableMessage
exception,
prepends the "clue" string passed as the first parameter to the beginning of the detail message
of that thrown exception, then rethrows it.
Executes the block of code passed as the second parameter, and, if it
completes abruptly with a ModifiableMessage
exception,
prepends the "clue" string passed as the first parameter to the beginning of the detail message
of that thrown exception, then rethrows it. If clue does not end in a white space
character, one space will be added
between it and the existing detail message (unless the detail message is
not defined).
This method allows you to add more information about what went wrong that will be reported when a test fails. Here's an example:
withClue("(Employee's name was: " + employee.name + ")") { intercept[IllegalArgumentException] { employee.getTask(-1) } }
If an invocation of intercept
completed abruptly with an exception, the resulting message would be something like:
(Employee's name was Bob Jones) Expected IllegalArgumentException to be thrown, but no exception was thrown
Run the passed test function in the context of a fixture established by this method.
Run the passed test function in the context of a fixture established by this method.
This method should set up the fixture needed by the tests of the
current suite, invoke the test function, and if needed, perform any clean
up needed after the test completes. Because the NoArgTest
function
passed to this method takes no parameters, preparing the fixture will require
side effects, such as reassigning instance var
s in this Suite
or initializing
a globally accessible external database. If you want to avoid reassigning instance var
s
you can use FixtureSuite.
This trait's implementation of runTest
invokes this method for each test, passing
in a NoArgTest
whose apply
method will execute the code of the test.
This trait's implementation of this method simply invokes the passed NoArgTest
function.
the no-arg test function to run with a fixture
A suite of tests. A
Suite
instance encapsulates a conceptual suite (i.e., a collection) of tests.This trait provides an interface that allows suites of tests to be run. Its implementation enables a default way of writing and executing tests. Subtraits and subclasses can override
Suite
's methods to enable other ways of writing and executing tests. This trait's default approach allows tests to be defined as methods whose name starts with "test
." This approach is easy to understand, and a good way for Scala beginners to start writing tests. More advanced Scala programmers may prefer to mix together otherSuite
subtraits defined in ScalaTest, or create their own, to write tests in the way they feel makes them most productive. Here's a quick overview of some of the options to help you get started:For JUnit 3 users
If you are using JUnit 3 (version 3.8 or earlier releases) and you want to write JUnit 3 tests in Scala, look at
AssertionsForJUnit
,ShouldMatchersForJUnit
, andJUnit3Suite
.For JUnit 4 users
If you are using JUnit 4 and you want to write JUnit 4 tests in Scala, look at
JUnitSuite
, andJUnitRunner
. WithJUnitRunner
, you can use any of the traits described here and still run your tests with JUnit 4.For TestNG users
If you are using TestNG and you want to write TestNG tests in Scala, look at
TestNGSuite
.For high-level testing
If you want to write tests at a higher level than unit tests, such as integration tests, acceptance tests, or functional tests, check out
FeatureSpec
.For unit testing
If you prefer a behavior-driven development (BDD) style, in which tests are combined with text that specifies the behavior being tested, look at
Spec
,FlatSpec
, andWordSpec
. Otherwise, if you just want to write tests and don't want to combine testing with specifying, look atFunSuite
or read on to learn how to write tests using this base trait,Suite
.To use this trait's approach to writing tests, simply create classes that extend
Suite
and define test methods. Test methods have names of the formtestX
, whereX
is some unique, hopefully meaningful, string. A test method must be public and can have any result type, but the most common result type isUnit
. Here's an example:You can run a
Suite
by invoking on it one of four overloadedexecute
methods. These methods, which print test results to the standard output, are intended to serve as a convenient way to run tests from within the Scala interpreter. For example, to runMySuite
from within the Scala interpreter, you could write:And you would see:
Or, to run just the
testAddition
method, you could write:And you would see:
Two other
execute
methods that are intended to be run from the interpreter accept a "config" map of key-value pairs (see Config map, below). Each of theseexecute
methods invokes arun
method takes seven parameters. Thisrun
method, which actually executes the suite, will usually be invoked by a test runner, such asorg.scalatest.tools.Runner
or an IDE. See the documentation forRunner
for more detail.Assertions and
Inside test methods in a
Suite
, you can write assertions by invokingassert
and passing in aBoolean
expression, such as:If the passed expression is
true
,assert
will return normally. Iffalse
,assert
will complete abruptly with aTestFailedException
. This exception is usually not caught by the test method, which means the test method itself will complete abruptly by throwing theTestFailedException
. Any test method that completes abruptly with aTestFailedException
or anyException
is considered a failed test. A test method that returns normally is considered a successful test.If you pass a
Boolean
expression toassert
, a failed assertion will be reported, but without reporting the left and right values. You can alternatively encode these values in aString
passed as a second argument toassert
, as in:Using this form of
assert
, the failure report will include the left and right values, thereby helping you debug the problem. However, ScalaTest provides the===
operator to make this easier. (The===
operator is defined in traitAssertions
which traitSuite
extends.) You use it like this:Because you use
===
here instead of==
, the failure report will include the left and right values. For example, the detail message in the thrownTestFailedException
from theassert
shown previously will include, "2 did not equal 1". From this message you will know that the operand on the left had the value 2, and the operand on the right had the value 1.If you're familiar with JUnit, you would use
===
in a ScalaTestSuite
where you'd useassertEquals
in a JUnitTestCase
. The===
operator is made possible by an implicit conversion fromAny
toEqualizer
. If you're curious to understand the mechanics, see the documentation forEqualizer
and theconvertToEqualizer
method.Expected results
Although
===
provides a natural, readable extension to Scala'sassert
mechanism, as the operands become lengthy, the code becomes less readable. In addition, the===
comparison doesn't distinguish between actual and expected values. The operands are just calledleft
andright
, because if one were namedexpected
and the otheractual
, it would be difficult for people to remember which was which. To help with these limitations of assertions,Suite
includes a method calledexpect
that can be used as an alternative toassert
with===
. To useexpect
, you place the expected value in parentheses afterexpect
, followed by curly braces containing code that should result in the expected value. For example:In this case, the expected value is
2
, and the code being tested isa - b
. This expectation will fail, and the detail message in theTestFailedException
will read, "Expected 2, but got 3."Intercepted exceptions
Sometimes you need to test whether a method throws an expected exception under certain circumstances, such as when invalid arguments are passed to the method. You can do this in the JUnit style, like this:
If
charAt
throwsIndexOutOfBoundsException
as expected, control will transfer to the catch case, which does nothing. If, however,charAt
fails to throw an exception, the next statement,fail()
, will be executed. Thefail
method always completes abruptly with aTestFailedException
, thereby signaling a failed test.To make this common use case easier to express and read, ScalaTest provides an
intercept
method. You use it like this:This code behaves much like the previous example. If
charAt
throws an instance ofIndexOutOfBoundsException
,intercept
will return that exception. But ifcharAt
completes normally, or throws a different exception,intercept
will complete abruptly with aTestFailedException
. Theintercept
method returns the caught exception so that you can inspect it further if you wish, for example, to ensure that data contained inside the exception has the expected values. Here's an example:Using other assertions
ScalaTest also supports another style of assertions via its matchers DSL. By mixing in trait
ShouldMatchers
, you can write suites that look like:If you prefer the word "
must
" to the word "should
," you can alternatively mix in traitMustMatchers
.If you are comfortable with assertion mechanisms from other test frameworks, chances are you can use them with ScalaTest. Any assertion mechanism that indicates a failure with an exception can be used as is with ScalaTest. For example, to use the
assertEquals
methods provided by JUnit or TestNG, simply import them and use them. (You will of course need to include the relevant JAR file for the framework whose assertions you want to use on either the classpath or runpath when you run your tests.) Here's an example in which JUnit's assertions are imported, then used within a ScalaTest suite:Nested suites
A
Suite
can refer to a collection of otherSuite
s, which are called nestedSuite
s. Those nestedSuite
s can in turn have their own nestedSuite
s, and so on. Large test suites can be organized, therefore, as a tree of nestedSuite
s. This trait'srun
method, in addition to invoking its test methods, invokesrun
on each of its nestedSuite
s.A
List
of aSuite
's nestedSuite
s can be obtained by invoking itsnestedSuites
method. If you wish to create aSuite
that serves as a container for nestedSuite
s, whether or not it has test methods of its own, simply overridenestedSuites
to return aList
of the nestedSuite
s. Because this is a common use case, ScalaTest provides a convenienceSuperSuite
class, which takes aList
of nestedSuite
s as a constructor parameter. Here's an example:If you now run
AlphabetSuite
, for example from the interpreter:You will see reports printed to the standard output that indicate nested suites—
ASuite
,BSuite
, andCSuite
—were run:Note that
Runner
can discoverSuite
s automatically, so you need not necessarily specifySuperSuite
s explicitly. See the documentation forRunner
for more information.Shared fixtures
A test fixture is objects or other artifacts (such as files, sockets, database connections, etc.) used by tests to do their work. If a fixture is used by only one test method, then the definitions of the fixture objects can be local to the method, such as the objects assigned to
sum
anddiff
in the previousMySuite
examples. If multiple methods need to share an immutable fixture, one approach is to assign them to instance variables. Here's a (very contrived) example, in which the object assigned toshared
is used by multiple test methods:In some cases, however, shared mutable fixture objects may be changed by test methods such that they need to be recreated or reinitialized before each test. Shared resources such as files or database connections may also need to be created and initialized before, and cleaned up after, each test. JUnit 3 offers methods
setUp
andtearDown
for this purpose. In ScalaTest, you can use theBeforeAndAfterEach
trait, which will be described later, to implement an approach similar to JUnit'ssetUp
andtearDown
, however, this approach usually involves reassigningvar
s between tests. Before going that route, you may wish to consider some approaches that avoidvar
s. One approach is to write one or more create-fixture methods that return a new instance of a needed object (or a tuple or case class holding new instances of multiple objects) each time it is called. You can then call a create-fixture method at the beginning of each test method that needs the fixture, storing the fixture object or objects in local variables. Here's an example:If different tests in the same
Suite
require different fixtures, you can create multiple create-fixture methods and call the method (or methods) needed by each test at the begining of the test. If every test method requires the same set of mutable fixture objects, one other approach you can take is make them simplyval
s and mix in traitOneInstancePerTest
. If you mix inOneInstancePerTest
, each test will be run in its own instance of theSuite
, similar to the way JUnit tests are executed.Although the create-fixture and
OneInstancePerTest
approaches take care of setting up a fixture before each test, they don't address the problem of cleaning up a fixture after the test completes. In this situation, one option is to mix in theBeforeAndAfterEach
trait.BeforeAndAfterEach
'sbeforeEach
method will be run before, and itsafterEach
method after, each test (like JUnit'ssetUp
andtearDown
methods, respectively). For example, you could create a temporary file before each test, and delete it afterwords, like this:In this example, the instance variable
reader
is avar
, so it can be reinitialized between tests by thebeforeEach
method.Although the
BeforeAndAfterEach
approach should be familiar to the users of most test other frameworks, ScalaTest provides another alternative that also allows you to perform cleanup after each test: overridingwithFixture(NoArgTest)
. To execute each test,Suite
's implementation of therunTest
method wraps an invocation of the appropriate test method in a no-arg function.runTest
passes that test function to thewithFixture(NoArgTest)
method, which is responsible for actually running the test by invoking the function.Suite
's implementation ofwithFixture(NoArgTest)
simply invokes the function, like this:The
withFixture(NoArgTest)
method exists so that you can override it and set a fixture up before, and clean it up after, each test. Thus, the previous temp file example could also be implemented without mixing inBeforeAndAfterEach
, like this:If you prefer to keep your test classes immutable, one final variation is to use the
FixtureSuite
trait from theorg.scalatest.fixture
package. Tests in anorg.scalatest.fixture.FixtureSuite
can have a fixture object passed in as a parameter. You must indicate the type of the fixture object by defining theFixture
type member and define awithFixture
method that takes a one-arg test function. (AFixtureSuite
has two overloadedwithFixture
methods, therefore, one that takes aOneArgTest
and the other, inherited fromSuite
, that takes aNoArgTest
.) Inside thewithFixture(OneArgTest)
method, you create the fixture, pass it into the test function, then perform any necessary cleanup after the test function returns. Instead of invoking each test directly, aFixtureSuite
will pass a function that invokes the code of a test towithFixture(OneArgTest)
. YourwithFixture(OneArgTest)
method, therefore, is responsible for actually running the code of the test by invoking the test function. For example, you could pass the temp file reader fixture to each test that needs it by overriding thewithFixture(OneArgTest)
method of aFixtureSuite
, like this:It is worth noting that the only difference in the test code between the mutable
BeforeAndAfterEach
approach shown previously and the immutableFixtureSuite
approach shown here is that two of theFixtureSuite
's test methods take aFileReader
as a parameter. Otherwise the test code is identical. One benefit of the explicit parameter is that, as demonstrated by thetestWithoutAFixture
method, aFixtureSuite
test method need not take the fixture. (Tests that don't take a fixture as a parameter are passed to thewithFixture
that takes aNoArgTest
, shown previously.) So you can have some tests that take a fixture, and others that don't. In this case, theFixtureSuite
provides documentation indicating which test methods use the fixture and which don't, whereas theBeforeAndAfterEach
approach does not.If you want to execute code before and after all tests (and nested suites) in a suite, such as you could do with
@BeforeClass
and@AfterClass
annotations in JUnit 4, you can use thebeforeAll
andafterAll
methods ofBeforeAndAfterAll
. See the documentation forBeforeAndAfterAll
for an example.The config map
In some cases you may need to pass information to a suite of tests. For example, perhaps a suite of tests needs to grab information from a file, and you want to be able to specify a different filename during different runs. You can accomplish this in ScalaTest by passing the filename in the config map of key-value pairs, which is passed to
run
as aMap[String, Any]
. The values in the config map are called "config objects," because they can be used to configuresuites, reporters, and tests.You can specify a string config object is via the ScalaTest
Runner
, either via the command line or ScalaTest's ant task. (See the documentation for Runner for information on how to specify config objects on the command line.) The config map is passed torun
,runNestedSuites
,runTests
, andrunTest
, so one way to access it in your suite is to override one of those methods. If you need to use the config map inside your tests, you can use one of the traits in theorg.scalatest.fixture
package. (See thedocumentation forFixtureSuite
for instructions on how to access the config map in tests.)Tagging tests
A
Suite
's tests may be classified into groups by tagging them with string names. When executing aSuite
, groups of tests can optionally be included and/or excluded. In this trait's implementation, tags are indicated by annotations attached to the test method. To create a new tag type to use inSuite
s, simply define a new Java annotation that itself is annotated with theorg.scalatest.TagAnnotation
annotation. (Currently, for annotations to be visible in Scala programs via Java reflection, the annotations themselves must be written in Java.) For example, to create a tag namedSlowAsMolasses
, to use to mark slow tests, you would write in Java:BECAUSE OF A SCALADOC BUG IN SCALA 2.8, I HAD TO PUT A SPACE AFTER THE AT SIGN IN ONE THE TARGET ANNOTATION EXAMPLE BELOW. IF YOU WANT TO COPY AND PASTE FROM THIS EXAMPLE, YOU'LL NEED TO REMOVE THE SPACE BY HAND, OR COPY FROM THE SUITE SCALADOC FOR VERSION 1.1 INSTEAD, WHICH IS ALSO VALID FOR 1.3. - Bill Venners
Given this new annotation, you could place a
Suite
test method into theSlowAsMolasses
group (i.e., tag it as beingSlowAsMolasses
) like this:The primary
run
method takes aFilter
, whose constructor takes an optionalSet[String]
s calledtagsToInclude
and aSet[String]
calledtagsToExclude
. IftagsToInclude
isNone
, all tests will be run except those those belonging to tags listed in thetagsToExclude
Set
. IftagsToInclude
is defined, only tests belonging to tags mentioned in thetagsToInclude
set, and not mentioned intagsToExclude
, will be run.Ignored tests
Another common use case is that tests must be “temporarily” disabled, with the good intention of resurrecting the test at a later time. ScalaTest provides an
Ignore
annotation for this purpose. You use it like this:If you run this version of
MySuite
with:It will run only
testAddition
and report thattestSubtraction
was ignored. You'll see:Ignore
is implemented as a tag. TheFilter
class effectively addsorg.scalatest.Ignore
to thetagsToExclude
Set
if it not already in thetagsToExclude
set passed to its primary constructor. The only difference betweenorg.scalatest.Ignore
and the tags you may define and exclude is that ScalaTest reports ignored tests to theReporter
. The reason ScalaTest reports ignored tests is as a feeble attempt to encourage ignored tests to be eventually fixed and added back into the active suite of tests.Pending tests
A pending test is one that has been given a name but is not yet implemented. The purpose of pending tests is to facilitate a style of testing in which documentation of behavior is sketched out before tests are written to verify that behavior (and often, the before the behavior of the system being tested is itself implemented). Such sketches form a kind of specification of what tests and functionality to implement later.
To support this style of testing, a test can be given a name that specifies one bit of behavior required by the system being tested. The test can also include some code that sends more information about the behavior to the reporter when the tests run. At the end of the test, it can call method
pending
, which will cause it to complete abruptly withTestPendingException
. Because tests in ScalaTest can be designated as pending withTestPendingException
, both the test name and any information sent to the reporter when running the test can appear in the report of a test run. (In other words, the code of a pending test is executed just like any other test.) However, because the test completes abruptly withTestPendingException
, the test will be reported as pending, to indicate the actual test, and possibly the functionality it is intended to test, has not yet been implemented.Although pending tests may be used more often in specification-style suites, such as
org.scalatest.Spec
, you can also use it inSuite
, like this:If you run this version of
MySuite
with:It will run both tests but report that
testSubtraction
is pending. You'll see:Informers
One of the parameters to the primary
run
method is anReporter
, which will collect and report information about the running suite of tests. Information about suites and tests that were run, whether tests succeeded or failed, and tests that were ignored will be passed to theReporter
as the suite runs. Most often the reporting done by default bySuite
's methods will be sufficient, but occasionally you may wish to provide custom information to theReporter
from a test method. For this purpose, you can optionally include anInformer
parameter in a test method, and then pass the extra information to theInformer
via itsapply
method. TheInformer
will then pass the information to theReporter
by sending anInfoProvided
event. Here's an example:If you run this
Suite
from the interpreter, you will see the message included in the printed report:Executing suites in parallel
The primary
run
method takes as its last parameter an optionalDistributor
. If aDistributor
is passed in, this trait's implementation ofrun
puts its nestedSuite
s into the distributor rather than executing them directly. The caller ofrun
is responsible for ensuring that some entity runs theSuite
s placed into the distributor. The-c
command line parameter toRunner
, for example, will causeSuite
s put into theDistributor
to be run in parallel via a pool of threads.Treatment of
java.lang.Error
sThe Javadoc documentation for
java.lang.Error
states:Because
Error
s are used to denote serious errors, traitSuite
and its subtypes in the ScalaTest API do not always treat a test that completes abruptly with anError
as a test failure, but sometimes as an indication that serious problems have arisen that should cause the run to abort. For example, if a test completes abruptly with anOutOfMemoryError
, it will not be reported as a test failure, but will instead cause the run to abort. Because not everyone usesError
s only to represent serious problems, however, ScalaTest only behaves this way for the following exception types (and their subclasses):java.lang.annotation.AnnotationFormatError
java.awt.AWTError
java.nio.charset.CoderMalfunctionError
javax.xml.parsers.FactoryConfigurationError
java.lang.LinkageError
java.lang.ThreadDeath
javax.xml.transform.TransformerFactoryConfigurationError
java.lang.VirtualMachineError
The previous list includes all
Error
s that exist as part of Java 1.5 API, excludingjava.lang.AssertionError
. ScalaTest does treat a thrownAssertionError
as an indication of a test failure. In addition, any otherError
that is not an instance of a type mentioned in the previous list will be caught by theSuite
traits in the ScalaTest API and reported as the cause of a test failure.Although trait
Suite
and all its subtypes in the ScalaTest API consistently behave this way with regard toError
s, this behavior is not required by the contract ofSuite
. Subclasses and subtraits that you define, for example, may treat allError
s as test failures, or indicate errors in some other way that has nothing to do with exceptions.Extensibility
Trait
Suite
provides default implementations of its methods that should be sufficient for most applications, but many methods can be overridden when desired. Here's a summary of the methods that are intended to be overridden:run
- override this method to define custom ways to run suites of tests.runNestedSuites
- override this method to define custom ways to run nested suites.runTests
- override this method to define custom ways to run a suite's tests.runTest
- override this method to define custom ways to run a single named test.testNames
- override this method to specify theSuite
's test names in a custom way.tags
- override this method to specify theSuite
's test tags in a custom way.nestedSuites
- override this method to specify theSuite
's nestedSuite
s in a custom way.suiteName
- override this method to specify theSuite
's name in a custom way.expectedTestCount
- override this method to count thisSuite
's expected tests in a custom way.For example, this trait's implementation of
testNames
performs reflection to discover methods starting withtest
, and places these in aSet
whose iterator returns the names in alphabetical order. If you wish to run tests in a different order in a particularSuite
, perhaps because a test namedtestAlpha
can only succeed after a test namedtestBeta
has run, you can overridetestNames
so that it returns aSet
whose iterator returnstestBeta
beforetestAlpha
. (This trait's implementation ofrun
will invoke tests in the order they come out of thetestNames
Set
iterator.)Alternatively, you may not like starting your test methods with
test
, and prefer using@Test
annotations in the style of Java's JUnit 4 or TestNG. If so, you can overridetestNames
to discover tests using either of these two APIs@Test
annotations, or one of your own invention. (This is in fact howorg.scalatest.junit.JUnitSuite
andorg.scalatest.testng.TestNGSuite
work.)Moreover, test in ScalaTest does not necessarily mean test method. A test can be anything that can be given a name, that starts and either succeeds or fails, and can be ignored. In
org.scalatest.FunSuite
, for example, tests are represented as function values. This approach might look foreign to JUnit users, but may feel more natural to programmers with a functional programming background. To facilitate this style of writing tests,FunSuite
overridestestNames
,runTest
, andrun
such that you can define tests as function values.You can also model existing JUnit 3, JUnit 4, or TestNG tests as suites of tests, thereby incorporating tests written in Java into a ScalaTest suite. The "wrapper" classes in packages
org.scalatest.junit
andorg.scalatest.testng
exist to make this easy. No matter what legacy tests you may have, it is likely you can create or use an existingSuite
subclass that allows you to model those tests as ScalaTest suites and tests and incorporate them into a ScalaTest suite. You can then write new tests in Scala and continue supporting older tests in Java.