An array of Fingerprint
s that specify how to identify ScalaTest's test classes during
discovery.
An array of Fingerprint
s that specify how to identify ScalaTest's test classes during
discovery.
SubclassFingerprint
for org.scalatest.Suite
and AnnotatedFingerprint
for org.scalatest.WrapWith
Test framework name.
Initiates a ScalaTest run.
Initiates a ScalaTest run.
the ScalaTest arguments for the new run
the ScalaTest remote arguments for the run in a forked JVM
a class loader to use when loading test classes during the run
a Runner
implementation representing the newly started run to run ScalaTest's tests.
IllegalArgumentException
when invalid or unsupported argument is passed
This class is ScalaTest's implementation of the new Framework API that is supported in sbt 0.13.
To use ScalaTest in sbt, you should add ScalaTest as dependency in your sbt build file, the following shows an example for using ScalaTest 2.0 with Scala 2.10.x project:
To pass argument to ScalaTest from sbt, you can use
testOptions
:If you are using multiple testing frameworks, you can pass arguments specific to ScalaTest only:
Supported arguments
Integration in sbt 0.13 supports same argument format as
Runner
, except the following arguments:-R
-- runpath is not supported because test path and discovery is handled by sbt-s
-- suite is not supported because sbt'stest-only
serves the similar purpose-A
-- again is not supported because sbt'stest-quick
serves the similar purpose-j
-- junit is not supported because in sbt different test framework should be supported by its correspondingFramework
implementation-b
-- testng is not supported because in sbt different test framework should be supported by its correspondingFramework
implementation-P
-- concurrent/parallel is not supported because parallel execution is controlled by sbt.-q
is not supported because test discovery should be handled by sbt, and sbt's test-only or test filter serves the similar purpose-T
is not supported because correct ordering of text output is handled by sbt-g
is not supported because current Graphic Reporter implementation works differently than standard reporterNew Features of New Framework API
New Framework API supports a number of new features that ScalaTest has utilized to support a better testing experience in sbt. The followings are summary of new features supported by the new Framework API:
Runner
per project run (non-fork), and a newdone
methodfork
modeIgnored
,Canceled
andPending
statusSpecified behavior of single instance of
Runner
per project run (non-fork), and a newdone
methodIn new Framework API, it is now a specified behavior that
Framework
'srunner
method will be called to get aRunner
instance once per project run. Arguments will be passed when callingFramework
'srunner
and this gives ScalaTest a good place to perform setup tasks, such as initializingReporter
s.There's also a new
done
onRunner
interface, which in turns provide a good spot for ScalaTest to perform cleanup tasks, such as disposing theReporter
s.HtmlReporter
depends on this behavior to generate itsindex.html
. In addition,done
can return framework-specific summary text for sbt to render at the end of the project run, which allows ScalaTest to return its own summary text.API to return nested
Suite
s as sbtTask
sIn new Framework API, a new concept of
Task
was introduced. ATask
has anexecute
method that can return moreTask
s for execution. ScalaTest does not utilize this feature, it always return empty array for sub-tasks.API to support test execution in
fork
modeForking was added to sbt since version 0.12, you can find documentation for forking support in sbt at Forking in sbt.
Although forking is already available in sbt since 0.12, there's no support in old Framework API, until it is added in new Framework API that is supported in sbt 0.13. With API provided with new Framework API, ScalaTest creates real
Reporter
s in the main process, and usesSocketReporter
in forked process to send events back to the main process, and get processed by realReporter
s at the main process. All of this is transparent to any customReporter
implementation, as only one instance of the customReporter
will be created to process the events, regardless of whether the tests run in same or forked process.Selector API to selectively run tests
New Framework API includes a set of comprehensive API to select tests for execution. Though new Framework API supports fine-grained test selection, current sbt's
test-only
andtest-quick
supports up to suite level selection only, orSuiteSelector
as defined in new Framework API. ThisFramework
implementation already supportsSuiteSelector
,NestedSuiteSelector
,TestSelector
andNestedTestSelector
, which should work once future sbt version supports them.Added new
Ignored
,Canceled
andPending
statusStatus
Ignored
,Canceled
andPending
are added to new Framework API, and they match perfectly with ScalaTest's ignored tests (now reported asIgnored
instead ofSkipped
), as well as canceled and pending tests newly added in ScalaTest 2.0.Added sbt Tagging support
Sbt supports task tagging, but has no support in old Framework API for test frameworks to integrate it. New Framework API supports it, and you can now use the following annotations to annotate your suite for sbt built-in resource tags:
CPU
Disk
Network
They will be mapped to corresponding resource tag
CPU
,Disk
andNetwork
in sbt.You can also define custom tag, which you'll need to write it as Java annotation:
which will be translated to
Tags.Tag("custom")
in sbt.