Class

org.scalatest.tools

Framework

Related Doc: package tools

Permalink

class Framework extends sbt.testing.Framework

This class is ScalaTest's implementation of the new Framework API that is supported in sbt 0.13.

To use ScalaTest in sbt, you should add ScalaTest as dependency in your sbt build file, the following shows an example for using ScalaTest 2.0 with Scala 2.10.x project:

"org.scalatest" % "scalatest_2.10" % "2.0" % "test"

To pass argument to ScalaTest from sbt, you can use testOptions:

testOptions in Test += Tests.Argument("-h", "target/html")  // Use HtmlReporter

If you are using multiple testing frameworks, you can pass arguments specific to ScalaTest only:

testOptions in Test += Tests.Argument(TestFrameworks.ScalaTest, "-h", "target/html") // Use HtmlReporter

Supported arguments

Integration in sbt 0.13 supports same argument format as Runner, except the following arguments:

New Features of New Framework API

New Framework API supports a number of new features that ScalaTest has utilized to support a better testing experience in sbt. The followings are summary of new features supported by the new Framework API:

Specified behavior of single instance of Runner per project run (non-fork), and a new done method

In new Framework API, it is now a specified behavior that Framework's runner method will be called to get a Runner instance once per project run. Arguments will be passed when calling Framework's runner and this gives ScalaTest a good place to perform setup tasks, such as initializing Reporters.

There's also a new done on Runner interface, which in turns provide a good spot for ScalaTest to perform cleanup tasks, such as disposing the Reporters. HtmlReporter depends on this behavior to generate its index.html. In addition, done can return framework-specific summary text for sbt to render at the end of the project run, which allows ScalaTest to return its own summary text.

API to return nested Suites as sbt Tasks

In new Framework API, a new concept of Task was introduced. A Task has an execute method that can return more Tasks for execution. ScalaTest does not utilize this feature, it always return empty array for sub-tasks.

API to support test execution in fork mode

Forking was added to sbt since version 0.12, you can find documentation for forking support in sbt at Forking in sbt.

Although forking is already available in sbt since 0.12, there's no support in old Framework API, until it is added in new Framework API that is supported in sbt 0.13. With API provided with new Framework API, ScalaTest creates real Reporters in the main process, and uses SocketReporter in forked process to send events back to the main process, and get processed by real Reporters at the main process. All of this is transparent to any custom Reporter implementation, as only one instance of the custom Reporter will be created to process the events, regardless of whether the tests run in same or forked process.

Selector API to selectively run tests

New Framework API includes a set of comprehensive API to select tests for execution. Though new Framework API supports fine-grained test selection, current sbt's test-only and test-quick supports up to suite level selection only, or SuiteSelector as defined in new Framework API. This Framework implementation already supports SuiteSelector, NestedSuiteSelector, TestSelector and NestedTestSelector, which should work once future sbt version supports them.

Added new Ignored, Canceled and Pending status

Status Ignored, Canceled and Pending are added to new Framework API, and they match perfectly with ScalaTest's ignored tests (now reported as Ignored instead of Skipped), as well as canceled and pending tests newly added in ScalaTest 2.0.

Added sbt Tagging support

Sbt supports task tagging, but has no support in old Framework API for test frameworks to integrate it. New Framework API supports it, and you can now use the following annotations to annotate your suite for sbt built-in resource tags:

They will be mapped to corresponding resource tag CPU, Disk and Network in sbt.

You can also define custom tag, which you'll need to write it as Java annotation:

import java.lang.annotation.Target;
import java.lang.annotation.Retention;
import org.scalatest.TagAnnotation;

@TagAnnotation("custom")
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE})
public @interface Custom {}

which will be translated to Tags.Tag("custom") in sbt.

Source
Framework.scala
Linear Supertypes
sbt.testing.Framework, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. Framework
  2. Framework
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new Framework()

    Permalink

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  5. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  6. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. def fingerprints(): Array[Fingerprint]

    Permalink

    An array of Fingerprints that specify how to identify ScalaTest's test classes during discovery.

    An array of Fingerprints that specify how to identify ScalaTest's test classes during discovery.

    returns

    SubclassFingerprint for org.scalatest.Suite and AnnotatedFingerprint for org.scalatest.WrapWith

    Definition Classes
    Framework → Framework
  10. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  11. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  12. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  13. def name(): String

    Permalink

    Test framework name.

    Test framework name.

    returns

    ScalaTest

    Definition Classes
    Framework → Framework
  14. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  15. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  16. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  17. def runner(args: Array[String], remoteArgs: Array[String], testClassLoader: ClassLoader): Runner

    Permalink

    Initiates a ScalaTest run.

    Initiates a ScalaTest run.

    args

    the ScalaTest arguments for the new run

    remoteArgs

    the ScalaTest remote arguments for the run in a forked JVM

    testClassLoader

    a class loader to use when loading test classes during the run

    returns

    a Runner implementation representing the newly started run to run ScalaTest's tests.

    Definition Classes
    Framework → Framework
    Exceptions thrown

    IllegalArgumentException when invalid or unsupported argument is passed

  18. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  19. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  20. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  21. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  22. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from sbt.testing.Framework

Inherited from AnyRef

Inherited from Any

Ungrouped