trait Retries extends AnyRef
Provides methods that can be used in withFixture
implementations to retry tests in various scenarios.
Trait Retries
is intended to help you deal with “flickers”—tests that usually pass, but
occasionally fail. The best way to deal with such tests is to fix them so they always pass. Sometimes, however, this is
not practical. In such cases, flickers can waste your time by forcing you to investigate test failures that turn
out to be flickers. Or worse, like the boy who cried wolf, the flickers may train you an your colleagues to not pay attention
to failures such that you don't notice real problems, at least not in a timely manner.
Trait Retries
offers methods that will retry a failed and/or canceled test once, on the same thread,
with or without a delay. These methods take a block that results in Outcome
,
and are intended to be used in withFixture
methods. You should be very selective about which tests you
retry, retrying those for which you have good evidence to conclude they are flickers. Thus it is recommended you
only retry tests that are tagged with Retryable
, and only tag tests as such once they have flickered
consistently for a while, and only after you invested a reasonable effort into fixing them properly.
Here's an example showing how you might use Retries
:
package org.scalatest.examples.tagobjects.retryable import org.scalatest._ import tagobjects.Retryable class SetSpec extends FlatSpec with Retries { override def withFixture(test: NoArgTest) = { if (isRetryable(test)) withRetry { super.withFixture(test) } else super.withFixture(test) } "An empty Set" should "have size 0" taggedAs(Retryable) in { assert(Set.empty.size === 0) } }
- Source
- Retries.scala
- Alphabetic
- By Inheritance
- Retries
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[java.lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
finalize(): Unit
- Attributes
- protected[java.lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
def
isRetryable(testData: TestData): Boolean
Indicates whether the test described by the given
TestData
includes the tagorg.scalatest.tags.Retryable
.Indicates whether the test described by the given
TestData
includes the tagorg.scalatest.tags.Retryable
.This method provides an easy way to selectively retry just tests that are flickering. You can annotated such problematic tests with
Retryable
, and just retry those. Here's what it might look like:override def withFixture(test: NoArgTest) = { if (isRetryable(test)) withRetry { super.withFixture(test) } else super.withFixture(test) }
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
def
withRetry(delay: Span)(blk: ⇒ Outcome): Outcome
Retries the given block with a given delay if the
Outcome
of executing the block is eitherFailed
orCanceled
.Retries the given block with a given delay if the
Outcome
of executing the block is eitherFailed
orCanceled
.The behavior of this method is defined in the table below. The first two rows show the main "retry" behavior: if executing the block initially fails, and on retry it succeeds, the result is
Canceled
. The purpose of this is to deal with "flickering" tests by downgrading a failure that succeeds on retry to a cancelation. Or, if executing the block initially results inCanceled
, and on retry it succeeds, the result isSucceeded
. The purpose of this is to deal with tests that intermittently cancel by ignoring a cancelation that succeeds on retry.In the table below, if the “Retry
Outcome
” has just a dash, the block is not retried. Otherwise, the block is retried on the same thread, after sleeping the given delay.First Outcome
Retry Outcome
Result Failed
Succeeded
Canceled
(theSucceeded
andFailed
are discarded; the exception from theFailed
is the cause of the exception in theCanceled
)Canceled
Succeeded
Succeeded
(theCanceled
is discarded)Succeeded
— Succeeded
(no retry)Pending
— Pending
(no retry)Failed
Failed
the first Failed
(the secondFailed
is discarded)Failed
Pending
the Failed
(thePending
is discarded)Failed
Canceled
the Failed
(theCanceled
is discarded)Canceled
Canceled
the first Canceled
(the secondCanceled
is discarded)Canceled
Pending
the Canceled
(thePending
is discarded)Canceled
Failed
the Failed
(theCanceled
is discarded)- delay
the amount of time to sleep before retrying
- blk
the block to execute and potentially retry
-
def
withRetry(blk: ⇒ Outcome): Outcome
Retries the given block immediately (with no delay) if the
Outcome
of executing the block is eitherFailed
orCanceled
.Retries the given block immediately (with no delay) if the
Outcome
of executing the block is eitherFailed
orCanceled
.The behavior of this method is defined in the table below. The first two rows show the main "retry" behavior: if executing the block initially fails, and on retry it succeeds, the result is
Canceled
. The purpose of this is to deal with "flickering" tests by downgrading a failure that succeeds on retry to a cancelation. Or, if executing the block initially results inCanceled
, and on retry it succeeds, the result isSucceeded
. The purpose of this is to deal with tests that intermittently cancel by ignoring a cancelation that succeeds on retry.In the table below, if the “Retry
Outcome
” has just a dash, the block is not retried. Otherwise, the block is retried on the same thread, with no delay.First Outcome
Retry Outcome
Result Failed
Succeeded
Canceled
(theSucceeded
andFailed
are discarded; the exception from theFailed
is the cause of the exception in theCanceled
)Canceled
Succeeded
Succeeded
(theCanceled
is discarded)Succeeded
— Succeeded
(no retry)Pending
— Pending
(no retry)Failed
Failed
the first Failed
(the secondFailed
is discarded)Failed
Pending
the Failed
(thePending
is discarded)Failed
Canceled
the Failed
(theCanceled
is discarded)Canceled
Canceled
the first Canceled
(the secondCanceled
is discarded)Canceled
Pending
the Canceled
(thePending
is discarded)Canceled
Failed
the Failed
(theCanceled
is discarded)- blk
the block to execute and potentially retry
-
def
withRetryOnCancel(delay: Span)(blk: ⇒ Outcome): Outcome
Retries the given block after the given delay if the
Outcome
of executing the block isCanceled
.Retries the given block after the given delay if the
Outcome
of executing the block isCanceled
.The behavior of this method is defined in the table below. The first row shows the main "retry" behavior: if executing the block initially results in
Canceled
, and on retry it succeeds, the result isSucceeded
. The purpose of this is to deal with tests that intermittently cancel by ignoring a cancelation that succeeds on retry.In the table below, if the “Retry
Outcome
” has just a dash, the block is not retried. Otherwise, the block is retried on the same thread, after the given delay.First Outcome
Retry Outcome
Result Canceled
Succeeded
the Succeeded
(theCanceled
is discarded)Succeeded
— Succeeded
(no retry)Pending
— Pending
(no retry)Failed
— Failed
(no retry)Canceled
Canceled
the first Canceled
(the secondCanceled
is discarded)Canceled
Pending
the Canceled
(thePending
is discarded)Canceled
Failed
the Failed
(theCanceled
is discarded)- delay
the amount of time to sleep before retrying
- blk
the block to execute and potentially retry
-
def
withRetryOnCancel(blk: ⇒ Outcome): Outcome
Retries the given block immediately (with no delay) if the
Outcome
of executing the block isCanceled
.Retries the given block immediately (with no delay) if the
Outcome
of executing the block isCanceled
.The behavior of this method is defined in the table below. The first row shows the main "retry" behavior: if executing the block initially results in
Canceled
, and on retry it succeeds, the result isSucceeded
. The purpose of this is to deal with tests that intermittently cancel by ignoring a cancelation that succeeds on retry.In the table below, if the “Retry
Outcome
” has just a dash, the block is not retried. Otherwise, the block is retried on the same thread, with no delay.First Outcome
Retry Outcome
Result Canceled
Succeeded
the Succeeded
(theCanceled
is discarded)Succeeded
— Succeeded
(no retry)Pending
— Pending
(no retry)Failed
— Failed
(no retry)Canceled
Canceled
the first Canceled
(the secondCanceled
is discarded)Canceled
Pending
the Canceled
(thePending
is discarded)Canceled
Failed
the Failed
(theCanceled
is discarded)- blk
the block to execute and potentially retry
-
def
withRetryOnFailure(delay: Span)(blk: ⇒ Outcome): Outcome
Retries the given block immediately with the given delay if the
Outcome
of executing the block isFailed
.Retries the given block immediately with the given delay if the
Outcome
of executing the block isFailed
.The behavior of this method is defined in the table below. The first row shows the main "retry" behavior: if executing the block initially fails, and on retry it succeeds, the result is
Canceled
. The purpose of this is to deal with "flickering" tests by downgrading a failure that succeeds on retry to a cancelation.In the table below, if the “Retry
Outcome
” has just a dash, the block is not retried. Otherwise, the block is retried on the same thread, after the given delay.First Outcome
Retry Outcome
Result Failed
Succeeded
Canceled
(theSucceeded
andFailed
are discarded; the exception from theFailed
is the cause of the exception in theCanceled
)Succeeded
— Succeeded
(no retry)Pending
— Pending
(no retry)Canceled
— the Canceled
(no retry)Failed
Failed
the first Failed
(the secondFailed
is discarded)Failed
Pending
the Failed
(thePending
is discarded)Failed
Canceled
the Failed
(theCanceled
is discarded)- delay
the amount of time to sleep before retrying
- blk
the block to execute and potentially retry
-
def
withRetryOnFailure(blk: ⇒ Outcome): Outcome
Retries the given block immediately (with no delay) if the
Outcome
of executing the block isFailed
.Retries the given block immediately (with no delay) if the
Outcome
of executing the block isFailed
.The behavior of this method is defined in the table below. The first row shows the main "retry" behavior: if executing the block initially fails, and on retry it succeeds, the result is
Canceled
. The purpose of this is to deal with "flickering" tests by downgrading a failure that succeeds on retry to a cancelation.In the table below, if the “Retry
Outcome
” has just a dash, the block is not retried. Otherwise, the block is retried on the same thread, with no delay.First Outcome
Retry Outcome
Result Failed
Succeeded
Canceled
(theSucceeded
andFailed
are discarded; the exception from theFailed
is the cause of the exception in theCanceled
)Succeeded
— Succeeded
(no retry)Pending
— Pending
(no retry)Canceled
— the Canceled
(no retry)Failed
Failed
the first Failed
(the secondFailed
is discarded)Failed
Pending
the Failed
(thePending
is discarded)Failed
Canceled
the Failed
(theCanceled
is discarded)- blk
the block to execute and potentially retry