Class TestRunnerBase<TContext, TTest>

Namespace

Xunit.v3

Assembly

xunit.v3.core.dll

A base class that provides default behavior when running a test. This includes support for skipping tests.

public abstract class TestRunnerBase<TContext, TTest> where TContext : TestRunnerBaseContext<TTest> where TTest : class, ITest

Type Parameters

TContext

The context type used by the runner

TTest

The test type used by the test framework. Must derive from ITest.

Inheritance

object

TestRunnerBase<TContext, TTest>

Derived

TestRunner<TContext, TTest>

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

This class does not make any assumptions about what it means to run an individual test, just that at some point, the test will be run. The intention with this base class is that it can serve as a base for non-traditional tests (e.g., tests that are not derived from invoking CLR methods).

Constructors

TestRunnerBase()

Initializes a new instance of the TestRunner<TContext, TTest> class.

protected TestRunnerBase()

Methods

GetAttachments(TContext)

Gets the attachments for the test. If the test framework did not collect attachments (or does not support attachments), then it should return null.

protected virtual ValueTask<IReadOnlyDictionary<string, TestAttachment>?> GetAttachments(TContext ctxt)

Parameters

ctxt TContext

The context that describes the current test

Returns

ValueTask<IReadOnlyDictionary<string, TestAttachment>>

Remarks

By default, this method returns Attachments from the current context. This method runs during Running and any exceptions thrown will contribute to test failure.

GetTestOutput(TContext)

Gets any output collected from the test after execution is complete. If the test framework did not collect any output, or does not support collecting output, then it should return Empty.

protected virtual ValueTask<string> GetTestOutput(TContext ctxt)

Parameters

ctxt TContext

The context that describes the current test

Returns

ValueTask<string>

Remarks

This method runs during Running and any exceptions thrown will contribute to test failure.

GetWarnings(TContext)

Gets the warnings that will be reported during test results. By default, returns null, indicating that there were no warnings

protected virtual ValueTask<string[]?> GetWarnings(TContext ctxt)

Parameters

ctxt TContext

The context that describes the current test

Returns

ValueTask<string[]>

Remarks

This method runs during Running and any exceptions thrown will contribute to test failure.

OnTestCleanupFailure(TContext, Exception)

This method is called when an exception was thrown while cleaning up, after the test has run. By default, this sends (like TestCleanupFailure).

protected virtual ValueTask<bool> OnTestCleanupFailure(TContext ctxt, Exception exception)

Parameters

ctxt TContext

The context that describes the current test

exception Exception

The exception that caused the cleanup failure (may be an instance of AggregateException if more than one exception occurred).

Returns

ValueTask<bool>

Return true if test execution should continue; false if it should be shut down.

Remarks

This method runs during CleaningUp and any exceptions thrown are converted into fatal exception messages (via IErrorMessage) and sent to the message bus in ctxt.

OnTestFailed(TContext, Exception, decimal, string, string[]?)

This method is called when a test has failed. By default, this sends TestFailed.

protected virtual ValueTask<(bool Continue, TestResultState ResultState)> OnTestFailed(TContext ctxt, Exception exception, decimal executionTime, string output, string[]? warnings)

Parameters

ctxt TContext

The context that describes the current test

exception Exception

The exception that caused the test failure

executionTime decimal

The time spent running the test

output string

The output from the test

warnings string[]

The warnings that were generated during the test

Returns

ValueTask<(bool Continue, TestResultState ResultState)>

Return true if test execution should continue; false if it should be shut down.

Remarks

This method runs during CleaningUp and any exceptions thrown will contribute to test cleanup failure.

OnTestFinished(TContext, decimal, string, string[]?, IReadOnlyDictionary<string, TestAttachment>?)

This method is called just after the test has finished running. By default, this sends TestFinished. Override this to enable any extensibility related to test finish.

protected virtual ValueTask<bool> OnTestFinished(TContext ctxt, decimal executionTime, string output, string[]? warnings, IReadOnlyDictionary<string, TestAttachment>? attachments)

Parameters

ctxt TContext

The context that describes the current test

executionTime decimal

The time spent running the test

output string

The output from the test

warnings string[]

The warnings that were generated during the test

attachments IReadOnlyDictionary<string, TestAttachment>

The attachments that were assocated with the test

Returns

ValueTask<bool>

Return true if test execution should continue; false if it should be shut down.

Remarks

This method runs during CleaningUp and any exceptions thrown will contribute to test cleanup failure.

OnTestNotRun(TContext, string, string[]?)

This method is called when a test was not run. By default, this sends TestNotRun.

protected virtual ValueTask<(bool Continue, TestResultState ResultState)> OnTestNotRun(TContext ctxt, string output, string[]? warnings)

Parameters

ctxt TContext

The context that describes the current test

output string

The output from the test

warnings string[]

The warnings that were generated during the test

Returns

ValueTask<(bool Continue, TestResultState ResultState)>

Return true if test execution should continue; false if it should be shut down.

Remarks

This method runs during CleaningUp and any exceptions thrown will contribute to test cleanup failure.

OnTestPassed(TContext, decimal, string, string[]?)

This method is called when a test has passed. By default, this sends TestPassed.

protected virtual ValueTask<(bool Continue, TestResultState ResultState)> OnTestPassed(TContext ctxt, decimal executionTime, string output, string[]? warnings)

Parameters

ctxt TContext

The context that describes the current test

executionTime decimal

The time spent running the test

output string

The output from the test

warnings string[]

The warnings that were generated during the test

Returns

ValueTask<(bool Continue, TestResultState ResultState)>

Return true if test execution should continue; false if it should be shut down.

Remarks

This method runs during CleaningUp and any exceptions thrown will contribute to test cleanup failure.

OnTestSkipped(TContext, string, decimal, string, string[]?)

This method is called when a test is skipped. By default, this sends TestSkipped.

protected virtual ValueTask<(bool Continue, TestResultState ResultState)> OnTestSkipped(TContext ctxt, string skipReason, decimal executionTime, string output, string[]? warnings)

Parameters

ctxt TContext

The context that describes the current test

skipReason string

The reason given for skipping the test

executionTime decimal

The time spent running the test

output string

The output from the test

warnings string[]

The warnings that were generated during the test

Returns

ValueTask<(bool Continue, TestResultState ResultState)>

Return true if test execution should continue; false if it should be shut down.

Remarks

This method runs during CleaningUp and any exceptions thrown will contribute to test cleanup failure.

OnTestStarting(TContext)

This method is called just before the test is run. By default, this sends TestStarting. Override this to enable any extensibility related to test start.

protected virtual ValueTask<bool> OnTestStarting(TContext ctxt)

Parameters

ctxt TContext

The context that describes the current test

Returns

ValueTask<bool>

Return true if test execution should continue; false if it should be shut down.

Remarks

This method runs during Initializing and any exceptions thrown will contribute to test failure (and will prevent the test from running). Even if this method records exceptions, OnTestFinished(TContext, decimal, string, string[]?, IReadOnlyDictionary<string, TestAttachment>?) will be called.

OnTestStarting(TContext, bool, int)

This is a helper that allows passing explicit and timeout values, since those are not part of the core object model.

protected ValueTask<bool> OnTestStarting(TContext ctxt, bool @explicit, int timeout)

Parameters

ctxt TContext

The context that describes the current test

explicit bool

A flag which indicates whether this is an explicit test

timeout int

The timeout for running this test

Returns

ValueTask<bool>

Run(TContext)

Runs the test.

protected ValueTask<RunSummary> Run(TContext ctxt)

Parameters

ctxt TContext

The context that describes the current test

Returns

ValueTask<RunSummary>

Returns summary information about the test that was run.

Remarks

This function is the primary orchestrator of test execution.

RunTest(TContext)

Override this method to run the test.

protected abstract ValueTask<TimeSpan> RunTest(TContext ctxt)

Parameters

ctxt TContext

The context that describes the current test

Returns

ValueTask<TimeSpan>

SetTestContext(TContext, TestEngineStatus, TestResultState?, object?)

Sets the test context for the given test state and engine status.

protected virtual void SetTestContext(TContext ctxt, TestEngineStatus testStatus, TestResultState? testState = null, object? testClassInstance = null)

Parameters

ctxt TContext

The context that describes the current test

testStatus TestEngineStatus

The current engine status for the test

testState TestResultState

The current test state

testClassInstance object

The instance of the test class

Remarks

This method must never throw. Behavior is undefined if it does. Instead, exceptions that occur should be recorded in the aggregator in ctxt and will be reflected in a way that's appropriate based on when this method is called.

ShouldTestRun(TContext)

Override this to determine whether a test should be run or not (meaning, if you return false, it will be reported with a status of NotRun). By default, this method will return true. This is typically used to implement the ability to exclude specific tests unless they've been explicitly asked to be run.

protected virtual bool ShouldTestRun(TContext ctxt)

Parameters

ctxt TContext

The context that describes the current test

Returns

bool

Remarks

This method runs during Running and any exceptions thrown will contribute to test failure.

UpdateTestContext(object?, TestResultState?)

Updates the test context values while the test is running, without swapping out the test context itself. This preserves the values in the existing context (notably, the cancellation token, which is wrapped and passed, and as such cannot be replaced).

protected void UpdateTestContext(object? testClassInstance, TestResultState? testState = null)

Parameters

testClassInstance object

testState TestResultState