Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New context-based state management model for allure-csharp #371

Merged
merged 24 commits into from
Aug 18, 2023
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
d0e332c
Add AllureContext lass to isolate allure state from threads and async…
delatrie Jul 12, 2023
03aa7d9
Add concurrency tests. Reformulate context in terms of active/inactive
delatrie Jul 14, 2023
152b66c
Add tests on the context capturing by child threads/tasks
delatrie Jul 15, 2023
5abf894
Modify lifecycle/storage to use new context model. Add context get/se…
delatrie Jul 18, 2023
ce1efb7
Make CoreStepsHelper use AllureLifecycle's context
delatrie Jul 19, 2023
a08046d
Make lifecycle thread safe. Fix multithreading issues in lifecycle tests
delatrie Jul 19, 2023
7ffe0bc
Several improvements on context. Allure-xunit migration
delatrie Jul 19, 2023
b89cb8b
Bump C# language version to 11 for all projects
delatrie Jul 19, 2023
50ef9ed
Enable null checks in some classes of allure-xunit
delatrie Jul 19, 2023
fbea57a
Make lifecycle methods with explicit uuids obsoleted
delatrie Jul 19, 2023
6fd9380
Minor changes in context and lifecycle
delatrie Jul 28, 2023
98d2d22
Rewrite Allure.NUnit to support the new context scheme
delatrie Jul 28, 2023
618843d
Fix Allure.XUnit usage of RunInContext
delatrie Jul 28, 2023
4e43ea0
Rewrite Allure.SpecFlowPlugin to support new context scheme
delatrie Jul 28, 2023
ee741ba
Separate context from storage
delatrie Jul 31, 2023
9a94cd0
Fix obsolete messages. Add file scoped namespaces
delatrie Jul 31, 2023
119c95b
Revert obsoleted StopFixtureSuppressTestCase to be intact
delatrie Aug 1, 2023
11a0567
Add new info to commons README
delatrie Aug 1, 2023
ea43161
Fix invalid test start. Remove rudimentary test start check
delatrie Aug 2, 2023
f271411
Remove dotnet SDK 3.1 installation from build pipeline
delatrie Aug 2, 2023
4b254be
Change .NET SDK version to 7.0 in build pipeline
delatrie Aug 2, 2023
06e8d59
Github pipelines fix
delatrie Aug 2, 2023
013fcb6
Rename a placeholder scenario
delatrie Aug 2, 2023
a00d452
Implement requested changes for PR #371
delatrie Aug 8, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,15 @@ jobs:
name: "Build"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3.0.2
- uses: actions/checkout@v3.5.3

- name: 'Setup .NET Core SDK'
uses: actions/setup-dotnet@v2.1.0
uses: actions/setup-dotnet@v3.2.0
with:
dotnet-version: |
3.1.x
6.0.x
7.0.x

- name: 'Restore packages'
run: dotnet restore ${{ env.SOLUTION_PATH }} --packages ${{ env.RESTORE_OUTPUT_PATH }}
Expand All @@ -38,5 +39,5 @@ jobs:

- name: 'Run tests'
run: |
dotnet test Allure.Net.Commons.Tests/Allure.Net.Commons.Tests.csproj --no-restore --configuration ${{ env.BUILD_CONFIGURATION }}
dotnet test Allure.SpecFlowPlugin.Tests/Allure.SpecFlowPlugin.Tests.csproj --no-restore --configuration ${{ env.BUILD_CONFIGURATION }}
dotnet test Allure.Net.Commons.Tests/Allure.Net.Commons.Tests.csproj --no-build --configuration ${{ env.BUILD_CONFIGURATION }}
dotnet test Allure.SpecFlowPlugin.Tests/Allure.SpecFlowPlugin.Tests.csproj --no-build --configuration ${{ env.BUILD_CONFIGURATION }}
5 changes: 3 additions & 2 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,15 @@ jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3.0.2
- uses: actions/checkout@v3.5.3

- name: 'Setup .NET Core SDK'
uses: actions/setup-dotnet@v2.1.0
uses: actions/setup-dotnet@v3.2.0
with:
dotnet-version: |
3.1.x
6.0.x
7.0.x

- name: 'Restore packages'
run: dotnet restore ${{ env.SOLUTION_PATH }} --packages ${{ env.RESTORE_OUTPUT_PATH }}
Expand Down
1 change: 1 addition & 0 deletions Allure.Features/Allure.Features.csproj
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp3.1</TargetFramework>
<LangVersion>11</LangVersion>
<IsPackable>false</IsPackable>
<OutputPath>bin</OutputPath>
</PropertyGroup>
Expand Down
2 changes: 1 addition & 1 deletion Allure.Features/TestData/After Feature Failure.feature
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Feature: After Feature Failure
Scenario: After Feature Failure 1
Given Step is 'passed'

@broken
@failed
Scenario: After Feature Failure 3
Given Step is 'failed'

Expand Down
2 changes: 1 addition & 1 deletion Allure.Features/TestData/Before Feature Failure.feature
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@
Feature: Before Feature Failure

@broken
Scenario: Unknown
Scenario: Feature hook failure placeholder
2 changes: 1 addition & 1 deletion Allure.Features/TestData/Invalid Steps.feature
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
Given Step is 'passed'
And I don't have such step too

@broken
@failed
Scenario: Failed step followed by invalid step
Given Step is 'failed'
Given I don't have such step
Expand Down
1 change: 1 addition & 0 deletions Allure.NUnit.Examples/Allure.NUnit.Examples.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>net6.0</TargetFramework>
<LangVersion>11</LangVersion>
<IsPackable>false</IsPackable>
<OutputType>Library</OutputType>
</PropertyGroup>
Expand Down
10 changes: 1 addition & 9 deletions Allure.NUnit.Examples/BaseTest.cs
Original file line number Diff line number Diff line change
@@ -1,19 +1,11 @@
using Allure.Net.Commons;
using NUnit.Allure.Attributes;
using NUnit.Allure.Attributes;
using NUnit.Allure.Core;
using NUnit.Framework;

namespace Allure.NUnit.Examples
{
[AllureNUnit]
[AllureParentSuite("Root Suite")]
public class BaseTest
{
[OneTimeSetUp]
neparij marked this conversation as resolved.
Show resolved Hide resolved
public void CleanupResultDirectory()
{
AllureExtensions.WrapSetUpTearDownParams(() => { AllureLifecycle.Instance.CleanupResultDirectory(); },
"Clear Allure Results Directory");
}
}
}
1 change: 1 addition & 0 deletions Allure.NUnit/Allure.NUnit.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>netstandard2.0</TargetFramework>
<LangVersion>11</LangVersion>
<Version>2.10-SNAPSHOT</Version>
<GeneratePackageOnBuild>false</GeneratePackageOnBuild>
<Company>Qameta Software</Company>
Expand Down
30 changes: 17 additions & 13 deletions Allure.NUnit/Attributes/AllureDisplayIgnoredAttribute.cs
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ namespace NUnit.Allure.Attributes
public class AllureDisplayIgnoredAttribute : NUnitAttribute, ITestAction
{
private readonly string _suiteName;
private string _ignoredContainerId;

public AllureDisplayIgnoredAttribute(string suiteNameForIgnoredTests = "Ignored")
{
Expand All @@ -22,13 +21,11 @@ public AllureDisplayIgnoredAttribute(string suiteNameForIgnoredTests = "Ignored"

public void BeforeTest(ITest suite)
{
_ignoredContainerId = suite.Id + "-ignored";
var fixture = new TestResultContainer
AllureLifecycle.Instance.StartTestContainer(new()
{
uuid = _ignoredContainerId,
uuid = suite.Id + "-ignored",
name = suite.ClassName
};
AllureLifecycle.Instance.StartTestContainer(fixture);
});
}

public void AfterTest(ITest suite)
Expand All @@ -37,12 +34,19 @@ public void AfterTest(ITest suite)
if (suite.HasChildren)
{
var ignoredTests =
GetAllTests(suite).Where(t => t.RunState == RunState.Ignored || t.RunState == RunState.Skipped);
GetAllTests(suite).Where(
t => t.RunState == RunState.Ignored
|| t.RunState == RunState.Skipped
);
foreach (var test in ignoredTests)
{
AllureLifecycle.Instance.UpdateTestContainer(_ignoredContainerId, t => t.children.Add(test.Id));
AllureLifecycle.Instance.UpdateTestContainer(
t => t.children.Add(test.Id)
);

var reason = test.Properties.Get(PropertyNames.SkipReason).ToString();
var reason = test.Properties.Get(
PropertyNames.SkipReason
).ToString();

var ignoredTestResult = new TestResult
{
Expand All @@ -66,12 +70,12 @@ public void AfterTest(ITest suite)
}
};
AllureLifecycle.Instance.StartTestCase(ignoredTestResult);
AllureLifecycle.Instance.StopTestCase(ignoredTestResult.uuid);
AllureLifecycle.Instance.WriteTestCase(ignoredTestResult.uuid);
AllureLifecycle.Instance.StopTestCase();
AllureLifecycle.Instance.WriteTestCase();
}

AllureLifecycle.Instance.StopTestContainer(_ignoredContainerId);
AllureLifecycle.Instance.WriteTestContainer(_ignoredContainerId);
AllureLifecycle.Instance.StopTestContainer();
AllureLifecycle.Instance.WriteTestContainer();
}
}

Expand Down
8 changes: 6 additions & 2 deletions Allure.NUnit/Core/AllureExtendedConfiguration.cs
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,14 @@ namespace NUnit.Allure.Core
{
internal class AllureExtendedConfiguration : AllureConfiguration
{
public HashSet<string> BrokenTestData { get; set; } = new HashSet<string>();
public HashSet<string> BrokenTestData { get; set; } = new();

[JsonConstructor]
protected AllureExtendedConfiguration(string title, string directory, HashSet<string> links) : base(title,
protected AllureExtendedConfiguration(
string title,
string directory,
HashSet<string> links
) : base(title,
directory, links)
{
}
Expand Down
59 changes: 34 additions & 25 deletions Allure.NUnit/Core/AllureExtensions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -58,15 +58,22 @@ public static void WrapSetUpTearDownParams(Action action, string customName = ""
/// Wraps Action into AllureStep.
/// </summary>
[Obsolete("Use [AllureStep] method attribute")]
public static void WrapInStep(this AllureLifecycle lifecycle, Action action, string stepName = "", [CallerMemberName] string callerName = "")
public static void WrapInStep(
this AllureLifecycle lifecycle,
Action action,
string stepName = "",
[CallerMemberName] string callerName = ""
)
{
if (string.IsNullOrEmpty(stepName)) stepName = callerName;
if (string.IsNullOrEmpty(stepName))
{
stepName = callerName;
}

var id = Guid.NewGuid().ToString();
var stepResult = new StepResult {name = stepName};
try
{
lifecycle.StartStep(id, stepResult);
lifecycle.StartStep(stepResult);
action.Invoke();
lifecycle.StopStep(step => stepResult.status = Status.passed);
}
Expand All @@ -88,14 +95,22 @@ public static void WrapInStep(this AllureLifecycle lifecycle, Action action, str
/// <summary>
/// Wraps Func into AllureStep.
/// </summary>
public static T WrapInStep<T>(this AllureLifecycle lifecycle, Func<T> func, string stepName = "", [CallerMemberName] string callerName = "")
public static T WrapInStep<T>(
this AllureLifecycle lifecycle,
Func<T> func,
string stepName = "",
[CallerMemberName] string callerName = ""
)
{
if (string.IsNullOrEmpty(stepName)) stepName = callerName;
var id = Guid.NewGuid().ToString();
if (string.IsNullOrEmpty(stepName))
{
stepName = callerName;
}

var stepResult = new StepResult {name = stepName};
try
{
lifecycle.StartStep(id, stepResult);
lifecycle.StartStep(stepResult);
var result = func.Invoke();
lifecycle.StopStep(step => stepResult.status = Status.passed);
return result;
Expand Down Expand Up @@ -125,13 +140,15 @@ public static async Task WrapInStepAsync(
[CallerMemberName] string callerName = ""
)
{
if (string.IsNullOrEmpty(stepName)) stepName = callerName;
if (string.IsNullOrEmpty(stepName))
{
stepName = callerName;
}

var id = Guid.NewGuid().ToString();
var stepResult = new StepResult { name = stepName };
try
{
lifecycle.StartStep(id, stepResult);
lifecycle.StartStep(stepResult);
await action();
lifecycle.StopStep(step => stepResult.status = Status.passed);
}
Expand Down Expand Up @@ -160,12 +177,15 @@ public static async Task<T> WrapInStepAsync<T>(
[CallerMemberName] string callerName = ""
)
{
if (string.IsNullOrEmpty(stepName)) stepName = callerName;
var id = Guid.NewGuid().ToString();
if (string.IsNullOrEmpty(stepName))
{
stepName = callerName;
}

var stepResult = new StepResult { name = stepName };
try
{
lifecycle.StartStep(id, stepResult);
lifecycle.StartStep(stepResult);
var result = await func();
lifecycle.StopStep(step => stepResult.status = Status.passed);
return result;
Expand All @@ -184,16 +204,5 @@ public static async Task<T> WrapInStepAsync<T>(
throw;
}
}

/// <summary>
/// AllureNUnit AddScreenDiff wrapper method.
/// </summary>
public static void AddScreenDiff(this AllureLifecycle lifecycle, string expected, string actual, string diff)
neparij marked this conversation as resolved.
Show resolved Hide resolved
{
var storageMain = lifecycle.GetType().GetField("storage", BindingFlags.NonPublic | BindingFlags.Instance).GetValue(lifecycle);
var storageInternal = storageMain.GetType().GetField("storage", BindingFlags.NonPublic | BindingFlags.Instance).GetValue(storageMain);
var keys = (storageInternal as System.Collections.Concurrent.ConcurrentDictionary<string, object>).Keys.ToList();
AllureLifecycle.Instance.AddScreenDiff(keys.Find(key => key.Contains("-tr-")), expected, actual, diff);
}
}
}
24 changes: 8 additions & 16 deletions Allure.NUnit/Core/AllureNUnitAttribute.cs
Original file line number Diff line number Diff line change
@@ -1,8 +1,5 @@
using System;
using System.Collections.Concurrent;
using Allure.Net.Commons;
using NUnit.Engine;
using NUnit.Engine.Extensibility;
using NUnit.Framework;
using NUnit.Framework.Interfaces;
using NUnit.Framework.Internal;
Expand All @@ -12,21 +9,11 @@ namespace NUnit.Allure.Core
[AttributeUsage(AttributeTargets.Interface | AttributeTargets.Class)]
public class AllureNUnitAttribute : PropertyAttribute, ITestAction, IApplyToContext
{
private readonly ConcurrentDictionary<string, AllureNUnitHelper> _allureNUnitHelper = new ConcurrentDictionary<string, AllureNUnitHelper>();
private readonly bool _isWrappedIntoStep;

static AllureNUnitAttribute()
{
//!_! This is essential for async tests.
//!_! Async tests are working on different threads, so
//!_! default ManagedThreadId-separated behaviour in some cases fails on cross-thread execution.
AllureLifecycle.CurrentTestIdGetter = () => TestContext.CurrentContext.Test.FullName;
}
private readonly ConcurrentDictionary<string, AllureNUnitHelper> _allureNUnitHelper = new();

[Obsolete("wrapIntoStep parameter is obsolete. Use [AllureStep] method attribute")]
public AllureNUnitAttribute(bool wrapIntoStep = true)
{
_isWrappedIntoStep = wrapIntoStep;
}

public AllureNUnitAttribute()
Expand All @@ -36,7 +23,11 @@ public AllureNUnitAttribute()
public void BeforeTest(ITest test)
{
var helper = new AllureNUnitHelper(test);
_allureNUnitHelper.AddOrUpdate(test.Id, helper, (key, existing) => helper);
_allureNUnitHelper.AddOrUpdate(
test.Id,
helper,
(key, existing) => helper
);

if (test.IsSuite)
{
Expand Down Expand Up @@ -64,7 +55,8 @@ public void AfterTest(ITest test)
}
}

public ActionTargets Targets => ActionTargets.Test | ActionTargets.Suite;
public ActionTargets Targets =>
ActionTargets.Test | ActionTargets.Suite;

public void ApplyToContext(TestExecutionContext context)
{
Expand Down
Loading