SpecFlow (Features) sync to Azure DevOps (TestCases)



Can be used for other frameworks like Java/Cucumber too (as the Feature files syntax remains the same)

installation

🚀 Visual Studio Extension (VSIX)

    🔰 Code-Editor (Feature file) ⇢ Context-menu 👉 Ctrl Alt Enter
    🔰 Code-Editor (Feature file) ⇢ Selection ⇢ Context-menu 👉 Ctrl Alt Enter
    🔰 Solution-Explorer ⇢ File / Folder ⇢ Context-menu 👉 Ctrl Shift Space
    🔰 Solution-Explorer ⇢ Project / Solution ⇢ Context-menu 👉 Ctrl Shift Space


☴ dotnet tool (auto-installed with VSIX)
                    
 # Install from nuget.org
 dotnet tool install -g --no-cache --ignore-failed-sources spex

 # Upgrade to latest version from nuget.org
 dotnet tool update -g --no-cache --ignore-failed-sources spex

 # Install a specific version from nuget.org
 dotnet tool install -g --no-cache --ignore-failed-sources spex --version 1.0.x

 # Uninstall
 dotnet tool uninstall -g spex
                        
            

If you don't have .NET 6.0 installed, download and install it.

If the Tool is not accessible post installation, add %USERPROFILE%\.dotnet\tools to the PATH env-var.


configuration

Spex.json (preferred)
                    
 {
     // "BaseUrl": "https://dev.azure.com", // Update in case of on-prem
     "Account": "", // Org - e.g. 'abc' in https://dev.azure.com/abc
     "Project": "", // e.g. 'xyz' in https://dev.azure.com/abc/xyz
     "PersonalAccessToken": "", // PAT (optional)
     "AreaPath": "", // {area-path}
     "IterationPath": "", // "{project}\\{iteration-path}"
     "DefaultAssignedTo": "", // testlead@abc.com
     "TestProjectPath": "", // .\\MyProject.Bdd.Tests.csproj
     "TestAssemblyPath": "", // .\\bin\\debug\\MyProject.Bdd.Tests.dll
     "FeatureFilters": [
     "Features"
     // "FeatureName1"
     // , "path2/FeatureName2"
     ],
     "ScenarioFilters": [
     // "ScenarioName1"
     // , "ScenarioName2"
     ],
     "AttachmentTags": [
     "@attach"
     //, @input
     ],
     "AttachmentsPath": "TestData",
     "AdditionalFields": [
     // "customField1=defaultValue1"
     // , "customField2=defaultValue2"
     ],
     "BddThenAsExpectedResult": false,
     "ScenarioOutlineExamplesInline": false,
     "QuiteMode": "" // "s", "m", "r"
 }
                        
            
App.config (alternative approach that auto-generates Spex.json)
                
 <Spex>
     <!-- <add key="BaseUrl" value="https://dev.azure.com" /> -->
     <add key="Account" value="{org}" />
     <add key="Project" value="{project}" />
     <add key="PersonalAccessToken" value="{pat}" />

     <add key="AreaPath" value="{areaPath}" />
     <add key="IterationPath" value="{IterationPath}" />
     <add key="DefaultAssignedTo" value="{default-user-email}" />

     <add key="TestProjectPath" value=".\MyProject.Bdd.Tests.csproj" />
     <add key="TestAssemblyPath" value=".\bin\debug\MyProject.Bdd.Tests.dll" />
     <add key="FeatureFilters" value="" /> <!-- Comma separated filters (e.g. Features,FeatureName1,path2/FeatureName2) -->
     <add key="ScenarioFilters" value="" /> <!-- Comma separated filters (e.g. ScenarioName1,ScenarioName2) -->

     <add key="AttachmentTags" value="@attach" /> <!-- Comma separated tags (e.g. @attach,@input) -->
     <add key="AttachmentsPath" value="TestData" />
     <add key="AdditionalFields" value="" /><!-- Comma separated fields (e.g. customField1=defaultValue1,customField2=defaultValue2) -->
     <add key="BddThenAsExpectedResult" value="false" />
     <add key="ScenarioOutlineExamplesInline" value="false" />
     <add key="QuiteMode" value="" />
 </Spex>
                
            
details
KEY TYPE VALUE DEFAULT
BaseUrl string [Optional] The base-url (with https:// prefix) that can be changed for on-prem AzDO Server https://dev.azure.com
Account string Azure DevOps Org Account name (https://dev.azure.com/{org})
Project string Azure DevOps Project under the Org (https://dev.azure.com/org/{project})
PersonalAccessToken string [Optional] Personal Access Token (PAT) (https://dev.azure.com/{org}/_usersSettings/tokens)
If empty, a prompt will appear for authentication
AreaPath string Area Path to be used when creating the Test-cases in AzDevOps
IterationPath string Iteration Path to be used when creating the Test-cases in AzDevOps
DefaultAssignedTo string The default Assigned To field to be used when creating the Test-cases in AzDevOps (can we overridden with @owner tag at individual Feature/Scenario level)
TestProjectPath string [Optional / .NET only] Relative path to the csproj file (w.r.t. the specified input folder)
TestAssemblyPath string [Optional / .NET only] Relative path to the dll file (w.r.t. the specified input folder)
FeatureFilters string[] [Optional] Path or name of the Features to filter out on (only Features matching this condition will be synced) Features
ScenarioFilters string[] [Optional] Name of the Scenarios to filter out on (only Scenarios matching this condition will be synced)
AttachmentTags string[] [Optional] List of tags to be considered for Test-case attachments
e.g. @attach=someFile.txt
@attach
AttachmentsPath string [Optional] Path to the root attachments folder
e.g. For @attach=someFile.txt tag, Spex will search for someFile.txt under TestData/Attach or TestData in the output folder
TestData
AdditionalFields string[] [Optional] Additional (custom) Work-item fields to be populated when creating AzDevOps Test-cases
BddThenAsExpectedResult bool [Optional] Populates BDD Thens in ExpectedResults column in AzDevOps Test-cases false
ScenarioOutlineExamplesInline bool [Optional] Populates Scenario-Outline Examples inline in the Steps (vs. Parameters) false
QuiteMode string [Optional] Just run (based on the information specified in the Settings file) and exit!
For valid values ("s"/"m"/"r"), see usage section below
e.g. When run in DevOps/CI pipelines

inputs

If no input (directory-path) is specified in the args, Spex assumes the current-dir as input.

When run from Visual Studio, additional args related to Feature-filters (f:) and Scenario-filters (s:) are passed to Spex tool, overriding the filters provided in Spex.json.


outputs

Results are saved under Spex folder under the specified input path.
You can .gitignore the generated Spex folder.


usage

 spex [bdd-test-project-folder-path]

    🔰 The specified path should contain a valid Spex.json settings file
    🔰 The tool creates a Spex folder under the specified path (for processing inputs/outputs)
    🔰 If no argument is supplied, Spex.json in the current-dir is used by default
    🔰 Any errors would be logged to %LOCALAPPDATA%\Spex folder (file: Spex_<datestamp>.log)
    🔰 Set QuiteMode to s / m / r in the Settings file to run once and exit (e.g. When run in DevOps/CI pipelines)
    👉 You can test / give it a first-spin using Azure DevOps Demo Generator
  • Enter s to sync SpecFlow Features to AzDevOps TestCases (with optional space-delimited suffixes)
                 -b to build the test-project before running AzDevOps sync
                 -r:false to skip regenerating the doc-cache, say if the project-files haven't changed
                 -d:true to demo/dry-run AzDevOps sync
                 -s:scenario-name-matches comma-separated scenario-filters to override global setting/input
                 -f:feature-name-or-path-matches comma-separated feature-filters to override global setting/input
  • Enter t to transform existing AzDevOps (non-BDD) Test-cases to BDD format (auto-generates .feature files under curr-dir\Spex\Features\ folder)
  • Enter m to generate AzDevOps TestCase-UserStory association matrix
  • Enter r to generate AzDevOps TestCases report
  • Enter u to auto-upgrade @version for updated Scenarios
  • Enter c to clear the console
  • Enter q to quit
  • Enter + to upgrade
supported tags on Features / Scenarios
    👉 Spaces (' ') in tag-values should be replaced with '^' (e.g. @area / @iteration below)
TAG DESCRIPTION
@testplan Mapped to Test-Plan ID in AzDevOps
e.g. @testplan=123
@testsuite Mapped to Test-Suite ID in AzDevOps
e.g. @testsuite=456
@testcase Mapped to Test-Case ID in AzDevOps (auto-populated post sync)
e.g. @testcase=789))
@sharedsteps Mapped to Shared-Steps ID in AzDevOps (used in case of Background; auto-populated post sync)
e.g. @sharedsteps=12))
@priority Mapped to Priority field in AzDevOps
e.g. @priority=2
@bvt Mapped to Priority field in AzDevOps with a value of 1 (same as setting @priority=1)
@manual Mapped to Automation-Status field in AzDevOps
e.g. @manual=planned or @manual=not-automated [same as empty/default @manual]) (spaces to be replaced with - [hyphen] in the reason/value)
@owner Mapped to Assigned To field in AzDevOps (overrides the DefaultAssignedTo value in Spex.json)
e.g. @owner=someone@something.com
@version Identifies if the Feature/Scenario needs to be re-synced to AzDevOps (if the tag value is greater than the one in AzDevOps)
e.g. @version=2
@attach File-names/paths (comma-separated) that need to be attached to the Test-case in AzDevOps
e.g. @attach=file1.txt,folder2/file2.json
@area Mapped to AreaPath field in AzDevOps (overrides the default value specified in Spex.json)
e.g. @area=My^Project\Some^Area
@iteration Mapped to IterationPath field in AzDevOps (overrides the default value specified in Spex.json)
e.g. @iteration=My^Project\Iteration^1
@ignore Features / Scenarios marked with this are ignored during sync
                            
                        dotnet tool install -g spex