A "no-code" (yet extensible) Azure API Performance benchmarking tool (for Developers) based on App-Insights (request-duration) - with multiple output-formats and customizations

💡 If App-Insights is configured, the durations are taken from Request-logs; else, from local timestamps

installation

                
 # Install from nuget.org
 dotnet tool install -g perfx --no-cache

 # Upgrade to latest version from nuget.org
 dotnet tool update -g perfx --no-cache

 # Install a specific version from nuget.org
 dotnet tool install -g perfx --version 1.0.x

 # Uninstall
 dotnet tool uninstall -g perfx
          
        

🔰 If you don't have .NET Core 3.1 installed, download and install it.

🔰 If the Tool is not accessible post installation, add %USERPROFILE%\.dotnet\tools to the PATH env-var.

configuration

Populate the following JSON and save it to your Documents/Perfx folder with the name: Perfx.Settings.json (or something like MyApp1.Settings.json)

                
 {
   "Tenant": "YOUR_COMPANY_NAME",
   "UserId": "",
   "Password": "",
   "ClientId": "",
   "ClientSecret": "",
   "ResourceUrl": "",
   "ReplyUrl": "",
   "ApiScopes": [
     "api://YOUR-API-SCOPES"
   ],
   "FormatArgs": {
     "api": "https://YOUR-{env}-API.COM"
   },
   "Endpoints": [
     "{api}/route1",
     "{api}/route2"
   ],
  "ResponseTimeSla": 5,
  "ResponseSizeSla": 200,
  "Iterations": 5,
  "OutputFormats": [
    "Excel",
    "Sql::Conn-string..."
  ],
  "AppInsightsAppId": "",
  "AppInsightsApiKey": "",
  "ReadResponseHeadersOnly": false,
  "InputsFile": "Perfx_Inputs.xlsx",
  "PluginClassName": null
 }
          
        

details

Order of Authentication:

• If Tenant is null or empty > No Authentication
• If Password is provided > Silent User-credentials authentication flow
  AcquireTokenByUsernamePassword(scopes, username, password)
• If ClientSecret is provided > Client-credentials authentication flow
  AcquireTokenAsync(resource, clientCredential)
• If ReplyUrl is provided > Interactive User-credentials authentication flow
  AcquireTokenAsync(resource, clientId, redirectUri, parameters)

logging

You can (optionally) populate the following JSON and save it to your 'Documents/Perfx' folder with the name: Perfx.Logging.json

                
 {
   "Logging": {
      "LogLevel": {
        "Default": "Warning"
        },
       "Console": {
        "IncludeScopes": true,
        "LogLevel": {
         "Default": "Warning"
       }
     },
     "Debug": {
       "LogLevel": {
         "Default": "Information"
       }
     }
   }
 }
          
        

"System.Net.Http.HttpClient": "Information"

inputs

Additional details (e.g. http-method (defaults to GET) / headers / body / query-params) for the Endpoints defined the Perfx.Settings.json (default) under Documents/Perfx can be provided in Perfx_Inputs.xlsx under the same folder (or using the Plugin model outlined below)

By default, if an entry/row for an Endpoint exists in Perfx_Inputs.xlsx, the corresponding row-index of that entry is considered for the specific iteration. If the number of entries/rows for an Endpoint do not match up with the number of iterations, the first entry is taken into consideration for the subsequent iterations

outputs

Results are saved to your Documents/Perfx with the name of the input settings JSON file (default is Perfx.Settings.json).
e.g. Perfx_Results.xlsx / Perfx_Results.csv & Perfx_Stats.csv / Perfx_Results.json & Perfx_Stats.json / [dbo.][Perfx_Results] & [dbo].[Perfx_Stats] Sql tables (depending on the specified OutputFormat in the settings file

usage

 perfx [MyApp1.Settings.json]

• If the specified path or the current-dir contains a valid Settings file, the tool uses/creates the Perfx folder under that path (for processing inputs/outputs); else, it falls back to the Perfx folder under My Documents
• If no argument is supplied, Perfx.Settings.json is used as input by default
• Set QuiteMode to true in the Settings file to run once and exit (e.g. DevOps Pipelines)

• Enter r:10 to run the benchmarks (10 times)
• Enter s to print the stats/details for the previous run
• Enter l:1h:10 to fetch app-insights request-duration logs for the previous run (in the last 1 hour with 10 retries)
• Enter c to clear the console
• Enter q to quit
• Enter ? to print this help

                        
                    dotnet tool install -g perfx
                    

The exit-code / %errorlevel% of the app will be equal to the count of results that match the following criteria:
90th-percentile of requests > response-time-sla (OR)
max-response-size > response-size-sla (OR)
status-codes other than 200 > 1%

plugins

• Create a .NET Standard project and add reference to Prefx.Core project
• Add a class that implements IPlugin interface (which in turn implements IOutput interface)

                    
 public Task IPlugin.GetAuthToken(Settings settings)
 {
     // NOTE: By default Perfx uses IPublicClientApplication's AcquireTokenSilent/AcquireTokenByUsernamePassword/AcquireTokenAsync (see 'Order of Authentication' note in the docs)
     // If you want to override that behavior and provide a custom implementation, go ahead...
     // If not, throw NotImplementedException or NotSupportedException, to trigger the default implementation
     var userId = settings.UserId;
     var pwd = settings.Password;

     // Get more settings as required...

     return Task.FromResult("someToken1");
 }

 public Task> IPlugin.GetEndpointDetails(Settings settings)
 {
     // NOTE: By default Perfx uses Documents/Perfx/Perfx_Inputs.xlsx
     // If you want to override that behavior and provide a custom implementation, go ahead...
     // If not, throw NotImplementedException or NotSupportedException, to trigger the default implementation
     var endpointDetails = new List();
     foreach (var endpoint in settings.Endpoints.Select((e, i) => (url: e, index: i)))
     {
         if (endpoint.url.Contains("odata"))
         {
             // Do whatever - based on the endpoint
             endpointDetails.Add(new Endpoint { Method = HttpMethod.Get.ToString(), Query = "?$top=10" });
         }
         else if (endpoint.url.EndsWith("route1"))
         {
             // Do whatever - based on the endpoint
             endpointDetails.Add(new Endpoint { Method = HttpMethod.Get.ToString(), Query = "/1" });
         }
     }

     return Task.FromResult(endpointDetails);
 }

public Task IOutput.Save(IEnumerable results, Settings settings)
{
    //  If you want to override that behavior and provide a custom implementation, go ahead...
    //  If not, throw NotImplementedException or NotSupportedException, to trigger the default implementation
    // Do something and return true
    return Task.FromResult(false);
}

public Task> IOutput.Read(Settings settings)
{
    //  If you want to override that behavior and provide a custom implementation, go ahead...
    //  If not, throw NotImplementedException or NotSupportedException, to trigger the default implementation
    // Do and return something
    return Task.FromResult(default(IList));
}
                        
                    

• Update the csproj file as follows:

                          
   ...
   <TargetFramework>netcoreapp3.1</TargetFramework>
   <Platforms>x64</Platforms>
   <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
   ...
   <ProjectReference Include="Perfx.Core.csproj">
   <Private>false</Private>
   <ExcludeAssets>runtime</ExcludeAssets>
   </ProjectReference>
   ...
                              
                          

• Build the project and copy the build-output to Documents/Perfx/Plugins folder (Documents can be some other base-folder depending on your usage)
• Optionally, if you have multiple IPlugin implementations, you can also update the value of PluginClassName with the specific implementation-class-full-name (e.g. MyPluginAssembly.MyPlugin1)

Sample: Perfx.SamplePlugin