Im starting to use Playwright to automate my tests for a Blazor WebAssembly application, i created a new NUnit Test project and following the docs on playwright.dev i installed:
dotnet add package Microsoft.Playwright.NUnit
doing so i can inherit the PageTest class and directly use a Page object already configured for me without needing to instantiate the browser and the context
now i need to see what happens during my tests so i wanted to enable headed mode, normally i would launch the browser with this options like this:
await playwright.Firefox.LaunchAsync(new BrowserTypeLaunchOptions
{
Headless = false,
SlowMo = 50,
});
but i cant do that here, the official docs suggest to use this commands on the console
set HEADED=1
dotnet test
but nothing happens, the tests are run headless mode regardless.
For the headless mode I first set the following in the terminal "$env:HEADED=1", so that it will open a browser. Keep in mind you have to set it to "0" if you want to run it in headless mode again.
From what I understood, after inheriting from PageTest, due to lack of config file, such things (headless/slowMo) should be set runtime from the terminal.
Use this
await using var browser = await playwright.Firefox.LaunchAsync(new BrowserTypeLaunchOptions { Headless = false, SlowMo = 50 });
var context = await browser.NewContextAsync();
var page = await context.NewPageAsync();
await page.GotoAsync("The site you want to go");
It should work
If you set the environment variable PWDEBUG before your tests run, they will run in headed mode.
e.g. by doing something like this in a one time set up.
Environment.SetEnvironmentVariable("PWDEBUG", "console");
More information about Run in Debug Mode
The playwright libary also provided test Runners frameworks like: mstest, NUnit, and Xunit.
With the testing runner framework - NUnit, you can use customizing Browser/launch options by using the run settings file - .runsettings.
See the docs: https://playwright.dev/dotnet/docs/test-runners
See it under - "customizing Browser/launch options under Nunit/mstest"
Also about .runsettings file
https://learn.microsoft.com/en-us/visualstudio/test/configure-unit-tests-by-using-a-dot-runsettings-file?view=vs-2022
for that to work with Nunit you need NUnit3TestAdapter. So your .csproj file contains:
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.3.2" />
<PackageReference Include="Microsoft.Playwright.NUnit" Version="1.28.0" />
<PackageReference Include="Microsoft.Playwright.TestAdapter" Version="1.28.0" />
<PackageReference Include="NUnit3TestAdapter" Version="4.3.1" />
</ItemGroup>
Related
I want to run my automated tests in TFS using a Custom Logger. Normally you would do this by adding something like /Logger:MyCustomLogger to the Other Console Options section of the Visual Studio Test Task.
You have the option to select tests by Assembly, Test Run ID, or Test Plan. If you select either Test Run or Test Plan, you are not able to set the Other Console Options value. Per the documentation:
Other console options that can be passed to vstest.console.exe, as documented here.
These options are not supported and will be ignored when running tests using the ‘Multi agent’ parallel setting of an agent phase or when running tests using ‘Test plan’ option. The options can be specified using a settings file instead.These options are not supported and will be ignored when running tests using the ‘Multi agent’ parallel setting of an agent phase or when running tests using ‘Test plan’ option. The options can be specified using a settings file instead.
I think thats because it uses TCM.exe instead of VSTest.Console.Exe, and TCM doesnt take the same console options, but not entirely sure.
According to the quote above, "The options can be specified using a settings file instead". My question is: What settings file allows you to provide a Logger? RunSettings doesnt support it (Though that may be addressed here).
So is there a workaround for this? Is there a way to provide a Logger while running a Test Plan?
Please check this documentation:
Runsettings via Logger node in the LoggerRunSettings section. Here
is a sample on how this can be specified:
<RunSettings>
<LoggerRunSettings>
<Loggers>
<Logger friendlyName="sampleLoggerwithParameters">
<Configuration>
<Key1>Value1</Key1>
<Key2>Value2</Key2>
</Configuration>
</Logger>
<Logger uri="logger://sample/sampleLoggerWithoutParameters1"
friendlyName="sampleLoggerWithoutParameters1" />
<Logger uri="logger://sample/sampleLoggerWithoutParameters2"
assemblyQualifiedName="Sample.Sample.Sample.SampleLogger,
Sample.Sample.Logger, Version=0.0.0.0, Culture=neutral,
PublicKeyToken=xxxxxxxxxxxxxxxx"
friendlyName="sampleLoggerWithoutParameters2" />
This loads and initializes:
Logger with friendlyName="sampleLoggerwithParameters". Key1=Value1 and Key2=Value2 are passed as dictionary parameters
to the logger while initialization. i.e.
SampleLoggerWithParameters.Initialize(TestLoggerEvents events,
Dictionary<string, string> parameters) is invoked with parameters =
{{"Key1", "Value1"}, {"Key2", "Value2"}}
Logger with uri="logger://sample/sampleLoggerWithoutParameters1". FriendlyName is ignored in this case as uri takes more precedence.
Logger with assemblyQualifiedName="Sample.Sample.Sample.SampleLogger,
Sample.Sample.Logger, Version=0.0.0.0, Culture=neutral,
PublicKeyToken=xxxxxxxxxxxxxxxx". Uri and friendlyName are ignored in
this case as assemblyQualifiedName takes more precedence.
I have added a MSBuild target to update the PublishUrl and then call the 'Publish' target, passing the new value in. This has allowed me to build multiple branches and have the corresponding ClickOnce app dropped in a branch specific share location.
<PropertyGroup>
<PublishUrl >\\someNetworkShare\</PublishUrl>
</PropertyGroup>
<Target Name="PublishClickOnce">
<PublishUrl>$(PublishUrl)$(BranchName)\</PublishUrl>
<MSBuild Projects="$(ProjectPath)" Properties="PublishUrl=$(PublishUrl); PublishDir=$(PublishUrl); Platform=AnyCPU" Targets="Publish" />
</Target>
The problem is that this just doesn't work on one of my build machines. It works perfectly fine on 2 of them, but not the 3rd.
Looking at the myApp.application file i can see that the deploymentProvider codebase points to the unchanged PublishedUrl (but it has been updated at the time of calling the 'Publish' target:
<deploymentProvider codebase="file://someNetworkShare/myApp.application" />
The above should be:
<deploymentProvider codebase="file://someNetworkShare/branchName/myApp.application" />
A few things i have tried:
Update the 'InstallUrl' to match the 'PublishUrl'
Added logging to ensure that the 'PublishUrl' has updated before calling the 'Publish' target.
Added a condition to the 'PublishUrl' to only update if it is empty
Any help would be much appreciated.
Thanks
-------------------------------- UPDATE --------------------------------
So a couple of things:
The working build machines are on W7, the failing machines are on a mixture of W7 and W8.
It turns out that you can exclude the deploymentProvider element (ProjectProperties -> Publish -> Manifest -> Exclude deployment provider URL). If excluded the system attempts to figure it out by its self at run-time (this is sufficient in most cases).
So I tested the build on a few more machines (a completely clean W7 VM, a W7 dev machine, W8.1 dev machine) and they both produced the incorrect deploymentProvider line.
Still curious to the actual root cause of the issue so I don't want to mark it as answered.
For some specific purpose, I need to install some fonts on the instances. It comes as no surprise when I choose StartUp Task to accomplish that goal. I've configured the Service Definitions as below:
<Startup>
<Task commandLine="Fonts\InstallFonts.vbs" executionContext="elevated" taskType="simple" />
</Startup>
Nothing special here. Click and run, it failed. However, if I changed the commandLine into a cmd file including just nonsense, namely "echo test", the instance would run without ado. So there must be some issue with my scripting:
Const FONTS = &H14&
Set objShell = CreateObject("Shell.Application")
Set objFolder = objShell.Namespace(CreateObject("Scripting.FileSystemObject").GetAbsolutePathName("."))
Set fontFolder = objShell.Namespace(FONTS)
Set rxTTF = New RegExp
rxTTF.IgnoreCase = True
rxTTF.Pattern = "\.ttf$"
Set fso = CreateObject("Scripting.FileSystemObject")
FOR EACH FontFile IN objFolder.Items()
IF rxTTF.Test(FontFile.Path) THEN
IF NOT fso.FileExists(fontFolder.Self.Path+"\\"+FontFile.Name) THEN
FontFile.InvokeVerb("Install")
END IF
END IF
NEXT
The script should come with no error because I've tested it either locally or on Azure via RDP.
Weirdly, when I put it in the startup, the role just won't start. The instance just keeps recycling and at last says "I'm unhealthy". Even if I deprecate the vbs into just one line of code - the first line Const FONTS = &H14&, it just won't start. Even if I wrap the invocation of the vbs into a cmd file, namely to put something like "cscript /B file.vbs", it won't run either.
So I'm concluding that there must be some issue regarding the communication between the script and the Windows Azure monitor. I'm not sure but I think the monitor might take the running script as a failed task. Besides, I'm wondering if there is any timeout for the startup task, which should be the problem though, because the script can guarantee that no UI interaction block the process.
Any idea would be greatly appreciated.
I am sure you must have but just for the sake of confirmation, have you checked that the InstallFonts.vbs file is exported with the package? I mean is the "Copy To Output Directory" is set to "Copy Always/Copy if newer"?
This is pretty much possible that it is not able to locate your file.
You need to write a cmd file as a start up task. In your cmd file, you can call the vbs file using the command line tool cscript.
Azure start up can compile only command line tools.
Oh god, I finally solved the problem.
Although the compiler does quite a good job usually, it allows to use subfolder as a source of command, I mean something like "Subfolder\command.cmd", which will not work always. I've seen examples in which people put whatever we do in cmd in commandLine property, such as "copy fileA fileB" and it really works. But as for vbs, you need to be cautious. Until now I still don't know what's under the cover, but there should be some problem with the path. And the solution is definitely simple, instead of doing the subfolder work for tidiness, just leave the command file in the root folder like most people do:
<Startup>
<Task commandLine="InstallFonts.vbs" executionContext="elevated" taskType="simple" />
</Startup>
And thank you all the same, Kunal. :)
How to get build definition from TFS and pass it to the external program
This is what we are doing manually:
1) Queue new build
2) Once build is completed go to the drop folder and get the exe name
3) pass this exe name to the test automation program and run it.`
I want to automate these 3 steps.
Is it possible to get the build definition programatically?
Create a custom Build Template. Use a copy of the default (or what ever you're using now) as your starting point. Look in the work flow where BuildDetail.CompilationStatus = BuildPhaseStatus.Succeeded. You will then have the opportunity to invoke another application, it would be a stub program/powershell script/any other executable process. you can pass the path of the build that you just completed by using BuildDetail.DropLocation.
Assuming that your step #1 has executed, this latest (successful!) build is reachable as the lastKnownGoodBuild of the specific build definition.With this in mind you can employ a console app that bases on the following:
using System;
using System.IO;
using Microsoft.TeamFoundation.Build.Client;
using Microsoft.TeamFoundation.Client;
namespace BuildDropLocation
{
class Program
{
static void Main()
{
TfsTeamProjectCollection teamProjectCollection = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(new Uri("http://yourTFSServerUri"));
var buildService = (IBuildServer)teamProjectCollection.GetService(typeof(IBuildServer));
IBuildDefinition myBuildDefinition = buildService.GetBuildDefinition("TeamProjectName", "BuildDefinitionName");
Uri lastKnownGoodBuild = myBuildDefinition.LastGoodBuildUri;
IBuildDetail myBuildDetail = buildService.GetBuild(lastKnownGoodBuild);
string[] myExeFiles = Directory.GetFiles(myBuildDetail.DropLocation, "*.exe",SearchOption.AllDirectories);
foreach (var exeFile in myExeFiles)
{
Console.WriteLine(myExeFile);
}
}
}
}
With the above you can retrieve the path to any *.exe under the drop location of the last build of build definition BuildDefinitionName that lives in Team Project TeamProjectName.This approach allows you to fully separate your TFS-Build with the execution of your tests. You can, for example, schedule this console-app to execute every night and invoke your runner to operate on the latest successful build.In case you would like the build and the testrun to be coupled in any way, you should proceed as #TimWagaman suggests by invoking your test runner during build. This 'coupling' might include:
The test results are contained in the build log
A failure generates a Bug
Test coverage is reportable
In this case, your tests will execute with each and every build that doesn't break in the compilation phase.
<MakeDir Directories="$(TemporaryFolder)" />
<Exec Condition=" '$(IsInTeamBuild)'=='True'" Command=""$(TfsTask)" history ../ /r /noprompt /stopafter:1 /version:W > "$(TemporaryFolder)\grab-changeset.txt"" />
<Exec Condition=" '$(IsInTeamBuild)'=='True'" Command=""$(TfsTask)" properties "$(MyMSBuildStartupDirectory)\all-companies-run-after-update.js" > "$(TemporaryFolder)\grab-properties.txt"" />
We use the above to extract: build#, branch, revision#
from the generated .txt files.
Here's some considerations when installing onto IIS-6:
Needs to register ASP.NET 4 (likely using aspnet_regiis.exe)
Needs to allow for both ASP.NET v2 and v4
Needs to register aspnet_isapi.dll with support for wildcard mapping
And here's what I have so far:
<iis:WebDirProperties Id='WebDirProperties' Script='yes' Read='yes
Execute='no' WindowsAuthentication='yes' AnonymousAccess='no'
AuthenticationProviders='NTLM,Negotiate' />
<!-- SO has some good posts on selecting the website from a dropdown -->
<iis:WebSite Id='SelectedWebSite' Directory='WWWROOT' SiteId='[WEBSITE_ID]' Description='[WEBSITE_DESCRIPTION]'>
<iis:WebAddress Id='AllUnassigned' Port='80' IP='*'/>
</iis:WebSite>
<Component Id="ProjWebApp" Guid="{B4BE9223-7109-4943-AE4E-8F72FA350D02}"
Win64="$(var.IsWin64)" NeverOverwrite="yes" Transitive="yes">
<CreateFolder/>
<iis:WebAppPool Id="ProjAppPool" Name="[APPPOOLNAME]" Identity="networkService"
ManagedRuntimeVersion="v4.0" ManagedPipelineMode="integrated" />
<iis:WebVirtualDir Id="ProjVDir" DirProperties="WebDirProperties"
Alias="[WEBAPPNAME]" Directory="WEBFILESDIR" WebSite="SelectedWebSite">
<iis:WebApplication Id="ProjApp" Name="[WEBAPPNAME]" WebAppPool="ProjAppPool">
<iis:WebApplicationExtension
CheckPath="no"
Script="yes"
Executable="[ASPNETISAPIDLL]"
Verbs="GET,HEAD,POST"
/>
</iis:WebApplication>
</iis:WebVirtualDir>
</Component>
<!-- other apps may start using it once installed so it must be permanent -->
<Component Id="EnableASPNet4Extension" Permanent="yes" Guid="{C8CDAB96-5DDC-4B4C-AD7E-CD09B59F7813}">
<iis:WebServiceExtension Id="ASPNet4Extension" Group="ASP.NET v4.0.30319"
Allow="yes" File="[ASPNETISAPIDLL]" Description="ASP.NET v4.0.30319"
UIDeletable="no"
/>
</Component>
And I have a custom action to register ASP.NET with IIS:
<?if $(var.Platform) = x64 ?>
<CustomAction Id="SetProperty_AspNetRegIIS_InstallNet40Cmd"
Property="AspNetRegIIS_InstallNet40Cmd"
Value=""[NETFRAMEWORK40FULLINSTALLROOTDIR64]aspnet_regiis.exe" -ir"/>
<?else?>
<CustomAction Id="SetProperty_AspNetRegIIS_InstallNet40Cmd"
Property="AspNetRegIIS_InstallNet40Cmd"
Value=""[NETFRAMEWORK40FULLINSTALLROOTDIR]aspnet_regiis.exe" -ir"/>
<?endif?>
The Problem
This almost works. There are two problems at this point:
The IIS extension doesn't respect the managed runtime version on IIS-6, so the application doesn't have an ASP.NET version set.
If I use aspnet_regiis.exe -s APP_PATH to register it once it's created, it overwrites the wildcard mapping (and I don't know a commandline that I can run to restore it).
Given the above shortcomings, how can I use WIX to install an ASP.NET MVC 3 application onto IIS-6 with a proper wildcard mapping when it already has ASP.NET 2 installed?
It turns out this was a dumb error on my part. The above is sufficient to make ASP.NET v4 applications work when the pieces that I didn't include (custom actions and property definitions) are correct.
In my case, I had accidentally quoted the path to the aspnet_isapi.dll, so it wasn't actually being picked up correctly.
The IIS extension doesn't respect the managed runtime version on IIS-6, so the application doesn't have an ASP.NET version set.
That's partially true. Although it doesn't use the managed runtime version when setting the App Pool, IIS actually picks up the ASP.NET version as soon as something is correctly mapped to the aspnet_isapi.dll. As soon as I fixed the path, everything worked correctly.
If I use aspnet_regiis.exe -s APP_PATH to register it once it's created, it overwrites the wildcard mapping (and I don't know a commandline that I can run to restore it).
You can use adsutil.vbs in order to manage this if need be:
C:\Inetpub\AdminScripts>adsutil.vbs enum w3svc/998577302/root/AppName
KeyType : (STRING) "IIsWebVirtualDir"
...
ScriptMaps : (LIST) (1 Items)
"*,C:\WINDOWS\Microsoft.NET\Framework64\v4.0.30319\aspnet_isapi.dll,1,GET,HEAD
,POST"
By using the set command in adsutil.vbs you can set the ScriptMaps property as needed.