Setting Test Runner Tool via configuration with SpecFlow - specflow

In a project using SpecFlow with this configuration:
<unitTestProvider name="MsTest.2010" />
The tests work normally when using only the default Visual Studio setup.
After installing ReSharper, the context menu commands "Run SpecFlow Scenarios" and "Debug SpecFlow Scenarios" stop working; when selected, nothing happens.
It seems this is because ReSharper sets its own test runner as default. The workaround is to open the options dialog (Tools | Options | SpecFlow | Default) and in the "Test Execution" section, set "Test Runner Tool" explicitly to "MsTest".
Unfortunately, when setting up new development machines, sometimes we forget to set this and the tests stop working after installing ReSharper, and someone has to remember this is the workaround.
The question is: would it be possible to set the "Test Runner Tool" option in the configuration file so it stays with the project, and not in the user settings? Is that what the runtimeProvider property of the unitTestProvider setting is for?

I know you asked about VS2010, but given the age of your question, I thought I'd try it in VS2012. I was able to get SpecFlow to work with MSTest without altering the Tools\Options\SpecFlow\Test Runner Tool setting (i.e. leaving it at "Auto"):
Install SpecFlow Extension to Visual Studio 2012
Select Tools\Extension and Updates
Search for "Specflow"
Install the extension
Re-start Visual Studio 2012 (SpecFlow menu items might not show up until you do this)
Create new class library project (e.g. MyProject.spec)
Add Specflow package via nuget
Right-click the project references node
Select Manage NuGet Packages
Search for "Specflow"
Select "Install"
Configure SpecFlow
Open the app.config file
Add the following entry:
Right-click the project references node
Add the following entry inside the <specFlow> node: <unitTestProvider name="MsTest" />
Create a new feature
Right-click the project node
Select Add\New Item\SpecFlow Feature File
Add the unit test assembly to the project
Right-click the project references node
Select "Add Reference"
Search Assemblies for "Microsoft.VisualStudio.QualityTools.UnitTestFramework"
Add the assembly
Generate Step Definitions
Double-click the features file to open it
Right click the white space at bottom of the features file
Select "Generate Step Definitions"
Run the tests
Select Test Explorer\Run All
Tests should run normally at this point

Related

TFS 2018 and AppCenter Test

I'm new to CI/CD and can't find any information on the internet on how to use AppCenter Test with TFS 2018. I have my Xamarin projects in TFS but I want to test them within the device cloud of AppCenter Test.
There are a couple documents on MS's docs that make it look like it's possible but for a newbie they give ZERO information on how to implement it. HERE is an example. If you look up top this document is relavant to TFS 2018.
In Azure Pipeline, you can use either YAML or the classic editor to define your pipelines. But YAML build is not supported in TFS 2018 yet.
So you need to use the classic editor to add App Center Test task to your pipeline and then refer to the Arguments in the documentation to define the task.
Here is what I did to get it working. No idea if it's correct or not, but hey, it works!
Xamarin.UITest
If you don't do the following step you'll run your tests locally and
nothing will happen, they'll just immediately exit.
First go to Tools > Options > Test > General > Active Solution then uncheck For improved performance, only use test adapters in test assembly folder or as specified in runsettings file
public class AppInitializer
{
public static IApp StartApp(Platform platform)
{
if (platform == Platform.Android)
{
return ConfigureApp.Android
// You may need to adjust the following relative path based on where you created your UITest project
.ApkFile(#"..\..\..\<AppName>\<AppName>.Android\bin\Release\<App Package Name>.apk")
// Uncomment if you are running locally and you want VS to launch/install app
//.PreferIdeSettings()
.StartApp();
}
return ConfigureApp.iOS.StartApp();
}
}
Do not add references to your Android and iOS projects like the MS
docs say. This will lead you down the path of hours of useless troubleshooting
Deploy your agent on a development windows box. Follow these instructions: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/v2-windows?view=azure-devops
TFS 2018 Configuration
In TFS click into the repository of your project
Click "Build and Release"
Up top right click "+New"
Click "Continue"
Scroll down and click "Xamarin.Android"
on the next screen set the Agent queue to whatever you created ealier, probably Default
List of my build tasks
Nuget Tool Installer
.NET Core
Nuget
Xamarin.Android
MSBuild
Android signing
Node Tool Installer
Nuget
App Center Test
Here are my build steps in detail
Nuget Tool Installer
Use Nuget
version of nuget to install : 4.9.3 (this is what I noticed my VS was using)
all else defaults
.NET Core
command: restore
path to projects: **/*.csproj
all else defaults
Nuget
Command: custom
Command and arguments: restore -MsbuildPath "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin"
All else defaults
Xamarin.Android
Project: **/*Droid*.csproj
Target: (empty)
Output directory: $(build.binariesdirectory)\$(BuildConfiguration)
Configuration: $(BuildConfiguration)
MSBuild: enable Specify Location
MSBuild Location: C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin
JDK Options, JDK 8 x64
all else defaults
MSBuild
Project: (fit this to match your uitest project): **/*UITest*.csproj
MSBuild version: Latest
Platform: (empty)
Configuration: $(BuildConfiguration)
MSBuild Arguments: /p:OutputPath="$(build.binariesdirectory)\$(BuildConfiguration)\test-assembly"
all else defaults
Android Signing
APK Files: $(build.binariesdirectory)\$(BuildConfiguration)\*.apk
Check sign the APK and put in your info
Jarsigner arguments: -verbose -sigalg MD5withRSA -digestalg SHA1
Check to Zipalign
Zipalign location: (empty)
all else defaults
Node Tool Installer
Version Spec: (choose whatever version you already have on your system) 10.11.0
all else defaults
Note: Prior to the next step create a new variable called
XamarinUITestVer and set the value to whatever value you want to use
for the Xamarin.UITest Nuget package you want to install. In my case
the value I set was: 2.2.7
Nuget
Command: custom
Command and Arguments (refer to note above): install Xamarin.UITest -Version $(XamarinUITestVer) -OutputDirectory "$(Agent.BuildDirectory)\Nuget"
all else defaults
App Center Test
Binary Application File Path: $(build.binariesdirectory)\$(BuildConfiguration)\*.apk
Artifacts Directory: $(Build.ArtifactStagingDirectory)\AppCenterTest
Check: 'Prepare Tests`
Test Framework: Xamarin UI Test
Build Directory: $(build.binariesdirectory)\$(BuildConfiguration)\test-assembly
Store File: (empty)
Store Password: (empty)
Key Alias: (empty)
Key Password: (empty)
Test Tools Directory: $(Agent.BuildDirectory)\Nuget\Xamarin.UITest.$(XamarinUITestVer)\tools\
Signing Information: (empty)
Additional Options: (empty)
Check: Run Tests
Authentication Method: App Center Connection
App Center Connection: (create a new connection)
App Slug: obtain this by "almost" creating a test in AppCenter. From your app in AppCenter create a New Test Run, select device(s), select Next and choose Xamarin.UITest and hit Next. Down below you'll see the --app string and this is what you use for this field.
Devices: obtain this by "almost" creating a test in AppCenter. From your app in AppCenter create a New Test Run, select device(s), select Next and choose Xamarin.UITest and hit Next. Down below you'll see the --devices string and this is what you use for this field.
all else defaults

TFS Build: accessing custom MSBuild file within sequence control flow

I am having hard times attempting to perform the simplest operation: run the custom MSBuild script (placed within dedicated "BuildScripts" folder) from the Lab test running build definition. What it is supposed to do prior to test execution is to trigger the child build, get the built sources and perform the modification of the config file by the means of MSBuild script. And that is what I struggle with. Supposing I have specified the build scripts folder among source settings (mapped to $(SourceDir)) and I use the MSBuild activity for running the script, what do I specify in the path for project file location?
Would appreciate any hints you might share.
Thank you.
If I understand your requirement correctly, you can select the MSBuild script file in Build Process as soon as you have uploaded the file into Version Control.
Clicking the button below:
You will get a dialog to select the solution or project you want to build:
And then click "Add..." button, you will be able to select the file from Version Control.
Never heard the "MSBuild script". If you mean to run powershell script during the build. So where do you want to specify the path for project file ? In the script or the configuration of the build template ? Suggest you to provide a screenshot.
Moreover, you can use "invokeprocess" workflow activity to invoke powershell script during the build.

TFS 2013 Build Output to both Server and Custom Drop Location

In TFS2013 I'm having the following dilemma, a build has always been dropped to a custom folder defined in the Build Definition. Suddenly, it is now needed to have it available online, so the "Copy build output to server" is the logic option.
However, I can't lose the "normal config", so I was wondering if it was a way to execute both Staging Locations. Tried with the .proj configuration file but I can't see a way to do it.
Any suggestions?
Thanks!
You can add a CopyDirectory activity in XAML build process template, or you can program a script to copy the outputs to a custom location in pre-build/post-build.
http://geekswithblogs.net/jakob/archive/2010/09/01/tfs-team-build-2010-how-to-place-the-build-output.aspx
https://msdn.microsoft.com/en-us/library/dn376353.aspx

running Grails integration tests from IntelliJ

I'm using IntelliJ 10.0.2 to develop a Grails project. From the Grails view, If I right-click on MyIntegrationTest.groovy and run it individually (by choosing "MyIntegration..." from the right-click menu) the tests therein pass. The tests in this class also pass when I use the "Run Grails target" feature and specify "test-app".
However, if I right-click on Tests:integration and select Run "All Tests", some of the tests in MyIntegrationTest fail because dependencies are not injected. The speed with which the tests run also suggests that the Spring app context is not being created. Is there a way to run all integration tests (or all integration & unit tests) from within IntelliJ without resorting to the Grails command line?
Thanks in advance!
If its the same case as mine, here's how I've solved it. I had a grails project imported into IDEA with .ipr project style. You can check if your project is .ipr style if you have Project.ipr, Project.iws, Project.iml and Project-grailsPlugins.iml files in your project's main directory.
Close IDEA, backup (for safety if something goes wrong) and remove these 4 files. Then open IDEA and choose New Project. Select "Import grails project from existing sources", select your sources and be sure to select .idea - project style. IDEA should create new project fine. Be sure to check Porject structure (Ctrl-Shift-Alt-S) -> Project Settings -> Modules -> Sources. You should have two directories test/unit and test/integration marked green as Test sources.
If all goes well you should be able to right-click on Tests:integration and Tests:unit project directories. There should be Run "Grails tests:integration" and Run "Grails test:unit" options that you were missing.
With Intellij you've a choise to either run tests as Grails:integration, Grails:unit tests or to run them as plain Unit tests. If you chose to run Unit tests it will work for test/unit only and won't create spring contexts for integration. Right click on test/unit and choose Run->Grails:unit or right click on test/integration and choose Grails:integration. Grails plugin for IDEA should be installed.
I encountered a very similar problem but I actually see a slightly different, almost opposite, behaviour in Idea 10.5.2. I have the '.ipr' style project setup. If I right click on the integration test class or the integration test package, I only get the standard 'Run MyIntegrationTests . . .' option and it fails as per the poster's error message (rather than actually fires up the integration context as per the poster's described behavour). However if I right click on the 'Tests: Integration' top level item in Grails view, I get the 'Run "Grails tests:integr..." option, and it works perfectly.
To me this is fine, and I do not have to rebuild my project into the 'directory' based structure.

how do I use ant build to execute exportReleaseBuild task in Flash Builder 4

I'm trying to do an Ant build with FlashBuilder 4 for an Export Release Build. There is supposed to be a new (in FB4) ant task fb.exportReleaseBuild that will execute the release build. Reference to the usage is here:
http://help.adobe.com/en_US/flashbuilder/using/WSbde04e3d3e6474c4-59108b2e1215eb9d5e4-8000.html
When I include a target
..target name="exportRelease"
fb.exportReleaseBuild project="${flexproject}" ...
I generate an error indicating that the task can't be found:
Problem: failed to create task or type fb.exportReleaseBuild
Cause: The name is undefined.
Action: Check the spelling.
Action: Check that any custom tasks/types have been declared.
Action: Check that any presetde/macrodef declarations have taken place.
I look in flexTasks.jar and it doesn't include any flex.ant.exportReleaseBuild class despite having the latest jar.
Am I wrong in assuming I can run the exportReleaseBuild from an ant script?
You have probably solved this by now, but here's the answer in case someone else needs it.
The Ant compiler is telling you that it can't find the ExportReleaseBuild task. The reason is that when Ant is run from the IDE, it runs as a separate process to Flash Builder. When you run it from the command line it runs in the same process, which is why it works.
To get this to work in the IDE:
Open the build.xml file in Flash Builder
Go to Run -> External Tools -> External Tools Configuration...
Select Ant, then click the New Launch Configuration button
Enter a name for the build task
Select the Main tab:
Browse to the location of the build file
Browse to the location of the workspace
Select the JRE tab, then tick "run in the same JRE as this Workspace" (see image below)
Click Apply, then click Run
In future you can run the the task using the icon on the toolbar:
Just came across this situation myself, and after tweeting w/ #renaun (his blog post has some info as well: http://renaun.com/blog/2010/09/command-line-build-a-flash-builder-4-premium-feature/), it's pretty much useless for a CI build, unless you want to install FB4 premium, you can't run this task.
I was able to get it to run on my OSX environment by following the Adobe docs for it, but it doesn't mention that you have to basically run it from the command line tool. The ant that is installed as a plugin w/ the premium version has an importexport-ant.jar that is being referenced by Eclipse (standalone or plugin).
The only way I got it to run was via cmd line/terminal. It is tedious, and not really useful for a CI build. I did try to just grab the jar file, place it in the right spot, reference that in my ant script and build directly, but that made my compiler angry :(
I think I cracked it by setting up a shell script, and calling the shell script from an ant task.
This way you * are * using the headless Flash Builder modus, from within Flash Builder.
Full working example at:
https://gist.github.com/1077715
Thanks for the link lordB8r, that one got me thinking : )

Resources