I am trying to run tests using SpecRun. I would like to run them using the .feature file, but so far I have not succeeded :(
I am using this as a filter:
/filter:testpath:Feature:FeatureFile.feature
but no tests are run.
How can I do to specify a .feature file in SpecRun?
Thanks
Documentation for filters are here: http://specflow.org/plus/documentation/Filter/
For your case you need to specify the parameter this way:
/filter:testpath:Feature:<FeatureTitle>
<FeatureTitle> is the title of your feature in the file. Not the file name.
Full Disclosure: I am one of the SpecFlow & SpecFlow+ developers.
Related
I am using jenkins to run jobs that uses maven with cucumber . my jobs are running succesfully. I have also managed the plugin Cucumber report in jenkins. But when i view the report i am getting the message as shown in below picture.
I have tried to install various plugins like Jenkins plugin, standalone but with no joy. These are shown as options in the bottom right of the report.
Any clue is helpful. I have tried to check this error in stackover flow also. I didnt get any clue
I could see the reports file in xml which is created by surefire plugin. Any way i can achive reporting part in my project
I managed to get reports in jenkins now.
Previously Jenkins is unable to find the json file in my project.
I have set the path of json file as target and i could able to see the reports.
This might help for those who are looking for answer.
#CucumberOptions(
features= "C:\\Users\\dd pc\\workspace\\PracticeCucumber\\src\\main\\java\\com\\qa\\feature\\Test.feature",
glue= {"com\\qa\\stepDefinition"},
format= {"pretty","html:test-output_1",
"json:target/cucumber-reports/CucumberTestReport.json"},
plugin = ("json:target/cucumber-reports/CucumberTestReport.json"),
monochrome= true,
dryRun= false
)
I am able to solve by providing **/*.json
under Post-build Actions/Cucumber Report/JSON Report Location/ File Include Pattern = **/*.json
First of all in runner file we need to provide following plugin as follows: plugin= {"json:target/cucumber.json" }
Now in Jenkins job, in "post build" section during configuration, add cucumber reports, click on advanced option and provide 'target' text in "JSON Reports Path" text field. Save the job, it will run and will work fine.
Note: target text is the name of folder which we have provided in runner file for generating reports in Json, also cucumber report plugin works on json so need to generate report in json.
I my case the problem was in two jobs over the same test repository (one for Chrome one for Firefox), where one deletes report files after the job while second job was already running and vice versa.
As far as I know, JMeter allows you to send multiple POST request with different parameters (e.g. { "value": "value1"}, {"value": "value2"}, ...) However, I'm more comfortable using a terminal-based interface similar to ab or siege. Basically, I need to load test a server simulating the case in which some requests are not previously cached.
Are there alternatives to JMeter for Linux that are able to use different parameters for a POST request?
UPDATE
As far as I can tell, JMeter requires the creation of a test plan (jmx file) in order to run via the command line. Unfortunately, this test plan needs to be built using the GUI, which is precisely what I want to avoid.
UPDATE 2
I will use JMeter because it offers dynamic parameters for POST requests and most alternatives depend on JMeter. However, if anyone knows of a standalone library that works exclusively from the terminal (similar to ab), please let me know.
you can use JMeter in terminal mode, it's called Non GUI mode.
To variabilize just use CsV dataset to load variables (varName for example )per thread, then use ${varName}
See :
http://jmeter.apache.org/usermanual/get-started.html#non_gui
http://jmeter.apache.org/usermanual/component_reference.html#CSV_Data_Set_Config
Nice report at end:
http://jmeter.apache.org/usermanual/generating-dashboard.html
If you don't want to use GUI even for building the test, then look at :
https://github.com/flood-io/ruby-jmeter
It allows you to generate the JMX from a DSL file.
Examples here:
https://github.com/flood-io/ruby-jmeter/tree/master/examples
DSL here:
https://github.com/flood-io/ruby-jmeter/blob/master/lib/ruby-jmeter/DSL.md
$LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
require 'ruby-jmeter'
test do
csv_data_set_config name:'MyCsv', filename: '/path to file', variableNames: 'myParam'
threads count: 10 do
visit name: 'Qwant Search', url: 'https://lite.qwant.com/?q=flood.io&t=web&p=${myParam}'
end
end.jmx(file: "path to your output plan")
Save file to ruby-jmeter-csv.rb
You can then generate the plan with:
ruby ruby-jmeter-csv.rb
And run it in non gui mode.
In fact JMeter GUI should be used for tests development and debugging only, when it comes to running the load test - it is recommended to run JMeter in command line mode, via Ant task or Maven plugin. Also there is a couple of more "geek" alternatives, i.e.:
JMeter .jmx scripts are basically XML files so you can use your favourite text editor to create or amend them
You can use JMeter API to create and kick off JMeter tests using Java language
If you're still looking for an alternative, here are few free and open source load testing tools
Grinder - you can write scripts in Jython
Gatling - you can write scripts in Scala-based DSL
Tsung - this guy exists for Linux and Unix-based platforms only, Erlang-based. Scripts are XML files.
Taurus - automation framework which supports all aforementioned tools (and some more), Python based, configuration files have simple YAML syntax.
See Open Source Load Testing Tools: Which One Should You Use? for more information on the above tools and comparison of them with JMeter
I have JMeter setup on Jenkins. I am running some performance test with it. On Jenkins, I have setup the TestLink plugin.
On JMeter, I have 'View Results Tree' tree setup, and export the results to a xml file. With the results with the following format:
<testResults version="1.2">
<httpSample t="81" lt="81" ts="1463588977147" s="true" lb="Unit_EdgeSpanTest_transmission-submit_Case1" rc="200" rm="OK" tn="CinemaCMSAPITest 1-1" dt="text" by="382">
<assertionResult>
<name>Response Assertion-ResponseCode</name>
<failure>false</failure>
<error>false</error>
</assertionResult>
<assertionResult>
<name>Response Assertion-TextResponse</name>
<failure>false</failure>
<error>false</error>
</assertionResult>
</httpSample>
Now I have problem get JMeter results report back to testLink. Googled, not much resource out there tell me what to do. I could only find one resource from Jenkins:https://wiki.jenkins-ci.org/display/JENKINS/Integrating+TestLink++Jenkins++JMeter
But it did not help much. I am not using Ant...
Does anyone has successfully made this work? And could give me some hint of what to do?
Thank you very much in advance!
I recall I had to submit JMeter results to TestLink some time ago. In my case each JMeter sampler represented an individual test case so I used the Beanshell Assertion and TestLink Java API Client combination. If you're not familiar with Beanshell - check out How to Use BeanShell: JMeter's Favorite Built-in Component
If you're not comfortable with coding I guess you can access the API endpoints using HTTP Request or SOAP/XML-RPC Request samplers.
There is also a number of discussions regarding the API on TestLink forum
Hope this helps.
I followed the instruction described in the question to configure the testLink, JMeter, and Jenkins. Just with one problem: I first tried to use the script from the link above, but it seems always returned the wrong result. I wrote a simple java script to transform JMeter XML test result file to TAP format. Except that the link from the question is a good instruction to follow to setup the integration between JMeter/Jenkins/TestLink:
To summarize:
From Jenkins:
install TestLink plugin
Configure TestLink from Jenkins > Configuration
Create a project > Configure:
Invoke TestLink: Put the TestLink Version/Test Project Name/Test Plan Name/Build Name/Custom Fields/Test Plan Custom Fields according to the settings from the TestLink.
Add build action to run the java function/shell script
Result Seeking Strategy: Add the tap directory in 'Include Pattern' Add JMeterTestCase to Key Custom Field.
From TestLink:
Create custom Fields,
Add the JMeterTestCase custom field to test specs. And set the test specs to 'Automated'
Assign the test specs/cases to the according test plan.
From JMeter:
Make sure the test result listener generates xml format result.
Have a script transform JMeter Test result from xml format to Tap format.
In the link from the question, it has a script for doing that, and I also attached a simple script wrote in Java for transforming JMeter Result xml file to TAP file in java:
https://github.com/yueran/jmeterResult_xmlToTap
I am using Jasmine JavaScript Tests and would like to include them on my buildserver in the cloud (TFS, visualstudio.online). Has everyone ever done that and can help me out?
Thanks and Cheers
Take a look at this article: http://blogs.msdn.com/b/visualstudioalm/archive/2012/07/09/javascript-unit-tests-on-team-foundation-service-with-chutzpah.aspx
You need to get http://chutzpah.codeplex.com/ (by NuGet), this enabled's you to run the javascript tests from the command line - then you setup your build controller properties and build definition to run js tests.
Only thing to remember, is that tests (specs) are depending on some files, and the run individually, so you need to add the
/// <reference path="
to the js specs. You could make a defaultReferenes.js with all the references, and then just add the defaultReferences to the specs.
My Jenkins job runs many tests that create log files. In case of failure, I want to look at the log of the failed test. I'd rather use Jenkins web-server to do it, even have a link in the email it sends me.
Is there any plugin that can do it? Or maybe another way?
You provide few details in your question, so it is impossible to give specific advice. In a general level: this is already possible. When your test framework creates JUnit XML files with test results, the test output can be included between the <failure> and </failure> tags. Usually test frameworks should take care of this automatically, so you are probably not using a test framework and are manually generating the XML files containing test results?
I recommend you adopt some test framework. It is usually well worth the effort.