Our bazel builds sometimes stuck and get timeouts, so we lose all build logs when VM is killed. To find the cause, we want to use Build event protocol to see which rules started to get executed, but did not finish (usually these are memory-eager tests).
This graph from official docs shows that TargetConfigured and TargetCompleted events are the only events between rule start and finish.
But in reality bazel configures all targets at the same time, so we cannot just subtract TargetCompleted time from TargetConfigured time.
Moreover, both events do not contain any timestamp. Here is the build event file from the sample repo (truncated):
{"id":{"targetConfigured":{"label":"//:B"}},"children":[{"targetCompleted":{"label":"//:B","configuration":{"id":"f157fdcaf05e7672fa1bf535fbb2c3edb004ce9e9a7f6d84d9bf031454e2fb64"}}}],"configured":{"targetKind":"java_binary rule","tag":["__JAVA_RULES_MIGRATION_DO_NOT_USE_WILL_BREAK__"]}}
{"id":{"targetConfigured":{"label":"//:main"}},"children":[{"targetCompleted":{"label":"//:main","configuration":{"id":"f157fdcaf05e7672fa1bf535fbb2c3edb004ce9e9a7f6d84d9bf031454e2fb64"}}}],"configured":{"targetKind":"java_library rule","tag":["__JAVA_RULES_MIGRATION_DO_NOT_USE_WILL_BREAK__"]}}
{"id":{"targetConfigured":{"label":"//:step1"}},"children":[{"targetCompleted":{"label":"//:step1","configuration":{"id":"f157fdcaf05e7672fa1bf535fbb2c3edb004ce9e9a7f6d84d9bf031454e2fb64"}}}],"configured":{"targetKind":"genrule rule"}}
{"id":{"progress":{"opaqueCount":2}},"children":[{"progress":{"opaqueCount":3}},{"namedSet":{"id":"0"}}],"progress":{"stderr":"\r\u001b[1A\u001b[K\u001b[32mAnalyzing:\u001b[0m 3 targets (0 packages loaded, 0 targets configured)\n\r\u001b[1A\u001b[K\u001b[32mINFO: \u001b[0mAnalyzed 3 targets (0 packages loaded, 0 targets configured).\n\n\r\u001b[1A\u001b[K\u001b[32mINFO: \u001b[0mFound 3 targets...\n\n\r\u001b[1A\u001b[K\u001b[32m[0 / 1]\u001b[0m [Prepa] BazelWorkspaceStatusAction stable-status.txt\n"}}
{"id":{"workspaceStatus":{}},"workspaceStatus":{"item":[{"key":"BUILD_EMBED_LABEL"},{"key":"BUILD_HOST","value":"mtymchuk"},{"key":"BUILD_TIMESTAMP","value":"1598888970"},{"key":"BUILD_USER","value":"mikhailtymchuk"}]}}
{"id":{"namedSet":{"id":"0"}},"namedSetOfFiles":{"files":[{"name":"B.jar","uri":"file:///private/var/tmp/_bazel_mikhailtymchuk/3bd90847b9f03e9e5c46f99d542eb754/execroot/__main__/bazel-out/darwin-fastbuild/bin/B.jar","pathPrefix":["bazel-out","darwin-fastbuild","bin"]},{"name":"B","uri":"file:///private/var/tmp/_bazel_mikhailtymchuk/3bd90847b9f03e9e5c46f99d542eb754/execroot/__main__/bazel-out/darwin-fastbuild/bin/B","pathPrefix":["bazel-out","darwin-fastbuild","bin"]}]}}
{"id":{"targetCompleted":{"label":"//:B","configuration":{"id":"f157fdcaf05e7672fa1bf535fbb2c3edb004ce9e9a7f6d84d9bf031454e2fb64"}}},"completed":{"success":true,"outputGroup":[{"name":"default","fileSets":[{"id":"0"}]}],"tag":["__JAVA_RULES_MIGRATION_DO_NOT_USE_WILL_BREAK__"],"importantOutput":[{"name":"B.jar","uri":"file:///private/var/tmp/_bazel_mikhailtymchuk/3bd90847b9f03e9e5c46f99d542eb754/execroot/__main__/bazel-out/darwin-fastbuild/bin/B.jar","pathPrefix":["bazel-out","darwin-fastbuild","bin"]},{"name":"B","uri":"file:///private/var/tmp/_bazel_mikhailtymchuk/3bd90847b9f03e9e5c46f99d542eb754/execroot/__main__/bazel-out/darwin-fastbuild/bin/B","pathPrefix":["bazel-out","darwin-fastbuild","bin"]}]}}
{"id":{"progress":{"opaqueCount":3}},"children":[{"progress":{"opaqueCount":4}},{"namedSet":{"id":"1"}}],"progress":{}}
{"id":{"namedSet":{"id":"1"}},"namedSetOfFiles":{"files":[{"name":"libmain.jar","uri":"file:///private/var/tmp/_bazel_mikhailtymchuk/3bd90847b9f03e9e5c46f99d542eb754/execroot/__main__/bazel-out/darwin-fastbuild/bin/libmain.jar","pathPrefix":["bazel-out","darwin-fastbuild","bin"]}]}}
{"id":{"targetCompleted":{"label":"//:main","configuration":{"id":"f157fdcaf05e7672fa1bf535fbb2c3edb004ce9e9a7f6d84d9bf031454e2fb64"}}},"completed":{"success":true,"outputGroup":[{"name":"default","fileSets":[{"id":"1"}]}],"tag":["__JAVA_RULES_MIGRATION_DO_NOT_USE_WILL_BREAK__"],"importantOutput":[{"name":"libmain.jar","uri":"file:///private/var/tmp/_bazel_mikhailtymchuk/3bd90847b9f03e9e5c46f99d542eb754/execroot/__main__/bazel-out/darwin-fastbuild/bin/libmain.jar","pathPrefix":["bazel-out","darwin-fastbuild","bin"]}]}}
{"id":{"progress":{"opaqueCount":4}},"children":[{"progress":{"opaqueCount":5}},{"namedSet":{"id":"2"}}],"progress":{}}
{"id":{"namedSet":{"id":"2"}},"namedSetOfFiles":{"files":[{"name":"step1_output.txt","uri":"file:///private/var/tmp/_bazel_mikhailtymchuk/3bd90847b9f03e9e5c46f99d542eb754/execroot/__main__/bazel-out/darwin-fastbuild/bin/step1_output.txt","pathPrefix":["bazel-out","darwin-fastbuild","bin"]}]}}
{"id":{"targetCompleted":{"label":"//:step1","configuration":{"id":"f157fdcaf05e7672fa1bf535fbb2c3edb004ce9e9a7f6d84d9bf031454e2fb64"}}},"completed":{"success":true,"outputGroup":[{"name":"default","fileSets":[{"id":"2"}]}],"importantOutput":[{"name":"step1_output.txt","uri":"file:///private/var/tmp/_bazel_mikhailtymchuk/3bd90847b9f03e9e5c46f99d542eb754/execroot/__main__/bazel-out/darwin-fastbuild/bin/step1_output.txt","pathPrefix":["bazel-out","darwin-fastbuild","bin"]}]}}
So, is it possible to extract target build start time from the build event protocol (or using another method)?
On the console, if that helps, you should be able to get this information combining --subcommands (or -s) which prints commands as they are being executed. And --show_timestamps which adds timestamps to all messages emitted.
It's not the same as what you're asking for (which I am not sure adding time to build event protocol could be trivially achieved just by configuration), but it may help with the debugging quest.
I am running jmeter using jenkins, i have duration assertion of 300ms in my script. The assertion is working fine, as my jmeter result is showing error but the jenkins build still passes.
Is there any way to fail my build in case of error in jmeter result ?
You need to define Error Threshold
The Error Threshold marks the the build as unstable or failed if the amount of errors will exceed the specified value.
Use error thresholds on single build – Define error thresholds in % for the current build.
The options are in:
Configure error threshold in Performance Plugin GUI
Jenkins considers build passed when it returns 0 exit status code, you can "tell" JMeter to exit with non-zero exit code by adding JSR223 Listener and using the following code:
if (!prev.isSuccessful()) {
System.out.println("Test failure, exiting...")
System.exit(1)
}
where prev stands for parent SampleResult class instance which gives you control over parent Sampler response code, message, data, etc. Check out Top 8 JMeter Java Classes You Should Be Using with Groovy article for more information on JMeter API shortcuts available for JSR223 Test Elements
it will result in the following behaviour:
and this 1 status code will cause Jenkins build failure
Alternative solution for point 2 would be using Taurus tool as a wrapper for your JMeter test, it provides handly Pass/Fail Criteria subsystem which provides flexible way of defining custom thresholds for considering your test as failed
I have a Microsoft Visual Studio Team Foundation Server (Version 15.117.26714.0) with predefined variable $(ProjectBuildNumber).
Is there any way to increment, during build process, value of variable with minor build number by +1?
$(ProjectBuildNumber) = 663
So, that on next build it will be:
$(ProjectBuildNumber) = 664
You can't reference variables in the build number of the Build Definition. But what you can do is override the build number in the build itself. You can either use a magic log command or use my VSTS Variables Task to set the Build.BuildNumber in the build itself. The Variables Task does expand variable references. You could probably just set the value to the current value to get it expanded.
To issue the log command yourself use a batch script, PowerShell or bash to output the following specific string to the console:
##vso[build.updatebuildnumber]build number
Update build number for current build. Example:
##vso[build.updatebuildnumber]my-new-build-number
Minimum agent version: 1.88
source: https://github.com/Microsoft/vsts-tasks/blob/master/docs/authoring/commands.md
An alternative option is to use the $(Rev) option:
Build.BuildNumber = 1.1.$(Rev:.r)
That will automatically increase the variable each time the build runs.
To update a variable in a Build Definition use yet another extension:
These things combined should be able to get what you want.
In the variable section,
set the value of ProjectBuildNumber to $[counter('', 663)].
This will queue build starting with 663 as ProjectBuildNumber and increments by 1 for the subsequent queue of builds.
Unfortunately counter function (Expressions) is not available in TFS 2018. In this old version the best solution for me is to use a PowerShell script as the first Task of the build. You can than have your parameter
$(ProjectBuildNumber)
as an input argument, and place this inline script:
$ProjectBuildNumber=$args[0]
$ProjectBuildNumber++
Write-Host "##vso[task.setvariable variable=ProjectBuildNumber;]$ProjectBuildNumber"
After this Task you can use your incremented ProjectBuildNumber variable in all subsequent Tasks.
I have a TFS 2017 (version 15.105.25910.0) build which also runs tests, but one test is taking a very long time and the whole build is cancelled due to a timeout set in the 'general' tab of the build edit page. TFS log is included below. How can I check which test is faulty?
Notice the time difference between the first and second log. I assume a faulty test is being run after ReportAnalyzer_Blabla_SomethingTest, but with over 1k tests it's hard to guess which it is.
2017-08-30T11:30:09.7614471Z Passed ReportAnalyzer_Blabla_SomethingTest
2017-08-30T11:53:52.1581687Z ##[debug]FindFiles.FindMatchingFiles(rootFolder = D:\TfsBuildAgents\RmsBuild\_work\8\s\TestResults, matchPattern = *.trx, includeFiles = True, includeFolders = False
2017-08-30T11:53:52.1581687Z ##[debug]FindFiles.GetMatchingItems(includePatterns.Count = 1, excludePatterns.Count = 0, includeFiles = True, includeFolders = False
2017-08-30T11:53:52.1581687Z ##[debug]FindFiles.FindMatchingFiles - Found 0 matches
2017-08-30T11:53:52.1581687Z ##[debug]Processed: ##vso[task.logissue type=warning;code=002003;]
2017-08-30T11:53:52.1581687Z
2017-08-30T11:53:52.1581687Z
2017-08-30T11:53:52.1737949Z ##[warning]No results found to publish.
2017-08-30T11:53:52.1737949Z ##[debug]Processed: ##vso[task.logissue type=warning]No results found to publish.
2017-08-30T11:53:52.2050485Z ##[error]The operation was canceled.
2017-08-30T11:53:52.2050485Z ##[debug]System.OperationCanceledException: The operation was canceled.
Normally the faulty test should be the first test after ReportAnalyzer_Blabla_SomethingTest. But as you said, it with over 1k tests, according to the log you posted, if you didn't split the tests we can not exactly identify which test is the faulty one. In this case, I'm afraid that you have to debug that one by one.
So, yo can try to split the tests then debug them accordingly.
You can also try to check if there are any other detailed logs to track that.
See Review continuous test results after a build for more information.
I've found a messy workaround which helped me find the failing test. In all test classes (the messy part) I've added a code which appends the currently running unit test name to a file - the last entry was what I was interested in.
[ClassInitialize]
public static void ClassInitialzie(TestContext testContext)
{
// This is just an example!
File.AppendAllText("testRunLog.txt", testContext.TestName + Environment.NewLine);
}
The closes thing to "run the code before each test in the whole test project" seems to be the ClassInitialize attribute.
https://msdn.microsoft.com/en-us/library/microsoft.visualstudio.testtools.unittesting.classinitializeattribute.aspx
I'm trying to convert xaml builds to TFS 2015 builds and running into a problem where the build seems to almost complete but then trows the following error:
And just in case you can't see the image, the error is:
Finishing task: VSBuild
System.NotSupportedException: The given path's format is not supported.
at System.Security.Permissions.FileIOPermission.QuickDemand(FileIOPermissionAccess access, String fullPath, Boolean checkForDuplicates, Boolean needFullPath)
at Microsoft.TeamFoundation.DistributedTask.Agent.Common.ContextExtensions.GetExpandedPath(ILogServiceContext context, String path, String defaultPathRoot)
at Microsoft.TeamFoundation.DistributedTask.Worker.JobRunner.ResolveInputs(IJobContext context, IJobExtension jobExtension, TaskWrapper task, IDictionary`2 variables)
at Microsoft.TeamFoundation.DistributedTask.Worker.JobRunner.Run(IJobContext jobContext, IJobRequest job, IJobExtension jobExtension, CancellationTokenSource tokenSource)
Worker Worker-44f37a2f-ff1b-43a7-b619-88373a8687c0 finished running job 44f37a2f-ff1b-43a7-b619-88373a8687c0
Finishing Build
It doesn't give me any clues as to where the code is that's causing this error so I really don't know where to look. When I look in the c:\Agent_work\5\a folder it looks like the project is building just fine (although I have no way of verifying that). But all of the files seem to be there including a subdirectory created by an .exe called in the Post-build event on the last .csproj file in the solution which I never even expected to be called and run in TFS 2015! Who knew - that's great! Now if I could just get my build to start working...
I figured it out. It was simply the trailing backslash on the source folder. I had $(build.artifactstagingdirectory)_PublishedWebsites\BOTWSitecoreWeb\ when it should have been $(build.artifactstagingdirectory)_PublishedWebsites\BOTWSitecoreWeb. I just happened to guess this was it - there was no help from the logs.