I am trying to adopt Bazel for a project which was recently converted to a monorepo.
(I am VERY new to Bazel and have no mentor handy, please excuse me if the questions I ask are answered with "rtfm"; I am happy to do so, please point me to the manual.)
This monorepo contains a bunch of libs and binaries. When a certain tag is set, the goal is to build everything (as in bazel build //...), pack all the binary files into a zip and attach the created archive to a GitHub release.
I am comfortable with GitHub Actions, triggers and the releases itself, however I am not sure what an appropriate approach would be to create the archive containing all binaries via Bazel. Can anyone point me in the right direction?
You should go through below links #1 #2, #3, #4.
Related
I'm struggling to get Azure Devops Server 2019 (on prem) to build a complicated project setup of mine.
I have multiple solutions that build various BizTalk apps. Some of these solutions references some of the projects in other solutions/repos. This works fine in Visual Studio (providing everybody names their repos as the default, which they do).
To automate this build, I've created a multi-stage build pipeline that builds each solution in order.
Whilst solution 1 will build successfully, when it comes to solution 2, the msbuild tasks will not reference the outputs for the projects that have been included as a project reference in the solution.
This seems to be because these referenced projects aren't marked for build - because they wouldn't be able to build themselves without them in turn referencing their other projects in the main solutions etc.
Bundling everything into one big super solution file is not feasible.
I'm wanting msbuild to /reference the projects that have already been built in the previous step, as per being included as project references. But msbuild is a complicated beast and I cannot figure out a way of achieving this.
Is what I'm trying to achieve even possible? Can anybody point me in the right direction?
TFS / msbuild, building project references
This is a known issue about project reference in different Repos, but unfortunately it doesn't have a best answer. Because the best solution is always to have a single repository.
Git thinks of files as the content of the whole repository, not as a collection of files. Therefore this is quite hard to do. As workaround, you can consider to use Git Submodules or Git Subtree:
Check the details info from here:
Git and Visual Studio project references
Besides, the solution we are using now is to put the shared code in the NuGet package. Then, you can use the package from any repository, and you don't have to perform any unstable git settings and keep access control as they should be.
Hope this helps.
It seems that msbuild was changed to only reference project references, and that you can revert that to reference all references by setting the property OnlyReferenceAndBuildProjectsEnabledInSolutionConfiguration to true when running msbuild.
Source: https://stackoverflow.com/a/25144169
Have confirmed this works.
I've finally revisited this and done some more experimentation.
The only way, that I can see, to get TFS/msbuild to /reference other projects outputs is to mark them in the solution build configuration to build.
Now if you do this in Visual Studio and try and build a solution that has projects from another solution then the build will fail if the previous solution hasn't been fully built yet (as the other chained dependencies won't have been built). This makes sense.
But with TFS/msbuild, the build will succeed. From what I can tell there is some magic going on that ensures the dependencies across all solutions are somehow resolved. This might be luck, it might be specific behaviour (it's working for me so far).
The problem of course is that it's incompatible with Visual Studio. So I have an extra "Build" definition setup in each Solution file that has all the projects set to build.
This seems the easiest way to manage lots of interrelated solutions without having one big one.
I want to display non-code differences between current build and the latest known successful build on Jenkins.
By non-code differences I mean things like:
Environment variables (includes Jenkins parameters) (set), maybe with some filter
Version of system tool packages (rpm -qa | sort)
Versions of python packages installed (pip freeze)
While I know how to save and archive these files as part of the build, the only part that is not clear is how to generate the diff/change-report regarding differences found between current build and the last successful build.
Please note that I am looking for a pipeline compatible solution and ideally I would prefer to make this report easily accessible on Jenkins UI, like we currently have with SCM changelogs.
Or to rephrase this, how do I create build manifest and diff it against last known successful one? If anyone knows a standard manifest format that can easily be used to combine all these information it would be great.
you always ask the most baller questions, nice work. :)
we always try to push as many things into code as possible because of the same sort of lack of traceability you're describing with non-code configuration. we start with using Jenkinsfiles, so we capture a lot of the build configuration there (in a way that still shows changes in source control). for system tool packages, we get that into the app by using docker and by inheriting from a specific tag of the docker base image. so even if we want to change system packages or even the python version, for example, that would manifest as an update of the FROM line in the app's Dockerfile. Even environment variables can be micromanaged by docker, to address your other example. There's more detail about how we try to sidestep your question at https://jenkins.io/blog/2017/07/13/speaker-blog-rosetta-stone/.
there will always be things that are hard to capture as code, and builds will therefore still fail and be hard to debug occasionally, so i hope someone pipes up with a clean solution to your question.
I have questions about how to best integrate Tensorflow with my team's existing iOS application.
I am currently adding Tensorflow to an existing iOS project. I built the library for iOS following the instructions in the makefile README.md, and was able to compile successfully.
I have now been trying to follow the instructions here to get the library integrated. These instructions tell you to add search paths to a number of folders in your Tensorflow build, which will cause problems as I need this project to be easily sharable with my team in git. I would prefer to not have to force anyone else working on this to run a 20-plus minute build before they can get up and running.
The options I've considered are below:
Option 1: Embed the entire Tensorflow library
The problems here would be making the size of the project enormous. In addition, the build time will be significant and unnecessary. How often would we build to keep updated? I would like to avoid this.
Option 2: Link to separate Tensorflow project
With this option we would statically link to a peer project and have the same structure on each developer's machine. This seems to imply we'd have to have each team member pull and build the Tensorflow library before being able to build our project at all. Is there a way with this option to copy over only the necessary output files when they're updated by someone who has built Tensorflow while at the same time not requiring developers to build Tensorflow on their machine?
I'm also curious if there is a 3rd option I haven't considered here.
The usual way I think about this is by treating TensorFlow as a framework. That means that you need to build a copy once, and can then share the resulting files as something you can distribute to other developers in your organization.
You should just be able to zip up the folder you've built TF in (the directory you cloned from github) after you've built it, and then have other developers install it in a known location on their machine. I think this is what you're saying with option #2, but the difference is that other developers can avoid building it, unless they want to.
This also has the advantage that you can make sure your team members are all on the same TensorFlow version, to make debugging a bit easier.
Does that help?
I'm currently looking at a TFS build server setup & I was trying to set up a process whereby I can set up a build template to build to a folder based on the version number of a .NET assembly that's part of the build (As per the assemblyinfo.cs file). I've got it building to the standard looking folder ("Release_20130502.1"), but that's not exactly useful in 4 months time when we want to find the build for the v1.1.0 release.
Basically I want to make a special build template which will create a major release, and I'd like the folders it makes to be more noticable as versions rather than timestamps. I'm also hoping to automatically label the release as that version too. I know how to copy the files & make labels, but I'm not sure how to get at the version numbers.
I'd also love if I could get this into the Build Name recorded in TFS somehow but I suspect that might be a bit optimistic.
Does anyone have any idea how to do this (Or alternatively any other technique that'll get me easily recognizable release version builds)?
Take a look at build number format property for build definitions this is used to generate the build number and in turn the build folder during build and label in sourcecontrol if its enabled, you can modify this manualy to pass desired build number. The mentioned drop folder can be manually given as well. And you can always copy the folder after build by hand.
You can use revision variable to create something that works similiar to what you want, Revision number gets incremented if there is build with same name in system
Format: Release v1.$(Rev).0 - this would give you Release v1.1.0, Release v1.2.0, ... on each build
You will have to customize build definition with custom activity for your defined goal to work without manual interaction. However with details you have provided this approach has issues - each project has its own assembly info - which one do you use, what if they are different? This should get you started.
http://www.ewaldhofman.nl/post/2010/04/20/Customize-Team-Build-2010-e28093-Part-1-Introduction.aspx
You can take a look at these may find something useful
http://tfsbuildextensions.codeplex.com/
It all comes down how often do these build take place, if its week or more then doing it by hand is perfecly valid aproach in my book.
We have an automated build and QA process for our software, using tfs/teambuild and msbuild, and we want to be able to know (for audit purposes) whether a component has gone through that process or not.
For example, if a library is installed on a user's machine, I'd like to be able to inspect it in some way to tell that it went through the build. In particular, I want to be able to distinguish it from components built directly on a developer's machine, and then manually installed.
What is the best way to do this? Code signing as part of the build process seems closest to these requirements, but presumably this would not cover any 3rd-party libraries that might be used? I also read about the ILMerge tool to merge all assemblies into one, but then I don't know enough to work out whether they can then be signed or not?
I'm sure we're not the first people to have the requirement, so casting around for any ideas or hints from others who might have done such a thing
Thanks!
Our developer builds are set to keep the versions at "0.0.0.0", but our build server marks the build based on a pre-configured version and automagically generated build string. "1.0.3.xxx". Your build server doesn't allow for this?
Your build process should be updating each of your projects assemblyinfo.cs files (or a global linked equivalent), you can do this with the TFS changeset number, so like the previous poster indicated you end up with the property on each dll of 1.0.changeset.buildno or something similar. You can do this easily in msbuild.
You could have the values of each assembly info file set in source control to be something obvious like 0 or 999.
A lot of what your asking is about process and training as well though.
If your using installers or zips to package your deliverables then you can also label them with the build number as part of your build process.
But if you have changeset you have the link from dll to code, so traceable, coupled with links to third party dll references as defined in each csproj.