Jenkins does not work properly on Mac M1 agent - jenkins

We want to use a Mac Mini with the M1 processor as an agent in our CI pipeline to make sure that developers with new Macs can compile the project. We duplicated the setup that we have for x86, but cmake cannot find libraries such as boost:
CMake Error at /opt/homebrew/Cellar/cmake/3.19.7/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:218 (message):
Could NOT find Boost (missing: Boost_INCLUDE_DIR container system
date_time)
Weirdly, this is not an issue when connecting to the machine directly via ssh. Even manually setting the environment variables does not change anything.

The Jenkins agent uses Java. If you install Java via brew (e.g., adoptopenjdk8), you might get an x64 version (as of March 2021) that is run using the Rosetta translation layer. You can verify this by executing sysctl sysctl.proc_translated. In the "normal" SSH connection, this should return 0, when executed as sh "sysctl sysctl.proc_translated" in Jenkins, you will see a 1.
Preferred Option (install native JDK)
Instead of using, e.g., adoptopenjdk8, you can install the zulu JDK, which is available natively, via
brew install --cask zulu
After this, you should be able to verify that proc_translated is 0 within Jenkins. You can verify that a native binary is built as described below.
Hotfix (without changing the JDK)
The following is a hotfix if you cannot install a native JDK.
We were able to force cmake to run natively (and thus to find the native libraries) by modifying our Jenkinsfile as follows:
old:
sh "cmake .."
sh "make hyriseTest"
new:
sh "PATH=/opt/homebrew/bin:$PATH arch -arm64 cmake .."
sh "arch -arm64 make hyriseTest"
Having to repeat the arch command over and over is not really pretty. Maybe someone has a better solution.
Verification of the result
We can verify that our generated binary is a native ARM binary:
sh "file hyriseTest"
Result:
+ file ./hyriseTest
./hyriseTest: Mach-O 64-bit executable arm64
Compiler warnings
Note: We are still seeing linker warnings about "ltmp3" and "ltmp4". I have not found any information about that online and it does not seem to affect the correctness of the result. Example:
ld: warning: direct access in function 'ltmp4' from file 'CMakeFiles/hyriseMvccDeletePlugin.dir/Unity/unity_0_cxx.cxx.o' to global weak symbol

Related

Using ZMQ on iOS

I am trying to build an iOS app that is exposed to a fairly extensive Rust project, but I am having issues with one of the dependencies in the project. The Rust project has a ZMQ dependency, and while I am able to build the project on my Mac, when I try to incorporate the Rust project I get the following error:
error: failed to run custom build command for `zmq-sys v0.8.3`
Caused by:
process didn't exit successfully: `/Users/lisa/company/project/project_detail/target/debug/build/zmq-sys-16dfc88dff84d855/build-script-build` (exit status: 101)
--- stdout
cargo:rerun-if-env-changed=LIBZMQ_NO_PKG_CONFIG
cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS_aarch64-apple-ios
cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS_aarch64_apple_ios
cargo:rerun-if-env-changed=TARGET_PKG_CONFIG_ALLOW_CROSS
cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS
cargo:rerun-if-env-changed=PKG_CONFIG_aarch64-apple-ios
cargo:rerun-if-env-changed=PKG_CONFIG_aarch64_apple_ios
cargo:rerun-if-env-changed=TARGET_PKG_CONFIG
cargo:rerun-if-env-changed=PKG_CONFIG
cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_aarch64-apple-ios
cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_aarch64_apple_ios
cargo:rerun-if-env-changed=TARGET_PKG_CONFIG_SYSROOT_DIR
cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
--- stderr
thread 'main' panicked at 'Unable to locate libzmq:
pkg-config has not been configured to support cross-compilation.
Install a sysroot for the target platform and configure it via
PKG_CONFIG_SYSROOT_DIR and PKG_CONFIG_PATH, or install a
cross-compiling wrapper for pkg-config and set it via
PKG_CONFIG environment variable.', /Users/lisa/.cargo/registry/src/github.com-1ecc6299db9ec823/zmq-sys-0.8.3/build.rs:31:17
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
I am trying to build the appropriate library for the iOS and iOS simulator architectures without relying on pkg-config, but if someone can provide useful instructions for configuring pkg-config for cross-compilation, I am not opposed to it, I just don't know how and can't find any up to date resources.
I have cloned the libzmq project from this github: https://github.com/zeromq/libzmq.git, then navigate into libzmq.
Inside libzmq, I run ./autogen.sh followed by ./configure --prefix [directory location] --target "aarch64-apple-ios", then make and make install.
The above procedure without the --target flag is how I got the project to compile on my Mac in the first place, and with the target flag in place make and make install both complete successfully, but when I try to build the project using PROJECT_DIR="${PWD}/ProjectName" ./ProjectName/build-rust.sh,
I get the previously posted error.
I am following the Swift-Bridge Book (https://chinedufn.github.io/swift-bridge/building/xcode-and-cargo/index.html) to try to load the Rust libraries into Xcode.
One thing to note: In the libzmq git hub, there is a folder called builds/ios/ and inside this folder is what looks to be a build file for all of the iOS archs, but I haven't been able to get the repo to build with that file. I think figuring out how to use the build_ios.sh build file as opposed to the standard build.sh provided might be the answer, but I've tried dropping build_ios.sh where build.sh is, and ./autogen.sh doesn't work, because it complains that it can't find configure.ac, even though this file does exist.

Espressif tool chain broken after macOS Monterey 12.3 beta 3 install

After Monterey 12.3 beta 2 installed, the Espressif tool chain is broken. If you compile with platformIO you'll get the following error:
^
Linking .pio/build/esp32dev/firmware.elf
/Users/stephanedeluca/.platformio/packages/toolchain-xtensa32/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/bin/ld: /Users/stephanedeluca/.platformio/packages/toolchain-xtensa32/bin/../libexec/gcc/xtensa-esp32-elf/5.2.0/liblto_plugin.so: error loading plugin: dlopen(/Users/stephanedeluca/.platformio/packages/toolchain-xtensa32/bin/../libexec/gcc/xtensa-esp32-elf/5.2.0/liblto_plugin.so, 0x0002): Library not loaded: /opt/osxcross/target/bin/../x86_64-apple-darwin12/lib/libstdc++.6.dylib
Referenced from: /Users/stephanedeluca/.platformio/packages/toolchain-xtensa32/libexec/gcc/xtensa-esp32-elf/5.2.0/liblto_plugin.0.so
Reason: tried: '/opt/osxcross/target/bin/../x86_64-apple-darwin12/lib/libstdc++.6.dylib' (no such file)
collect2: error: ld returned 1 exit status
*** [.pio/build/esp32dev/firmware.elf] Error 1
Looking at my Mac, there is no /opt/osxcross.
I have searched on another Mac with the beta 1 where the tool chain was still working the presence of the folder, and guess what?! there's none!
Any idea what as actually broken by the beta?
Finally, here is a work around: we are going to provide the missing /opt/osxcross folder and contents.
If you do not have gcc installed (this should be unlikely though, install gcc to get the folder content, as follows:
$ brew install gcc
Create the missing /opt/osxcross folder as follows:
$ sudo mkdir -p /opt/osxcross/target/bin/../x86_64-apple-darwin12/lib/
Note that you'll be prompted for your password.
And, finally, copy the folder contents from the gcc build as follows:
$ sudo cp /usr/local/Cellar/gcc/11.2.0_3/lib/gcc/11/libstdc++.6.dylib /opt/osxcross/target/x86_64-apple-darwin12/lib/libstdc++.6.dylib
This completes the fix.
You must restart vscode, Arduino IDE or whatever tool you use. And then build your project, which should compile successfully.
And voila.

How to use meson to build glib

I need to upgrade glib for a specific project. It currently uses glib 2.28.8. I have three problems.
I've never used meson and ninja before, so I checked glib's INSTALL.in and it just said to run meson _build followed by ninja -C _build. So I ran meson _build and got the following output:
$ meson _build
The Meson build system
Version: 0.47.2
Source dir: /srv/devel/build/glib-2.65.0
Build dir: /srv/devel/build/glib-2.65.0/_build
Build type: native build
meson.build:227: WARNING: Identifier 'in' will become a reserved keyword in a future release. Please rename it.
meson.build:227:14: ERROR: Expecting eol got id.
if vs_crt_opt in ['mdd', 'mtd']
So the basic build doesn't work. Why?
For our purposes, we use the following configure command:
PKG_CONFIG_PATH=$(OUTPUT_DIR)/lib/pkgconfig ./configure --prefix=$(OUTPUT_DIR) --disable-dtrace --disable-selinux ac_cv_path_MSGFMT=/bin/true CPPFLAGS="-fPIC -I$(OUTPUT_DIR)/include" LDFLAGS="-L$(OUTPUT_DIR)/lib" --enable-static --disable-shared
How do I specify that in meson?
I will also need to build in Windows. Any gotchas there?
Thanks!
EDIT: I tried older versions of glib, going back to 2.62.0 and when I run meson _build I get the error meson.build:1:0: ERROR: Meson version is 0.47.2 but project requires >= 0.49.2.. So that's probably a big part of the problem for question (1). This is running on CentOS 6 & 7, so I'll probably have to get and install a current meson package.
So the basic build doesn't work. Why?
You correctly figured this out in your edit: GLib 2.64 requires Meson 0.49.2, and it seems that Meson 0.47.2 is so old as to not be able to correctly parse GLib’s meson.build.
It looks from your build output that you’re trying to build GLib 2.65.0. Note that 2.65 is an unstable release series. Even minor versions of GLib (2.62.x, 2.64.x, etc.) are stable; odd ones are unstable. Using an unstable release is fine, as long as you know what you’ve signed up for: it may contain bugs, and new APIs introduced in that unstable series may change or be removed before the first stable release (in the case of 2.65.x, the corresponding first stable release will be 2.66.0).
For our purposes, we use the following configure command:
You’ll want something like:
meson --prefix "$(OUTPUT_DIR)" -Dselinux=disabled -Ddefault_library=static _build
You can see from the b_staticpic option’s default value that -fPIC is the default for static libraries, so (I believe) doesn’t need to be explicitly specified.
There should be no need to disable dtrace support since it’s disabled by default. If you did need to disable it, you’d do that with -Ddtrace=false.
The custom -L and -I arguments should be covered by use of --prefix.
Overriding the msgfmt tool to disable internationalisation is not a supported way of building GLib and you’re on your own with that one.
There is some good documentation on the built-in options in Meson here and here.
I will also need to build in Windows. Any gotchas there?
That’s too broad a question to be answered on StackOverflow.

Code::Blocks 10.05 and Environmentmental Variables

I'm probably missing something really obvious, but I don't have the Environmental Variables dialog shown in Configuring environmental variables. I've been through Settings -> Environment... (and other submenus from Settings) and its just missing.
OpenSSL FIPS Capable is particular about having CC and FIPSLD_CC set to embed the module's fingerprint. I even tried running the following as a prebuild step with no joy (the echo's are empty):
Running project pre-build steps
export CC=`find /usr/local/ssl -name fipsld`
export FIPSLD_CC=`find /usr/bin -name g++`
echo $CC
echo $FIPSLD_CC
I'm working on Debian 7.3 (x64) fully patched, if it matters. Code::Blocks was installed from Debian's repos. OpenSSL was custom built and installed in /usr/local/ssl.
Where does one set environmental variables in Code::Blocks 10.05?
The Environment Variables Editor is an optional plugin. Debian packages can be found at Package: codeblocks-contrib (10.05-2.1).
To install it you need to install package codeblocks-contrib as well as codeblocks:
sudo apt-get install codeblocks codeblocks-contrib
Close and restart Code::Blocks, if it is running. The dialog you looking for will then appear in Settings -> Environment

Erlang: Building Issue of not finding Ncurses on Solaris 10

I am trying to build Erlang on Solaris 10. The build process fails with the message that it can not find libncurses.so.5.
I have installed libncurses from sunfreeware.com, which I have build from scratch and has installed itself in /usr/local/lib/.
I have tried to set LDFLAGS with -L/usr/local/lib/ but have still had no luck.
What am I missing so that make picks up the library?
I am using GNU Make 3.81 and GCC 3.4.6.
Could you post the relevant bit from config.log?
Did you also set the include path with -I in either CPPFLAGS or CFLAGS?
Update: You could also try installing Erlang from OpenCSW instead.

Resources