For a small C helloworld program, Travis-CI uses 18 seconds to build. Why is it so slow? My files:
.travis.yml
language: C
script: make
Makefile
all:
gcc hello.c
hello.c
int main(){
return 0;
}
It doesn't only have to build the C code. It also has to create the enviroment in which it has access to a C complier to build your code!
Creating this enviroment will take some time.
Related
I have built libstdc++ with no modifications yet:
cd gccsrcdir/libstdc++-v3/build
../configure --prefix=$PWD/../install
make && make install
I am using Ubuntu 21.10 and I set the following environment variables:
export LIBRARY_PATH=gccsrcdir/libstdc++-v3/install/lib
export LD_LIBRARY_PATH=gccsrcdir/libstdc++-v3/install/lib
export CPLUS_INCLUDE_PATH=gccsrcdir/libstdc++-v3/install/include/c++/13.0.0
When I then use the system's GCC, I get no problems. When I use the system's Clang, it produces a symbol lookup error - even with no parameters:
clang++
clang++: symbol lookup error: /lib/x86_64-linux-gnu/libicuuc.so.67: undefined symbol: _ZSt15__once_callable, version GLIBCXX_3.4.11
In fact I only need to update LD_LIBRARY_PATH to arrive here. What am I doing wrong?
The symbol -- std::__once_callable is defined in your system libstdc++.so.6 (it has version GLIBCXX_3.4.11 in my build, which means it was added in GCC-4.4.0).
Your build of libstdc++.so.6 should define this symbol as well, but for some reason does not. That is a problem -- any binary which uses this symbol will fail at runtime when using your build of libstdc++.so.6 (which is happening because you've pointed LD_LIBRARY_PATH to it).
Note: in your case it's the clang++ binary that is failing to run -- any flags you add to it (such as -femulated-tls) are irrelevant -- they only affect the binary that would have been generated IF clang++ itself didn't fail.
I just repeated your configure && make steps, and the library built this way also doesn't define this symbol.
I then repeated the configure && make, but starting from top-level GCC directory, and libstdc++.so.6 built that way does define the symbol.
Conclusion: libstdc++ is configured differently during "normal" GCC build.
The definition comes from mutex.o, which is built from ./libstdc++-v3/src/c++11/mutex.cc, and which has this chunk of code:
#ifdef _GLIBCXX_HAS_GTHREADS
namespace std _GLIBCXX_VISIBILITY(default)
{
_GLIBCXX_BEGIN_NAMESPACE_VERSION
#ifdef _GLIBCXX_HAVE_TLS
__thread void* __once_callable;
__thread void (*__once_call)();
...
So it sounds like either _GLIBCXX_HAS_GTHREADS or _GLIBCXX_HAVE_TLS is not defined when doing configure && make in the libstdc++-v3 directly.
Digging further, I see that libstdc++-v3 determines _GLIBCXX_HAS_GTHREADS by trying to compile #include "gthr.h", and that file is available in libgcc/gthr.h, but not in "standard" installed GCC.
../libstdc++-v3/configure && grep _GLIBCXX_HAS_GTHREADS config.h
/* #undef _GLIBCXX_HAS_GTHREADS */
TL;DR: correctly configuring libstdc++.so is complicated, and you will be better off building complete GCC.
Once you have a complete build, you will have a libstdc++-v3 directory properly configured, and can just rebuilt in that directory:
grep _GLIBCXX_HAS_GTHREADS ./x86_64-pc-linux-gnu/libstdc++-v3/config.h
#define _GLIBCXX_HAS_GTHREADS 1
Setup
I have a simple helloworld program:
// content of main.c
#include <stdio.h>
#include <limits.h>
int main() {
for (int i = 0; i < INT_MAX; ++i) {
printf("simply helloworld!\n");
}
return 0;
}
I compile a baseline version with clang 13.0.0 using clang -flto=thin -fvisibility=hidden -fuse-ld=lld main.c
To experiment with CFI, I compile another version using clang -flto=thin -fsanitize=cfi -fsanitize-cfi-cross-dso -fno-sanitize-cfi-canonical-jump-tables -fsanitize-trap=cfi -fvisibility=hidden -fuse-ld=lld main.c
Expectation
I am expecting negligible performance overhead as I am only calling into a shared library that I expect will run the same code for both. The disassembly for main function for both binaries look the same.
Reality
The baseline version completes execution in ~27s while the cfi version completes execution in ~32s. Using perf stat -e instructions <binary> I can see that the cfi version runs ~100,000,000,000 more instructions. With perf record then perf diff, I can see that the difference is primarily in two functions _pthread_cleanup_push_defer and _pthread_cleanup_pop_restore that the cfi version runs. Using gdb, these functions are called as the call stack of printf gets deeper.
Question
How do I begin to explain the performance difference between these two binaries? What makes a simple call to printf call two different versions of itself for two different binaries?
I have a somewhat complex SCons build script that an some point does the following two steps:
# 1: builds unit tests (googletest, shell executable)
compile_tests = some_environment.Program(executable_path, test_sources)
# 2: runs unit tests (call earlier compiled program)
run_tests = other_environment.Command(
source = executable_path,
action = executable_path + ' --gtest_output=xml:' + test_results_path,
target = test_results_path
)
Depends(run_tests, compile_tests)
This is working fine if I run scons with this build script on its own.
If I, however, invoke it via environment.SConscript() from another SConstruct file one directory level up, then step 1 adjusts the path to the project's location while step 2 doesn't. See this output:
scons: Building targets ...
g++ -o Nuclex.Game.Native/obj/gcc-7-amd64-release/NuclexGameNativeTests -z defs -Bsymbolic Nuclex.Game.Native/obj/gcc-7-amd64-release/Tests/Timing/ClockTests.o -LNuclex.Game.Native/obj/gcc-7-amd64-release -LReferences/googletest/gcc-7-amd64-release -lNuclexGameNativeStatic -lgoogletest -lgoogletest_main -lpthread
obj/gcc-7-amd64-release/NuclexGameNativeTests --gtest_output=xml:bin/gcc-7-amd64-release/gtest-results.xml
sh: obj/gcc-7-amd64-release/NuclexGameNativeTests: No such file or directory
Line 2 builds the executable into Nuclex.Game.Native/obj/gcc-7-amd64-release/ while line 3 tries to call it in obj/gcc-7-amd64-release/, forgetting the project directory.
Should I use another way to invoke my unit test executable? Or can I query the SCons environment for its base path?
Update: reproduction case, place https://pastebin.com/W08yZuF9 as SConstruct in root directory, create subdirectory somelib and place https://pastebin.com/eiP63Yxh as SConstruct therein, also create main.cpp with a "Hello World" or other dummy program.
A SCons Action (the action parameter in the Command) will use the SCons variables to substitute sources and targets in correctly, taking into account VariantDirs and SConscript directories automatically. You can find more info on these source and target substitutions here: https://scons.org/doc/HTML/scons-man.html#variable_substitution
There is a section which explains using this in regards to SConscript and VariantDirs:
SConscript('src/SConscript', variant_dir='sub/dir')
$SOURCE => sub/dir/file.x
${SOURCE.srcpath} => src/file.x
${SOURCE.srcdir} => src
So in your example I think you want to replace executable_path with $SOURCE in the action string:
# 2: runs unit tests (call earlier compiled program)
run_tests = other_environment.Command(
source = executable_path,
action = '$SOURCE --gtest_output=xml:$TARGET',
target = test_results_path
)
Setting up Deep Learning Framework [Lua, Torch]:
I need to set up Lua running with Torch
on Windows 10 and the ZeroBrane IDE, with limited possibilities of installing Software and restricted download rights.
It took me so Long, so I thought I might share a recipe for you guys. I would be glad if it helped you.
SETTING UP
(Admin) Download/Install tdm64/gcc/5.1.0-2.exe Compiler
(Admin) Download/Install ZeroBrane (Lua IDE)
Download lua/5.3.4.tar.gz (https://www.lua.org/download.html)
Write batch file build.cmd
#echo off
setlocal
:: you may change the following variable's value
:: to suit the downloaded version
set lua_version=5.3.4
set work_dir=%~dp0
:: Removes trailing backslash
:: to enhance readability in the following steps
set work_dir=%work_dir:~0,-1%
set lua_install_dir=%work_dir%\lua
set compiler_bin_dir=%work_dir%\tdm-gcc\bin
set lua_build_dir=%work_dir%\lua-%lua_version%
set path=%compiler_bin_dir%;%path%
cd /D %lua_build_dir%
mingw32-make PLAT=mingw
echo.
echo **** COMPILATION TERMINATED ****
echo.
echo **** BUILDING BINARY DISTRIBUTION ****
echo.
:: create a clean "binary" installation
mkdir %lua_install_dir%
mkdir %lua_install_dir%\doc
mkdir %lua_install_dir%\bin
mkdir %lua_install_dir%\include
copy %lua_build_dir%\doc\*.* %lua_install_dir%\doc\*.*
copy %lua_build_dir%\src\*.exe %lua_install_dir%\bin\*.*
copy %lua_build_dir%\src\*.dll %lua_install_dir%\bin\*.*
copy %lua_build_dir%\src\luaconf.h %lua_install_dir%\include\*.*
copy %lua_build_dir%\src\lua.h %lua_install_dir%\include\*.*
copy %lua_build_dir%\src\lualib.h %lua_install_dir%\include\*.*
copy %lua_build_dir%\src\lauxlib.h %lua_install_dir%\include\*.*
copy %lua_build_dir%\src\lua.hpp %lua_install_dir%\include\*.*
echo.
echo **** BINARY DISTRIBUTION BUILT ****
echo.
%lua_install_dir%\bin\lua.exe -e"print [[Hello!]];print[[Simple Lua test successful!!!]]"
echo.
pause
SETTING UP TORCH UNDER LUA ON WINDOWS
--- Quick and dirty ---
Download and unzip the desired binary build from: https://github.com/hiili/WindowsTorch
Generate user.lua file in C:\Users\Name.zbstudio:
path.lua = [[C:\app\tools\torch\bin\luajit.exe]]
Move the C:\app\tools\torch\lua folder to C:\app\tools\torch\bin
--- Untested alternatives ---
Not tested, but I encourage you: https://github.com/torch/torch7/wiki/Windows#cmder
Maybe second best option is to build a virtual environment with linux
Note: More information on Torch can be found here
https://github.com/soumith/cvpr2015/blob/master/cvpr-torch.pdf
GET STARTED WITH LUA AND TORCH
http://torch.ch/docs/tutorials.html
I recommend Torch Video Tutorials to get the basics straight (https://github.com/Atcold/torch-Video-Tutorials)
This is a Torch Cheetsheet for further reading (https://github.com/torch/torch7/wiki/Cheatsheet):
- Newbies
- Installing and Running Torch
- Installing Packages
- Tutorials, Demos by Category
- Loading popular datasets
- List of Packages by Category
Everyone who searched how to include a static library in SDK, surely read this thread from 2014. I tried what they suggested, but that didn't work.
Reading the yocto mega manual version 2.1 (yocto morty), I found in chapter 5.9.12. (Poky Reference Distribution Changes), that they added DISABLE_STATIC variable, to disable generation of static libraries. I tried adding this to my recipe, and it didn't enable adding static library to SDK:
DISABLE_STATIC = ""
I can see the library in the sysroot when building the image. But it is not getting in the SDK.
So, what exactly do I need to do to get a static library and the headers in SDK?
What worked is adding the staticdev package to ´IMAGE_INSTALL´ in local.conf, but I don't want to have to do that.
I created an example recipe, which demonstrates my problem. The directory structure is like this:
example-staticlib/
example-staticlib/example-staticlib_0.1.bb
example-staticlib/files/
example-staticlib/files/lib.c
example-staticlib/files/lib.h
example-staticlib/files/Makefile
example-staticlib_0.1.bb :
DESCRIPTION = "example stared library"
LICENSE = "LGPLv2"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/LGPL-2.0;md5=9427b8ccf5cf3df47c29110424c9641a"
SRC_URI = "file://lib.c \
file://lib.h \
file://Makefile"
PR = "r0"
S = "${WORKDIR}"
ALLOW_EMPTY_${PN} = "1"
do_install () {
oe_runmake install DEST=${D}
}
TOOLCHAIN_TARGET_TASK += "example-staticlib-dev"
TOOLCHAIN_TARGET_TASK += "example-staticlib-staticdev"
lib.c:
int foo()
{
return 42;
}
lib.h:
int foo();
Makefile:
TARGET=libexample.a
all:$(TARGET)
install :
#install -d $(DEST)/usr/lib/
#install -m 0644 $(TARGET) $(DEST)/usr/lib/
#install -d $(DEST)/usr/include/
#install -m 0644 lib.h $(DEST)/usr/include/
$(TARGET) : lib.c
$(CC) -c lib.c -o lib.o
$(AR) rcs $# lib.o
clean:
rm -rf lib.o $(TARGET)
How exactly to modify the recipe, in order to get the static library in the SDK?
Following your added example.
Adding the following line to your image recipe (or to an .bbappend, eg core-image-minimal.bbappend)
TOOLCHAIN_TARGET_TASK += "example-staticlib-staticdev"
should work for you. That will give you the .a file in the SDK, after running bitbake core-image-minimal -c populate_sdk. (Again assuming that the image used is core-image-minimal).
That your experiment to add the .a file to ${PN}-dev didn't work, is duet to the order of how files are put into packages. The order is ${PN}-dbg ${PN}-staticdev ${PN}-dev ${PN}-doc ${PN}-locale ${PACKAGE_BEFORE_PN} ${PN}. Thus, the .a file will, regardless, be put into ${PN}-staticdev, as that packages is handled prior to {PN}-dev.
Note, you add this line, TOOLCHAIN_TARGET_TASK += "example-staticlib-staticdev" to your image recipe, thus, you need to write the package name instead of PN.
I tried a way which doesn't require editing the image recipe.
example-staticlib_0.1.bb :
After do_install. I didn't use TOOLCHAIN_TARGET_TASK
FILES_${PN}-staticdev += "${libdir}/libexample.a"
RDEPENDS_${PN}-dev += "${PN}-staticdev"
BBCLASSEXTEND = "native nativesdk"