I am getting the error Illegal instruction (core dumped) on many commands run inside an arm64 docker container. Even simply typing pip results in the error. I am using buildroot so the error could be related to my kernel configurations. The container does run on raspberry pi with nearly the same buildroot configuration. I compiled the container on qemu, and AWS. I have made the container work by building it on target, but this is slow and runs out of room easily. What can I do to make other build machines work?
NodeMCU documentation states
NodeMCU firmware build now automatically generates a luac.cross image
as standard in the firmware root directory; this can be used to
compile and to syntax-check Lua source on the Development machine for
execution under NodeMCU Lua on the ESP8266.
Where do I get luac.cross from and how do I install it?
Do I build NodeMCU firmware from source on Mac and is luac.cross created as part of that process? I have been using the cloud service to create custom firmware. Is luac.cross available via cloud build?
Straight lua code has overwhelmed the MakerFocus NodeMCU board resulting runtime panic with out of memory issue. Hoping compiled code will reduce RAM needs.
Where do I get luac.cross from and how do I install it?
You gave the answer in the quote from the documentation you posted. Specifically this
NodeMCU firmware build now automatically generates a luac.cross image...
So, if you build the NodeMCU manually on your platform the build process will also create a lua.cross for your platform. That's the reason you cannot download or install lua.cross - it has to fit your platform i.e. OS et.al.
The logical next question would then be: how do I manually build NodeMCU on macOS?
I don't know the answer to that as I build with the Docker image (from yours truly) on macOS. Running the Docker build creates a luac.cross in the firmware root directory. However, as macOS is just the host OS for Docker in this setup luac.cross is for Linux rather than native for macOS. To use it you would start the Docker container again and run bash in it to get a shell to execute the Lua cross compilation: docker run --rm -ti -vpwd:/opt/nodemcu-firmware marcelstoer/nodemcu-build bash.
Straight lua code has overwhelmed the MakerFocus NodeMCU board resulting runtime panic with out of memory issue. Hoping compiled code will reduce RAM needs.
I hate to disillusion you, but if I had to bet I would expect that savings won't be significant enough to yield the expected results. As you already started reading documentation I'd like to point you to the relevant FAQ: How is NodeMCU Lua different to standard Lua? and Techniques for Reducing RAM
And maybe using LFS will be your life saver.
In case you want to use this tool regardless of the platform - you can use my API to build it:
curl -d #yourscript.lua -X POST https://nodemcu-luacross-run-64l7ehzjta-uc.a.run.app/compile > output.luac
I'm building an elixir/phoenix application using a docker container.
This has been working for some time now, but recently it stopped working, with the error always being associated with a lack of memory.
For instance, the most frequent point of failure is during the mix compile task of Elixir (the most time heavy task in the Dockerfile), which crashes with the error:
eheap_alloc: Cannot allocate 147852528 bytes of memory (of type "old_heap").
Crash dump is being written to: erl_crash.dump...done
Sometimes it might be able to get through that step, but will again fail at a later step, like brunch build which compiles the frontend code. Sometimes it just fails at some other step with no specific error message, just saying:
Killed
While this is happening, I can easily check htop and see that I'm using 3 or 4GB of RAM, out of 16GB total, so there's no lack of physical RAM at all.
After some digging, I found that sudo sysctl vm.overcommit_memory=1 could help, but no luck there either.
The exact same build runs fine on my other computer, which runs Arch Linux, while this one runs Ubuntu 16.04
I'm wanting to install the minimal cuda runtime files into alpine linux and create a much smaller docker base with cuda than that provided by nvidia themselves. The nvidia official ones are enormous as usual.
How do I obtain these runtime files without pulling the entire cuda 8 toolkit during docker build?
I can't speak as to what other files might be needed. However, Nvidia drivers are compiled with glibc, and alpine uses musl to maintain its small footprint. You would likely need the nvidia driver's source code so you could recompile it with musl, or an alpine baseimage that implements glibc such as this one. I haven't tried using this yet, but I was able to sucessfully compile libcudacore with musl and gcc/make on an alpine 3.8 container. I have not yet been able to compile the entire Nvidia/Cuda toolkit yet. I will attempt to test this more when I have more time.
The reality is that Nvidia/CUDA is not supported in any way with Alpine Linux Musl or its libc port, and you will end up with a flaky image nevertheless even if you succeed with your alchemist venture.
Nvidia drivers and CUDA Toolkits are incredibly complex systems that honestly I can't see the point to compile it yourself for an unsupported system library or an unsupported port for libc, with all the unexpected to happen even in the case it compiles. Use Debian's slim images or Ubuntu minimal and install official supported files manually, as this is the smallest you can go. Or even better use the "huge" Nvidia DockerHub images (ubuntu LTS based).
Anyway, beyond this question, the Nvidia DockerHub ones are the best way to go, they are supported by the creators of CUDA Toolkit itself and they are no brainers. If you want to be picky go to their Gitlab's repository for dockers, you can build up Debian/Ubuntu by hand pretty easily and quick.
Yes they Nvidia DockerHub images are 1-2 gig's large, but normally you only have to download them once, as you use the image as a base, if you add your code to it only those layers of your code which are normally small to dozens of Mbi are to be recurrently pulled/pushed, not the entire image, so honestly I can't see a reason why people is so much concerned about image sizes, small is better no doubt but up to a point, spending your valuable time in your actual needs is far better.
somebody's solution for alpine-cuda:
https://arto.s3.amazonaws.com/notes/cuda
Drivers
https://developer.nvidia.com/vulkan-driver
$ lsmod | fgrep nvidia
$ nvidia-smi
Driver Installation
https://us.download.nvidia.com/XFree86/Linux-x86_64/390.77/README/
https://github.com/NVIDIA/nvidia-installer
Driver Installation on Alpine Linux
https://github.com/sgerrand/alpine-pkg-glibc
https://github.com/sgerrand/alpine-pkg-glibc/releases
https://wiki.alpinelinux.org/wiki/Running_glibc_programs
$ apk add sudo bash ca-certificates wget xz make gcc linux-headers
$ wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://raw.githubusercontent.com/sgerrand/alpine-pkg-glibc/master/sgerrand.rsa.pub
$ wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.27-r0/glibc-2.27-r0.apk
$ wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.27-r0/glibc-bin-2.27-r0.apk
$ wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.27-r0/glibc-dev-2.27-r0.apk
$ wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.27-r0/glibc-i18n-2.27-r0.apk
$ apk add glibc-2.27-r0.apk glibc-bin-2.27-r0.apk glibc-dev-2.27-r0.apk glibc-i18n-2.27-r0.apk
$ /usr/glibc-compat/bin/localedef -i en_US -f UTF-8 en_US.UTF-8
$ bash NVIDIA-Linux-x86_64-390.77.run --check
$ bash NVIDIA-Linux-x86_64-390.77.run --extract-only
$ cd NVIDIA-Linux-x86_64-390.77 && ./nvidia-installer
Driver Uninstallation
$ nvidia-uninstall
Driver Troubleshooting
Uncompressing NVIDIA Accelerated Graphics Driver for Linux-x86_64 390.77NVIDIA-Linux-x86_64-390.77.run: line 998: /tmp/makeself.XXX/xz: No such file or directory\nExtraction failed.
$ apk add xz # Alpine Linux
bash: ./nvidia-installer: No such file or directory
Install the glibc compatibility layer package for Alpine Linux.
ERROR: You do not appear to have libc header files installed on your system. Please install your distribution's libc development package.
$ apk add musl-dev # Alpine Linux
ERROR: Unable to find the kernel source tree for the currently running kernel. Please make sure you have installed the kernel source files for your kernel and that they are properly configured
$ apk add linux-vanilla-dev # Alpine Linux
ERROR: Failed to execute `/sbin/ldconfig`: The installer has encountered the following error during installation: 'Failed to execute `/sbin/ldconfig`'. Would you like to continue installation anyway?
Continue installation.
Toolkit
https://developer.nvidia.com/cuda-toolkit
https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/
Toolkit Download
https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1604&target_type=runfilelocal
$ wget -c https://developer.nvidia.com/compute/cuda/9.2/Prod2/local_installers/cuda_9.2.148_396.37_linux
Toolkit Installation
https://docs.nvidia.com/cuda/cuda-installation-guide-linux/
Toolkit Installation on Alpine Linux
$ apk add sudo bash
$ sudo bash cuda_9.2.148_396.37_linux
# You are attempting to install on an unsupported configuration. Do you wish to continue? y
# Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 396.37? y
# Do you want to install the OpenGL libraries? y
# Do you want to run nvidia-xconfig? n
# Install the CUDA 9.2 Toolkit? y
# Enter Toolkit Location: /opt/cuda-9.2
# Do you want to install a symbolic link at /usr/local/cuda? y
# Install the CUDA 9.2 Samples? y
# Enter CUDA Samples Location: /opt/cuda-9.2/samples
$ sudo ln -s cuda-9.2 /opt/cuda
$ export PATH="/opt/cuda/bin:$PATH"
Toolkit Uninstallation
$ sudo /opt/cuda-9.2/bin/uninstall_cuda_9.2.pl
Toolkit Troubleshooting
Cannot find termcap: Can't find a valid termcap file at /usr/share/perl5/core_perl/Term/ReadLine.pm line 377.
$ export PERL_RL="Perl o=0"
gcc: error trying to exec 'cc1plus': execvp: No such file or directory
$ apk add g++ # Alpine Linux
cicc: Relink `/usr/lib/libgcc_s.so.1' with `/usr/glibc-compat/lib/libc.so.6' for IFUNC symbol `memset'
https://github.com/sgerrand/alpine-pkg-glibc/issues/58
$ scp /lib/x86_64-linux-gnu/libgcc_s.so.1 root#alpine:/usr/glibc-compat/lib/libgcc_s.so.1
$ sudo /usr/glibc-compat/sbin/ldconfig /usr/glibc-compat/lib /lib /usr/lib
Compiler
https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/
$ nvcc -V
Please define what you actually mean by "into Alpine Linux".
Regardless whether you're running the workloads directly on the host or in a container or chroot - you need to install the whole NVidia driver stack (including Cuda libs, kernel drivers, etc) on the host. Also kernel and userland drivers are two sides of the same product, both have to have the same version.
This means: whatever the host OS actually is, it has to be exactly one of those directly supported by NVidia. You have to use exactly the kernel versions (and configurations) that Nvidia built their proprietary/binary-only drivers for. Using a different kernel version or recompiling it with different configuration MIGHT POSSIBLY work, but it's DANGEROUS. Even with exactly the officially supported distros, it's still gambling, and depending on moon phase or whether some Chinese rice bag fallen over. It often works, but when it doesn't anymore, you're most likely out of luck.
Now when you're putting your workloads into some separate OS image, e.g. chroot or container, you also have to have the same driver package version in that image, too. One of the primary reasons for using containers or chroots - isolating and decoupling applications from host OS (so you don't need to fit them in anymore and do upgrades independently, even have container images independent from the host OS) - is now immediately voided. Host and workload need to fit together exactly.
In short: if you wanna have a CUDA workload, both host OS as well as workload image (container, chroot, etc) need to be supported by that, and they both need to have the same driver version installed. Anything else is just russian roulette.
Since somebody mentioned "nvidia-docker". This breaks the security isolation that docker is originally meant for. (just look at the source, which actually is available somewhere on github). It's nothing but a better chroot. And still, host and docker image need to have the same driver stack version installed.
Finally, I'd like to ask the question, what your actual use case is here.
Be warned: this all might be okay for playing games on an totally unimportant home computer, but really not suited for anything professional, where stability and security matter. If you're bound to certain data security / privacy regulations like GDPO, keep far away from this - you just cannot comply to these regulations with those proprietary drivers. Legally dangerous.
--mtx
Addendum: why do proprietary kernel drivers never work reliably ?
Express answer: the Linux kernel was never ever made for that, this just isn't supported.
Longer answer: kernel modules are NOT external programs, that are executed in some isolated environment (like eg. done with userland programs) - they are (by definition) integral pieces of the kernel that just happen to be lazily loaded when needed. (they are not even like shared libraries / DLLs). This means that they have to fit - on binary level - exactly to the actual build of the kernel you're running. When compiling the kernel, there're lots of config options that influence the actual internal binary layout in subtle ways, e.g. enabling/disabling some features can change the layout of certain data structures, cpu specific optimizations can change datastructures, calling conventions, locking mechanisms, and much much more.
And those things also change from kernel version to another. We're e.g. doing lots of internal refactorings (e.g. in data structures, macros and inline functions) after which the same piece source code generates very different binary code.
Therefore, any kernel modules always need to be compiled exactly for a specific kernel image (with the same config options, against the same includes, with the same compiler flags), or you risks horrible failures that could lead to lockups, security flaws, data corruptions or even total data loss.
You have been warned.
To clarify, this is just the driver. Not cuda. That's another story.
In fact this turns out to be much easier than expected. I just didn't quite /understand how far nvidia-docker project had come and quite how it worked.
Basically, download and install the latest nvidia-docker. From the nvidia-docker project.
https://github.com/NVIDIA/nvidia-docker/releases
Then create an alpine linux Dockerfile.
FROM alpine:3.5
LABEL com.nvidia.volumes.needed="nvidia_driver"
ENV PATH /usr/local/nvidia/bin:/usr/local/cuda/bin:${PATH}
ENV LD_LIBRARY_PATH /usr/local/nvidia/lib:/usr/local/nvidia/lib64
RUN /bin/sh
Build it.
docker build -t alpine-nvidia
Run
nvidia-docker run -ti --rm alpine-nvidia
Note the use of the nvidia-docker cli instead of the normal docker cli.
nvidia-docker calls docker cli with extra parameters.
I'm baffled. I have a VM running Ubuntu 14.04. I've followed procedures here: http://clang.llvm.org/docs/LibASTMatchersTutorial.html and am at the step to run ninja. This builds llvm and clang. Now, my VM is no slouch, I gave it 6GB of RAM and 4 CPUs and a 20GB swap file. The biggest problem comes at link time - it seems to start a large number of ld processes, each using at least 3-4GB or virtual memory, and at some point a lot of CPU each. But the the swap file grew to over 12GB and the processes are all IO bound, but I don't know if they are doing something useful, or thrashing. All I know is the disk is getting hammered and the jobs run forever. I've actually just dropped the CPU count to the VM to 1, to see if it might be more efficient with less parallelism, as I surmised the issue may be thrashing.
I suppose my disk could be slow... Any ideas? Should I be using make instead of ninja? My expertise is not Linux (although I'm getting there :-) ) So I'm following the tutorial but perhaps it is not recommended the "best" way to build the clang / llvm programs.
I have been there, It's happening with the latest svn release (but not if you get clang 3.8 or older releases). What is happening is that since during development a lot of debug information is also being generated for each compilation unit the file sizes are becoming big.
The solution is to turn off all the debug info that's been attached by default. You probably are not going to debug clang, so won't need it. SO instead of just doing this
cmake -G Ninja ../llvm -DLLVM_BUILD_TESTS=ON
What you should do is
cmake -G Ninja ../llvm -DLLVM_BUILD_TESTS=ON -DCMAKE_BUILD_TYPE=Release
All the other steps remain the same. Now I have not tested this with ninja, but have verified it with make on ubuntu (this tutorial, I modified the same thing in step 7). This should owkr as weel.