Parallel make builds for parallel serverless deployments, errors STDOUT - serverless

I have a mono repo that has multiple serverless deployment stacks. From codebuild, we are doing all deployments. And currently with a make file we are running the deployment. Deployments are successful.
To speed up the deployments, I plan to run these sls deployments parallelly as they are independent of each other. So created a make target for each deployment and created another target with all these independent targets as prerequisites and run parallel make builds with make -j option. Here I face errors like below
Makefile
all-deploy: sls1 sls2 sls3
sls1:
cd backend && sls deploy -c /path/to/sls1-config
sls2:
cd backend && sls deploy -c /path/to/sls2-config
sls3:
cd backend && sls deploy -c /path/to/sls3-config
And invoke the deployment from a bash file like
make -j3 all-deploy
Error
Error: STDOUT:
STDERR: find: '/root/.cache/serverless-python-requirements/a92be47b63e9e4444f3ed7b447b3a846a5de81273cb952775bb672275a820a07_slspyc/numpy/core/__pycache__': No such file or directory
find: '/root/.cache/serverless-python-requirements/a92be47b63e9e4444f3ed7b447b3a846a5de81273cb952775bb672275a820a07_slspyc/aws_lambda_powertools-1.24.0.dist-info': No such file or directory
find: '/root/.cache/serverless-python-requirements/a92be47b63e9e4444f3ed7b447b3a846a5de81273cb952775bb672275a820a07_slspyc/numpy-1.21.5.dist-info': No such file or directory
find: '/root/.cache/serverless-python-requirements/a92be47b63e9e4444f3ed7b447b3a846a5de81273cb952775bb672275a820a07_slspyc/boto3-1.18.65.dist-info': No such file or directory
find: '/root/.cache/serverless-python-requirements/a92be47b63e9e4444f3ed7b447b3a846a5de81273cb952775bb672275a820a07_slspyc/wrapt-1.13.3.dist-info': No such file or directory
find: '/root/.cache/serverless-python-requirements/a92be47b63e9e4444f3ed7b447b3a846a5de81273cb952775bb672275a820a07_slspyc/idna-3.3.dist-info': No such file or directory
find: '/root/.cache/serverless-python-requirements/a92be47b63e9e4444f3ed7b447b3a846a5de81273cb952775bb672275a820a07_slspyc/pytz-2021.3.dist-info': No such file or directory
find: '/root/.cache/serverless-python-requirements/a92be47b63e9e4444f3ed7b447b3a846a5de81273cb952775bb672275a820a07_slspyc/six-1.16.0.dist-info': No such file or directory
at /codebuild/output/src472528929/src/node_modules/serverless-python-requirements/lib/pip.js:331:13
at Array.forEach (<anonymous>)
at installRequirements (/codebuild/output/src472528929/src/node_modules/serverless-python-requirements/lib/pip.js:318:28)
at installRequirementsIfNeeded (/codebuild/output/src472528929/src/node_modules/serverless-python-requirements/lib/pip.js:576:3)
at ServerlessPythonRequirements.installAllRequirements (/codebuild/output/src472528929/src/node_modules/serverless-python-requirements/lib/pip.js:655:29)
at ServerlessPythonRequirements.tryCatcher (/codebuild/output/src472528929/src/node_modules/bluebird/js/release/util.js:16:23)
at Promise._settlePromiseFromHandler (/codebuild/output/src472528929/src/node_modules/bluebird/js/release/promise.js:547:31)
at Promise._settlePromise (/codebuild/output/src472528929/src/node_modules/bluebird/js/release/promise.js:604:18)
at Promise._settlePromise0 (/codebuild/output/src472528929/src/node_modules/bluebird/js/release/promise.js:649:10)
at Promise._settlePromises (/codebuild/output/src472528929/src/node_modules/bluebird/js/release/promise.js:729:18)
at _drainQueueStep (/codebuild/output/src472528929/src/node_modules/bluebird/js/release/async.js:93:12)
at _drainQueue (/codebuild/output/src472528929/src/node_modules/bluebird/js/release/async.js:86:9)
at Async._drainQueues (/codebuild/output/src472528929/src/node_modules/bluebird/js/release/async.js:102:5)
at Immediate.Async.drainQueues [as _onImmediate] (/codebuild/output/src472528929/src/node_modules/bluebird/js/release/async.js:15:14)
at processImmediate (internal/timers.js:461:21)

Related

Could not build wheels for backports.zoneinfo, which is required to install pyproject.toml-based projects

I am trying to build a django project to push it to an ECR instance but I am having trouble building the project through docker.
Running this command:
docker build -t stuffkeep-docker-app .
Results in this error and I have not found any way to resolve this yet.
Building wheels for collected packages: backports.zoneinfo, django-allauth, django-rest-framework, fcm-django, gcloud, http-ece, jws, py-vapid, pywebpush
Building wheel for backports.zoneinfo (pyproject.toml): started
Building wheel for backports.zoneinfo (pyproject.toml): finished with status 'error'
error: subprocess-exited-with-error
× Building wheel for backports.zoneinfo (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [43 lines of output]
/tmp/pip-build-env-9x4p5_f7/overlay/lib/python3.10/site-packages/setuptools/config/setupcfg.py:459: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
warnings.warn(msg, warning_class)
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-310
creating build/lib.linux-x86_64-cpython-310/backports
copying src/backports/__init__.py -> build/lib.linux-x86_64-cpython-310/backports
creating build/lib.linux-x86_64-cpython-310/backports/zoneinfo
copying src/backports/zoneinfo/_tzpath.py -> build/lib.linux-x86_64-cpython-310/backports/zoneinfo
copying src/backports/zoneinfo/_version.py -> build/lib.linux-x86_64-cpython-310/backports/zoneinfo
copying src/backports/zoneinfo/_zoneinfo.py -> build/lib.linux-x86_64-cpython-310/backports/zoneinfo
copying src/backports/zoneinfo/_common.py -> build/lib.linux-x86_64-cpython-310/backports/zoneinfo
copying src/backports/zoneinfo/__init__.py -> build/lib.linux-x86_64-cpython-310/backports/zoneinfo
running egg_info
writing src/backports.zoneinfo.egg-info/PKG-INFO
writing dependency_links to src/backports.zoneinfo.egg-info/dependency_links.txt
writing requirements to src/backports.zoneinfo.egg-info/requires.txt
writing top-level names to src/backports.zoneinfo.egg-info/top_level.txt
reading manifest file 'src/backports.zoneinfo.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.png' under directory 'docs'
warning: no files found matching '*.svg' under directory 'docs'
no previously-included directories found matching 'docs/_build'
no previously-included directories found matching 'docs/_output'
adding license file 'LICENSE'
adding license file 'licenses/LICENSE_APACHE'
writing manifest file 'src/backports.zoneinfo.egg-info/SOURCES.txt'
copying src/backports/zoneinfo/__init__.pyi -> build/lib.linux-x86_64-cpython-310/backports/zoneinfo
copying src/backports/zoneinfo/py.typed -> build/lib.linux-x86_64-cpython-310/backports/zoneinfo
running build_ext
building 'backports.zoneinfo._czoneinfo' extension
creating build/temp.linux-x86_64-cpython-310
creating build/temp.linux-x86_64-cpython-310/lib
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/include/python3.10 -c lib/zoneinfo_module.c -o build/temp.linux-x86_64-cpython-310/lib/zoneinfo_module.o -std=c99
lib/zoneinfo_module.c: In function ‘zoneinfo_fromutc’:
lib/zoneinfo_module.c:600:19: error: ‘_PyLong_One’ undeclared (first use in this function); did you mean ‘_PyLong_New’?
600 | one = _PyLong_One;
| ^~~~~~~~~~~
| _PyLong_New
lib/zoneinfo_module.c:600:19: note: each undeclared identifier is reported only once for each function it appears in
error: command '/usr/bin/gcc' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for backports.zoneinfo
and
ERROR: Could not build wheels for backports.zoneinfo, which is required to install pyproject.toml-based projects
I am very aware of this question which has the same error but the poster of that is building for Heroku which I am not using and the solutions given are heroku focused.

8th wall web app setup child compilation failed

I am new to 8th wall. I have cloned 8th wall web from git and executed below steps properly
# cd <directory_where_you_saved_sample_project_files>
# cd serve
# npm install
# cd ..
# ./serve/bin/serve -d <sample_project_location>
but on execution of last step which is for ex.
./serve/bin/serve -n -d gettingstarted/xraframe/ -p 7777
I am getting below errors
Failed to compile.
Error: Child compilation failed: Entry module not found: Error:
Can't resolve
'C:\8thWall_Project\web\serve\bin\gettingstarted\xraframe"
\index.html' in 'C:\8thWall_Project\web\serve': Error: Can't resolve
'C:\8thWall_Project\web\serve\bin\gettingstarted\xraframe"
\index.html' in 'C:\8thWall_Project\web\serve'
compiler.js:79 childCompiler.runAsChild
[serve]/[html-webpack-plugin]/lib/compiler.js:79:16
Compiler.js:306 compile
[serve]/[webpack]/lib/Compiler.js:306:11
Compiler.js:631 hooks.afterCompile.callAsync.err
[serve]/[webpack]/lib/Compiler.js:631:15
Hook.js:154 AsyncSeriesHook.lazyCompileHook
[serve]/[tapable]/lib/Hook.js:154:20
Compiler.js:628 compilation.seal.err
[serve]/[webpack]/lib/Compiler.js:628:31
Hook.js:154 AsyncSeriesHook.lazyCompileHook
[serve]/[tapable]/lib/Hook.js:154:20
Compilation.js:1325 hooks.optimizeAssets.callAsync.err
[serve]/[webpack]/lib/Compilation.js:1325:35
Any idea or pointers what is missing?
Thanks
I don't know why, but bat file don't want to be opened by path. Just go to the serve\bin directory and launch bat from here, like that:
7777 is unnecessary. problem was, that it can't find path to your xraframe
project, as you are in another directory, so you have to go tow directories up in ypur path for xraframe
It seems as if you're attempting this on a Windows computer. The serve process for Windows is slightly different than on macOS.
Instead of the normal serve script, use the serve.bat executable.
serve\bin\serve.bat -n -d gettingstarted\xraframe -p 7777
https://docs.8thwall.com/web/#locally-from-windows

$(pkg_build_dir) value in OpenWrt

What does $(build_dir), $(pkg_build_dir) has in OpenWrt.
I have create the sample helloworld component and created the Makefile by following steps in openwrt.org.
When I try:
make package/helloworld/install -j1 V=s
it throws below error:
Warning: Your configuration is out of Sync, Please run make memuconfig, oldconfi or defconfig!
Entering Directory '.../openwrt'
***no rule to make target 'pckage/helloworld/install'. Stop
Leaving Directory .../openwrt
/home../toplevel.mk:216: recipe for target 'package/helloworld/install' failed.
Can someone help me in resolving this?
$(build_dir), $(pkg_build_dir) are variable defaults used by OpenWRT.
$(build_dir), is the directory that OpenWRT builds to and
$(pkg_build_dir) is a package build directory.
$(build_dir) does not need to be defined in the makefile
$(pkg_build_dir) can be defined in the makefile as:
PKG_NAME:=helloworld
PKG_VERSION:=1.0
PKG_RELEASE:=1
PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_NAME)-$(PKG_VERSION)
With regards to your issue I would also try including the source directory in the makefile:
SOURCE_DIR:=/home/johndoe/helloworld
and updating the package feeds
cd /home/buildbot/source
./scripts/feeds update mypackages
./scripts/feeds install -a -p mypackag

My package in OpenWRT does not compile

I have downloaded OpenWRT source from git and build it. I created a simple helloworld package with the required Makefile in package/helloworld directory and build it. But it does not seem to be doing anything. Following is the log:
sonal#sonal-ThinkPad:~/openwrt$ make V=99 package/helloworld/compile
WARNING: your configuration is out of sync. Please run make menuconfig, oldconfig or defconfig!
make[1]: Entering directory '/home/sonal/openwrt'
make[2]: Entering directory '/home/sonal/openwrt/package/libs/toolchain'
if [ -f /home/sonal/openwrt/staging_dir/target-mips_34kc_uClibc-0.9.33.2/pkginfo/toolchain.default.install.clean ]; then rm -f /home/sonal/openwrt/staging_dir/target-mips_34kc_uClibc-0.9.33.2/pkginfo/toolchain.default.install /home/sonal/openwrt/staging_dir/target-mips_34kc_uClibc-0.9.33.2/pkginfo/toolchain.default.install.clean; fi; echo "libc" >> /home/sonal/openwrt/staging_dir/target-mips_34kc_uClibc-0.9.33.2/pkginfo/toolchain.default.install
if [ -f /home/sonal/openwrt/staging_dir/target-mips_34kc_uClibc-0.9.33.2/pkginfo/toolchain.default.install.clean ]; then rm -f /home/sonal/openwrt/staging_dir/target-mips_34kc_uClibc-0.9.33.2/pkginfo/toolchain.default.install /home/sonal/openwrt/staging_dir/target-mips_34kc_uClibc-0.9.33.2/pkginfo/toolchain.default.install.clean; fi; echo "libgcc" >> /home/sonal/openwrt/staging_dir/target-mips_34kc_uClibc-0.9.33.2/pkginfo/toolchain.default.install
make[2]: Leaving directory '/home/sonal/openwrt/package/libs/toolchain'
make[2]: Entering directory '/home/sonal/openwrt/package/helloworld'
make[2]: Leaving directory '/home/sonal/openwrt/package/helloworld'
make[1]: Leaving directory '/home/sonal/openwrt'`enter code here`
Thanks
Sonal
http://wiki.prplfoundation.org/wiki/Creating_an_OpenWrt_package_for_a_web_page
This above link will guide you how to build own package. I had also made own luci pages with above link. It will guide you from scratch.
If you want to create only hello world page then no need to create package, you just make new directory name as files in '/home/sonal/openwrt/'
and then go to files directory and make new directory as www in '/home/sonal/openwrt/files/'. No custom packages requires for it.

asm/socket.h: No such file or directory cross compiling Dart for Raspberry pi

I'm cross-compiling the Dart runtime using the instruction here.
I've installed all the dependencies as specified. I've also cloned the git repository with the necessary tool chain.
I'm running the runtime compilation with this command:
./tools/build.py -m release -a arm --toolchain=../tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/arm-linux-gnueabihf runtime
The compilation starts with no problem then it stops with this error:
LINK(target) out/ReleaseXARM/libdart_dependency_helper.target
CXX(host) out/ReleaseXARM/obj.host/libdart_lib_withcore/runtime/vm/bootstrap.o
In file included from /usr/include/sys/socket.h:38:0,
from /usr/include/netinet/in.h:23,
from /usr/include/arpa/inet.h:22,
from runtime/platform/globals.h:56,
from runtime/platform/assert.h:16,
from runtime/vm/allocation.h:8,
from runtime/vm/bootstrap.h:9,
from runtime/vm/bootstrap.cc:5:
/usr/include/bits/socket.h:345:24: fatal error: asm/socket.h: No such file or directory
#include <asm/socket.h>
^
compilation terminated.
CXX(host) out/ReleaseXARM/obj.host/libdart_lib_withcore/gen/async_gen.o
In file included from /usr/include/sys/socket.h:38:0,
from /usr/include/netinet/in.h:23,
from /usr/include/arpa/inet.h:22,
from runtime/platform/globals.h:56,
from runtime/platform/assert.h:16,
from runtime/vm/allocation.h:8,
from runtime/vm/bootstrap.h:9,
from out/ReleaseXARM/obj/gen/async_gen.cc:5:
/usr/include/bits/socket.h:345:24: fatal error: asm/socket.h: No such file or directory
#include <asm/socket.h>
^
compilation terminated.
runtime/libdart_lib_withcore.host.mk:978: recipe for target 'out/ReleaseXARM/obj.host/libdart_lib_withcore/runtime/vm/bootstrap.o' failed
make: *** [out/ReleaseXARM/obj.host/libdart_lib_withcore/runtime/vm/bootstrap.o] Error 1
make: *** Waiting for unfinished jobs....
runtime/libdart_lib_withcore.host.mk:986: recipe for target 'out/ReleaseXARM/obj.host/libdart_lib_withcore/gen/async_gen.o' failed
make: *** [out/ReleaseXARM/obj.host/libdart_lib_withcore/gen/async_gen.o] Error 1
BUILD FAILED
Am I missing any dependency or package?
I hit the same problem. On my ubuntu 14.04 system /usr/include/asm didn't exist. It was called asm-generic instead. I sym-linked it and the build was able to continue.
cd /usr/include
sudo ln -s asm-generic/ asm
The build was able to continue after that.
This is probably because you're trying to build an application without some of the include paths correctly set, for example using a 32-bit gcc on a 64-bit platform.
To resolve:
sudo apt-get install gcc-multilib
I'm not sure why this happens, but sometimes /usr/include/asm gets deleted. My teammates who looked at their Ubuntu x86-64 workstations found that the asm symlink was:
0 lrwxrwxrwx 1 root root 20 May 22 2013 /usr/include/asm -> x86_64-linux-gnu/asm
And the command to recreate it is:
$ cd /usr/include
$ sudo ln -s x86_64-linux-gnu/asm asm
The files in /usr/include/asm-generic are sometimes, but not always, equivalent to the files in the x86-64 specific directory; so it's difficult to recommend symlinking it as a workaround.

Resources