I'm running a workflow pipeline in GitHub which uses ImageMagick.
When I run tests which use ImageMagick on PDF files, I get the following error:
Command failed: convert "/tmp/test.pdf[0]" "/tmp/test-0.png"
convert-im6.q16: attempt to perform an operation not allowed by the security policy `PDF' # error/constitute.c/IsCoderAuthorized/408.
convert-im6.q16: no images defined `/tmp/test-0.png' # error/convert.c/ConvertImageCommand/3258.
I found the following posts which present a solution
ImageMagick security policy 'PDF' blocking conversion
https://bugs.archlinux.org/task/60580
But as this is ran in GitHub workflow, I was wondering maybe there's a simpler solution?
Related
I am trying to run bazel from a company-installed location. It decides to use my home directory for some caching and extracting installation files. There is no much space (company policy). So, the question is: how to redirect it to a different location? I could not find much on the web.
WARNING: Output base '/home/name/.cache/bazel/_bazel_name/b05102fc551c33214ce89fb43ea90837' is on NFS. This may lead to surprising failures and undetermined behavior.
Extracting Bazel installation...
FATAL: Failed to extract embedded binaries: Failed to write zipped file '/home/name/.cache/bazel/_bazel_name/install/f6ca571514ebc1be8327564b4455aae2.tmp.10086/_embedded_binaries/A-server.jar': (error: 122): Disk quota exceeded
Also, not sure why it thinks that this is NFS. I try to run it onlinux.
found it: bazel --output_user_root=<loc> fixes the issue.
Issue Summary:
Hi,
I am using avro version 1.11.0 for parsing an avro file and decoding it. We have a custom requirement, so i am not able to use ReadFromAvro. When trying this with dataflow there arises a dependency issues as avro-python3 with version 1.82 is already available. The issue is of class TimestampMillisSchema which is not present in avro-python3. It fails stating Attribute TimestampMillisSchema not found in avro.schema. I then tried passing a requirements file with avro==1.11.0 but now the dataflow was not able to start giving error "Error syncing pod" which seems to be because of dependencies conflicts.
To Solve the issue , we set an experiment flag (--experiments=no_use_multiple_sdk_containers ) which ran fine.
I want to know a better solution of my issue and also does the above flag will effect the pipeline performance.
Please try with the dataflow run command:
--prebuild_sdk_container_engine=cloud_build --experiments=use_runner_v2
this would use cloud build to build the container with your extra dependencies and then would use it within the dataflow run.
I have looked at many other posts related to this issue and have tried each solution. None have worked in my case, including copying over the makevars from Rcpp. Anyhow, when building on Travis I get the following error
undefined symbol: dpotrf_’
The interesting note is that the package installs fine on windows, macOS, and linux.
here is my repo R package
I can reproduce the failure on a very standard Debian testing system (which I use for the extensive reverse dependency checks on Rcpp and RcppArmadillo).
After installing packages bain and BFpack (I had the rest) I attempted to build the tar.gz from your pristine just-checked-out sources. And I get:
*** installing help indices
*** copying figures
** building package indices
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘BGGM’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/tmp/Rinst106c6ed5251a/00LOCK-BGGM/00new/BGGM/libs/BGGM.so':
/tmp/Rinst106c6ed5251a/00LOCK-BGGM/00new/BGGM/libs/BGGM.so: undefined symbol: dpotrf_
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/tmp/Rinst106c6ed5251a/BGGM’
-----------------------------------
ERROR: package installation failed
This appears to be a moderately complex and large enough package so please pardon me for not diving in and debugging. I would suggest you simplify with smaller mock packages to see what may be wrong. (dpotrf is a fairly standard LAPACK routine so something somewhere calls it. Maybe you call it explicitly. Maybe you did a Fortran-to-C mapping wrong. Maybe you have something wrong in how you interface with RcppArmadillo. Hard to tell...)
Edit: You committed compiled code and a Windows library. "Don't do that." When Travis builds it also starts from a git checkout as I did. That may be the difference.
Edit 2: It wasn't, but your R code mixes .Call() with generated entry points (ie via RcppExports.cpp and RcppExports.R). I have seen that blow up for other people. That may be something to look into.
Disclaimer: I work with D_Williams, but I figured out the problem, and others may find it useful.
A functioning configure.ac was present, and a Makevars.in is present.
The problem is that the configure file was not yet generated. This is an autotools/autoconf setup. To resolve it, I ran autoconf in the package directory, which generated the configure file. That configure file is then executed when R builds the package. The configure file modifies the Makevars.in and creates Makevars. That Makevars file ultimately defines where to find libraries, includes, compilers, compiler options, etc.
If you do not generate the configure file from configure.ac using autoconf, then there is no configure file to be executed, and no Makevars to define the needed options at compile time. Therefore, the compiler is not fully configured, and it will fail.
TLDR: If you have an configure.ac, you must run autoconf on it, and commit that configure file to your repo. R needs to execute it to have a functioning Makevars.
I am exploring the new EMR 6.0.0 with Docker support in order to make decision if we want to use it. One of our projects is written in Scala 2.11. But EMR 6.0.0 comes with Spark built from Scala 2.12. So I switched to try 6.00-beta, which is Spark 2.4.3 built from Scala 2.11. If it works on 6.0.0-beta, then we will upgrade our code to Scala 2.12 and use 6.0.0.
A few issues I am having are when I tried to run my Scala spark job:
When it tried to read parquet from S3, I got error: java.lang.RuntimeException: Cannot create temp dirs: [/mnt/s3]
When I tried to make API call with https, I got error: usun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target.
When it tried to read files from S3, I got error: Class com.amazon.ws.emr.hadoop.fs.EmrFileSystem not found. I was able to able to hack this one by passing the path by --jars. Maybe not the best solution.
I am guessing there must be something I need to set either during bootstrap or in the Docker file.
Can someone please help? Thanks!
I figure out the S3 issue. In beta version, /mnt/s3 is not mounted and given the read and write permission.
So I need to add the "docker.allowed.rw-mounts" to the container-executor configuration like below:
docker.allowed.rw-mounts=/etc/passwd,/mnt/s3
I am trying to migrate a huge project having visual studio and maven projects to bazel. I need to access our in house maven server which is encrypted. To get access I need the load the maven_jar skylark extension since the default impl does not support encryption (get error 401). using the extension leads to a lot of troubles, like:
ERROR: BUILD:4:1: no such package '#org_bouncycastle_bcpkix_jdk15on//jar': Traceback (most recent call last):
File ".../external/bazel_tools/tools/build_defs/repo/maven_rules.bzl", line 280
_maven_artifact_impl(ctx, "jar", _maven_jar_build_file_te...)
File ".../external/bazel_tools/tools/build_defs/repo/maven_rules.bzl", line 248, in _maven_artifact_impl
fail(("%s: Failed to create dirs in e...))
org_bouncycastle_bcpkix_jdk15on: Failed to create dirs in execution root.
The main issue seems to be the shell that needs to be provided to bazel in BAZEL_SH environment variables:
I am working under windows
I am using bazel 0.23.2
bazel seems to run a bash command using "bash" directly and not the one provided by env variable.
I got a ubuntu shell installed in windows. bazel was using everything from ubuntu, especially when using maven (settings.xml was using from ubuntu ~/.m2 and not from windows user)
after uninstalling ubuntu and making sure that bash in a cmd ends up in "command not found" I also removed the BAZEL_SH env var and bazel throws the message above
after setting the BAZEL_SH variable again it fails with the same error message
I am assuming that bazel gets a bash from somewhere or is ignoring the env variable. My questions are:
1. How to setup a correct shell?
2. Is BAZEL_SH needed when using current version?
For me the doc at bazel website about setup is outdated.
Cheers
Please consider using rules_jvm_external to manage your Maven dependencies. It supports both Windows and private repositories using HTTP Basic Authentication.
For me the doc at bazel website about setup is outdated.
The Bazel team is aware of this and will be updating our docs shortly.