I'm trying to use ImageMagick to read SVG files. This works just fine locally, since I have librsvg (https://github.com/GNOME/librsvg) installed on my machine. However, the package doesn't seem to be available through yum, which is what Elastic Beanstalk uses.
So in my backend (Rails/Postgresql) I have something like:
def save_svg_as_image
file = params[:uploaded_svg]
#svg_image = MiniMagick::Image.read(file)
self.svg = #svg_image
end
Again, this works fine on local thanks to librsvg, but when testing with my EB (64bit Amazon Linux/2.11.4), I get:
MiniMagick::Invalid (`identify /tmp/mini_magick20210205-24760-bygqeh` failed with error:
identify: no decode delegate for this image format `/tmp/mini_magick20210205-24760-bygqeh' # error/constitute.c/ReadImage/544.
):
I've attempted using the following solution: Taking the code from https://gist.github.com/whyvez/1e0212a35da97aa8f1b1 and using it in a packages.config file, but I'm continually getting the following error:
Command failed on instance. Return code: 2 Output: wget: unrecognized option '--prefix=/usr'
Related
When I tried to run a Batch transform job in AWS SageMaker, I met below error:
ImportError: No module named cv2
Please note that, I am able to "import CV2" in the notebook instance. The jupter can run "import CV2" in notebook instance. But failed to run it in endpoints during inference time. I have tried below method using "env" as the link AWS Sagemaker - Install External Library and Make it Persist
but it still not work.
anyone have good way to solve it? Thanks!
my codes are:
env = {
'SAGEMAKER_REQUIREMENTS': 'requirements.txt', # path relative to `source_dir` below.
}
image_embed_model = MXNetModel(model_data=model_data,
entry_point='sagemaker_entrypoint.py',
role=role,
source_dir = 'src',
env = env,
py_version='py3',
framework_version='1.6.0')
transformer = image_embed_model.transformer(instance_count=1, # Please pay attention here!!!
instance_type='ml.m4.xlarge',
output_path=output_path,
assemble_with = 'Line',
accept = 'text/csv'
)
transformer.transform(batch_input,
content_type='text/csv',
split_type='Line',
input_filter='$[0:]',
join_source='Input',
wait=False)
You can follow https://github.com/aws/sagemaker-python-sdk/blob/master/doc/using_mxnet.rst#use-third-party-libraries to import third party libraries to your batch transform instances. Make sure the requirement.txt file is saved under the right directory before packaging the model data.
I am trying to run hadoop using docker provided here:
https://github.com/big-data-europe/docker-hadoop
I use the following command:
docker-compose up -d
to up the service and am able to access it and browse file system using: localhost:9870. Problem rises whenever I try to use pyhdfs to put file on HDFS. Here is my sample code:
hdfs_client = HdfsClient(hosts = 'localhost:9870')
# Determine the output_hdfs_path
output_hdfs_path = 'path/to/test/dir'
# Does the output path exist? If not then create it
if not hdfs_client.exists(output_hdfs_path):
hdfs_client.mkdirs(output_hdfs_path)
hdfs_client.create(output_hdfs_path + 'data.json', data = 'This is test.', overwrite = True)
If test directory does not exist on HDFS, the code is able to successfully create it but when it gets to the .create part it throws the following exception:
pyhdfs.HdfsIOException: Failed to find datanode, suggest to check cluster health. excludeDatanodes=null
What surprises me is that my code is able to create the empty directory but fails to put the file on HDFS. My docker-compose.yml file is exactly the same as the one provided in the github repo. The only change I've made is in the hadoop.env file where I change:
CORE_CONF_fs_defaultFS=hdfs://namenode:9000
to
CORE_CONF_fs_defaultFS=hdfs://localhost:9000
I have seen this other post on sof and tried the following command:
hdfs dfs -mkdir hdfs:///demofolder
which works fine in my case. Any help is much appreciated.
I would keep the default CORE_CONF_fs_defaultFS=hdfs://namenode:9000 setting.
Works fine for me after adding a forward slash to the paths
import pyhdfs
fs = pyhdfs.HdfsClient(hosts="namenode")
output_hdfs_path = '/path/to/test/dir'
if not fs.exists(output_hdfs_path):
fs.mkdirs(output_hdfs_path)
fs.create(output_hdfs_path + '/data.json', data = 'This is test.')
# check that it's present
list(fs.walk(output_hdfs_path))
[('/path/to/test/dir', [], ['data.json'])]
Getting the error Error: [$injector:unpr] Unknown provider: IdleProvider in my application when it is deployed to our staging server using dokku but I am not getting it when running it on my local machine. I'm using ng-idle 1.2.1
I've found this question asked a number of times but the cause was always related to the changes made in version 1.0.0 where the service names were changed. The only thing I can think of is that the minification of the code is the problem but as far as I can see the code should be ok but I am not an expert. Any help would be greatly appreciated.
It's written in Coffeescript
configuration = (RestangularProvider, $logProvider, growlProvider, IdleProvider, KeepaliveProvider) ->
.
.
.
return
configuration.$inject = [
'RestangularProvider'
'$logProvider'
'growlProvider'
'IdleProvider'
'KeepaliveProvider'
]
angular
.module 'vssApp.config', [
'restangular'
]
.config configuration
EDIT
While trying to replicate the problem on my local machine I removed the 'ngIdle' module in the modules array below. This resulted in the same behavior so I am assuming that the problem stems from the ngIdle module not being loaded correctly here. I still feel that minification could be causing the problem but, again, I'm not sure why or how to fix it.
modules = [
'ui.router'
'ui.bootstrap'
'ui.select'
'ngAnimate'
'ngMessages'
'ngSanitize'
'ngCookies'
'smart-table'
'angularMoment'
'templates'
'angular-storage'
'angular-growl'
'vssApp.core.auth'
'vssApp.core.loading'
'ngIdle'
'cgPrompt'
'vssApp.filters'
]
runBlock.$inject = [
'$templateCache'
]
angular
.module 'vssApp.core', modules
.run runBlock
EDIT 2
Here's the full output from the error message I'm getting
Error: [$injector:modulerr] Failed to instantiate module vssApp due to:
Error: [$injector:modulerr] Failed to instantiate module vssApp.config due to:
Error: [$injector:unpr] Unknown provider: IdleProvider
http://errors.angularjs.org/1.3.16/$injector/unpr?p0=IdleProvider
at https://SERVER/assets/application-85a5fd382c73380bf2a71b66e581c941.js:3:18814
at https://SERVER/assets/application-85a5fd382c73380bf2a71b66e581c941.js:4:16489
at getService (https://SERVER/assets/application-85a5fd382c73380bf2a71b66e581c941.js:4:14903)
at Object.invoke (https://SERVER/assets/application-85a5fd382c73380bf2a71b66e581c941.js:4:15466)
at runInvokeQueue (https://SERVER/assets/application-85a5fd382c73380bf2a71b66e581c941.js:4:13793)
at https://SERVER/assets/application-85a5fd382c73380bf2a71b66e581c941.js:4:14062
at forEach (https://SERVER/assets/application-85a5fd382c73380bf2a71b66e581c941.js:3:19482)
at loadModules (https://SERVER/assets/application-85a5fd382c73380bf2a71b66e581c941.js:4:13587)
at https://SERVER/assets/application-85a5fd382c73380bf2a71b66e581c941.js:4:13964
at forEach (https://SERVER/assets/application-85a5fd382c73380bf2a71b66e581c941.js:3:19482)
http://errors.angularjs.org/1.3.16/$injector/modulerr?p0=vssApp.config&p1=E…net%2Fassets%2Fapplication-85a5fd382c73380bf2a71b66e581c941.js%3A3%3A19482)
at https://SERVER/assets/application-85a5fd382c73380bf2a71b66e581c941.js:3:18814
at https://SERVER/assets/application-85a5fd382c73380bf2a71b66e581c941.js:4:14406
at forEach (https://SERVER/assets/application-85a5fd382c73380bf2a71b66e581c941.js:3:19482)
at loadModules (https://SERVER/assets/application-85a5fd382c73380bf2a71b66e581c941.js:4:13587)
at https://SERVER/assets/application-85a5fd382c73380bf2a71b66e581c941.js:4:13964
at forEach (https://SERVER/assets/application-85a5fd382c73380bf2a71b66e581c941.js:3:19482)
at loadModules (https://SERVER/assets/application-85a5fd382c73380bf2a71b66e581c941.js:4:13587)
at createInjector (https://SERVER/assets/application-85a5fd382c73380bf2a71b66e581c941.js:4:16844)
at doBootstrap (https://SERVER/assets/application-85a5fd382c73380bf2a71b66e581c941.js:3:28466)
at bootstrap (https://SERVER/assets/application-85a5fd382c73380bf2a71b66e581c941.js:3:28995)
http://errors.angularjs.org/1.3.16/$injector/modulerr?p0=vssApp&p1=Error%3A…net%2Fassets%2Fapplication-85a5fd382c73380bf2a71b66e581c941.js%3A3%3A28995)
A rule of thumb is to load modules dependencies in each place where they are used. This allows to decouple them. And this eliminates race condition with service provider injection.
If the app looks like this
angular.module('vssApp', ['vssApp.config', 'ngIdle', ...])..
angular.module('vssApp.config', ['restangular'])...
service provider for Idle service is not defined at the time when vssApp.config module is loaded.
While this
angular.module('vssApp', ['ngIdle', 'vssApp.config', ...])
angular.module('vssApp.config', ['restangular'])...
avoids race condition but still indicates code smell.
It should be
angular.module('vssApp.config', ['restangular', 'ngIdle'])...
This issue applies to service providers only and config phase. Service instances can be injected for any module order.
Finally found the cause and solution to this, it seems to have been a bower issue.
It's a Rails app, so I specified ng-idle 1.2.1 in the bower.json file but for some reason the bower file was ignored when the app was being deployed using Dokku and the last installed version 0.3.5 remained, which meant that the pre-1.0.0 ng-idle services naming convention was still being used where all service names were preceded with a $. This resulted in the Unknown provider: IdleProvider error because $IdleProvider was the actual service name.
In the end I had to connect to the docker container and remove and reinstall all bower components. Running bower update as part of the deployment was not enough for some reason. When I have more time I will investigate what caused this behavior and I will report here.
I'm in a big trouble, I can't start Elasticsearch and I need it for run my rails locally, please tell me what's going on. I installed Elasticsearch in the normal fashion then I did the following:
elasticsearch --config=/usr/local/opt/elasticsearch/config/elasticsearch.yml
But it shows the following error: [2015-11-01 20:36:50,574][INFO ][bootstrap] es.config is no longer supported. elasticsearch.yml must be placed in the config directory and cannot be renamed.
I tried several alternative ways of run it, like:
elasticsearch -f -D
But then I get the following error, and I can't find any useful for solve it, it seems to be related with file perms but not sure:
java.io.IOException: Resource not found: "org/joda/time/tz/data/ZoneInfoMap" ClassLoader: sun.misc.Launcher$AppClassLoader#33909752
at org.joda.time.tz.ZoneInfoProvider.openResource(ZoneInfoProvider.java:210)
at org.joda.time.tz.ZoneInfoProvider.<init>(ZoneInfoProvider.java:127)
at org.joda.time.tz.ZoneInfoProvider.<init>(ZoneInfoProvider.java:86)
at org.joda.time.DateTimeZone.getDefaultProvider(DateTimeZone.java:514)
at org.joda.time.DateTimeZone.getProvider(DateTimeZone.java:413)
at org.joda.time.DateTimeZone.forID(DateTimeZone.java:216)
at org.joda.time.DateTimeZone.getDefault(DateTimeZone.java:151)
at org.joda.time.chrono.ISOChronology.getInstance(ISOChronology.java:79)
at org.joda.time.DateTimeUtils.getChronology(DateTimeUtils.java:266)
at org.joda.time.format.DateTimeFormatter.selectChronology(DateTimeFormatter.java:968)
at org.joda.time.format.DateTimeFormatter.printTo(DateTimeFormatter.java:672)
at org.joda.time.format.DateTimeFormatter.printTo(DateTimeFormatter.java:560)
at org.joda.time.format.DateTimeFormatter.print(DateTimeFormatter.java:644)
at org.elasticsearch.Build.<clinit>(Build.java:51)
at org.elasticsearch.node.Node.<init>(Node.java:135)
at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:145)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:170)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
[2015-11-01 20:40:57,602][INFO ][node ] [Centurius] version[2.0.0], pid[22063], build[de54438/2015-10-22T08:09:48Z]
[2015-11-01 20:40:57,605][INFO ][node ] [Centurius] initializing ...
Exception in thread "main" java.lang.IllegalStateException: failed to load bundle [] due to jar hell
Likely root cause: java.security.AccessControlException: access denied ("java.io.FilePermission" "/usr/local/Cellar/elasticsearch/2.0.0/libexec/antlr-runtime-3.5.jar" "read")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at java.security.AccessController.checkPermission(AccessController.java:884)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
at java.lang.SecurityManager.checkRead(SecurityManager.java:888)
at java.util.zip.ZipFile.<init>(ZipFile.java:210)
at java.util.zip.ZipFile.<init>(ZipFile.java:149)
at java.util.jar.JarFile.<init>(JarFile.java:166)
at java.util.jar.JarFile.<init>(JarFile.java:103)
at org.elasticsearch.bootstrap.JarHell.checkJarHell(JarHell.java:173)
at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:340)
at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:113)
at org.elasticsearch.node.Node.<init>(Node.java:144)
at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:145)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:170)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
Refer to the log for complete error details.
Thanks for your help.
There are some changes with libexec with Elasticsearch/homebrew installation and that is why it is failing to start. There is a PR #45644 currently being worked on. Till the PR gets accepted, you can use the same formula to fix the installation of Elasticsearch.
First uninstall the earlier/older version. Then edit the formula of Elasticsearch:
$ brew edit elasticsearch
And use the formula from the PR.
Then do brew install elasticsearch, it should work fine.
To start Elasticsearch, just do:
$ elasticsearch
config option is no longer valid. For custom config, use path.config:
$ elasticsearch --path.conf=/usr/local/opt/elasticsearch/config
In my project I upload audiofiles to GridFS using CarrierWave gem. After uploading file is saved to GridFS properly but in my application I am unable to get it from GridFS with MongoFiles Tool or with GridFS-nginx module.
mongofiles get audiotracks/4dfb70d6bcd73f3488000002/data
command leads to this error:
assertion: 13325 couldn't open file: audiotracks/4dfb70d6bcd73f3488000002/data
The only way to get file is to use rails console and it works fine:
cc = Mongo::GridFileSystem.new(Mongo::Connection.new.db("test")).open('audiotracks/4dfb70d6bcd73f3488000002/data', 'r')
cc.read
So if you have encountered problem like this or have some ideas - plz let me know.
mongofiles get will try to write the file to disk with the same name and path as in GridFS.
Assertion 13325 happens when GridFS can't write the file like this.
You should check if the file path exists and you have the permission to write the file. Alternatively you could just provide a file name with the --local parameter.
mongofiles --local mytrack.mp3 get audiotracks/4dfb70d6bcd73f3488000002/data