I am using GraphicsMagick 1.3.26 on server.
Due, to performance impact. I want to remove OpenMP feature from it.
Kindly provide me the steps to remove the same for windows machine.
Thanks.
Never tried on Windows, and unable to test, but apparently you can either:
add -limit threads 1 to each command, or
rebuild with --disable-openmp and reinstall, or
set environment variable OMP_NUM_THREADS to 1 before running
Reference
Use:
gm identify -version
to see how your instance was built.
Related
Introduction:
I have to create a pip wheel of Tensorflow 2.2.0 with cuda libraries dynamically linked(specifically cudart.so). To accomplish this i am currently using the tensorflow-dev docker image.
I am able to build the tf wheel file, an able to install and use it while inside the build container.
Issue:
The issue is that importing the generated wheel file in a CentOS server, i get the following error:
ImportError: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by /home1/private/mavridis/Vineyard/tensorflowshared/test/lib64/python3.6/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
Having looked around, the issue is caused by the build container using a newer libc:
ldd --version
ldd (Ubuntu GLIBC 2.27-3ubuntu1) 2.27
Compared to CentOS older version:
ldd --version
ldd (GNU libc) 2.17
Expected behavior:
Having already tried the 'vanilla' tenorflow 2.2.0 version with no issues, installed using pip:
pip install tensorflow==2.2.0
I expected my own build to also work.
So i assume there is some configuration option or docker configuration to allow me to use the docker built wheel file, in a CentOS setup, just like the pip installed version. As this wheel file is intended to be deployed to setups beyond my control, solutions involving alternate OSes and/or libc replacement are not applicable.
Build configuration:
During build i use the following configuration/ command line:
export TF_NEED_CUDA=1
export TF_USE_XLA=0
export TF_SET_ANDROID_WORKSPACE=0
export TF_NEED_OPENCL_SYCL=0
export TF_NEED_ROCM=0
bazel build --config=opt --config=cuda --output_filter=DONT_MATCH_ANYTHING --linkopt=-L/usr/local/cuda/lib64 --linkopt=-lcudart --linkopt=-static-libstdc++ //tensorflow/tools/pip_package:build_pip_package
Regarding options used:
--output_filter=DONT_MATCH_ANYTHING : Silence warnings
--linkopt=-L/usr/local/cuda/lib64 --linkopt=-lcudart : Dynamic linking of cudart.so
--linkopt=-static-libstdc++ : Static link libstc++ as libstc++ also caused the libc error, this however is not possible for libm
I expected my own build to also work.
That expectation is (obviously) incorrect. The symbols your program or library requires from GLIBC depend on exactly which functions you call.
Consider the following program:
int main() { exit(0); }
When compiled/linked on a GLIBC-2.30 system, this program only depends on GLIBC_2.2.5 (because it doesn't call any newer symbols).
Now change the program slightly:
int main() { gettid(); exit(0); }
Compile/link it again, and all of a sudden this program now requires GLIBC_2.30 (because that's where gettid() was added to GLIBC), and will not work on any system which has older GLIBC.
So i assume there is some configuration option or docker configuration
Sure: your Docker image must have GLIBC that is not newer than what your target system have, i.e. GLIBC-2.17. Your current image contains GLIBC-2.27 (or newer).
You need a different Docker image, and you'll likely have to build it yourself, since GLIBC-2.17 is over 7 years old, and predates TensorFlow by many years.
Update:
What i don't understand is how come the pip tensorflow package (which i assumed was build with the docker image i am using) works with CentOS?
It works by accident, just like my first program would work on CentOS, but the second one wouldn't.
In short i wanted to generate a pip package that would work on 'any' linux/libc version
That is an impossible goal: Linux predates GLIBC, and it is impossible to build a single package that will work on a Linux distribution which didn't include GLIBC and on a distribution that did.
You have to draw a line somewhere. The developers of tensorflow-dev docker image drew a line at GLIBC-2.27. Packages built on this image should work on any system with 2.27 or later, and might (but are not at all guaranteed to) work on older systems.
just like the pip installed version.
You claim that the pip installed version has no "only GLIBC-xx or later" requirement, but that is not true. I am 99.9% sure that it requires at least GLIBC-2.14.
To find which GLIBC versions that package requires, run this command:
readelf -WV _pywrap_tensorflow_internal.so | grep GLIBC_
I assumed, the pip installed version was built using the publicly available tensorflow-devel docker image.
That is quite likely. And like I said, it happens to work on CentOS, but minute changes may make it not work anymore.
Update 2:
So running the readelf command as you suggested, does show the most recent required versions to be: - pip version: GLIBC_2.12 - mine : GLIBC_2.27 So from what i understand the pip version uses an older version even from CentOS, which explains why it works.
It doesn't "use" older version, it uses whatever version is available.
It requires a minimum version 2.12, while your build requires a minimum version 2.27.
How do they achieve this? Do they use a different image that has an older libc? If so, where can i get it? Or do they use the public image, but build with some bazel flag, that 'limits' symbols to the ones contained up to libc 2.12?
You are still not getting it.
The version that your program requires depends on exactly which functions you call. In my example program, if I only call exit, my program requires vesion 2.2.5, but if I also call gettid, then my program requires version 2.30. Note: these two programs are built on the same system with the same flags.
So no: they (most likely) didn't use a different Docker image, and didn't use "magic" bazel flags. They just happened to not call any functions which require GLIBC version > 2.12, and you did.
P.S. You can find which symbol(s) are causing "bad" dependency in your build like so:
readelf -Ws _pywrap_tensorflow_internal.so | egrep 'GLIBC_2.2[0-9]'
readelf -Ws _pywrap_tensorflow_internal.so | egrep 'GLIBC_2.1[89]'
This would produce output similar to (using my second program):
readelf -Ws a.out | egrep 'GLIBC_2.[23][0-9]'
2: 0000000000000000 0 FUNC GLOBAL DEFAULT UND gettid#GLIBC_2.30 (2)
48: 0000000000000000 0 FUNC GLOBAL DEFAULT UND gettid##GLIBC_2.30
The output above shows that the only symbol my binary requires from GLIBC 2.20 or above is gettid.
To make a counter point to what Employed Russian wrote:
The version that your program requires depends on exactly which functions you call. In my example program, if I only call exit, my program requires vesion 2.2.5, but if I also call gettid, then my program requires version 2.30. Note: these two programs are built on the same system with the same flags.
I don't think that's quite accurate. My understanding, which is corroborated by https://github.com/wheybags/glibc_version_header, is that things work like so (quoting that project, emphasis mine):
Glibc uses something called symbol versioning. This means that when you use e.g., malloc in your program, the symbol the linker will actually link against is malloc#GLIBC_YOUR_INSTALLED_VERSION (actually, it will link to malloc from the most recent version of glibc that changed the implementaton of malloc, but you get the idea).
So my guess (I have not checked) would be that the Tensorflow releases are built against an older glibc (perhaps by way of being built on an older release of their target Linux distro).
I'm trying to setup ImageMagick so I can use the Paperclip gem for Ruby on Rails. I've looked at pretty much every question on StackOverflow about the issue, but they all just skip over my issue, that none of the ImageMagick commands are recognized on the CommandPrompt.
I downloaded and then installed ImageMagick-7.0.1-1-Q16-x64-dll.exe on this page
I installed the program to my C: directory, so that there wouldn't be an issue with spaces in the Path, and I used all default installation selections and even selected "Install development headers" which some answers said to do. My issue is that when ImageMagick finishes installing, and it says to test some commands (like "convert -version) on Command Prompt, they all result in
"'convert' is not recognized as an internal or external command ...."
On the command prompt. What could be an issue? Every step-by-step tutorial automatically assumes that these commands are recognized. Most troubleshooting involves steps afterwards.
Here's the documentation for paperclip. All I have to do is make sure the gem has access to ImageMagick, by putting in the development.rb file:
Paperclip.options[:command_path] = "/usr/local/bin/"
In order to find that path above, however, the documentation says to type "which convert". I've researched this and apparently that's a Unix command, and not something for Windows.
So basically, what steps do I need to take so that paperclip has access to ImageMagick? Why aren't basic ImageMagick commands recognized, even after a completed installation?
When you install ImageMagick under Windows, they is a button you need to check that tells the script to install "legacy" utilities such as convert. Reinstall, click on this button, and convert is ready to use. Alternatively, install ImageMagick 6.X which deploys convert automatically.
Windows users, I've answered most of this here.
The windows command line won't run which convert. You'll need to use something like git bash.
Reinstalling image magick won't hurt. You could try another file. I know the dll one (you mentioned) is top of the list, the next one is 64bit as well. You must check every option at install as paperclip will use lots of image magick functionality behind the scenes so check all options. Then restart the system.
in command before typing convert you have to type in magick then convert
keep in mind you are using a newer version of IM hence you have to start by typing 'magick' and then follow other command
I'm trying to deploy a grails application on cloudbees.
In order to do that, I have a Jenkins job that builds the war and it deploy it.
My application uses ImageMagick in order to resize pictures.
After reading on cloudbees site that their servers have already ImageMagick installed, I did a test:
In my Jenkiks job a have a "execute shell" part where I put :
convert --version
When running the job, everything works fine and I see on the console output:
+ convert --version
Version: ImageMagick 6.7.5-6 2012-08-11 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2012 ImageMagick Studio LLC
Features: OpenMP
My application uses im4java wrapper.
When I try to resize a picture in my application I have the following error:
org.im4java.core.CommandException: java.io.IOException: Cannot run program "convert": error=2, No such file or directory
So, I don't understant why does not works since ImageMagick is installed.
Can you please give me some idea ?
Thanks.
C.C.
I guess the problem here is that ImageMagick is installed as a build tool on the development environment (DEV#cloud) but not on the stack you are using for the runtime environment (RUN#cloud). I cannot see any ImageMagick dependency added into the Tomcat 6-7 stack.
Only way to have imagemagick on runtime is to create a specific new stack which makes it available for you. So need to fork the git repo and use genapp as a development tool for the new stack.
I am a newb with python and just learning what to do.
I am using pyscripter and have been for a while whilst learning.
I am now going through an online course which is taught in 2.6, yet my pyscripter uses the latest.
I need to know how to change it to use an older version, I have seen replies about changing the PATH variable but not where it is or how to do it.
I have 3 versions of python on my machine, 25,26 and 33.
I don't know if this is the best way to do it, but those are the two ways I did it:
WAY 1 (The best of two)
Go to PyScripter>>Tools>>Options...>>Custom Parameters... and add the following values
1. PythonDir = C:\Program Files\CustomPythonInstallation
2. PythonExe = C:\Program Files\CustomPythonInstallation\python.exe
3. PythonVer = 3.3.3
Note: Adapt the Name = Value pairs above to your case.
And close the window with OK button.
Now select PyScripter>>Run>>Python Engine>>Remote and your are ready to go.
WAY 2 (The more temporary solution)
Go to PyScripter>>Run>>Configure External Run...
set the "Application:" field to your python.exe file
Close the window with OK button.
Make sure you run your scripts with PyScripter>>Run>>External Run (Alt+F9)
I hope this helped, good luck.
The easiest way I know (on Windows) is, having used the installer executable, I select from the Start menu's PyScripter folder whichever version of Python I want to run.
You can modify the PYTHONPATH (under Pyscripter>>Tools, for instance)
You can modify your External Python Interpreter with Pyscripter>>Modify Tools>>Python &Interpreter>>Modify
You can modify the default Python engine used with Pyscripter>>Options>>IDE Options>>Python Interpreter>>Python Engine Type
You can simply redirect Pyscripter to see the environment of a different Python distribution.
In Windows, do this by assigning PYTHONDLLPATH in the Pyscripter shortcut. You can r-click on the shortcut, access its properties and then set the target to:
[Pyscripter executable dir] --PYTHONDLLPATH [Python distribution dir]
See this image to help you out:
setting a shortcut target
For example, in my Win10 64-bit computer I have a Python 2.7.8 installation back from when I installed ArcGIS, which is automatically recognized by my 32-bit Pyscripter installation.
In the same computer, I also have Anaconda installed with two environments that feature two 64-bit Python distributions:
2.7.14 in "C:\ProgramData\Anaconda2"
3.6 in "C:\Users\bouzi\AppData\Local\conda\conda\envs\py3"
When I installed a 64-bit version of Pyscripter, that Pyscripter version couldn't even open, as it couldn't find the conda distributions. I had to point them to it by replacing the shortcut target to:
"C:\Program Files\PyScripterx64\PyScripter.exe" --PYTHONDLLPATH "C:\ProgramData\Anaconda2"
You can create three Pyscripter shortcuts that point to these different installations of Python within your system. It's probably not the optimal way to deal with this but it works, and allows you to combine Anaconda environments with Pyscripter.
You can also read more on opening non-standard python distributions with PyScripter from this link.
Run->Python Versions -> setup Python Versions -> Add... select folder
p.s.
python 3.7.3 - ok,
still python 3.10.5 could not be identified by PyScripter in such a way (actually works with WAY_1 Solution in this thread but pip install under such env. not succeed afterwards)
I keep getting the same type of error messages and I have really done all I can to get rid of them. The message is as follows, only the module can vary at times:
ruby 5740 child_info_fork::abort: address space needed by 'date_core.so' (0x600000) is already occupied
These messages occur at various times when trying to run Rails, such as “rails console”.
I am running the following environment:
Windows 7 Ultimate 64-bit
Cygwin Net using setup 2.774 (installed from Internet mirrors)
Ruby 1.9.3p392
Rails 3.2.11
I have taken the following steps attempting to fix this:
Deleted all BLODA and restarted, including Norton 360 and Malwarebytes. (BLODA is Big List of Dodgy Apps.)
Deleted and reinstalled Cygwin.
Executed “rvm implode” and rebuilt the RVM and Ruby on Rails environment.
Used CMD & ASH to run “peflagsall” & “rebaseall”. Ran “rubyrebase”. Restarted.
Used FIND to build a file of all *.so & *.dll files. Ran “rebase -T”. Restarted. This has also been done using the “-b 0x77000000” switch.
Used same file to create “to_peflags” script issuing peflags against each individual file. Restarted.
All repair attempts have been futile. At times, “rebase -si |grep ‘*’” would be clean. Any attempt to run “rails console” or similar commands results in Address Space Occupied messages. Subsequent “rebase -si” would then again show conflicts.
I have been trying to run “peflags” against the Ruby modules, but I only get “skipped because could not open” error messages there. File permissions and ACL both show correct permissions. The same error occurs whether or not Administrator is used.
This is pretty frustrating. I would like to fix my current environment, but I am considering moving to MSYS or SFU, but I don’t know if either of those are decent options. I am also considering going native UNIX, but I’ll leap off the cliff when I get to it.
For now, does anyone have a fix for this specific error for me? Thanks…
I have used procexp to monitor "rails console" execution. Bash is calling cmd.exe to run the rails.bat file. Cmd.exe runs in ASLR mode. Used Microsoft EMET 3.0 to turn off mandatory ASLR and bottom up ASLR for both bash and cmd. Cmd.exe still runs in ASLR mode. It must be compiled that way.
Updated registry to set HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\MoveImages to 0, -1 then 1. 0 caused system problems. -1 had no effect. 1, which was supposed to be the default, helped somewhat as long as EMET changes were in place. But, it was unreliable as Address Space Occupied would eventually recur.
Decided to attempt to install a command processor that either didn’t require ASLR or allowed me to disable it using PE flags. I installed TCC/LE and attempted to get Cygwin Bash to use it to process .bat files. I tried using the path variable, the COMSPEC variable and changes to batfile and cmdfile in the registry all to no effect. Cygwin Bash must either have it hard coded or some nonobvious control.
My fix is to replace my environment. I know I will eventually migrate to some Unix, but for now I am converting to MinGW/MSYS and RubyStack. All I want to do is to get a stable RoR (Ruby on Rails) environment running for today. I don’t see that happening using Cygwin with Windows 7 Ultimate. I keep running into too many problems and this one is a very, very tough nut to crack. Thanks…