I'm doing an internship (= yes I'm a newbie). My supervisor told me to create a conda environment. She passed me a log file containing many packages.
A quick qwant.com search shows me how to create envs via the
conda env create --file env_file.yaml
The file I was give is however NOT a yaml file it is structured like so:
# packages in environment at /home/supervisors_name/.conda/envs/pancancer:
#
# Name Version Build Channel
_libgcc_mutex 0.1 main
bedtools 2.29.2 hc088bd4_0 bioconda
blas 1.0 mkl
bzip2 1.0.8 h7b6447c_0
The file contains 41 packages = 44 lines including comments above. For simplicity I'm showing only the first 7.
Appart from adding env name (see 2. below), is there a way to use the file as it is to generate an environment with the packages?
I ran the cmd using
conda env create --file supervisors.log.txt
SpecNotFound: Environment with requirements.txt file needs a name
Where in the file should I put the name?
alright, so, it seems that they give you the output of conda list rather than the .yml file produced by conda with conda env export > myenv.yml. Therefore you have two solutions:
You ask for the proper file and then proceed to install the env with conda built-in pipeline
If you do not have any access on the proper file, you could do one of the following:
i) Parse with python into a proper .yml file and then do the conda procedure.
ii) Do a bash script, downloading the packages listed in the file she gave you.
This is how I would proceed, personally :)
Because there is no other SO post on this error, for people of the future: I got this error just because I named my file conda_environment.txt instead of conda_environment.yml. Looks like the yml extension is mandatory.
I want to make some revisions to the code of package/kernel/mac80211.
I am new to OpenWrt and after some research, I think I should change the PKG_SOURCE_URL to my own GitHub repository, which is my own copy of /linux/kernel/projects/backports/stable/v4.19.120.
So I change package/kernel/mac80211/Makefile like following:
PKG_SOURCE_PROTO:=git
PKG_SOURCE_URL:=https://github.com/sheep94lion/openwrt.git
PKG_SOURCE_VERSION:=168bae33318ebd14d8c035b543a2583ea31f9f52
PKG_MIRROR_HASH:=skip
# PKG_SOURCE_URL:=#KERNEL/linux/kernel/projects/backports/stable/v4.19.120/
# PKG_HASH:=2bafd75da301a30a5f2b98f433b6545d7b58c1fc3af15e9e9aa085df7f9db1d4
My question is: am I in the right direction? What is the right/proper way to revise an existing kernel package?
I copy the source files to package/kernel/mac80211/src directory (create the directory first), and then revise Makefile to use local source files instead of downloading and unpack from the official URI.
The revisions in Makefile is as following:
# comment out the configurations to download the tarball.
# PKG_SOURCE_URL:=#KERNEL/linux/kernel/projects/backports/stable/v4.19.120/
# PKG_HASH:=2bafd75da301a30a5f2b98f433b6545d7b58c1fc3af15e9e9aa085df7f9db1d4
# PKG_SOURCE:=backports-$(PKG_VERSION).tar.xz
......
define Build/Prepare
rm -rf $(PKG_BUILD_DIR)
mkdir -p $(PKG_BUILD_DIR)
# do not unpack the downloaded tarbar.
# $(PKG_UNPACK)
# instead, copy files under src to the build directory.
$(CP) ./src/* $(PKG_BUILD_DIR)/
$(Build/Patch)
When I want to release the code, I think I should use the patch.
What is the correct way to specify x11 dependency in a homebrew formula?
The default superenv removes /opt/X11/lib from its arguments.
I am writing a formula for a package that I can build outside of homebrew with the usual configure, make install.
So I have this install function:
def install
ENV["PKG_CONFIG_PATH"] = "/usr/local/opt/qt/lib/pkgconfig"
# ENV["PATH"] = "/usr/local/bin:/usr/bin:/bin" <--- work around
Dir.chdir("codebase")
system "./configure", "--disable-dependency-tracking", "--prefix=#{prefix}"
system "make install"
end
The link phase that gets echoed shows
/bin/sh ../../../../libtool --tag=CXX --mode=link clang++ .... -I /opt/X11/include ..... -L/opt/X11/lib ...
But the link fails with
ld: library not found for -lX11
If I add this to the top of the class definition, the build is successful
env :std
Alternatively, I can set PATH inside the build function and the build succeeds.
This makes sense since within the context of brew install, /usr/local/Homebrew/Library/Homebrew/shims/super appears at the start of the PATH, and that directory has a clang++ which among other things strips /opt/X11 components out.
I assume there is a good reason for this behavior, and am curious what is the best way to specify that X11 library.
The easiest way to know how to do something in writing Hombrew formulas is to look at existing formulas. For your case you can look at MuPDF a lightweight PDF and XPS viewer depending on X11. In its formula you will find the solution:
depends_on :x11
Context
I have some functions defined in my ~/.bashrc which I'd like to turn into a Homebrew package. Currently, these functions act as custom commands on my command line:
# .bashrc
function foo() {
# do something interesting
}
# at terminal
$ foo
# => does the interesting thing
Approach
I've created a homebrew formula using brew create. My current approach is as follows:
Move the function definitions into a separate file, script, within a directory, brew-script
Make brew-script downloadable as a tarball, brew-script.tar.gz
Have my brew formula append text to the end of ~/.bash_profile to include script when terminal session starts
Concerns
Is modifying .bash_profile in a brew formula bad practice? (eg. when uninstalling with brew uninstall script, brew should somehow remove the text that was appended to .bash_profile... Parsing .bash_profile doesn't seem very fun.)
Is there a convention already established for including functions in bash scripts so that they are available from the command line?
Is it common to simply ask the user to add some text to their .bash_profile or .bashrc?
Desired result
Should be able to install cleanly with brew and then run foo as a command:
$ brew install script
$ foo
# => does the interesting thing
(Assume the brew formula is already installed locally. I'll worry about auditing and pushing the formula to homebrew later)
Refer https://github.com/Homebrew/homebrew/issues/50232 and https://github.com/Homebrew/homebrew/issues/50231.
I have a script that safely‡ modifies ~/.bash_profile as part of a homebrew install process. https://github.com/paul-hammant/homebrew-tap/blob/master/switchjdk.rb
‡ allegedly
Without using homebrew:
to put your bash scripts in some file such as bashrc or any other name works, then put the following line:
source "path/to/brew-script/script"
somewhere in your bash profile.
Then you just have to make sure you refresh or reload your bash profile by running . ~/.bash_profile or source ~/.bash_profile.
How homebrew installs work:
When you installed homebrew it added a line to your bash_profile that modifies your $PATH variable to include the path to the homebrew install repo, so that whenever brew installs something it becomes findable through your PATH.
If you use brew create you must have your script uploaded somewhere on the internet, because the argument brew install takes is a URL. I.e if I create my script at my_bash_function.tar.gz then I would do
brew create http://web.mit.edu/dianah13/www/my_bash_function.tar.gz
It also templates a pull request to include your package in homebrew's main repo.
I was trying to set up a system similar to heroku where I would store secret keys in environmental variables and then access them from my rails app like this:
secret = ENV['EMAIL_PASSWORD']
I know heroku lets you do heroku config:add EMAIL_PASSWORD=secret, and I wanted to do something like that for my own ubuntu box running nginx and Passenger.
Should I add these variables as exports in .bashrc or .bash_login so that on system reboot these variables are automatically set?
I'm not sure when each of those files gets read in.
You can use dotenv gem which loads the .env file as environmental variables. You can generate the .env file for different environments, and need not be rather should not checked into your repository.
Keep in mind that nginx may not be running under the same environment as you are, and usually (pronounced "Apache") we add env-vars in the server config file via SetEnv. However, nginx doesn't have such a feature... nor does it need one, I believe.
sudo -E /usr/local/sbin/nginx
When running nginx for it to be aware of your own user env vars.
Or, check out the env command (see here):
env EMAIL_PASSWORD=secret
To answer your question, yes, you should use export statements in your shell config files.
This is documented in nginx. It removes all environment variables except TZ when running the workers. If you want to add an environment variable, add the following to the top of the nginx configuration:
# The top of the configuration usually has things like:
user user-name;
pid pid-file-name;
# Add to this:
env VAR1=value1;
env VAR2=value2;
# OR simply add:
env VAR1;
# To inherit the VAR1 from whatever you set in bash
The normal export or anything you do in bash has no guarantee of getting passed on to nginx, due to the way the init scripts are written (we don't know if they're using sudo with a clean environment, etc). So I'd rather put these in the nginx configuration file itself, rather than depending on the shell to do it.
Edit: Fix link
(this is probably a overkill, but maybe it'll be useful)
Some things to keep in mind:
Environment variables are somewhat public, and can be seen by other processes as easily as added an option to the ps(1) command (like ps e $$ in bash) or looking at /proc/*/environ, though both are restricted at least to the same user (or root) on modern systems. Don't rely on them being secret if you have another fairly easy option available.
~/.bashrc is the wrong place for environment variables, since they can be computed once at login in ~/.bash_login, ~/.bash_profile, or ~/.profile, depending on your usage, and passed down to all descendent shells. In contrast, ~/.bashrc actions tend to be recomputed on every shell invocation (unless explicitly disabled).
Putting bash code in the ~/.profile can confuse other sh-descendent shells and non-shell tools which try to read that file, so having the bash-specific ~/.bash_login or -_profile contain the bash-specific things, and using . ~/.profile for the more general things (LESS, EDITOR, VISUAL, LC_COLLATE, LS_COLORS, etc), is friendlier to the other tools.
Environment variables in ~/.profile should be in the old Bourne shell form (VAR=value ; export VAR). On Linux, this isn't usually critical, though on other Unixen this can be a big issue when an older version of "sh" tries to read them.
Some X sessions will only read ~/.profile, not ~/.bash_login or the others mentioned above. Some will look for a ~/.xsession file will need to be modified to have . $HOME/.profile if it doesn't already somehow.
System-wide settings would be put instead in something like /etc/profile.d/similar-to-heroku.sh. Note that the ".sh" is only present since the file will be used with "." or "source" - shell scripts should never have command-name extensions in any form of Unix/Linux.
Most environment variables get ditched when one sudos to root, as ybakos points out. Similar issues show up in crontabs, at jobs, etc. When in doubt, adding env | sort > /tmp/envvars or the like a suspect script can really help in debugging.
Be aware some distributions have shell startup scripts so contorted they end up actually defying the order given in the bash(1) manual page. Anytime you find a default user ~/.profile checking for $BASH or $BASH_VERSION, you may be in one of these, um..., "interesting" environments, and may have to read through them to figure out where the control flow goes (they should be using a bash-specific ~/.bash_profile or ~/.bash_login, which includes the more generic ~/.profile by reference, thus letting the bash executable do the work instead of having to write $BASH checks in shell code).
~/.bash_profile (or ~/.bash_login) can certainly include . ~/.bashrc, but the environment variables belong in the ~/.bash_profile (if bash-specific) or the ~/.profile included from it (if you're using this mechanism and have envvars for everything else in there) as DeWitt says, just remember to put the . ~/.bashrc AFTER the .bash_profile's . ~/.profile and other environment variables, so that both login and all other invocations of the ~/.bashrc can rely on the envvars already being set. An Example ~/.bash_profile:
# .bash_profile
[ -r ~/.profile ] && . ~/.profile # envvars
[ -r ~/.bashrc ] && . ~/.bashrc # functions, per-tty settings, etc.
#---eof
The [ -r ... ] && ... works in any Bourne shell descendent and doesn't cause errors/aborts if the .profile is missing (I personally have a ~/.profile.d/*.sh setup as well, but this is left as an entirely optional exercise).
Note that bash only reads the first file of these three which it finds:
~/.bash_profile
~/.bash_login
~/.profile
...so once you have that one, the use of the other two is entirely under control of the user, from bash's perspective.
I put them in my nginx config, specifically in the server definition for the app using the passenger_env_var command:
server {
server_name www.foo.com;
root /webapps/foo/public;
passenger_enabled on;
passenger_env_var DATABASE_USERNAME foo_db;
passenger_env_var DATABASE_PASSWORD secret;
passenger_env_var SECRET_KEY_BASE the_secret_keybase;
}
This works for me. See the phusion passenger docs for more info.
I have a script in /usr/local/bin folder that sets some env vars and then executes Ruby. I define the path to Ruby in my (Apache, not Nginx) conf file to that file in /usr/local/bin.
example:
#!/bin/sh
# setup env vars here
export FOO=bar
export PATH_TO_FOO=/bar/bin
export PATH=$PATH:PATH_TO_FOO
# and execute Ruby with any arguments passed to this script
exec "/usr/bin/ruby" "$#"
You should read this response to another question, it will help:
https://stackoverflow.com/a/11765775/1217298
EDITED :
Ok sorry i read it too fast, you can check how to save your ENV variables here :
https://help.ubuntu.com/community/EnvironmentVariables
http://www.cyberciti.biz/faq/set-environment-variable-linux/
If you use Nginx as server on your local computer, you can define your env variable into your nginx config file.
location / {
...
fastcgi_param EMAIL_PASSWORD secret; #EMAIL_PASSWORD = secret
...
}
I'm using rbenv as a version manager. Good solution to store environment variables for the project was installing the rbenv-vars plugin and putting them in .rbenv-vars file.
Here is a useful post:
Deploying app ENV variables with Rbenv, Passenger and Capistrano
For those battling this that are using RVM. Make sure that your default environments file is including your user's .bashrc and .profile files
file: $rvm_path/environments/default
to find the path run this command:
ls -lah `whereis rvm`/environments/default
add these two lines before the first line in that file:
source $HOME/.bashrc
source $HOME/.profile
The best place to keep env variables for your project is /etc/profile.d/YOUR_FILE.sh,
Here you can find the documentation which explains in details where to keep env variables for different scenarios.
In case anyone had the same type of question as I did, here's a nice little writeup about the different .bash* files: http://www.joshstaiger.org/archives/2005/07/bash_profile_vs.html
In summary:
For the most part:
.bash_profile is read when you log into the computer and .bashrc is read when you start a new terminal. For Mac OSX .bash_profile is read with every terminal window you start.
So, the recommended procedure is to source .bashrc from .bash_profile so all the variables are set when you login to the computer. Just add this to .bash_profile:
if [ -f ~/.bashrc ]; then
source ~/.bashrc
fi
You have to add the export lines into your .profile file under your home folder...
Environment variables are being set on login...