I am trying to use Python to call a Google Cloud AI platform training API. The path to my service account key JSON file is "/Users/my_mac_username/service_account_key.json", and I added the export statement in the bash_profile file so that it looks like:
# Setting PATH for Python 3.8
# The original version is saved in .bash_profile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/3.8/bin:${PATH}"
export PATH
export GOOGLE_APPLICATION_CREDENTIALS="/Users/my_mac_username/service_account_key.json"
However, I still got the error:
google.auth.exceptions.DefaultCredentialsError: Could not automatically determine credentials.
Besides, I have google-cloud-sdk folder installed. I would like to know in which file I should set the environment variable. Thanks for your help!
Assuming your terminal shell is bash, you have your environment variable setup correctly. The new variable will not be available until you reopen the terminal or run source ~/.bash_profile in the current terminal.
If you are running zsh, add the export in your ~/.zshrc file.
To determine which shell you're currently running, use: echo $SHELL from the command line
When I use vim to update my environmental variables (in ~/.bashrc), PyCharm does not get the updates right away. I have to shut down the program, source ~/.bashrc again, and re-open PyCharm.
Is there any way to have PyCharm source the changes automatically (or without shutting down)?
When any process get created it inherit the environment variables from it's parent process (the O.S. itself in your case). if you change the environment variables at the parent level, the child process is not aware of it.
PyCharm allows you to change the environment variables from the Run\Debug Configuration window.
Run > Edit Configurations > Environment Variables ->
In my case pycharm does not take env variables from bashrc even after restarting
Pycharm maintains it's own version of environment variables and those aren't sourced from the shell.
It seems that if pycharm is executed from a virtualenv or the shell containing said variables, it will load with them, however it is not dynamic.
the answer below has a settings.py script for the virtualenv to update and maintain settings. Whether this completely solves your question or not i'm not sure.
Pycharm: set environment variable for run manage.py Task
I recently discovered a workaround in windows. Close Pycharm, copy the command to run Pycharm directly from the shortcut, and rerun it in a new terminal window: cmd, cmder, etc.
C:\
λ "C:\Program Files\JetBrains\PyCharm 2017.2.1\bin\pycharm64.exe"
I know this is very late, but I encountered this issue as well and found the accepted answer tedious as I had a lot of saved configurations already.
The solution that a co-worker told me is to add the environment variables to ~/.profile instead. I then had to restart my linux machine and pycharm picked up the new values. (for OSX, I only needed to source ~/.profile and restart pycharm completely)
One thing to be aware is that another coworker said that pycharm would look at ~/.bash_profile so if you have that file, then you need the environment variables added there
In case you are using the "sudo python" technique, be aware that it does not by default convey the environment variables.
To correctly pass on the environment variables defined in the PyCharm launch configuration, use the -E switch:
sudo -E /path/to/python/executable "$#"
This is simply how environment variables work. If you change them you have to re-source your .bashrc (or whatever file the environment variables are located in).
from dotenv import load_dotenv
load_dotenv(override=True)
Python-dotenv can interpolate variables using POSIX variable expansion.
With load_dotenv(override=True) or dotenv_values(), the value of a variable is the first of the values defined in the following list:
Value of that variable in the .env file.
Value of that variable in the environment.
Default value, if provided.
Empty string.
With load_dotenv(override=False), the value of a variable is the first of the values defined in the following list:
Value of that variable in the environment
Value of that variable in the .env file.
Default value, if provided.
Empty string.
I am writing Ansible playbooks to setup and install our applications on Solaris servers.
The problem is that the (bash) scripts which I need to execute all assume that a certain directory lies on the PATH, namely /data/bin - which would normally not be a problem were it not for Ansible ignoring all the .profile and .bashrc config.
Now, I know that you can specify the environment for shell tasks via the environment flag, for example like this:
- shell: printenv
environment:
PATH: /usr/bin:/usr/sbin:/data/bin
This will properly path the /data/bin folder, and the printenv command will correctly display (or my bash scripts would correctly run).
But. There are two problems however:
First of all it is very annoying to have to specify the environment over and over again. I know that you can define the environment in some playbook base file variable and the reference that, but you still have to set environment: ... on every single shell task.
Secondly, the above example does not allow me to specify the path dynamically, e.g. as PATH: $PATH:/data/bin - because Ansible executes this in a way which does not resolve $PATH, thus the command fails catastrophically. So essentially this will override any other changes to PATH.
I am looking for a solution where
the additional PATH entry should only be added once
the additional PATH entry should not override entries added by other tasks
P.S. I found this nice explanation on how to do this on Linux, but it makes use of /etc/environment which does not exist on Solaris. (And /etc/profile is once again ignored by Ansible.)
try adding -o SendEnv=PATH to ssh_args in ansible.cfg. Requires that
the shell in which you run ansible has /data/bin in PATH. Or however ansible allows you to modify the current/local PATH variable.
remote machine has AcceptEnv set correctly.
I was trying to set up a system similar to heroku where I would store secret keys in environmental variables and then access them from my rails app like this:
secret = ENV['EMAIL_PASSWORD']
I know heroku lets you do heroku config:add EMAIL_PASSWORD=secret, and I wanted to do something like that for my own ubuntu box running nginx and Passenger.
Should I add these variables as exports in .bashrc or .bash_login so that on system reboot these variables are automatically set?
I'm not sure when each of those files gets read in.
You can use dotenv gem which loads the .env file as environmental variables. You can generate the .env file for different environments, and need not be rather should not checked into your repository.
Keep in mind that nginx may not be running under the same environment as you are, and usually (pronounced "Apache") we add env-vars in the server config file via SetEnv. However, nginx doesn't have such a feature... nor does it need one, I believe.
sudo -E /usr/local/sbin/nginx
When running nginx for it to be aware of your own user env vars.
Or, check out the env command (see here):
env EMAIL_PASSWORD=secret
To answer your question, yes, you should use export statements in your shell config files.
This is documented in nginx. It removes all environment variables except TZ when running the workers. If you want to add an environment variable, add the following to the top of the nginx configuration:
# The top of the configuration usually has things like:
user user-name;
pid pid-file-name;
# Add to this:
env VAR1=value1;
env VAR2=value2;
# OR simply add:
env VAR1;
# To inherit the VAR1 from whatever you set in bash
The normal export or anything you do in bash has no guarantee of getting passed on to nginx, due to the way the init scripts are written (we don't know if they're using sudo with a clean environment, etc). So I'd rather put these in the nginx configuration file itself, rather than depending on the shell to do it.
Edit: Fix link
(this is probably a overkill, but maybe it'll be useful)
Some things to keep in mind:
Environment variables are somewhat public, and can be seen by other processes as easily as added an option to the ps(1) command (like ps e $$ in bash) or looking at /proc/*/environ, though both are restricted at least to the same user (or root) on modern systems. Don't rely on them being secret if you have another fairly easy option available.
~/.bashrc is the wrong place for environment variables, since they can be computed once at login in ~/.bash_login, ~/.bash_profile, or ~/.profile, depending on your usage, and passed down to all descendent shells. In contrast, ~/.bashrc actions tend to be recomputed on every shell invocation (unless explicitly disabled).
Putting bash code in the ~/.profile can confuse other sh-descendent shells and non-shell tools which try to read that file, so having the bash-specific ~/.bash_login or -_profile contain the bash-specific things, and using . ~/.profile for the more general things (LESS, EDITOR, VISUAL, LC_COLLATE, LS_COLORS, etc), is friendlier to the other tools.
Environment variables in ~/.profile should be in the old Bourne shell form (VAR=value ; export VAR). On Linux, this isn't usually critical, though on other Unixen this can be a big issue when an older version of "sh" tries to read them.
Some X sessions will only read ~/.profile, not ~/.bash_login or the others mentioned above. Some will look for a ~/.xsession file will need to be modified to have . $HOME/.profile if it doesn't already somehow.
System-wide settings would be put instead in something like /etc/profile.d/similar-to-heroku.sh. Note that the ".sh" is only present since the file will be used with "." or "source" - shell scripts should never have command-name extensions in any form of Unix/Linux.
Most environment variables get ditched when one sudos to root, as ybakos points out. Similar issues show up in crontabs, at jobs, etc. When in doubt, adding env | sort > /tmp/envvars or the like a suspect script can really help in debugging.
Be aware some distributions have shell startup scripts so contorted they end up actually defying the order given in the bash(1) manual page. Anytime you find a default user ~/.profile checking for $BASH or $BASH_VERSION, you may be in one of these, um..., "interesting" environments, and may have to read through them to figure out where the control flow goes (they should be using a bash-specific ~/.bash_profile or ~/.bash_login, which includes the more generic ~/.profile by reference, thus letting the bash executable do the work instead of having to write $BASH checks in shell code).
~/.bash_profile (or ~/.bash_login) can certainly include . ~/.bashrc, but the environment variables belong in the ~/.bash_profile (if bash-specific) or the ~/.profile included from it (if you're using this mechanism and have envvars for everything else in there) as DeWitt says, just remember to put the . ~/.bashrc AFTER the .bash_profile's . ~/.profile and other environment variables, so that both login and all other invocations of the ~/.bashrc can rely on the envvars already being set. An Example ~/.bash_profile:
# .bash_profile
[ -r ~/.profile ] && . ~/.profile # envvars
[ -r ~/.bashrc ] && . ~/.bashrc # functions, per-tty settings, etc.
#---eof
The [ -r ... ] && ... works in any Bourne shell descendent and doesn't cause errors/aborts if the .profile is missing (I personally have a ~/.profile.d/*.sh setup as well, but this is left as an entirely optional exercise).
Note that bash only reads the first file of these three which it finds:
~/.bash_profile
~/.bash_login
~/.profile
...so once you have that one, the use of the other two is entirely under control of the user, from bash's perspective.
I put them in my nginx config, specifically in the server definition for the app using the passenger_env_var command:
server {
server_name www.foo.com;
root /webapps/foo/public;
passenger_enabled on;
passenger_env_var DATABASE_USERNAME foo_db;
passenger_env_var DATABASE_PASSWORD secret;
passenger_env_var SECRET_KEY_BASE the_secret_keybase;
}
This works for me. See the phusion passenger docs for more info.
I have a script in /usr/local/bin folder that sets some env vars and then executes Ruby. I define the path to Ruby in my (Apache, not Nginx) conf file to that file in /usr/local/bin.
example:
#!/bin/sh
# setup env vars here
export FOO=bar
export PATH_TO_FOO=/bar/bin
export PATH=$PATH:PATH_TO_FOO
# and execute Ruby with any arguments passed to this script
exec "/usr/bin/ruby" "$#"
You should read this response to another question, it will help:
https://stackoverflow.com/a/11765775/1217298
EDITED :
Ok sorry i read it too fast, you can check how to save your ENV variables here :
https://help.ubuntu.com/community/EnvironmentVariables
http://www.cyberciti.biz/faq/set-environment-variable-linux/
If you use Nginx as server on your local computer, you can define your env variable into your nginx config file.
location / {
...
fastcgi_param EMAIL_PASSWORD secret; #EMAIL_PASSWORD = secret
...
}
I'm using rbenv as a version manager. Good solution to store environment variables for the project was installing the rbenv-vars plugin and putting them in .rbenv-vars file.
Here is a useful post:
Deploying app ENV variables with Rbenv, Passenger and Capistrano
For those battling this that are using RVM. Make sure that your default environments file is including your user's .bashrc and .profile files
file: $rvm_path/environments/default
to find the path run this command:
ls -lah `whereis rvm`/environments/default
add these two lines before the first line in that file:
source $HOME/.bashrc
source $HOME/.profile
The best place to keep env variables for your project is /etc/profile.d/YOUR_FILE.sh,
Here you can find the documentation which explains in details where to keep env variables for different scenarios.
In case anyone had the same type of question as I did, here's a nice little writeup about the different .bash* files: http://www.joshstaiger.org/archives/2005/07/bash_profile_vs.html
In summary:
For the most part:
.bash_profile is read when you log into the computer and .bashrc is read when you start a new terminal. For Mac OSX .bash_profile is read with every terminal window you start.
So, the recommended procedure is to source .bashrc from .bash_profile so all the variables are set when you login to the computer. Just add this to .bash_profile:
if [ -f ~/.bashrc ]; then
source ~/.bashrc
fi
You have to add the export lines into your .profile file under your home folder...
Environment variables are being set on login...