There are older versions of Google Cloud Platform tools: Docker - docker

After updating gcloud I get this warning, but how do I do it(Should I remove Docker)?
WARNING: There are older versions of Google Cloud Platform tools on your system PATH.
Please remove the following to avoid accidentally invoking these old tools:
/Applications/Docker.app/Contents/Resources/bin/kubectl
I have this in my .zshrc:
# The next line updates PATH for the Google Cloud SDK.
if [ -f '/Users/<NAME>/google-cloud-sdk/path.zsh.inc' ]; then source '/Users/<NAME>/google-cloud-sdk/path.zsh.inc'; fi
# The next line enables shell command completion for gcloud.
if [ -f '/Users/<NAME>/google-cloud-sdk/completion.zsh.inc' ]; then source '/Users/<NAME>/google-cloud-sdk/completion.zsh.inc'; fi
[ -f ~/.fzf.zsh ] && source ~/.fzf.zsh

This happens because docker-for-mac installs a bin for kubectl, and gcloud-sdk also installs another bin with gcloud components install kubectl.
My recommendation is to uninstall kubectl as component from gcloud, overwrite the symlink from docker-for-mac, and only use the homebrew installed bin.
Try this commands:
gcloud components remove kubectl
brew install kubernetes-cli
brew link --overwrite kubernetes-cli

TLDR
/usr/local/bin/kubectl is a link installed by Docker: ls -l /usr/local/bin/kubectl => /usr/local/bin/kubectl -> /Applications/Docker.app/Contents/Resources/bin/kubectl. Removing the link has no side effects and solves the conflict:
rm /usr/local/bin/kubectl
Justification
The conflict is with the Docker-provided version of kubectl so it makes sense to check what Docker docs have to say about it. https://docs.docker.com/desktop/kubernetes/#use-the-kubectl-command
Extract:
If you installed kubectl using Homebrew, or by some other method, and experience conflicts, remove /usr/local/bin/kubectl.

Here is my case you can refer to. After running gcloud components update, I got this warning:
WARNING: There are older versions of Google Cloud Platform tools on your system PATH.
Please remove the following to avoid accidentally invoking these old tools:
/usr/local/Cellar/kubernetes-cli/1.10.2/bin/kubectl
I check this tool using brew list
☁ issue [master] brew list
coreutils gdbm git-lfs icu4c kops kubectx libpng mtr openssl python#2 sqlite tree wxmac
erlang geoip git-redate jpeg kube-ps1 kubernetes-cli libtiff node pcre readline telnet watchman
After reading the doc. I decided to uninstall kubernetes-cli and its dependencies kops, kube-ps1, and kubectx to avoid the conflicts.
☁ issue [master] brew uninstall kops kube-ps1 kubectx
Uninstalling /usr/local/Cellar/kops/1.9.0... (5 files, 129.8MB)
Uninstalling /usr/local/Cellar/kube-ps1/0.6.0... (6 files, 29.0KB)
Uninstalling /usr/local/Cellar/kubectx/0.5.0... (12 files, 27.8KB)
☁ issue [master] brew uninstall kubernetes-cli
Uninstalling /usr/local/Cellar/kubernetes-cli/1.10.2... (178 files, 52.8MB)
☁ issue [master] gcloud components update
All components are up to date.
This warning is gone.

I just went into the Docker file's bin folder and moved the kubectl to the trash.

Do echo $PATH and check which folder takes precedence. In my case it is like .../Users/myname/google-cloud-sdk/bin:/usr/local/bin:/usr/bin:.... Here the kubectl in gcloud is actually before the kubectl from Docker Desktop (which is in /usr/local/bin/kubectl) so there is no problem. If this is also your case you don't need to do anything.
Of course, if you want to completely remove confusion you can just delete the link /usr/local/bin/kubectl -> /Applications/Docker.app/Contents/Resources/bin/kubectl, or rename it.
Update:
In the recent docker desktop releases they actually provided another link /usr/local/bin/kubectl.docker -> /Applications/Docker.app/Contents/Resources/bin/kubectl to differentiate it from other kubectl so it is not a bad idea to just simply delete the link /usr/local/bin/kubectl -> /Applications/Docker.app/Contents/Resources/bin/kubectl

Related

Docker rootless mode setup tool script can't find slirp4netns binary

I am trying to set up Docker rootless mode on Ubuntu 18.04, using the dockerd-rootless-setuptool.sh script. But I'm struggling to get the script to find the slirp4netns binary I downloaded (slirp4netns provides user-mode networking for unprivileged network namespaces). On Ubuntu 20 this is installable as a package using apt, but for lower Ubuntu versions a pre-built binary is provided here.
I am running the script (as non-root) with:
dockerd-rootless-setuptool.sh install
But it complains:
Either slirp4netns (>= v0.4.0) or vpnkit needs to be installed
I have downloaded the latest stable binary for slirp4netns (v1.1.12) as per the instructions:
curl -o slirp4netns --fail -L https://github.com/rootless-containers/slirp4netns/releases/download/v1.1.12/slirp4netns-$(uname -m)
chmod +x slirp4netns
I moved the binary (as root) to /usr/local/lib. What am I doing wrong? Do I need to update an environment variable? I don't really want to upgrade my entire OS just to get a single library.
Thanks.
It was my own stupidity. As the person says in the comment to my question, all I needed to do was to move the binary to /usr/local/bin. I had assumed /usr/local/lib was on PATH, but it wasn't.

Link hadoop installed with sdkman to brew

I installed hadoop with sdkman and now I'm trying to install Hive with homebrew but brew wants to install hadoop again because it doesn't know hadoop is already installed on my computer.
I use --ignore-dependencies flag as workaround but it's not a best practice.
Do you know how can I link my hadoop installation done with sdkman to brew?
It is not possible to use a non-Homebrew hadoop with Homebrew hive, see https://docs.brew.sh/Building-Against-Non-Homebrew-Dependencies
To improve quality and reduce variation, Homebrew now exclusively supports using the default formula, as an ordinary dependency, and no longer supports using arbitrary alternatives.
You will have to install Hive manually: https://cwiki.apache.org/confluence/display/hive/gettingstarted#GettingStarted-InstallingHivefromaStableRelease
Installing Hive from a Stable Release
Start by downloading the most recent stable release of Hive from one of the Apache download mirrors (see Hive Releases).
Next you need to unpack the tarball. This will result in the creation of a subdirectory named hive-x.y.z (where x.y.z is the release number):
$ tar -xzvf hive-x.y.z.tar.gz
Set the environment variable HIVE_HOME to point to the installation directory:
$ cd hive-x.y.z
$ export HIVE_HOME={{pwd}}
Finally, add $HIVE_HOME/bin to your PATH:
$ export PATH=$HIVE_HOME/bin:$PATH

Anyone else get this error "Error: invalid option: --with-passenger"

Can not get NGINX to install with passenger
Following the steps you are supposed to...
brew install passenger
Then you run the following command...
brew install nginx --with-passenger
But I get:
Error: invalid option: --with-passenger
Could not find any resources about this online, so asking here.
$ brew install nginx --with-passenger
Usage: brew install [options] formula
Install formula.
formula is usually the name of the formula to install, but it can be specified
in several different ways.
-d, --debug If brewing fails, open an interactive
debugging session with access to IRB or a
shell inside the temporary build directory
--env If std is passed, use the standard build
environment instead of superenv.If super
is passed, use superenv even if the formula
specifies the standard build environment.
--ignore-dependencies Skip installing any dependencies of any
kind. If they are not already present, the
formula will probably fail to install.
--only-dependencies Install the dependencies with specified
options but do not install the specified
formula.
--cc Attempt to compile using provided
compiler. compiler should be the name
of the compiler's executable, for instance
gcc-7 for GCC 7. In order to use LLVM's
clang, use llvm_clang. To specify the
Apple-provided clang, use clang. This
parameter will only accept compilers that
are provided by Homebrew or bundled with
macOS. Please do not file issues if you
encounter errors while using this flag.
-s, --build-from-source Compile the specified formula from source
even if a bottle is provided. Dependencies
will still be installed from bottles if
they are available.
--force-bottle Install from a bottle if it exists for the
current or newest version of macOS, even if
it would not normally be used for
installation.
--include-test Install testing dependencies required to
run brew test.
--devel If formula defines it, install the
development version.
--HEAD If formula defines it, install the HEAD
version, aka. master, trunk, unstable.
--fetch-HEAD Fetch the upstream repository to detect if
the HEAD installation of the formula is
outdated. Otherwise, the repository's HEAD
will be checked for updates when a new
stable or development version has been
released.
--keep-tmp Don't delete the temporary files created
during installation.
--build-bottle Prepare the formula for eventual bottling
during installation.
-f, --force Install without checking for previously
installed keg-only or non-migrated
versions.
-v, --verbose Print the verification and postinstall
steps.
--display-times Print install times for each formula at the
end of the run.
-i, --interactive Download and patch formula, then open a
shell. This allows the user to run
./configure --help and otherwise
determine how to turn the software package
into a Homebrew package.
-g, --git Create a Git repository, useful for
creating patches to the software.
-h, --help Show this message.
Error: invalid option: --with-passenger
This is supposed to work... so yeah.
A late answer. Obviously the documentation on the passenger site is outdated at time of this writing.
According to phusion's github site
https://github.com/phusion/passenger/issues/2187#issue-416881033
the config option --nginx-with-passenger is not valid anymore.
Instead do:
brew install nginx passenger
Addendum:
When using google search for results currently the old page comes up at the top of the list. Here's the currently maintained page with the accurate information:
https://www.phusionpassenger.com/docs/advanced_guides/install_and_upgrade/nginx/install/oss/osx.html

lack MOSQ_1.5 for mosquitto

I am using mosquitto as one of my project tool. After i successfully compile (make binary) mosquitto via source code and try to run mosquitto_sub/pub, its shows that
./mosquitto_sub: /usr/lib/i386-linux-gnu/libmosquitto.so.1: version `MOSQ_1.5' not found (required by ./mosquitto_sub)
while Mosquitto and Mosquitto_psswd is able is run without any problem. Besides, my mosquitto version is 1.5.
I have no idea about this problem.
Any Help appreciated .
It sounds like you have an earlier version of mosquitto installed and it's libraries are on the system path.
Make sure you have uninstalled any earlier versions of mosquitto installed on the machine.
Also make sure you have run make install as root in your build directory to copy the libraries to the correct locations
I had a similar issue building on rpi model B+ using GNU make 4.2.1 and cc (Raspbian 8.3.0-6+rpi1) 8.3.0
Error reported: ./mosquitto_sub: /usr/lib/i386-linux-gnu/libmosquitto.so.1: version `MOSQ_1.6' not found (required by ./mosquitto_sub)
Note: I left the existing distribution in place so that I could leverage off the systemd mosquitto.service that gets installed with the debian packages mosquitto and mosquitto-clients.
Resolution:
mkdir proj
git clone https://github.com/eclipse/mosquitto.conf
cd mosquitto
# my build failed on master branch, so I picked a tag that worked for me
git checkout -b 1.6.9 tags/1.6.9
make WITH_WEBSOCKETS=yes WITH_DOCS=yes WITH_SRV=yes
# overwrite the existing binaries
sudo make install
sudo ldconfig
# verify that the newly installed binaries have overwritten the originals.
mosquitto --help|grep -w 'mosquitto version'
mosquitto_pub --help|grep -w 'mosquitto_pub version'
mosquitto_sub --help|grep -w 'mosquitto_sub version'

`docker-credential-gcloud` not in system PATH

After the latest updates to gcloud and docker I'm unable to access images on my google container repository. Locally when I run: gcloud auth configure-docker as per the instructions after updating gcloud, I get the following message:
WARNING: `docker-credential-gcloud` not in system PATH.
gcloud's Docker credential helper can be configured but it will not work until this is corrected.
gcloud credential helpers already registered correctly.
Running which docker-credential-gcloud returns docker-credential-gcloud not found.
I have no other gcloud-related path issues and for the life of me can't figure out how to install/add docker-credential-gcloud to path. Here's what I have installed (shown via gcloud version):
Google Cloud SDK 197.0.0
beta 2017.09.15
bq 2.0.31
container-builder-local
core 2018.04.06
docker-credential-gcr
gsutil 4.30
I also have Docker CE Version 18.03.0-ce-mac60 (23751).
Here's my $PATH:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
I also ran source /usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/path.zsh.inc on original gcloud install.
Notice: All docker-credential-gcr below can be replaced with docker-credential-gcloud. I think it is just different versions of gcloud, I might be wrong.
I used Homebrew Cask to install gcloud too. I installed docker-credential-gcr with
$ gcloud components install docker-credential-gcr
And then like you said, which docker-credential-gcr doesn't gave you anything.
So I ran which gcloud to find there is a symlink to gcloud in /usr/local/bin. This symlink is created by Homebrew when you installed gcloud at first place. Now docker-credential-gcr wasn't installed by Homebrew but by gcloud itself, so there isn't a symlink.
I called readlink /usr/local/bin/gcloud and found out gcloud is installed in /usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/bin/.
Then:
$ ls /usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/bin
There you should see docker-credential-gcr listed there.
I simply linked it to /usr/local/bin:
$ ln -s \
/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/bin/docker-credential-gcr \
/usr/local/bin/
Then run:
$ docker-credential-gcr configure-docker
It should succeed.
Just had the same issue on Windows, running Docker with Linux containers, Docker engine v19.03.8. Using docker compose. I do not use gcloud for my dockerfiles...
DT1001 dockerpycreds.errors.InitializationError:
docker-credential-gcloud not installed or not available in PATH
Option 1: Edit the docker configuration file and remove all gcloud entries from there.
Windows c:/Users/<your account>/.docker/config.json
Linux & MacOS ~/.docker/config.json
Option 2: Go to Troubleshoot -> Reset to factory defaults.
After this my docker compose was creating containers and running the images without any issues.
On MacOS
Step 1:
Install gcloud and docker-credential-gcr,
following this tutorial
Step 2:
$ ln -s /usr/local/google-cloud-sdk/bin/docker-credential-gcr /usr/local/bin/docker-credential-gcloud
Step 3:
$ rm -rf ~/.docker
Step 4:
$ docker-compose build --pull
Finished!
Never found a way to directly resolve the docker-credential-gcloud issue, but the following got me up and running again. WARNING: the following will delete all your existing docker images and install a bunch of gcloud utilities:
gcloud components install docker-credential-gcr,
Restart the terminal completely
docker-credential-gcr configure-docker.
screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty
umount /var/lib/docker/overlay2
rm -rf /var/lib/docker
Restart the terminal completely.
The new version of google-cloud-sdk has only docker-credential-gcr but not docker-credential-gcloud anymore. On the other hand one of my python packages always requested docker-credential-gcloud.
The solution was to symlink docker-credential-gcloud to docker-credential-gcr:
ln -s /path/to/google-cloud-sdk/bin/docker-credential-gcr /usr/local/bin/docker-credential-gcloud
ls -l /usr/local/bin | grep docker should now print:
...
docker-credential-gcloud -> /path/to/google-cloud-sdk/bin/docker-credential-gcr
...
Usually, this error indicates that your $PATH variable has been clobbered by a package or program you have recently installed so that the Google Cloud SDK can't be found.
$PATH is altered by many programs when they install by altering ~/.profile, ~/.bash_profile or ~/.bashrc or their non-bash equivalents. With a bad $PATH, Google Cloud SDK is configured in docker but can't be seen as executables so we get this error. This assumes you have used the Google Cloud SDK in the past, but if gcloud is configured with your docker then you probably have. Don't reinstall gcloud or disable it, you already have it on your system and that is fine.
The solution then is to fix your $PATH, not to install anything.
echo $PATH
This should be a pretty long : delimited list of directories that your files are in. Do you see a google-cloud-sdk/bin in the string? Is the string way too short given all the trouble you've gotten into in your life on this computer? You use NVM but it is missing? Use Homebrew but it is missing? Try brew from the command line, does it work?
If the answer is "no" to any of the above, inspect the files above to see if there are any new entries at the bottom of each that might have broken things. Did you just install anything new?
Something is clobbering your $PATH and you need to figure out what that is. For me it is usually something to do with Anaconda Python via the conda init command. For you it might be nvm or something else. Figure out what it is and fix the problem. Don't start over with a new $PATH and install the same stuff over again or disable gcloud authentication.
It really seems to be something with the Homebrew Cask. I uninstalled the cask and then reinstalled the Google Cloud SDK by manually downloading the tar ball and running the packaged install script as described there.
Now docker-credential-gcloud is in my path:
$ which docker-credential-gcloud
/Users/moritz/google-cloud-sdk/bin/docker-credential-gcloud
I can't figure out what Google is trying to achieve here. On Linux there is docker-credential-gcloud and on Windows there is docker-credential-gcr.exe, and then there is docker-credential-gcloud.cmd which calls gcloud auth docker-helper. This is kind of a nightmare if you're trying to write portable build scripts or gradle rules because not everything seems capable of finding and calling docker-credential-gcloud.cmd when you exec docker-credential-gcloud... it might work from the dos prompt, but in general doesn't work.
After a ton of fooling around with .bat scripts, cygwin scripts, .cmd scripts and so forth, I found the best solution was to go into the gcloud installation and just copy docker-credential-gcr.exe docker-credential-gcloud.exe ... not a very satisfying solution, but is the only thing I found that would do the trick.
I got the issue when I tried to SSH from Google Cloud Build into an Engine VM Instance, so I had
steps:
- name: 'gcr.io/cloud-builders/gcloud'
args: ['compute', 'ssh',
'--project', '$PROJECT_ID',
'--zone', 'asia-southeast1-b',
'--strict-host-key-checking=no',
'username#instance-1',
'--command' ,'sh start.sh'
My start.sh
#!/bin/sh
echo "Started: $(date --iso-8601=seconds)"
docker pull gcr.io/aaa/bbbc/cccc
echo "Finished: $(date --iso-8601=seconds)"
The issue was How to set PATH when running a ssh command?
https://unix.stackexchange.com/questions/332532/how-to-set-path-when-running-a-ssh-command
So I just faced the same problem where I am trying to pull an image from GCR to an GCP instance and want to share my solution.
I ran gcloud auth configure-docker and got the warning:
WARNING: `docker-credential-gcloud\` not in system PATH.
gcloud's Docker credential helper can be configured but it will not work until this is corrected.
I applied the accepted answer for this thread and ran gcloud components install docker-credential-gcr and got a long error:
ERROR: (gcloud.components.install) You cannot perform this action because this Cloud SDK installation is managed by an external package manager.
Please consider using a separate installation of the Cloud SDK created through the default mechanism described at: https://cloud.google.com/sdk/
When no solution was working, I uninstalled the Google provided google-cloud-sdk package that was installed via snap and instlled with distro specifice package manager, for me that is apt-get as instructed in the Installing Google Cloud SDK: Installation options page and re-ran the gcloud auth configure-docker and this time it solved my problem.
In my case the problem was due to how WSL 1 works with Docker on Windows. At first I only installed and initialized gcloud in WSL Ubuntu, not in Windows. However as Docker daemon is actually run by Windows, you need to install gcloud for Windows as well (and don't forget to run all of the inits and authorizations there).
On Windows 10/11, you need to ensure that C:\Users\USERNAME\AppData\Local\Google\Cloud SDK\google-cloud-sdk\bin\ is added to your system $PATH environment variable. It may not have been added if the Google Cloud SDK was not able to add it during GCloud installation. So add it manually like this:
Windows Task Bar ➔ Press the search icon 🔍 or the search bar
Type "environment" ➔ and click on "Edit the System Environment Variables" (ensure that you have Administrator access)
At the bottom of the dialog, click the Environment Variables... button
System Variables ➔ click Path ➔ Edit... ➔ New ➔ paste in C:\Users\USERNAME\AppData\Local\Google\Cloud SDK\google-cloud-sdk\bin\ (replace "USERNAME" with your username)
Close and restart any open Command Prompt windows.
Then verify on the Git Bash for Windows console:
Optional: Note that the AppData folder is hidden by default, so you may want to unhide AppData first, to see its contents.
Restart the Git Bash Terminal window
echo $PATH ➔ This should print a long string that contains: :/c/Users/USERNAME/AppData/Local/Google/Cloud SDK/google-cloud-sdk/bin
where docker-credential-gcloud ➔ This should print C:\Users\USERNAME\AppData\Local\Google\Cloud SDK\google-cloud-sdk\bin\docker-credential-gcloud.cmd

Resources