Caching Homebrew bottles and formulas on Travis CI - homebrew

I need to install a newer version of a dependency via Brew than what is contained in the Travis CI image. Running brew update adds more than 100 seconds to every build job, so I'd like to find a way to cache the installation, or at least cache the Bottle and Formula.
It seems like there should be a simple recipe for caching the Bottles and Formulas from Brew, but so far I have not found that recipe.
To be specific, I'm using pyenv on TravisCI and for the build jobs that runs tests with Python 2.6 I need pyenv to be the latest (1.2.4), since get_pip fails on earlier versions due to recent changes on PyPI.
I added a conditional to my .travis.yml and it works fine:
if [[ "$pyver" == 2.6 ]]; then
brew upgrade pyenv &>/dev/null
fi
However, the command takes a while to run. I've split out the upgrade command and timed it:
time brew update
time brew upgrade pyenv
The brew update takes more than 100 seconds, the brew upgrade takes 10 seconds.
That finding led to the idea to cache the Bottle and Formula and on every subsequent build run:
HOMEBREW_NO_AUTO_UPDATE=1 brew upgrade pyenv
There's a question on how to cache Brew sources. That works fine.
cache:
directories:
- "$HOME/Library/Caches/Homebrew"
In $HOME/Library/Caches/Homebrew, I find the bottle cached:
pyenv-1.1.5.sierra.bottle.tar.gz
pyenv-1.2.4.sierra.bottle.tar.gz
However even with the bottle cached, running HOMEBREW_NO_AUTO_UPDATE=1 brew upgrade pyenv doesn't succeed, presumably because the updated formula is not cached. The command yields:
Error: pyenv 1.1.5 already installed
I added caching of the formula:
cache:
directories:
- "$HOME/Library/Caches/Homebrew"
- /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core
and implemented the following algorithm:
Run HOMEBREW_NO_AUTO_UPDATE=1 brew upgrade pyenv to upgrade pyenv using the latest cached formula/bottle.
Test if pyenv >= 1.2.4 is installed. If the pyenv 1.2.4 formula and bottle were cached, it will be installed in step (1) without brew update having been run.
If pyenv >= 1.2.4 was not cached, then run brew upgrade pyenv and save the cached bottle and formula.
and the following script:
function ver { printf "%03d%03d%03d" $(echo "$1" | tr '.' ' '); }
HOMEBREW_NO_AUTO_UPDATE=1 brew upgrade pyenv &>/dev/null
# Update brew and upgrade pyenv if necessary
if [ $(ver $(pyenv --version | cut -d ' ' -f2)) -lt $(ver 1.2.4) ]; then
brew upgrade pyenv &>/dev/null
fi
However, I question whether it's a good idea to cache /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core. The examples in the Travis CI documentation are always caching directories under $HOME. It seems like I might experience problems when Travis updates the image. Does Travis CI delete the branch caches after the image is updated? If not, it seems like I would need to do that manually.
Is there a better way? It seems like Travis CI should offer an easy way to cache Brew formula and bottles.
Should I not bother with Brew and just install pyenv from the source?

I cached the brew formula under $HOME by copying the file on the first build:
cache:
directories:
- "$HOME/Library/Caches/Homebrew"
[...]
before_install:
- |
if [ "$TRAVIS_OS_NAME" = osx ]; then
# Pyenv >= 1.2.4 is needed with Python 2.6
if [[ "$pyver" == 2.6 ]]; then
# Upgrade pyenv and save formula/bottle in cache.
formula=$(brew --repository)/Library/Taps/homebrew/homebrew-core/Formula/pyenv.rb
formula_cached=$(brew --cache)/pyenv.rb
if [[ ! -f $formula_cached ]]; then
brew upgrade pyenv &>/dev/null
cp $formula $formula_cached
else
HOMEBREW_NO_AUTO_UPDATE=1 brew upgrade $formula_cached
fi
fi

Related

How can I downgrade Homebrew to an older version?

Currently I have the latest version 2.2.17, I want to downgrade to version 2.2.12.
Brew does not have a downgrade option, just an upgrade. Is the process to uninstall brew and re-install it again?
If you used the normal installation way via the install script:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
Then take a look at the brew install script (it has its own repo).
It downloads the actual Homebrew repo to /usr/local/Homebrew.
...
HOMEBREW_PREFIX="/usr/local"
HOMEBREW_REPOSITORY="/usr/local/Homebrew"
...
BREW_REPO="https://github.com/Homebrew/brew"
...
ohai "Downloading and installing Homebrew..."
(
cd "${HOMEBREW_REPOSITORY}" >/dev/null || return
# "git remote add" will fail if the remote is defined in the global config
execute "git" "config" "remote.origin.url" "${BREW_REPO}"
execute "git" "config" "remote.origin.fetch" "+refs/heads/*:refs/remotes/origin/*"
...
execute "ln" "-sf" "${HOMEBREW_REPOSITORY}/bin/brew" "${HOMEBREW_PREFIX}/bin/brew"
...
}
...
==> This script will install:
/usr/local/bin/brew
/usr/local/share/doc/homebrew
/usr/local/share/man/man1/brew.1
/usr/local/share/zsh/site-functions/_brew
/usr/local/etc/bash_completion.d/brew
/usr/local/Homebrew
...
The script installs the latest copy of the Homebrew repo. So the answer to "Is the process to uninstall brew and re-install it again?" would be a "NO". Instead, install it first normally, and then after, go over to /usr/local/Homebrew and checkout the version you want.
~$ cd /usr/local/Homebrew
Homebrew$ brew --version
Homebrew 2.3.0
Homebrew/homebrew-core (git revision d41d92; last commit 2020-05-29)
Homebrew$ git fetch --tags
Homebrew$ git checkout 2.2.12
Note: switching to '2.2.12'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:
git switch -c <new-branch-name>
Or undo this operation with:
git switch -
Turn off this advice by setting config variable advice.detachedHead to false
HEAD is now at 2ae26808a Merge pull request #7301 from Bo98/cmake-sdkroot
Homebrew$ brew --version
Homebrew 2.2.12
Homebrew$ cd /usr/local/bin
bin$ ./brew --version
Homebrew 2.2.12
Homebrew/homebrew-core (git revision d41d92; last commit 2020-05-29)
I recommend doing a brew doctor after just to make sure everything's still OK.
Then, unfortunately, Homebrew always auto-updates itself every time you install something. So it will always update the repo at /usr/local/Homebrew up again to the latest version. You can disable that by exporting HOMEBREW_NO_AUTO_UPDATE as mentioned here https://github.com/Homebrew/brew/issues/1670.
~$ export HOMEBREW_NO_AUTO_UPDATE=1
~$ brew install python#3.8
You'll need to create an alias or auto-export that env var to your bash_profile (or whatever is the equivalent for the terminal you use), so that it takes effect every time you use brew.

Homebrew does not upgrade pyenv

I'm trying to upgrade pyenv with homebrew for getting new Python releases.
Here is the problem:
$ brew upgrade pyenv
Error: pyenv 1.2.7 already installed
Is there a kind of update latency in homebrew package versioning? I am aware of the existence of other ways to install pyenv, but I'd like to use homebrew.
pyenv 1.2.8 is not yet officially released.
On the master branch the last tag is 1.2.7.
The commit you mentioned in your question is in preparation for release 1.2.8.
Version 1.2.7 is here https://github.com/pyenv/pyenv/tree/v1.2.7
Version 1.2.8 should be here https://github.com/pyenv/pyenv/tree/v1.2.8 but nothing is there (yet).
Verify which pyenv your system is using.
I thought I had this issue but then realized I had two different installs on my computer and my system was not actually using the brew installed pyenv --version.
Grepping my shell history, I saw that at some point I installed pyenv with curl using pyenv-installer for some reason. The command I found in my history was curl https://pyenv.run | bash. I followed the pyenv-installer directions to uninstall. Make sure to restart your shell once you uninstall!
I then reinstalled pyenv with brew. brew reinstall pyenv though I'm not sure if that's necessary.
I now see the newer versions of python available for install with pyenv install --list

homebrew - how to install older versions

I'm trying to install memcached with older versions (ex: 1.4.5) but I'm not sure how to do it.
brew install memcached installs the latest.
I also tried brew install memecached1.4.5 but it didn't work.
Usually, you can check if multiple versions are available and you can specify the version with #. e.g. brew install package#2.8
$ brew info memcached
memcached: stable 1.4.24
High performance, distributed memory object caching system
https://memcached.org/
Conflicts with:
mysql-cluster (because both install `bin/memcached`)
Not installed
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/memcached.rb
...
If is not available the version you want you can go to the repo, and check the history
cd "$(brew --repo homebrew/core)"
git log master -- Formula/memcached.rb
Then you can find the commit you are looking for
commit 5ec463decefeaab3d1825b923ad2dbee73ffc6dc
Author: Adam Vandenberg <flangy#gmail.com>
Date: Fri Apr 9 21:19:48 2010 -0700
Update memcached to 1.4.5
Checkout that version and install:
cd "$(brew --repo homebrew/core)" && git checkout 5ec463decefeaab3d1825b923ad2dbee73ffc6dc
HOMEBREW_NO_AUTO_UPDATE=1 brew install memcached
Once you get the version installed:
git checkout master # bring brew back to its latest version
brew pin memcached # [optional] prevent formula from updating
and, that's it!
I would do
brew unlink memecached
brew install memecached#1.4.5
brew link memecached#1.4.5 --force
A more expanded version of the good answer from Adrian is also here on SO.
https://stackoverflow.com/a/53766019/3794873
One thing to keep in mind is that if you are installing an older Formula the Homebrew API/methods may have changed since that time so you should brew edit appFormula against the current version and compare to the brew edit app#your.version if you encounter any errors trying to brew install app#your.version after the brew extract command in the answer linked.
2021 update:
$ curl https://raw.githubusercontent.com/Homebrew/homebrew-cask/<commit-hash>/Casks/<FORMULA>.rb > $(find $(brew --repository) -name <FORMULA>.rb)
$ brew reinstall <FORMULA>
You can find the cask or formula file URL here and the commit message should describe the file version to grab the URL for.
https://github.com/Homebrew/homebrew-core/commits/master/Formula/<formula>.rb
https://github.com/Homebrew/homebrew-cask/commits/master/Casks/<cask>.rb
Credit: https://remarkablemark.org/blog/2017/02/03/install-brew-package-version/
Sharing an interactive oneliner I made for myself of Adrian's answer now:
It does a simple parsing, depending only on "/blob/" in the github URL, of "From" in brew info, and does a bash select of commit from git log.
brew install can be replaced with brew reinstall as needed.
Tested (for cask "name=powershell") in bash 5.1 (from macports ;P) and env -i /bin/bash 3.2 on macOS (High Sierra) 10.13.6 against brew 3.3.11. (I might have done some tap'ing the other day.)
(name= && { [[ $name ]]||read -ep "Name: " name;} && [[ $(brew info "$name"|sed -Ene'/^From: (.*)/s//\1/p') =~ ([^/]*)/blob/([^/]*)/([^/]*)/([^/]*) ]] && V=("${BASH_REMATCH[#]:1}") && N=(repo current dir file) && for s in ${!N[#]};do eval "${N[s]}=\${V[s]}" && declare -p >&2 "${N[s]}";done && cd "$(brew --repo "${repo//-//}")" && IFSO="$IFS" && IFS=$'\n' && select line in $(git log --oneline master -- "$dir/$file");do IFS="$IFSO" && commit=($line) && break;done && set -x && git checkout "$commit" && HOMEBREW_NO_AUTO_UPDATE=1 brew install "$name" && git checkout master)
Skål, salud, cheers!

Can't re-install homebrew

I'm having a problem re-installing homebrew. I am running Mavericks
It started with me trying to install express.js. Homebrew had been installed on my mac for quite some time, never had any trouble with it. I used sudo npm install -g express, but when I ran express --version, I got command not found. Eventually, I found a possible solution online, and just tried to reinstall npm with brew remove npm and brew install npm. Again, I ran into errors (Unfortunately, I didn't save the exact errors) I used brew doctor to try and figure it out, looked up a ton of tutorials and mucked around with homebrew, and I guess I broke something.
Now, I figure I should just try and reinstall homebrew.
I run the install command for homebrew:
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
It returns this message:
It appears Homebrew is already installed. If your intent is to reinstall you
should do the following before running this installer again:
rm -rf /usr/local/Cellar /usr/local/.git && brew cleanup
So, I run
rm -rf /usr/local/Cellar /usr/local/.git && brew cleanup
This returns:
-bash: brew: command not found
I try running the install command again, and this time, it doesn't tell me homebrew is already installed.
So again, I run the install command for homebrew:
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
And now I get this:
==> This script will install:
/usr/local/bin/brew
/usr/local/Library/...
/usr/local/share/man/man1/brew.1
Press RETURN to continue or any other key to abort
==> /usr/bin/sudo /bin/chmod g+rwx /Library/Caches/Homebrew
Password:
==> Downloading and installing Homebrew...
remote: Counting objects: 136, done.
remote: Total 136 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (136/136), 10.37 MiB | 1.53 MiB/s, done.
Resolving deltas: 100% (44/44), done.
From https://github.com/Onomanatee/portfolio_Yuri_Legiest
* [new branch] master -> origin/master
HEAD is now at c253261 Commit4
==> Installation successful!
==> Next steps
Run brew doctor before you install anything
Run brew help to get started
Running brew doctor or brew help , however, results in
-bash: brew: command not found
I really don't know how to fix this. And why is the brew installer referring to a repository I used as a test for a little web project? (right above Installation succesful!)

How can I brew link a specific version?

I have a few kegs of the same package in /usr/local/Cellar/libfoo like /usr/local/Cellar/libfoo/1.0.1, /usr/local/Cellar/libfoo/HEAD and /usr/local/Cellar/libfoo/mycopy
How can I brew link to a specific version?
This is probably the best way as of 11.1.2022:
To install a specific version, e.g. postgresql 9.5 you simply run:
$ brew install postgresql#9.5
To list the available versions run a search with #:
$ brew search postgresql#
==> Formulae
postgresql postgresql#11 postgresql#13 postgresql#9.5 qt-postgresql
postgresql#10 postgresql#12 postgresql#9.4 postgresql#9.6 postgrest
==> Casks
navicat-for-postgresql
DEPRECATED in Homebrew 2.6.0 (December 2020):
The usage info:
Usage: brew switch <formula> <version>
Example:
brew switch mysql 5.5.29
You can find the versions installed on your system with info.
brew info mysql
And to see the available versions to install, you can provide a dud version number, as brew will helpfully respond with the available version numbers:
brew switch mysql 0
Update (15.10.2014):
The brew versions command has been removed from brew, but, if you do wish to use this command first run brew tap homebrew/boneyard.
The recommended way to install an old version is to install from the homebrew/versions repo as follows:
$ brew tap homebrew/versions
$ brew install mysql55
For detailed info on all the ways to install an older version of a formula read this answer.
Sadly brew switch is deprecated in Homebrew 2.6.0 (December 2020)
$ brew switch
Error: Unknown command: switch
TLDR, to switch to package version 10:
brew unlink package
brew link package#10
To use another version of a package, for example node:
First, ensure that the specific version is installed using brew list. My package here is node (16) and node#14.
➜ ~ brew list
==> Formulae
node
node#14
➜ ~ node -v
v16.1.0
Unlink the current package: brew unlink node.
➜ ~ brew unlink node
Unlinking /usr/local/Cellar/node/16.1.0... 7 symlinks removed.
Link the correct version
➜ ~ brew link node#14
Linking /usr/local/Cellar/node#14/14.16.1_1... 3857 symlinks created.
If you need to have this software first in your PATH instead consider running:
echo 'export PATH="/usr/local/opt/node#14/bin:$PATH"' >> ~/.zshrc
➜ ~ node -v
v14.16.1
Homebrew removed brew switch subcommand in Homebrew 2.6.0. Get it back from here.
brew tap laggardkernel/tap
brew switch --help
name#version formula
There's mainly two ways to switch to an old version of an app.
If it's a bigger version change. Homebrew may have created a versioned package in the repo. Like go, go#1.10, they are two different formulae, installed into two different locations.
# install the old one
brew install go#1.10
# link the executable into /usr/local/bin, or /opt/homebrew/bin
brew link --overwrite --force go#1.10
brew switch
But not every package has a versioned variant. If you just upgraded to the new version and the old one is still in your system, skip step 1, 2.
In this situation, search in the homebrew-core repo and download the specific formula. e.g. mysql 8.0.23
Download the raw file, and install from it brew install /path/to/downloaded/mysql.rb.
Now both the latest and the old 8.0.23 (same formula mysql) exist, switch (link out) the old version with brew switch mysql 8.0.23
brew info mysql will list all the old version still exist.
Step 1, 2 could be replaced by brew extract, but that requires user maintain its own tap. I won't cover it here, just search it if you're interested.
If you have installed, for example, php 5.4 it could be switched in the following way to php 5.5:
$ php --version
PHP 5.4.32 (cli) (built: Aug 26 2014 15:14:01)
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2014 Zend Technologies
$ brew unlink php54
$ brew switch php55 5.5.16
$ php --version
PHP 5.5.16 (cli) (built: Sep 9 2014 14:27:18)
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2014 Zend Technologies
brew switch libfoo mycopy
You can use brew switch to switch between versions of the same package, if it's installed as versioned subdirectories under Cellar/<packagename>/
This will list versions installed ( for example I had Cellar/sdl2/2.0.3, I've compiled into Cellar/sdl2/2.0.4)
brew info sdl2
Then to switch between them
brew switch sdl2 2.0.4
brew info
Info now shows * next to the 2.0.4
To install under Cellar/<packagename>/<version> from source you can do for example
cd ~/somewhere/src/foo-2.0.4
./configure --prefix $(brew --Cellar)/foo/2.0.4
make
check where it gets installed with
make install -n
if all looks correct
make install
Then from cd $(brew --Cellar) do the switch between version.
I'm using brew version 0.9.5
In case brew switch produces an error (in this example trying to switch to node version 14):
> brew switch node 14
Error: Calling `brew switch` is disabled! Use `brew link` #-versioned formulae instead.
The correct way switching versions would be :
> brew link --overwrite node#14
if #simon's answer is not working in some of the mac's please follow the below process.
If you have already installed swiftgen using the following commands:
$ brew update
$ brew install swiftgen
then follow the steps below in order to run swiftgen with older version.
Step 1: brew uninstall swiftgen
Step 2: Navigate to: https://github.com/SwiftGen/SwiftGen/releases
and download the swiftgen with version: swiftgen-4.2.0.zip.
Unzip the package in any of the directories.
Step 3:
Execute the following in a terminal:
$ mkdir -p ~/dependencies/swiftgen
$ cp -R ~/<your_directory_name>/swiftgen-4.2.0/ ~/dependencies/swiftgen
$ cd /usr/local/bin
$ ln -s ~/dependencies/swiftgen/bin/swiftgen swiftgen
$ mkdir ~/Library/Application\ Support/SwiftGen
$ ln -s ~/dependencies/swiftgen/templates/ ~/Library/Application\ Support/SwiftGen/
$ swiftgen --version
You should get: SwiftGen v0.0 (Stencil v0.8.0, StencilSwiftKit v1.0.0, SwiftGenKit v1.0.1)

Resources