How can I set up Homebrew without destroying my system? - homebrew

Question

Succumbing to pressure, I've installed Homebrew and given it a whirl. But I'm surprised at the experience so far. My impression of Homebrew is that it serves as an easy to use, safe, and self-contained package manager for OS X. But this has not been my experience.
(1) The first thing it does is change a bunch of permissions on a number of scary looking directories:
==> The following directories will be made group writable:
/usr/local/.
/usr/local/bin
/usr/local/include
/usr/local/lib
/usr/local/lib/pkgconfig
/usr/local/share
/usr/local/share/man
/usr/local/share/man/man1
/usr/local/share/man/man3
/usr/local/share/man/man7
/usr/local/share/info
/usr/local/share/doc
==> The following directories will have their group set to admin:
/usr/local/.
/usr/local/bin
/usr/local/include
/usr/local/lib
/usr/local/lib/pkgconfig
/usr/local/share
/usr/local/share/man
/usr/local/share/man/man1
/usr/local/share/man/man3
/usr/local/share/man/man7
/usr/local/share/info
/usr/local/share/doc
(2) The next thing it does (following the recommended procedure to run brew doctor) is ask me to delete a whole bunch of scary looking files:
Warning: Some directories in /usr/local/share/man aren't writable.
This can happen if you "sudo make install" software that isn't managed
by Homebrew. If a brew tries to add locale information to one of these
directories, then the install will fail during the link step.
You should probably `chown` them:
/usr/local/share/man/de
/usr/local/share/man/de/man1
/usr/local/share/man/mann
Warning: Broken symlinks were found. Remove them with `brew prune`:
/usr/local/share/ghostscript/9.05/Resource/Font/blex.pfb
[hundreds...]
Warning: Unbrewed dylibs were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
Unexpected dylibs:
/usr/local/lib/libasan.1.dylib
/usr/local/lib/libasan.2.dylib
/usr/local/lib/libatomic.1.dylib
/usr/local/lib/libcdt.5.dylib
/usr/local/lib/libcgraph.6.dylib
/usr/local/lib/libcilkrts.5.dylib
/usr/local/lib/libgcc_ext.10.4.dylib
/usr/local/lib/libgcc_ext.10.5.dylib
/usr/local/lib/libgcc_s.1.dylib
/usr/local/lib/libgcc_s.10.4.dylib
/usr/local/lib/libgcc_s.10.5.dylib
/usr/local/lib/libgfortran.2.0.0.dylib
/usr/local/lib/libgfortran.3.dylib
/usr/local/lib/libgmp.10.dylib
/usr/local/lib/libgmpxx.4.dylib
/usr/local/lib/libgomp.1.dylib
/usr/local/lib/libgvc.6.dylib
/usr/local/lib/libgvpr.2.dylib
/usr/local/lib/libitm.1.dylib
/usr/local/lib/liblkdynam.dylib
/usr/local/lib/liblkrealt.dylib
/usr/local/lib/liblksec.dylib
/usr/local/lib/liblksock.dylib
/usr/local/lib/libmpc.3.dylib
/usr/local/lib/libmpfr.4.dylib
/usr/local/lib/libpathplan.4.dylib
/usr/local/lib/libquadmath.0.dylib
/usr/local/lib/libssp.0.dylib
/usr/local/lib/libstdc++.6.dylib
/usr/local/lib/libtcl8.6.dylib
/usr/local/lib/libtk8.6.dylib
/usr/local/lib/libubsan.0.dylib
/usr/local/lib/libxdot.4.dylib
Warning: Unbrewed header files were found in /usr/local/include.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
Unexpected header files:
/usr/local/include/c++/5.0.0/backward/auto_ptr.h
[hundreds...]
Warning: Unbrewed .la files were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
Unexpected .la files:
/usr/local/lib/libasan.la
/usr/local/lib/libatomic.la
/usr/local/lib/libcilkrts.la
/usr/local/lib/libgfortran.la
/usr/local/lib/libgmp.la
/usr/local/lib/libgmpxx.la
/usr/local/lib/libgomp.la
/usr/local/lib/libitm.la
/usr/local/lib/libmpc.la
/usr/local/lib/libmpfr.la
/usr/local/lib/libquadmath.la
/usr/local/lib/libssp.la
/usr/local/lib/libssp_nonshared.la
/usr/local/lib/libstdc++.la
/usr/local/lib/libsupc++.la
/usr/local/lib/libubsan.la
Warning: Unbrewed .pc files were found in /usr/local/lib/pkgconfig.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
Unexpected .pc files:
/usr/local/lib/pkgconfig/libcdt.pc
/usr/local/lib/pkgconfig/libcgraph.pc
/usr/local/lib/pkgconfig/libgvc.pc
/usr/local/lib/pkgconfig/libgvpr.pc
/usr/local/lib/pkgconfig/libpathplan.pc
/usr/local/lib/pkgconfig/libxdot.pc
/usr/local/lib/pkgconfig/tcl.pc
/usr/local/lib/pkgconfig/tk.pc
Warning: Unbrewed static libraries were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
Unexpected static libraries:
/usr/local/lib/libatomic.a
/usr/local/lib/libcilkrts.a
/usr/local/lib/libgfortran.a
/usr/local/lib/libgmp.a
/usr/local/lib/libgmpxx.a
/usr/local/lib/libgomp.a
/usr/local/lib/libitm.a
/usr/local/lib/libmpc.a
/usr/local/lib/libmpfr.a
/usr/local/lib/libquadmath.a
/usr/local/lib/libssp.a
/usr/local/lib/libssp_nonshared.a
/usr/local/lib/libstdc++.a
/usr/local/lib/libsupc++.a
/usr/local/lib/libtclstub8.6.a
/usr/local/lib/libtkstub8.6.a
(3) It fails to install, for example, attempting to install matplotlib-basemap gives me
==> Installing matplotlib-basemap from homebrew/homebrew-python
==> Installing dependencies for matplotlib-basemap: numpy, pkg-config, libpng, freetype, matplotlib, jpeg, libtiff, little-cms2, webp, pillow
==> Installing matplotlib-basemap dependency: numpy
==> Using Homebrew-provided fortran compiler.
This may be changed by setting the FC environment variable.
==> Downloading https://downloads.sourceforge.net/project/numpy/NumPy/1.9.1/numpy-1.9.1.tar.gz
Already downloaded: /Library/Caches/Homebrew/numpy-1.9.1.tar.gz
==> Patching
==> python setup.py build --fcompiler=gnu95 install --prefix=/usr/local/Cellar/numpy/1.9.1
File "/private/tmp/numpy-PSE07t/numpy-1.9.1/numpy/distutils/fcompiler/gnu.py", line 197, in get_flags_opt
v = self.get_version()
File "/private/tmp/numpy-PSE07t/numpy-1.9.1/numpy/distutils/fcompiler/__init__.py", line 434, in get_version
raise CompilerNotFound()
numpy.distutils.fcompiler.CompilerNotFound
couldn't understand kern.osversion `14.0.0'
even though I have the ScyPy stack and gfortran already installed and working fine.
Is there a way out of this mess? How do I get from a working configuration (Xcode, Python, Python packages maintained in site-packages with pip, etc.) to one that also uses Homebrew (and continues to work)? Do I really need to follow all of the Doctor's recommendations and delete all those files in order to proceed; is it safe to do so?

Answer 1

The issues you show aren't indicative of a general problem.
The fact that Homebrew "takes over" /usr/local/ is a well-documented design decision. It might be "scary" at first, but it works that way for everybody. It is possible to install Homebrew into a different directory, if you really want to do that.
brew doctor warnings are also normal if you have existing non-brewed software installed under /usr/local/. Note that these are warnings that say something to the effect of, if you are having problems, these could be causes. It doesn't say, delete all of these files before doing anything else. (plenty of questions on SO on this issue)
and 4. are just bad luck AFAICT. These are somewhat off-the-beaten-path packages, so they might be having some issues right now.
If you can install mainstream packages like, I don't know, git or make or lynx, then everything is probably fine and you just need to work out the issues with the individual packages.

Answer 2

/usr/local is yours — I mean — it's not a system location, though it is specific to your system. Of f**k. Lemme start over.
The reason of Homebrew being in /usr/local is because that place is reserved for modifications that shouldn't be touched by macOS updates.
As for the permissions, just claim the whole thing (/usr/local) to yourself (you are an admin, right?). In your account, in Bash, not elevated (prompt $ not #), do:
sudo chown -fR "$(id -u):$(id -g)" /usr/local
Maybe try lowering its permissions to a common group. If you pay attention to the commands Homebrew tells you to run to fix permissions, they all specify a username, it's not great in machines with multiple users, regardless if all of them are you. If you have multiple accounts in the system you have to make sure they all share access to Homebrew's directories under /usr/local. In this case not the whole /usr/local (unless, again, all users are yours).
Add yourself to the admin groups (works for Active Directory accounts too, though dsconfigad is supposed to do that for you):
sudo dscl . -append /groups/wheel GroupMembership $USER
sudo dscl . -append /groups/admin GroupMembership $USER
If $USER (the command dscl requires elevation so to check elevation is needed too: sudo echo $USER) somehow doesn't match your account, id -u(n) is evaluated on the spot.
sudo dscl . -append /groups/wheel GroupMembership "$(id -u)"
sudo dscl . -append /groups/admin GroupMembership "$(id -u)"
When problems start appearing about Homebrew not being able to delete thing it's always about permissions. Check your file structure. Homebrew has a caching location in your home Library ($HOME/Library/Caches/Homebrew), if you're not running, Docker, Spotify or iCloud, this should be one of the biggest folders there.
You can just nuke it if you're not in the mood, things will be downloaded again. ~/Library is a heavily accessed place, it is the reason why Macs can't use networked home directories anymore. It is prone to corruption specially when the system is busy, despite Apple's APFS claims of being on top of things.
It wouldn't hurt to be more aggressive with things, delete whole structures, break things. It's your system*! Make it your b****. You'll learn a lot. As long as you keep backups, the worst that can happen is you waste an hour reinstalling macOS.
*: If you're using anything past Mojave, then forget it. Apple owns your system.