I use a gem, my_gem stored on github in an app, my_app. The Gemfile looks like this:
source 'https://rubygems.org'
git_source(:github) { |repo| "https://github.com/#{repo}.git" }
ruby '2.7.4'
gem 'rails', '~> 5.2.6'
...
gem 'my_gem', git: "https://#{ENV['PRIVATE_LIB_TOKEN']}:x-oauth-basic#github.com/obromios/my_gem.git", branch: 'master'
About a year ago, I was able to bundle install the app and push to heroku but this no longer works. I reissued an new private access token from Github, giving it access to the repo, and set it in my .env file as follows
PRIVATE_LIB_TOKEN=<token>
When I type bundle install I get the following error message
remote: Repository not found.
fatal: Authentication failed for 'https://github.com/obromios/my_gem.git/'
Retrying `git clone https://x-oauth-basic#github.com/obromios/my_gem.git /Users/my_name/.rvm/gems/ruby-2.7.4/cache/bundler/git/my_gem-<some_other_token> --bare --no-hardlinks --quiet` at /Users/my_name/xyz/my_app due to error (2/4): Bundler::Source::Git::GitCommandError Git error: command `git clone https://x-oauth-basic#github.com/obromios/my_gem.git /Users/my_name/.rvm/gems/ruby-2.7.4/cache/bundler/git/my_gem-<some_other_token> --bare --no-hardlinks --quiet` in directory my_app has failed.
If this error persists you could try removing the cache directory '/Users/my_name/xyz/my_app'
Looking at /Users/my_name/.rvm/gems/ruby-2.7.4/cache/bundler/git/ there was a directory named my_gem-<yet_another_token>. The tokens are all different but are random hashes of similar length. I removed the my_gem-<yet_another_token> directory and this did not help. I also deleted /Users/xyz/my_app and reinstalled the app, but this did not work.
I note that since last time that this worked, I had set up two factor authentication on Github.
How do I fix this?
This is not an answer to the question, but does provide a workaround as suggested by Sachin Sing. This SO answer points out that accessing the gem using a command like
gem 'my_gem', git: "https://#{ENV['PRIVATE_LIB_TOKEN']}:x-oauth-basic#github.com/obromios/my_gem.git"
causes the token to be written to the Gemfile.lock and so appears in the repository. It is suggested a better way is to put the following in the Gemfile
gem 'my_gem', git: "https://github.com/obromios/my_gem.git", branch: 'master'
and then doing the following
export MY_OAUTH_KEY=<token>
bundle config github.com $MY_OAUTH_KEY
This allowed me to bundle install the Gemfile. In order to deploy to heroku, I needed to set the following config variable
heroku config:set BUNDLE_GITHUB__COM=<token>
Then I can just deploy the app to heroku and the gem is installed properly.
Related
I'm starting to build a series of plugins and engines in a project I'm developing and I've run into the issue of having to list the paths to dependencies in all of the Gemfiles for the main application and the plugins/engines if I want rake to work.
Rake works fine for the main application because it's Gemfile lists the relative paths to the plugins/engines I want, but when a plugin/engine is dependent on another and does not have the relative paths all listed, using rake rdoc I'll get an error like the following (presumably I'll get the same error trying to run tests/the dummy application/etc):
Bundler could not find compatible versions for gem "user":
In Gemfile:
auth (>= 0) ruby depends on
user (>= 0) ruby
Could not find gem 'user (>= 0) ruby', which is required by gem 'auth (>= 0) ruby', in any of the sources.
Rather than having to use paths, I've tried specifying the git repository in the plugins/engines like so:
# User engine
gem 'user', git: 'https://localhost/git/project/user.git', branch: 'master'
And then using bundler config local.user /path/to/local/repo command to make it point to a local repository for development. This appeared to work perfectly... until I change the version of the plugin in the local repo, then it spits out this error in any dependent plugin/engine/application:
Could not find user-0.0.1 in any of the sources
Run `bundle install` to install missing gems.
Whilst that isn't really much of an issue-- the version number will be changed at the end anyway --it also turns out that it will throw the following error if you're on a branch in the local repo instead of master:
Local override for user at /path/to/local/repo is using branch deleteme but Gemfile specifies master
And omitting the branch option from the Gemfile leaves me with this error:
Cannot use local override for user at /path/to/local/repo because :branch is not specified in Gemfile. Specify a branch or use `bundle config --delete` to remove the local override
So am I just stuck with having , path: "../local-repo-directory" strewn about all of the Gemfiles for plugins/engines with dependencies on one another whilst in development or is there a way of developing multiple interdependent plugins/engines for Rails at the same time that doesn't use a really sloppy/inelegant solution?
I'm drawing blanks on other ways to do this, so any help would be greatly appreciated. I hope I've explained this well enough, but if there's anything else I can clarify, let me know.
Thanks!
Stick with specifying git repositories in your Gemfile and using bundle local overrides to work on them.
Possible problems with solutions:
1. Local override for user at /path/to/local/repo is using branch deleteme but Gemfile specifies master
If your Gemfile specifies branch "master" then the local override should have branch "master" checked out. This is because the goal of a local override is so that you can work on a gem in a local folder while running and testing the application. In production it will check out the branch and revision specified in the Gemfile and Gemfile.lock and it should offcourse be exactly what you are running with your local override.
2. Could not find user-0.0.1 in any of the sources
Run `bundle install` to install missing gems.
When you run bundle install it places a version number for each gem in the Gemfile.lock. This file is added to git so that other developers and your production server runs the same versions of the gems as you do. Your Gemfile.lock needs to specify the same version of your gem as what is returned in the local override gemspec file. If you incremented the version number in the local override you need to run "bundle update gemname" in the application that uses it so that it updates the Gemfile.lock. Note that bundler caches everything in a gemspec file until you increment the version. Thus you can't add new dependencies or change dependency versions in a gemspec if you don't increase the version number in that gemspec.
The git-bundle gem
If you use a git repository in your Gemfile with local overrides bundler will store git revision hashes in your Gemfile.lock as mentioned above. Example:
Gemfile:
gem 'example', git: 'https://github.com/your_name/example.git', branch: :master
Bundle config shell command:
bundle config local.example /path/to/local/git/repository
Gemfile.lock (auto generated):
GIT
remote: https://github.com/your_name/example.git
revision: b9270e61abb89e1ff77fb8cfacb463e4d04388ad
branch: master
After you commit in the local override repository you will need to run bundle install on your main application so that it rebuilds the Gemfile.lock to include the new revision hash and commit it. I recommend using the gem below as it automates this process for you and also aids in other scenarios. See the gem page for exact detail.
https://github.com/EPI-USE-Labs/git-bundle
The activesupport-decorators gem
As a sidenote, when you need to extend classes between your gems you can use the decorator pattern which is implemented elegantly by the activesupport-decorators gem:
https://github.com/EPI-USE-Labs/activesupport-decorators
I have a Gem on my local machine that I declare in my Gem file like this:
group :assets do
gem 'my_gem', path: "/Users/me/path/to/my_gem"
end
This works great locally, but when I push to staging on Heroku, the build fails because the gem isn't available.
Currently I'm having to comment/uncomment this gem between deploys which is a real pain.
I've also tried adding it to my development group, but this doesn't help.
Why is Heroku looking for this gem?
Bundler always needs to resolve all of the gems in your Gemfile. You shouldn't commit a Gemfile that contains a local path.
Instead, push your gem to a git repository that is reachable from Heroku and point to that in your Gemfile.
For development, you can use a local path override: http://bundler.io/v1.3/git.html#local
you can try placing the gem in vendor/gems directory, create it if it doesn't exist.then in your Gemfile do like this:
gem 'rails_multisite', path: 'vendor/gems/rails_multisite'
and make sure you run bundle update so Heroku can Pickup the changes
I have a rails app which requires a gem. I host this gem on bitbucket in a private repository.
In my Gemfile I added the gem like following:
gem "my-gem", :git => "git#bitbucket.org:my-username/my-gem.git", :branch => 'master'
I want to deploy my rails app on heroku with
git push heroku master
Now I always get following error
Fetching git#bitbucket.org:my-username/my-git-repo.git
Host key verification failed.
fatal: The remote end hung up unexpectedly
I understand the error, because the repository is set to private. But how can I solve this problem?
I already read this question: Deploying to Heroku using git on bitbucket, but I don´t really get the answer :)..
Bitbucket allows for HTTP basic auth on repository URLs similar to github. Specify the URL for the gem as https://username:password#bitbucket.org/username/gemrepo.git.
It does mean having your username and password in your Gemfile, which itself is version controlled, and that's not a good practice, but on the other hand that's what Heroku recommends, so...
The proper way to achieve this is using bundle config, which saves the configuration on your home directory .bundle/config so it stays outside the repo.
bundle config bitbucket.org user:pwd
And then on Heroku you have to set a simple configuration in a special way:
heroku config:set BUNDLE_BITBUCKET__ORG=user:pwd
And in your Gemfile you just use the URL without the credentials.
gem 'gemname', git: "https://bitbucket.org/User/gemname.git"
I had the same problem, but I ended up doing the following as a workaround to providing the Bitbucket password in the Gemfile.
The basic idea is to clone the gem from Bitbucket into a local directory, add it to your app and package it into vendor/cache so Heroku can use it. My exact steps are below:
Clone your gem to a local directory:
git clone git#bitbucket.org:me/my_private_gem.git /home/me/my_private_gem
Add the gem to your Gemfile as a 'fake' Bitbucket repo:
gem 'my_private_gem', :git => 'git#bitbucket.org:me/my_private_gem.git', :branch => 'master' # this repo will not be used
Configure Bundler to work against the local repository (where you cloned the gem in step 1):
bundle config local.my_private_gem /home/me/my_private_gem
Run bundle install as usual, you should see something like this:
Using my_private_gem (0.0.1) from git#bitbucket.org:me/my_private_gem.git (at /home/me/my_private_gem)
Package all your gems into /vendor
bundle package --all
Add /vendor to your repo
git add vendor && git commit -m 'add my_private_gem to /vendor/cache'
Push to Heroku (don't forget to commit your updated Gemfile and Gemfile.lock first), you should see something like the following:
Using my_private_gem (0.0.1) from git://github.com/my_private_gem/my_private_gem.git (at /tmp/build_19fmj3tup0zy2/vendor/cache/my_private_gem-8bc6f436e2c8)
References:
Bundler - Local Git Repos
Bundler - package
I would suggest to use ENV vars instead of a new user like:
https://#{ENV['BITBUCKET_USER']}:#{ENV['BITBUCKET_PWD']}....
Then set them using:
heroku config:add BITBUCKET_X=value
For your development environment you can use the dotenv gem to define the credentials.
See also: How can I specify a gem to pull from a private github repository?
I'm running bundler v1.3.0.pre.2 and trying to get bundler to bundle a local git repository holding a gem, into a rails app.
Gemfile :
gem 'mygem', :github => 'myrepo/mygem', :branch => 'develop'
Config :
bundle config local.mygem /path/to/local/git/repo
Bundle install throws the error :
Local override for mygem at /path/to/local/git/repo is using branch
develop but Gemfile specifies develop
Is this a bug ? I see that the branch names are exactly the same. What could be wrong ? Got the same error for Bundler v1.2.3 as well
You can use a local gem and even the branch you are working on like this
gem 'rspec-rails', path: '~/forks/rspec-rails', branch: 'feature-branch'
Just change github to path, Then
bundle install
Also as far as bundler config goes, although it is in the docs, I have never needed to change the local config like you did above.
With this method I have never needed to remove Gemfile.lock, it just picks up the changes to your local repo.
Although I don't recommend it, you could disable the branch checking feature used by bundler to make sure you are using the correct branch when developing
bundle config disable_local_branch_check true
If you just need to point to a branch within a local repository you're using like a gem, in gemfile:
gem 'gem-name', :path=> 'relative/path/to/directory'
Then create bundler configuration for that "gem" before bundle-ing
(Keep in mind the repository name may be different than the gem's name, although this is not the norm).
bundle config local.repository-name relative/path/to/directory
Magically, whatever branch is currently checked out locally will be active when you start a local server. You'll likely need to restart your server if you need to make changes or change branches (unless you have some type of automatic reloading feature).
One gotcha is when a configuration setting is already established (let's say via the mechanism above), and you need to use/reference a remote git gem - you don't need bundler configuration to use a remote git gem (if you have an active configuration, you'll get the local override error).
Check to make sure you don't have local settings for given gem:
bundle config local.gem-name
If you do, remove configuration for that gem:
bundle config --delete local.gem-name
Then point to remote repo and branch, in gemfile:
gem 'gem-name', :path => 'https://github.com/reponame.git', :branch => 'branch_name'
More information:
Git Gems: http://bundler.io/v1.7/git.html
"bundle config": http://bundler.io/v1.7/man/bundle-config.1.html
This GitHub issue suggests that the workaround may be to remove Gemfile.lock and do bundle install again.
The branch you reference in your Gemfile needs to be the same as you have checked out in the local repository. Note that after you commit in the local repository you will need to run bundle install on your main application so that it rebuilds the Gemfile.lock to include the new revision hash and commit it. I recommend using the gem below as it automates this process for you and also aids in other scenarios. See the gem page for exact detail.
https://github.com/EPI-USE-Labs/git-bundle
Full detail of what happens:
When you use a git repository in your gemfile you can use bundle local overrides for development which will store revision hashes in your Gemfile.lock. On production these exact revision hashes will be checked out when it runs bundle install.
Gemfile:
gem 'example', git: 'https://github.com/your_name/example.git', branch: :master
Bundle config shell command:
bundle config local.example /path/to/local/git/repository
Gemfile.lock (auto generated):
GIT
remote: https://github.com/your_name/example.git
revision: b9270e61abb89e1ff77fb8cfacb463e4d04388ad
branch: master
I am suffering from the infamous SQLite3 Vs Heroku error while trying to deploy a simple Rails app.
Initially my Gemfile looked like
gem 'sqlite3'
...
After googling on the topic, I updated it to look like this:
group :development, :test do
gem 'sqlite3'
end
...
and then did a bundle install and surprisingly (at least for me),
$ git status --short
M Gemfile
The Gemfile.lock did NOT change!
Now heroku keeps giving this SQLite error again and again because
the Gemfile.lock is same as before and bundle install keeps failing
on their server!
Where am I going wrong?
Damn! It was a git issue.
I was working on a branch named 'deploy' and trying to push the 'master' branch and hence bundle install failed every time!
$git push heroku deploy:master
This made it work :)