I can shell into a Heroku app using the CLI command:
heroku run -a app-name bash
This works beautifully, however, I cannot seem to be able to specify which dyno I want to shell into. I have one web and one worker dyno, each with their own Docker image, and the run command always goes into the web.
Is there a solution to shell into a worker dyno?
I found the answer myself. Based on Docker's documentation:
If your app is composed of multiple Docker images, you can target the process type when creating a one-off dyno:
$ heroku run bash --type=worker
This works exactly as expected.
Related
I am currently using visual studio build an console applocation that has docker support, the problem with this is the application does not seem to start in a external command prompt, but the
seem to outputting in the internal console window of visual studio, how do i make it execute in a command prompt window?
It seems that the commands it uses forces it to outputted into the dev console window
docker exec -i -w "/app" b6375046a58cba92571a425d937a16bd222d87b537af1c1d64ca6b4c845616c9 sh -c ""dotnet" --additionalProbingPath /root/.nuget/fallbackpackages2 --additionalProbingPath /root/.nuget/fallbackpackages "bin/Debug/netcoreapp3.1/console.dll" | tee /dev/console"
how do i the exec command line such that it outputs to a different window?
And is it somehow possible to deploy these containered application into an locally running kubernetes cluster?
Thus utilizing kubernetes services - instead of specifying ip address and etc?
There is no meaining "different window".
You can run your app in foreground or in detached mode(-d).
To start a container in detached mode, you use -d=true or just -d option.
In foregroung you shouldn't spesified the -d flag
In foreground mode (the default when -d is not specified), docker run can start the process in the container and attach the console to the process’s standard input, output, and standard error
And, of course, you can deploy your applications into kubernates cluster. Without any troubles try minikube to achieve all what you need.
And kubernets services that is another way to represent your app to the world or other local place.
An abstract way to expose an application running on a set of Pods as a network service.
I have a scrapy spider that uses splash which runs on Docker localhost:8050 to render javascript before scraping. I am trying to run this on heroku but have no idea how to configure heroku to start docker to run splash before running my web: scrapy crawl abc dyno. Any guides is greatly appreciated!
From what I gather you're expecting:
Splash instance running on Heroku via Docker container
Your web application (Scrapy spider) running in a Heroku dyno
Splash instance
Ensure you can have docker CLI and heroku CLI installed
As seen in Heroku's Container Registry - Pushing existing image(s):
Ensure docker CLI and heroku CLI are installed
heroku container:login
docker tag scrapinghub/splash registry.heroku.com/<app-name>/web
docker push registry.heroku.com/<app-name>/web
To test the application: heroku open -a <app-name>. This should allow you to see the Splash UI at port 8050 on the Heroku host for this app name.
You may need to ensure $PORT is set appropriately as the EXPOSE docker configuration is not respected (https://devcenter.heroku.com/articles/container-registry-and-runtime#dockerfile-commands-and-runtime)
Running Dyno Scrapy Web App
Configure your application to point to <app-host-name>:8050. And the Scrapy spider should now be able to request to the Splash instance previously run.
Run at the same problem. Finally, I succesfully deployed splash docker image on Heroku.
This is my solution:
I cloned the splash proyect from github and changed the Dockerfile.
Removed command EXPOSE because it's not supported by Heroku
Replaced ENTRYPOINT by CMD command.
CMD python3 /app/bin/splash --proxy-profiles-path
/etc/splash/proxy-profiles --js-profiles-path /etc/splash/js-profiles
--filters-path /etc/splash/filters --lua-package-path /etc/splash/lua_modules/?.lua --port $PORT
Notice that I added the option --port=$PORT. This is just to listen at the port specified by Heroku instead of the default (8050)
A fork to the proyect with this change its avaliable here
You just need to build the docker image and push it to the heroku's registry, like you did before.
You can test it locally first but you must pass the environment variable "PORT" when running the docker
sudo docker run -p 80:80 -e PORT=80 mynewsplashimage
Like most docker users, I periodically need to connect to a running container and execute various arbitrary commands via bash.
I'm using 17.06-CE with an ubuntu 16.04 image, and as far as I understand, the only way to do this without installing ssh into the container is via docker exec -it <container_name> bash
However, as is well-documented, for each bash shell process you generate, you leave a zombie process behind when your connection is interrupted. If you connect to your container often, you end up with 1000s of idle shells -a most undesirable outcome!
How can I ensure these zombie shell processes are killed upon disconnection -as they would be over ssh?
One way is to make sure the linux init process runs in your container.
In recent versions of docker there is an --init option to docker run that should do this. This uses tini to run init which can also be used in previous versions.
Another option is something like the phusion-baseimage project that provides a base docker image with this capability and many others (might be overkill).
New to Docker; trying to follow along with two COdeShip tutorials for Rails+Postgres inside Docker containers. Database container starts and stops with app container under control of docker-compose up -d and docker-compose rm. Postgres commands are not available inside app container, however, so docker-compose run app bin/rake db:create chokes reporting that createdb could not be found.
I've written this up more fully in this Gist.
I've been stuck for 3 days and counting; help would be greatly appreciated!
I'm working on deploying a Rails application to Elastic Beanstalk using docker and so far everything has worked out. I'm at the point where the application needs to run migrations and seeding of the database, and I'm having trouble figuring out exactly how I need to proceed. It appears that any commands in the /.ebextensions folder run in the context of the host machine and not the docker container. Is that correct?
I'm fine with running a command to execute migrations inside of the docker container after startup, but how do I ensure that the migrations only run on a single instance? Is there an environment variable or some other way I can tell what machine is the leader from within the docker container?
Update: I posted a question in the Amazon Elastic Beanstalk forums asking how to run "commands from Docker host on the container" on the 6th/Aug/15'. You can follow the conversations there as well as they are useful.
I'm not sure the solution you have proposed is going to work. It appears that the current process for EB Docker deployment runs container commands before the new docker container is running, which means that you can't use docker exec on it. I suspect that your commands will execute against the old container which is not yet taken out of service.
After much trial and error I got this working through using container commands with a shell script.
container_commands:
01_migrate_db:
command: ".ebextensions/scripts/migrate_db.sh"
leader_only: true
And the script:
if [ "${PROCESS}" = "WEB" ]; then
. /opt/elasticbeanstalk/hooks/common.sh
EB_SUPPORT_FILES=$(/opt/elasticbeanstalk/bin/get-config container -k support_files_dir)
EB_CONFIG_DOCKER_ENV_ARGS=()
while read -r ENV_VAR; do
EB_CONFIG_DOCKER_ENV_ARGS+=(--env "$ENV_VAR")
done < <($EB_SUPPORT_FILES/generate_env)
echo "Running migrations for aws_beanstalk/staging-app"
docker run --rm "${EB_CONFIG_DOCKER_ENV_ARGS[#]}" aws_beanstalk/staging-app bundle exec rake db:migrate || echo "The Migrations failed to run."
fi
true
I wrap the whole script in a check to ensure that migrations don't run on background workers.
I then build the ENV in exactly the same way that EB does when starting the new container so that the correct environment is in place for the migrations.
Finally I run the command against the new container which has been created but is not yet running - aws_beanstalk/staging-app. It exits at the end of the migration and the --rm removes the container automatically.
Update: This solution, though seemingly correct, doesn't work as intended (it seemed it was at first though). For reasons best explained in nmott's answer below. Will leave it here for posterity.
I was able to get this working using container_commands via the .ebextensions directory config files. Learn more about container commands here. And I quote ...
The commands in container_commands are processed in alphabetical
order by name. They run after the application and web server have been
set up and the application version file has been extracted, but before
the application version is deployed. They also have access to
environment variables such as your AWS security credentials.
Additionally, you can use leader_only. One instance is chosen to be
the leader in an Auto Scaling group. If the leader_only value is set
to true, the command runs only on the instance that is marked as the
leader.
So, applying that knowledge ... the container_commands.config will be ...
# .ebextensions/container_commands.config
container_commands:
01_migrate_db:
command: docker exec `docker ps -l -q -f 'status=running'` rake db:migrate RAILS_ENV=production
leader_only: true
ignoreErrors: false
02_seed_db:
command: docker exec `docker ps -l -q -f 'status=running'` rake db:seed RAILS_ENV=production
leader_only: true
ignoreErrors: false
That runs the migrations first and then seeds the database. We use docker exec [OPTIONS] CONTAINER_ID COMMAND [ARG...] which runs the appended COMMAND [ARG...] in the context of the existing container (not the host). And we get CONTAINER_ID by running docker ps -q.
Use .ebextensions/01-environment.config:
container_commands:
01_write_leader_marker:
command: touch /tmp/is_leader
leader_only: true
Now add directory /tmp to volumes in Dockerfile / Dockerrun.aws.json.
Then check set all initialization commands like db migration in sh script that first check if file /tmp/is_leader exists and executes them only in this case.
Solution 1: run migration when you start server
In the company I work for we have literally equivalent for this line to start the production server:
bundle exec rake db:migrate && bundle exec puma -C /app/config/puma.rb
https://github.com/equivalent/docker_rails_aws_elasticbeanstalk_demmo_app/blob/master/puppies/script/start_server.sh.:
https://github.com/equivalent/docker_rails_aws_elasticbeanstalk_demmo_app/blob/master/puppies/Dockerfile
And yes this is Load balanced environment (3 - 12 instances depending on load) and yes they all execute this script. (we do load balance by introducing 1 instance at a time during deployment)
The thing is the first batch of deployment (first instance up ) will execute the bundle exec rake db:migrate and run the migrations (meaning it will run the DB changes)
and then once done it will run the server bundle exec puma -C /app/config/puma.rb
The second deployment batch (2nd instance) will
also run the bundle exec rake db:migrate but will not do anything (as there are no pending migrations).
It will just continue to the second part of the script bundle exec puma -C /app/config/puma.rbo
So honestly I don't think this is the perfect solution but is pragmatic and works for our team
I don't believe there is any generic "best practice" for EB out there for Rails running migrations as some
application teams don't want to run the migrations after the deployment while others (like our team) they
do want to run them straight after deployment.
Solution 2: background worker Enviromnet to run migrations
if you have Worker like Delayed job, Sidekiq, Rescue on own EB enviroment you can configure them to run
the migrations:
bundle exec rake db:migrate && bundle exec sidekiq)
So first you willdeploy the worker and once the worker is deployed then deploy webserver that will not run the migrations
e.g.: just bundle exec puma
Solution 3 Hooks
I agree that using EB hoos ore ok far this but
honestly I use eb hooks only for more complex
devops stuff (like pulling ssl certificates for the Nginx web-server) not for running migrations)
anyway hooks were already covered in this SO question so I'll not repeat the solution. I will just reference this article that will help you understand them:
https://blog.eq8.eu/article/aws-elasticbeanstalk-hooks.html
Conclusion
It's really up to you to figure out what is the best for your application. But honestly EB is really simple tool
(compared to tools like Ansible or Kubernetes) No mater what you implement as long as it works its ok :)
One more helpful link for EB for Rails developers:
talk: AWS Elastic Beanstalk & Docker for Rails developers