I'm a developer who uses RoR-CoffeeScript-Sass-Passenger-Apache. We use EC2 for our deployment and we have Macbook Airs for development. While the rails community is very much Mac-friendly, because of the whole deployment stack difference in dev vs. prod, I'm using a virtualbox+ubuntu while my peers are developing on OS X native.
Having on OS X Native adds more problem as we have more dependencies in the stack (Solr, Beanstalk, Mongodb and more which works well in Ubuntu)
I'm looking for suggestions on how Rails developers using Mac and Amazon EC2 can have their dev and prod environment setup.
Would also like feedback on use of vagrant for distribution of development environments for this use case.
A common practice would be to replicate your stack as a "staging" environment. With EC2, you can just create AMI's of your existing machines and duplicate them, turning them on only to test deploys, and run your tests to make sure everything is running properly before deploying it to production. Or often you may wish to leave it on permanently so developers can quickly deploy updates or patches to test as need be.
Doing it this way ensures that you have an exact replica of your production system to test before rolling out, thereby eliminating any (catastrophic) issues pertaining to the deploy sneaking out into production.
Our team has been been developing on Macs and deploying to Ubuntu on EC2 for three years now with very few issues. Several things have helped make this a smooth process:
We can run the entire app stack** on a Mac. Between macports, homebrew, and building from source when necessary, we have managed to get every piece of technology that we run in prod working on our dev boxes. The way the pieces are configured and fit together is different locally (in prod, for example, we auto-discover our memcached instance, whereas locally it's hard coded) but every integration can be tested on Macs first before going to prod.
Our continuous build system is on the same setup as our prod boxes. This means if you check in some code that depends on some piece of local magic it's discovered quickly.
We run a soak (some people call this staging or integ) stack that is configured identically to production. This sometimes causes some development overhead but has so many benefits that it's well worth it. All code goes through this stack before being pushed to prod.
This setup has worked well enough that over time we've allowed more parts of the setup to drift apart. We used to run passenger locally (like we do in prod) but now use Pow. We regularly experiment with new ruby versions in development for some time before upgrading the rest of the stack.
I've had to develop using a virtualized environment for other projects (OSX + CentOS in VirtualBox) and definitely found it more painful that all-native. For one, it felt like managing two machines instead of one. Everything also felt sloooooowww.
If there's a piece of the stack that is painful to run on the Mac, I would definitely prefer to take the hit of either a) spending the time of getting it working locally or b) abstracting that piece away, rather than pay the tax of dealing with a virtual environment.
** I'm only including the Rails app and direct dependencies in this discussion. For example, we use puppet to configure our EC2 fleet, but don't run it on our dev boxes.
Related
I am working on moving our existing websites from a shared hosting to a VPS and then they will be redeveloped and improved using Laravel. My background is not software development, I have however a decent understanding of web development (enough to make a blog, CMS etc) BUT I have never worked in a web dev team so I don't know how things should be done "properly".
Locally I have always used XAMPP and remotely it has always been as basic as publishing files via Filezilla.
Now I have been required to do:
Version control - the changes to the website should be reviewed by a second (non technical person) before going live
Develop a system based on Laravel
What I am struggling to understand is how Ubuntu Server, GIT, Docker, Kubernetes, NGINX etc. all work together. Basically I don't know what "the big picture" looks like, how a decent workflow should look like.
So far I have manually installed all the necessary software to run Laravel on the VPS (the LAMP stack) but soon after I started to run into problems (libraries that are activated locally are not activated remotely). It has also become clear that software updates and differences between my local environment and remote (production environment) will make the issues worst over time.
Can someone explain in VERY general terms, how things should fit together so that my setup is both resilient, robust and scalable? For example:
Install docker on the server and on your computer
Download such and such image
connect GIT in such and such way
Enable unattended-upgrades
The more I read the more I get confused.
What I would like is a simple guide/idea on how things should be done properly.
I am trying to find the best way to achieve the following scenario;
I am currently working on getting a complex enterprise web application that consist on:
DB
BPM Engine
SOA Engine
Reporting Engine
Web Application Server
IDE
The applications is currently running in non-prod and prod environment but each environment is independent (no infra as a code, and deployments go from dev -> ... -> prod).
When a new developer comes in, they can't run the system in their local machine as it involves too many components (will come to this later). So they do development in their local machine and to test, they need to publish and deploy to dev. Test, rinse and repeat.
I am currently working on reverse engineer the whole thing so I can get it working on my local machine provided that I can install and run all the components. I am nearly there after fiddling with a lot of configuration, settings, etc.
This work I would like others to use, so they can also run the project in their local machines. In fact, since we will be migrating soon, I would like to pack the whole thing in a way that I can deploy it anywhere (the app already working and configured) and parametrised somehow whether is DEV, SYS, UAT, PROD. This, according to my understanding is what a docker image would do for you correct? You do all the work and then you create an image out of it? Then you can have this image running in a container and that way, other people can 'reuse' your work?
Is this the correct way of doing it? Any hints / comments would be appreciated
Apologies for my writing.
I am working on an open source project and there is an issue where an upload error only seems to occur on the production side that is running apache and unicorn.
Due to privacy issues and risks, I am unable to mess around in the actual production side (such as creating a temp id for myself with various privileges.
Is there anyway that I can quickly create a VM with such setup in my own computer that would mirror the live site?
The site is running in RoR 4, latest unicorn, and latest stable version of apache.
There are a couple of ways you can be able to accomplish this. You can use vagrant and also with the rising popularity of Docker containers, you can easily model production environment on your computer. Since you indicated a faster way to get the VM up and running, I would recommend using railsbox.io. Its amazing and it saves you a lot of time. According to their webiste -
Fast and easy Ruby on Rails virtual machines. Streamline your
development workflow in no time by creating production-like virtual
machine with your development environment. Try this extremely simple
to use VM configuration tool to create new Ruby on Rails server using
vagrant and ansible.
The app helps you setup a VM with ease.
From the beginning, I am a Windows master. I started with MS-DOS. I put up Windows 2.1 and every Windows since. I have 10 different Windows boxes running in my house right now, from Windows 7 Ultimate to varied flavors of Windows Servers. I haven't done Windows 8 and don't want to go there.
I have UNIX experience with both servers and varied software, but it hasn't been my preferred environment. However, I guess I am converting. I've tried to pretend to run UNIX under Windows using Cygwin and MSYS. My purpose is to build a development environment. Both have failed me. I have spent more time trying to fix a series of technical issues than I have developing. That is unacceptable.
My Ruby on Rails development environment is by far my highest current priority. I have websites to build, right now.
At this point, I have two options. One is to find a UNIX development environment in a cloud. The other is to convert one of my many machines to a true UNIX system. So, I need advice. I don't really want to build and babysit a system. The idea of a cloud-based development environment is very interesting, with the caveat that I don't chase it down another rat hole like I have with Cygwin and MSYS.
Here are the questions. Is there a solid cloud-based Ruby on Rails friendly development environment out there? Failing that, should I put up an Ubuntu-based system. If I go there, do I convert a workstation or a server?
Thanks...
I highly recommend Vagrant. I use this to do development on my Windows systems.
As you found out, Windows is terrible for RoR development. Your best option would be to use a VM like VirtualBox to run a Linux/Unix instance. There are other VM options, but VirtualBox is free.
Failing that, just convert once of your boxes to Linux/Unix. For development it does not matter one bit if it is a workstation or a server.
Mac OSX
The unofficial standard for RoR production is Linux, but for development it is Mac OSX. There has been a big migration of developers to the Apple platform that has been going on for many years now.
It gives you the best of both worlds: it is Unix underneath but it's also a commercial platform, a polished UI, and an available software ecosystem.
Yes, it's expensive, but people should ask themselves, why are people willing to pay so much? If you can afford 10 boxes for Windows, how about finding $ for one Mac? Then, you will have not just a workable RoR environment, but the best.
Or go VM
But if you don't take that advice, you may want to install a Hypervisor like Xen or XenServer for free, and then you can run both Windows and Linux on the same machine. This is slightly different than running a VM under Windows.
Externally, I have received a recommendation for EngineYard.com as an outstanding RoR environment and will ultimately consider it as my primary development/migration/production environment.
I have a working Debian system now and am building it out as a local RoR environment. It just seems to be right for a serious RoR development environment. I will go there as soon as it is fully built out because RoR is just meant for UNIX.
However, at least temporarily, I have found that RubyStack is a seriously usable Windows RoR development environment. It is 100% usable as a standalone system that doesn't require UNIX-style environments shoehorned onto Windows. Trying to run UNIX on Windows was a constant source of frustration, so this meets my immediate needs.
I admit that I don't follow much of anything "right" on deploying test versus production code. I have been using ASP.NET, and I typically run it locally in Visual Studio, it works, I upload it, I test it again on the production server.
I have read several people say that deploying Rails apps is harder and there are special programs/ways on the ruby site about deploying RoR. I've only toyed with RoR. What is special about deployment? You don't just copy and paste the code and run it (from development machine to the production)? Is it because one is in Apache and the other running on the built in server?
This will be on a Mac Server if it matters.
Deploying RoR is not difficult anymore, especially with Phusion Passenger.
What is somewhat difficult, is getting a automated production environment setup with capistrano, vlad, etc. If you don't mind simply copying your code to the server, you can do that just fine. Most people choose not to do it that way because you lose out on a lot of the benefits that the automated deployment tools give you.
I guess people consider a Rails app harder to deploy than say some PHP apps or such where you just plop the code somewhere and point Apache or whatever at it. But, as mentioned above, you could do that now with Phusion Passenger.
We use Nginx+Passenger, but not for simplicity of deployment. Capistrano is our deploy tool of choice, and really, unless you have a very simple app, you're going to want something like Capistrano anyway. For example, with our deploy, we do a slew of things:
run any database migrations
generate release notes automatically, based on all the commits to Git between the last deploy and this one
notify various people via email (with differing lists depending on whether the deploy is to our staging environment or production) - we do this via cap_gun which integrates with Capistrano.
Notify New Relic RPM of the deploy so it can mark it in our RPM analysis
Notify Hoptoad of the deploy, so it too can have that data when reporting any exceptions
produce our sitemap.xml file, and ping Google to tell them there's a new one
update crontab files (I store our crontab files for each server in our git repo, and then on deploy it sees if there is a new version and updates accordingly, etc.).
flush/restart memcached
There are other ways aside from Capistrano, but it's a proven tool, with a lot of flexibility, yet pretty simple to setup a vanilla configuration.
So, my take is that once you get into any app that is beyond just the very simplest of apps, you're going to need/want to be doing things other than just simply updating the code. In the beginning though, if you just need the code updates, and maybe Rails migrations, then you can do simpler things like Passenger and code sync, or look at tools like Heroku or Engine Yard's stuff where they do a deploy by doing a Git clone (and then offer some additional abilities).
Another super easy way to deploy is with http://heroku.com/
Some of the issues you face with deploying rails to production:
Database connection.
You need to be sure that the database connector is set up for the production environment.
Database migrations.
You have to run database migrations against the production database even though you may have already run them in production/testing/staging
Ruby version. The version or sub-version or Ruby can trip you up, e.g. An error occurred while installing debugger-linecache (1.1.1), and Bundler cannot continue
Gem dependency.
Your production environment may have different packages and gems from development. Bundler will figure this out for the most part and install the dependencies but occasionally there are still issues that you have to resolve manually.
Dependencies.
Some gems on some machine have particular dependencies. I have seen frequent problems with using gems on my unix box that work on OSX and vice-versa.
Note the last 3 shouldn't affect you if on the same machine but I included them based on the title and to be comprehensive.
It's not especially hard. If you stick to conventions then with a little bit of configuration it boils down to this:
cap deploy
...however there is sometimes a bit of effort needed up front to get the workflow in place.
The good news is that lots of people have packaged up solutions and stacks for RoR that you can just plug and play. For example, google ec2onrails - this is a packaged Ubuntu image and set of capistrano tasks for running rails apps in Amazon's EC2 cloud, with lots of common stuff set up already out of the box.
Choose a good hosting provider and you should be able to find something similar for that also.
An easy way to deploy Rails apps is to use Phusion Passenger. Deployment doesn't get much easier than that for any programming language or framework. You can do that on a Mac server.
Another really easy way to deploy rails is with jruby and the glassfish gem.