I am currently working on a Rails 3 project that is divided up into four parts:
The public facing website
The administration website/backend
The models
The API for third party data access
As the models are shared between the three key components I want to keep them away from being in one main project, however each part needs access to the models, but I don't want to repeat the code and have different versions everywhere.
Currently I have the model code in a gem, and in each project's Gemfile I am referencing them with the following line:
gem "my_models", :path => "../my_models/"
However when I deploy to our test servers for my co-workers to evaluate the system on I need to pull the models from an external repository, so I swap out the above line with the following:
gem "my_models", :git => "git#private.repository.com:username/my_models.git"
This in its self works well, but its quite clunky in terms of 'versions' (i.e. I need to bump the version every time I wish to deploy the changes to the test servers), switch the line over to use git instead of local, and make sure that I'm pushing the files properly.
Previously I was using a shared git submodule, but this was just as awkward.
I would rather not build everything into one mega-project, as these tend to become monstrous and difficult to maintain, and I would also like to separate concerns if possible, so any changes I make to the administration site doesn't have much of a chance to impact the other components - obviously the models have the potential to cause issues, but that is a risk I have considered and understand.
What would people out there suggest when it comes to something like this? Or, am I going about it completely the wrong way?
Some additional background:
This app is a rewrite of an existing website which followed the model of 'lump everything into the one project' - unfortunately there are two issues here:
The app was badly developed - I inherited this project and when I first picked it up the load times were ~2 minutes per page with a single user - this has since been reduced but still has issues throughout
We are currently at our capacity limit of the current site and we anticipate that we will need to take on more load in the next 6 months - however scaling out with an 'all in one' app means we'll be wasting resources on scaling out the back end of the site which doesn't need it.
Essentially there are two things I want to separate - the Front end (being the public website and the API) and the back end - everything I know about software development tells me that combining all this together is not an ideal solution (and past history shows me that splitting these two is a good move in terms of ensuring front end performance).
Perhaps I need to look at this from another angle - keep the models in each project, and instead of sharing them between projects have a cut-down subset of functionality for each functional area (i.e. the backend needs to know who created a post, but the front end doesn't really care about that, so omit that logic when reading in the model).
drop the models project(put models into one of other parts, i'd suggest whatever you consider "more important"), put all projects into single repository(separate project folders) and make symlinks to models/libs/apis/whatever
your code is highly coupled together and you often need to make changes to few projects at once(like updating models and updating APIs that use them, etc)
one nice thing about single-repo-symlink setup is that your commits will be less fragmented and will usually represent full feature implementation - easier to track bugs, read history and maintain codebase
also when you deploy you don't need to read from many repositories - one less point of failure right there
release process is also simpler with such model as branch will now hold the scope of all projects
there are some drawbacks like symlinks dont work that well on windows and whatnot but for me it works perfectly
You can create a mountable engine that contains the shared models and create a gem out of it. This will handle the name spacing issues elegantly. Other nice aspect here is you get to share your assets also.
Watch this railscast for more details.
You'll still have to manage the 'versions' by pushing changes that need to be tested to a remote repo, but you can use the new local config of Bundler 1.2
http://gembundler.com/man/bundle-config.1.html#LOCAL-GIT-REPOS
This way it will pick up your local commits and you won't have to keep change your Gemfile upon deployment.
I know that this is not an solution for your particular problem. But I really suggest you to merge all projects into one. It is very usual to have all this parts in one application and there is no overhead. I think there is no not-awkward solution for this problem.
Take look at Git subtree.
This may work for you..
http://igor-alexandrov.github.io/blog/2013/03/28/using-git-subtree-to-share-code-between-rails-applications/
OR
You can write Rake task..
Example:-
namespace :sync do
desc 'Copy common models and tests from Master'
task :copy do
source_path = '/home/project/src-path'
dest_path = '/home/project/dest-path'
# Copy all models & tests
%x{cp #{source_path}/app/models/*.rb #{dest_path}/app/models/}
%x{cp #{source_path}/spec/models/*_spec.rb #{dest_path}/spec/models/}
# Database YML
%x{cp #{source_path}/config/database.yml #{dest_path}/config/database.yml}
end
See the below link.
http://hiltmon.com/blog/2013/10/14/rails-tricks-sharing-the-model/
Does your project have enough code coverage? If it does, I would try to separate the logic where it makes sense, and if a model is used in different projects, just pick one that fits best and write an API on top of that.
Then you could use that API to access those models (preferably using something like ActiveModel) on the other project. You would still have a simple CRUD, but all the core model logic would be handled externally.
Be sure to think well before splitting them up, though. You want to keep your domain tight on each app you create out of the Behemoth you want to torn apart.
Regarding engines:
I have used Engines for the same issue and it does help, but I also had to change my Gemfile to either point to a local path when developing, pushing the gem, then pulling it on the current project, which is the behavior you're not fond of.
Related
Now that college is out for the summer and my classes have come to an end I need to be able to keep my mind sharp over the summer. I've decided to try my hand at making a nice enough ruby on rails app from start to finish.
I've tried doing this before but the problem I always run into is:
"I can't really finish this until that's done but it seems like it won't work unless I finish this other thing that need the first thing to work."
Basically my project gets out of hand really fast because I have no direction regarding what to work on and when.
My question to you, oh great and wise community of SO, is what is the most accepted way to go about developing a rails app.
Do I start with models, views, or controllers?
Is it better to make the HTML first or the server code?
What is the best way to get things done?
Thanks in advance!
The Rails community strongly promotes some form of test driven development. When you're just getting started, it probably doesn't make sense to start out writing tests, but you can still keep the same approach.
So, essentially you would just try to do what you want in the browser and write what you need as you need it to accomplish that.
So getting started, just generate a new rails app and start the server. It will give you the default Rails environment information. You don't want that to be the home page, so you go to the routes.rb file and define your root address (e.g. root 'welcome#index').
Now you reload the page you get the error "Uninitialized constant WelcomeController'. So now you need a controller. Add the controller.
Now you get an error 'The action 'index' could not be found for WelcomeController'. Define the index action on your WelcomeController.
Now you get the error 'Missing template welcome/index'. Add your views/welcome/index.html.erb view.
etc. etc.
Soon you'll find yourself needing models. Reference the model as you'd like to interact with it from the view. e.g. #articles.each do |article| etc.You'll get an 'undefined method on nil' or similar.
Then you define your model as you'd like to intereact with it in your controller (e.g. #articles = Article.all). Then you'll get an error that your model doesn't exist. Create the model.
Add the attributes to the model that you need in the view.
Repeat the process...
Design
Decide which features you are going to implement
Decide which gems you are going to use to implement those features
Configure and implement those gems/features in isolation - Use Git
Realize that you are spending a lot of time here.
Consider trimming the features or simplifying the design. Maybe go back to the top.
Start coding your master app - Use Git
Configure those gems together - Use Git
Write the Models - Use Git
Try to write controllers and views(Use scaffolding to get started, that way you'll be doing more hacking and less coding) - Use Git
Run into a problem in last three steps, or discover a new feature that you must implement to make everything work.
Solve that problem in isolation (in a separate rails app) - Definitely Use Git
Repeat
About Using Git
Damien Roche has suggested in the comments to use Git branches instead of testing new tools in isolation.
I have been using Git with Rails from the beginning, and I recommend using Git branches and testing new tools in isolation.
e.g.: You can see a public Github repository of mine here: https://github.com/spundun/emblem-coffee-emberjs-rails-starter-kit . It's a simple single branch repository where I document each step I'm taking to incrementally shape my project.
But when using a new tool or library, many times things will break, and you will start with no clue as to why things don't work the way they say it will, in the tutorial. Did I use the wrong version of the library? wrong version of Rails? Did I add tools in the wrong order?
To make sense of such problems, many times you'll want to compare a working directory that works with working directory that doesn't. Meaning you will want to have two working directories side-by-side.
Of course you can achieve the above by cloning one directory into the other and checking out two different branches, but we are getting ahead of ourselves. Before making a gem co-exist with a dozen other gems that you are using, you should make sure that you know how to get it to work in the simplest of the scenarios. Meaning with vanilla rails, with the bare minimum that is required.
So first you will make this isolated branch incorporating the new gem into a vanilla rails app. This branch is not related to your main branch in anyway, except it should use the same version of rails that you are using for your app. You can host it in the same repository or a separate repository, really doesn't matter.
Once you feel confident that you know how it's supposed to work in isolation, you can trace all the steps you took using the commit log of that isolated branch and incorporate those commits as appropriate in a feature branch on top of your master branch. And when you are done fixing bugs and making all the gems work in harmony, your feature branch is ready to be pulled into the master branch.
HTH.
I think it's a good question and I expect to see some insightful answers here.
In my opinion it's something that comes with practice, and you will constantly switch between M, V and C. I think the Model is usually the first part to implement, as it is often determined by the real world problem domain and therefore less fluid and subjective. Then the Controller and the View next. If you just implement the smallest working part with each iteration you get to see results sooner and it helps for not getting bogged down. (Although I like to try and define all fields in my model up front).
But really the best way to not get lost is step back and take pen and paper and sketch out the main workflow and entities in your system, in whatever notation works for you. Keep it simple so that you can visualise the main stuff your system does end to end, on just a page or two. Then when you're implementing, those designs can serve as a map for where you are in your build.
Of course you could also try to specify system tests up front (or rspec feature specs) to define what the system should do. This has never worked for me to be honest.
Many software development methodologies exist and continue to evolve albeit at a slower rate than the programming languages they implement. I'd recommend researching the Agile Manifesto and memorizing it's principles.
My typical Rails application workflow:
Write and implement a feature (Cucumber).
Write and implement a spec (RSpec).
Write a benchmark and refactor.
when i got to this project there were cucumber tests in "features/enhanced", which ran with javascript and a few in "features/plain" which did not require js. with the later development of per-scenario #javascript, this doesn't make sense. and as the number of features files we have grows and grows, it'd be awesome if this stayed tidy.
so, in best practice land:
1) how long should .feature files be? i try to keep each narrow and specific with 1 or 2 "Scenarios".
2) what folder/file structure should one keep them in?
2a) how might one group similar features?
1) Once you've done them for a few months you'll soon find what works best for you. My advice is you should make them small ish. We have often split our earlier features down into smaller chunks, but have never ended up combining them. It's handy for making use of backgrounds etc...
2) We had a big problem with this and spent ages doing it one way then another. In the end we gunned to group them by the services that our company provides. e.g. payments, customer registration, stock management
Inconveniently, features don't always conform to a hierarchical tree view of the world, so make liberal use of tagging and your primary grouping of features is less important.
Have you tried yard? There's an example here We've just built it into our CI, it lets you pull together sets of scenarios based on tags, you can do unions, intersections etc... well worth it :)
I would keep the JavaScript and non-JavaScript versions of a scenario together, since they should be very similar.
Anything more than 8 scenarios in a feature file is probably too much.
A useful approach is to have a folder to represent the high-levels features (sometimes call epics or themes), and separate feature files within those folders for the different aspects of the behaviour.
For example, you may have a feature "Employee Directory" which would have separate feature files contains scenarios for a photograph, office location, job title, etc.
Depending on the size and complexity of your app, you could group those folders into other folders.
(Note that none of the above is specific to Rails apps).
I’m writing an application with Ruby on Rails. This application will be delivered to a minimum of two different customer types. The base is always the same but some of the views differ. That’s mostly it. For now.
Is there any way, for example using branches, to use the same code base and separate only the views, for example? I read the Git manual for branching but am still not sure if this is the best way to achieve what I need.
Another idea would be forking. But, is that clever? If I change something in the code of fork A, is it easy to merge these changes into fork B?
Branching and forking in git is not bad at all, as the merge support is great (possible the best of al VCMs).
Personally, I don't like the idea of branching or forking a project to provide different customization as it can very quickly become really difficult, e.g. what are you going to do if you have 15 different deployments?
I think a better approach is to build the application so it behaves differently depending on some parameters. I'm well aware that sometimes the implementations are very different, so this might not be useful anymore.
Another approach is to build the core of your app in a GEM which acts as a service to the application, and the only thing you customize per client are the views. Of course, the GEM should be generic enough to provide all the services you need.
Please share with us what you decided, as there's no best answer for your question.
It would probably be better to make you product select between the types at either build or runtime, that way you can use a single set of source.
Otherwise it is possible with branches, and merging, but you'll have more difficulty managing things. Forking is basically branching at this level.
I agree with #Augusto. You could run your app in 2 different environments, ie production_A and production_B. From there, you could use SettingsLogic to define configurations based on Rails.env, then reference those settings in your app when selecting which view to use for example.
We have a very lage Rails app that has two distinct sections: the front end and the CMS/Admin. We would like to break up the app into two pieces (for maintenance, as we have distinct teams that work on the front end vs. back end and they could have different release cycles).
One thought was to start a new Admin 2.0 app that has access to the models/schema from the original application, but has its own controllers/views and its own models that extend the original models until it is safe to fully decouple. Is this advisable? If not, what would be an appropriate plan to migrate away from one monolithic codebase?
warning, this is a bit ranty, and does not go anywhere.
Having worked on a very large app that operates in the manor you describe (for scalability reasons), I still have mixed opinions (an no conclusive answers).
Currently we operate 3 major apps (+ one or two smaller ones that use a fragment of the schema).
RVW (our admin app): This is the only app that writes, runs on a single server, and is responsible for maintaining the schema.
reevoo.com: ecommerce, price comparison, stuff like that. This (for historic reasons runs on a slightly different schema, run on a read only slave of RVW, with database views to map the schemas. All writes are done by sticking things on queues that RVW picks up and acts on. This works very well, although the number of random db related issues (mostly related to the views) is an issue. The main problem with this app is the difficulty sharing code (gems work well, I've often dreamed of bringing the schemas into line and sharing the core models in a gem!). We share code between apps using ruby gems. And test using lots of integration tests that cross app boundaries (using drunit (presentation on this available)).
reevoomark: very high load b2b app. This has many servers each with a full stack (db server, app server one per node). These have their databases populated with a db export - import batch job. This works very well in the short term, the shear flexibility of it is just ace, but integration testing between apps is very hard.
My advice would be to avoid splitting the apps at all costs, keeping things DRY quickly becomes a major challenge. My advice would be to stick with one app, two sets of routes (selected at startup by environment variables).
This gives you all the advantages of the other solutions, while making code sharing implicit. Splitting your test packs out would make your test cycles shorter and make things more manageable for the two teams. I would avoid working on different code bases, as doing this promotes the apps drifting apart and making code sharing tricky (as in .com).
If you decide do split, have a good set of high level cross app tests. Custom (per app) extensions to a core set of models sounds like a good plan, although with distinct code bases and teams you may still end up with duplicate code. Rails engines should be a good way of sharing the models, but be prepared for model reloading to become a little schizophrenic.
Good luck!
Have you namespaced your admin controllers? That would be a relatively easy point of subdivision and also avoid many of the negative side effects of forking your code into two apps.
Have you considered looking at Rails Engines? Added to Rails in 2.3.
The Grails framework has a lot of constructs/features that allows for adhering to the DRY principle ("don't repeat yourself") within a project. That is: within a specific project you're seldom required to repeat identical blocks of settings or code. So far so good.
However, the more I've worked with Grails the more of I've observed that I repeat code not within the same project but between projects. That is project A has controllers, GSP:s and images that overlaps with project B. This is a maintenance nightmare since bug fixes in project A must also be fixed in project B, etc.
I'd like to take DRY to the next level by not duplicating code between my projects.
My question: How do you tackle this problem (violated "inter-projects DRY") in your own internal Grails projects?
Please be very specific/concrete. If possible try to include specific code examples on how you solve it in practice.
Writing a custom plugin is the best way. You don't need to release it to the public repository, as you can use a private repository somewhere within your own network.
I haven't had enough duplicated code yet to pull out a plugin (most of the code repeated in my projects seem to be covered by the various public plugins), but a plugin can be as simple as a few common domain classes or services.
I agree with Lee, Using common/shared plugins is probably the best way to go. At one place that I worked we had quite a few internal plugins for this very reason.
The most common pattern is to put your common domain objects into their own plugin. This works really well for domain classes or services. We didn't end up refactoring the controllers, views, and static resources into a plugin, but the same principle should apply.
Long story short: Reuse of Grails artifacts = use a plugin.
To add to Lee and Colin's points, which are both valid, I think thinking in terms of multiple plugins can yield other benefits.
For example, you can split up your application functinality into multiple pieces, and have different people work on them. Or it can yield results during deployment, if, say, you need to have two layers of access to an app - user-level and admin - if your domain model is in a separate plugin, as Colin suggested, you can easily build two applications and deploy them separately.
For my app, I have several plugins specific to my project - domain classes plugin, one that is a bunch of code for importing data (which I can run easily against my site), some other plugins for graphing and customization of scaffolding. It takes a bit more thinking, but I expect this factoring will yield dividends in the future as we bring on more people to the team.