I am in serious need of some suggestions on how to improve the way I am doing things right now. I currently manage over 100 websites and they are all on a Dedicated Server, I just log on the FTP and make the necessary changes and upload it.. so I always work live.
I know that is a bad idea and I really want to find a good solution on how to use a development server and then push to production when ready, and keep everything up to date, etc.
I also have two more people on the team go in and occasionally make changes to websites, so this is something that would work for all of us so we are all on the same page and have the most up to date code.
I have been doing a ton of research on this but a lot of it confuses me and I can't find a clear answer on what would be best for me. I have tried git before, and it works great but not sure how that would work with 100+ websites.
I'm not sure where to even start.. what would be the best option? Are there any services out there that I could pay for to make the process easier?
Any help is appreciated. Thank you!
The first thing I would do is create a local development copy of the websites. Stop messing with code on your production server!
Next I would put the sites into version control. You can use any of the open source tools available (Subversion, Git, etc.). Just get this done.
You can write a script to deploy each site to the production server or you can play it safe and use a diff tool to copy over the changes only.
Related
I've been learning Travis CI and I want to use it to help automate tests on a MEAN application, then deploy it. However, there are some ways to go about this.
After reading, I learned I can create two separate repositories, thus maintaining two separate applications: a client application and a backend application. Since they are separate repositories, I can have separate .travis.yml files on each and perform continuous integration on the client application and backend application. However, I need advice on this approach because I have questions:
For the client app, I have to write tests. Since I'll be using angular, I want to test responsiveness and if components are working as intended. The client application also has to communicate with the backend application and I want to see if it is properly getting the correct results (such as clicking a button triggers a GET request and see if I'm getting the correct response body). Since the client app is on a separate repository, and when I build it on TravisCI, how will I connect the client application to the backend application if it exists on a separate repository?
I read around and I can use submodules in git. Thus, the client application and the backend application can be submodules for a 'master repository'. Therefore, how will the trigger in TravisCI work? Will I have separate travis.yml files in each submodule, or will I have to have one in the "master repository"?
If I were to get everything to work properly and have the client application and backend application both successfully deploy and the two are hosted on different servers, how will I fix the cross-domain issue?
The other approach is to host the static files produced by ng build --prod and have the node backend application host them. When Travis CI is triggered, I can first build the node backend application and run the tests on it first and then run the tests on the angular client application. After all of the tests are passed, where do I deploy? I know I have to deploy the node application since it will host the static files, so I how exactly will I deploy the backend application in Travis CI?
I know this is going to push it, but I'll ask it anyway. In the future, I want to learn how to implement microservices, and I want to use Nginx for the purpose of load balancing. How will I go about that? Docker can help me create a production environment where I can see if the Nginx server and node application are working well, but how do I include that in Travis CI?
If my question is a bit vague, please let me know what parts of it are vague so I can edit it that way I can make more sense of what I'm asking for. Thank you, and I look forward to an answer :)
Question is ultra-broad. You should solve one problem at a time, because by the time you solve 1 and 2 I doubt that 3 will be your only concern, and all of these issues are not really related.
try spending a bit of time reading Travis CI documentation, but also how to write tests and what different types of tests will do for you. Your issue is less about Travis than about what are unit tests vs. what are integration tests. So write simple standalone tests for your frontend, simple standalone tests for your backend, maybe run integration tests manually for a while, and when it becomes a real issue, then you will have better knowledge of how everything works together and you will find a way. Long story short: there is no single best way to run integration tests and it mostly depends on many, many things in your app (how does it run, what type of DB do you use, etc.)
you should read about submodules. Perhaps you need them, perhaps not. There is no way to tell. You can use submodules with Travis CI, but you can also not use submodules. Depends on what you want to achieve. Focus on the end goal for your architecture, not on what Travis CI needs!
what cross-domain issue? Again, this is a very different problem, and probably not the most prominent one you will face. Since I have no idea what server technology you will use there is no way I can answer that question properly. If you use express, this may be what you are looking for: https://expressjs.com/en/resources/middleware/cors.html
General bit of advice: all of your questions boil down to experience. Try solving one problem at a time, get started with your project and when you hit a specific issue, it's much much easier to solve than asking about "microservices". There are many ways to do microservices right, each solving a different issue. Without any knowledge of what your application is about and what issues you want to solve, microservices may or may not be what you are looking for, but there are also many other components that can affect your stack. Just get started and don't think about all of this too much for now - it's better to have something soon that you can test and learn upon, than think for weeks about something that you will never get because it is only theory.
What is the best way to get a ballpark idea of what kind of resources an application needs if it is running on Heroku (how many dynos and what db plan you should be running)?
My non-technical friend had his site built in rails and it is currently being hosted on Linode by the shop that built the app. They aren't providing the support he was hoping for, and is interested in having me move it over to Heroku, but we are trying to get an idea of how much more that is going to cost him. I tried contacting the dev shop directly to find out what the current instance is running on, but they aren't getting back to me.
I have access to their analytics, and the current repo, so from this can I glean what kind of initial setup we are probably going to need, and if so what should I be looking for? I realize that this is very situation dependent and not an easy question to answer, but any insight would be greatly appreciated.
dyno-blitzer will help you find out how many dynos you'll need to support 1,000 users, but there are a lot of questions you need to answer in order to try and ballpark what you'll need with Heroku:
How big is the database right now? How fast is it growing?
How many users do you need to support? If you only need 5, the numbers dyno-blitzer gives you probably won't be anywhere near what you need.
How much delayed processing work is being done?
What sort of things does the app need that have to be provided through add-ons with Heroku?
Once you know these, you should be able to get a rough ballpark, but it won't be laser accurate - this will heavily depend on the app and exactly what it's doing.
I know this is not a really programmic question, but which one should I deploy my app too?
Basically, I will have a straight forward Rails app with a a decent database usage. Heroku is obviously a great platform and has a lot of gimmicks. Duostack however seems to getting bigger and bigger while still in beta, and I really like their autoscaling feature, since I wouldn't have to monitor my site like 24/7 to be as cost efficient as possible.
Secretly, I just hope that Amazon will extend Elastic to Rails, but that would probably take a while
Well you have to get an invite to use Duostack, I don't know how hard that is. Plus, if you're looking to do a production app, they're still in Beta so there's no 100% guarantee things will be stable, or that there won't be API changes.
ALso, is duostack going to offer a free usage tier?
If you're comfortable with the answers to all those questions, then just go for whichever one you like better.
One nice thing about Heroku is it is well-established enough to have a lot of third party integration, and that can make development just blissful if it includes the integrations you need :)
Probably not a wrong choice between the two, though. Request a beta invite and if you get in, try both. Since all you have to do is "git push" to deploy to both, it ought to be pretty easy to do a direct side-by-side comparison of the workflow.
I come from a traditional programming background by which I mean C, Java, C#, C++ and a little python and VBA.
Now I'm trying to create a small CRUD application for the purpose of taking a form and turning it into an online form and database for later querying.
My initial thinking seems to lead me to Ruby on Rails given the fact that there is a lot of good stuff about it on the internet and my greatest strength is that I pick up languages fairly quickly so the fact that I have never seen Ruby code until 10 mins ago is no big deal. Now having said that I'm looking for cheap infrastructure to a) host an svn repository and a web server to allow me to develop and learn what I need and eventually deploy.
In short;
1) Where can I go for this cheap infastracture for the purposes of learning and eventual deployment?
2) Where should I go for infrastructure to host and SVN repository? (I haven't coded in a while, but will be needing this for a multitude of things and am not in a place to run this off a home desktop/server)
3) If you don't agree with my Ruby on Rails conclusion, what would you recommend and why?
Is SVN mandatory? If you don't mind trying something like git, you can get a small application up and running using heroku in no time. And, for free.
There might be a few stumbling blocks at first–getting your local development environment all set up–but you'll be able to get going fairly quickly.
unfuddle has great SVN and GIT hosting for free, you actually get private repos with SSH.
If you haven't given GIT a try, I would. SVN was the first source control I used, but after messing with git a little I immediately switched, it is SO much faster.
I'm working on a cms and wanted the ability to offer custom extentions for certain accounts. Like having a plugin with custom code that is only available or only used by that account. These custom extentions would be specific to the business needs of an account and perhaps unlikely that any other accounts would need it, but maybe. Is there a way that this could be done and to be loaded without having to restart the whole app, thereby creating downtime for the other accounts?
In terms of per-client plugin code, you could store the code in a data model and then eval() the code to dynamically execute it. (Of course you would want to do some serious sanity-checking / scrubbing on update to ensure the code does not contain malicious calls). Another approach could be to develop a custom tag library, much along the lines of what the Radiant CMS developers have built ... and then let your users "program" their behaviour using the tags provided. This gives you more control and better security at the expense of less flexibility.
In terms of the downtime question, if you are using a modern rails deployment approach I don't see how this should be an issue. The eval() approach above doesn't require a restart (unless your custom code calls "include ..." on libraries that are not installed at the time of the last boot - but getting these libraries installed is also an "out of band" problem that you would need to solve.
Passenger gives you the restart.txt file that you can touch to force a refresh. Similarly there are recipes for mongrel (like see saw) that allow you to progressively restart your mongrel stack to avoid downtime. I would pull these two issues apart mentally if I were you, as the dependencies between the two are not that great. Hope this helps.
I built a cms and added plugin support for it. Best thing you can do is have it be all database driven, the plugin exists for everyone, technically, but you can only make use of it if you've "purchased" it, or some other way of turning it on.. Which is really just a db record.
That'd be 0 downtime. :) Then again, I have no idea what the rest of your setup looks like. I'd think your solution is going to be pretty specifically tailored to your cms system design.
how long would this downtime be really? i mean running migrations and stuff would be a pain for a system that allows any tom dick or harry to upload a plugin. You'd have to verify that the migrations were set up correctly etc. if you aren't getting that 'fancy' and just allow them to do something 'neat' in js, then i guess it's a question of restarting passenger, which is what 5 secs?
I'd check out other 'famous' CMS like radiant or something to see if/how they do it, personally. good luck.
Im not if this is exactly what you are trying to achieve, but have you checked out pancake?
Pancake is a tool & framework to let you stack and loosely couple Rack-based webapps.
http://www.rubyinside.com/pancake-rack-webapps-stacking-2863.html