I'm building an arsenal of different things to attack my site with, to see how I can protect it. I was curious if there were gems available for this.
If you want to simulate a DDos attack you may want to spin up a bunch of Amazon EC2 instances and run ApacheBench on them. That should do it :)
Disclaimer
I'm not sure if the Apache terms say anything about using EC2 in this way. Please make sure you don't get your account banned for DDOSing your own site!
you can use something like fail2ban to avoid too much request from IP
Related
I am new to this, what is a best approach to implement microservices?
I found fw like seneca but it is little bit confusing...
Is there any tut how to create jwt auth, mongodb and other staff in microservices?
Take a look on Docker.
With docker-compose you can play with several services with an easy integration without worrying about the IP addresses to connect them.
Also if you add nginx to your stack, it's gonna be very easy to scale those services, there are several videos and tutorials that you can lookup to help you get started.
I've heard aboutseneca, but I haven't used, I think you shouldn't depend on a specific framework because one of the ideas behind of Microservices is the low coupling.
To make the jump into the real micro-services world is not trivial. It's not about plumbing some APIs, but a radical change in architecture thinking that, well, at the beginning will make you a bit uncomfortable (e.g. every service with its own database) :)
The best book I have read so far about micro-services is The Tao of Microservices, by Richard Rodger the author of Seneca himself. It exposes very well the shift from monolithic and object-oriented software towards micro-services.
I have personally struggled a bit with Seneca because of the average quality of documentation (inconsistencies, etc...). I would rather recommend Hemera, which took its inspiration from the message-pattern approach from Seneca, but is better documented and much more production-ready.
1) Build services and deploy it with Docker Containers
2) Let them communicate via gRPC coz it is really fast for inter services communication.
3) Use error reporter like Bugsnag or Rollbar. Error reporting is really important to catch error quickly.
4) Integrate tracing using opentracing or opencensus. Tracing is important too because it will be so hard to monitor all microservices with logs only.
We are creating a new version of payment gateway processor and We want to use docker container with kubernetes but we are worried if Kubernetes and docker container follow the PCI DSS requirements.
We don't find anything clear in PCI DSS specifications.
Re-iterating Tim's comment above: As far as I know nobody has implemented a fully PCI-compliant kubernetes install yet (they might have done and not told us). I don't know of anything specific to Docker or Kubernetes that would prevent you from getting your deployment certified.
PCI-DSS can be achieved by 3rd party solutions.
(Disclaimer - I'm an employee of Twistlock, which has brought a PCI-DSS solution, if you're interested in it please check the following link - https://info.twistlock.com/guide-to-pci-compliance-for-containers)
I've implemented and got PCI-DSS Level 1 certified a K8S Cluster as Cardholder Data Environment for the company I work for.
Don't get intimidated by the requirements, there's always a way to make them "not applicable" or meet them with some elbow oil.
The basics you need to meet to make it compliant are:
use COS so you can skip all the Nodes hardening hassle.
use the --enable-master-authorized-networks flag (Beta) although I haven't had any problems with it yet.
manage the network CIDRs yourself as you need to provide classes that don't change for the docs and show how only
those are authorized to access the cluster.
you must implement a NAT gateway cluster and pass all the K8S traffic through it and setup on the systems a silly
outgoing IDS/IPS solution, I used Suricata. (it's silly, I know)
you need to whitelist all outgoing traffic IPs for any API you're eventually calling from your apps and deny everything else.
PS: I know it sounds like BS, but you gotta do it if you wanna pass compliance.
PPS: remember to harden the NAT gateways, I used ansible with the STIG playbook.
These were the trickiest parts, everything else was cumbersome but manageable. Glad to help.
see this article. While the author is referring to "public iaas" it seems that one could substitute "private kubernetes".
We have a very simple function (We look something up from a third party database and return an answer. It's literally five lines of code.) We would like to offload this task from our main server because we expect a high volume of traffic for this one function and would like to optimize it.
We are thinking about testing the promise of many cloud/PaaS providers, where they handle scaling and performance responsibilities.
We're most interested in Rails environments, but are curious to hear experiences from others about any company in the space.
Here are the PaaS companies we found that supports Rails:
1) Heroku
2) DotCloud
3) Duostack
Questions:
1) Do you know of other Rails-specific companies? Also feel free to list non-Rails companies since we're interested in following other companies in case they eventually provide Rails support.
2) How has your experience been with these companies?
Foreword and disclaimer: I work for DotCloud; so the following might be biased. You've been warned.
DotCloud could be interesting for you if you like the following features:
run something else than Ruby (what about some Django or Pylons code talking with your SQL DB? Or even some PHP blog like WordPress or Drupal, using the same user authentication database?)
experiment with databases like Redis or MongoDB, or background ruby workers, without paying for add-ons
SSH access, crontab access (without requiring an add-on)
cheaper workers (I didn't come up with this one; some of our users coming from the Heroku world told us that workers were insanely expensive there)
Duostack is indeed very nice if you want to mix specifically Rails and Node.js. I've been told that they had awesome auto-configuration facilities.
Finally, if you only plan to do Rails and nothing else, ever, you might as well stick with Heroku since they've been in that business for a while, and are probably more mature than the first two of the batch.
Shameless plug: DotCloud is offering a beta test drive; so if you want to see what it looks like, just subscribe to the beta and you will be quickly enough be able to see for yourself. Heroku has a free tier as well.
You could add EngineYard in the mix - but i'd be inclined to use Heroku as my first choice, Dotcloud second (it's a newish product, and is very good but still in development)
If you want more control over your app/servers or want to run it on any cloud or your own infrastructure without having to download/deploy anything, you can try Cloud 66 (www.cloud66.com)
Disclaimer: I work for Cloud 66
A lot has changed on the scene since this question was asked. We recently looked into these services and settled on Heroku, but even more recently decided to continue managing my own deployments directly on EC2. Here are some points not mentioned in the other answers.
Heroku
Now supports much more than just ruby
Has really great-looking support for PostgreSQL
Uses LXC for process containers, like DotCloud
DotCloud
Is now Docker, and is putting a lot of manpower into developing docker.io
Doesn't have a free tier any more
I'm not sure if DotCloud is using Docker internally or not, since the docs say explicitly it isn't production-ready yet.
Our decision to stick with plain EC2 was motivated by the fact that it's cheaper and affords a lot more flexibility. For example, we use local-only http servers behind our public server to do some of our request processing, which doesn't really fit into the PaaS models out there. We would have had to reimplement all our back-end components as redis workers, and pay for them as additional dynos. The fact that Amazon RDS now supports PostgreSQL was also a compelling factor. Incidentally, Amazon has a full-stack PaaS offering as well, Elastic Beanstalk.
Just stumbled upon the question. There are similar ones around here. The problem is also: The PaaS scene is changing very quickly. New vendors are popping in every week or so.
Nowadays OpenShift from Red Hat might also be mentioned here as a Ruby PaaS.
OFFTOPIC + shameless plug: I have compiled a list of PHP PaaS here: http://blog.fortrabbit.com/comparing-cloud-hosting-platforms/
How can i write a cloud-aware application? e.g. an application that takes benefit of being deployed on cloud. Is it same as an application that runs or a vps/dedicated server? if not then what are the differences? are there any design changes? What are the procedures that i need to take if i am to migrate an application to cloud-aware?
Also i am about to implement a web application idea which would need features like security, performance, caching, and more importantly free. I have been comparing some frameworks and found that django has least RAM/CPU usage and works great in prefork+threaded mode, but i have also read that django based sites stop to respond with huge load of connections. Other frameworks that i have seen/know are Zend, CakePHP, Lithium/Cake3, CodeIgnitor, Symfony, Ruby on Rails....
So i would leave this to your opinion as well, suggest me a good free framework based on my needs.
Finally thanks for reading the essay ;)
I feel a matrix moment coming on... "what is the cloud? The cloud is all around us, a prison for your program..." (what? the FAQ said bring your sense of humour...)
Ok so seriously, what is the cloud? It depends on the implementation but usual features include scalable computing resource and a charge per cpu-hour, storage area etc. So yes, it is a bit like developing on your VPS/a normal server.
As I understand it, Google App Engine allows you to consume as much as you want. The back-end resource management is done by Google and billed to you and you pay for what you use. I believe there's even a free threshold.
Amazon EC2 exposes an API that actually allows you to add virtual machine instances (someone correct me please if I'm wrong) having pre-configured them, deploy another instance of your web app, talk between private IP ranges if you wish (slicehost definitely allow this). As such, EC2 can allow you to act like a giant load balancer on the front-end passing work off to a whole number of VMs on the back end, or expose all that publicly, take your pick. I'm not sure on the exact detail because I didn't build the system but that's how I understand it.
I have a feeling (but I know least about Azure) that on Azure, resource management is done automatically, for you, by Microsoft, based on what your app uses.
So, in summary, the cloud is different things depending on which particular cloud you choose. EC2 seems to expose an API for managing resource, GAE and Azure appear to be environments which grow and shrink in the background based on your use.
Note: I am aware there are certain constraints developing in GAE, particularly with Java. In a minute, I'll edit in another thread where someone made an excellent comment on one of my posts to this effect.
Edit as promised, see this thread: Cloud Agnostic Architecture?
As for a choice of framework, it really doesn't matter as far as I'm concerned. If you are planning on deploying to one of these platforms you might want to check framework/language availability. I personally have just started Django and love it, having learnt python a while ago, so, in my totally unbiased opinion, use Django. Other developers will probably recommend other things, based on their preferences. What do you know? What are you most comfortable with? What do you like the most? I'd go with that. I chose Django purely because I'm not such a big fan of PHP, I like Python and I was comfortable with the framework when I initially played around with it.
Edit: So how do you write cloud-aware code? You design your software in such a way it fits on one of these architectures. Again, see the cloud-agnostic thread for some really good discussion on ways of doing this. For example, you might talk to some services on GAE which scale. That they are on GAE (example) doesn't really matter, you use loose coupling ideas. In essence, this is just a step up from the web service idea.
Also, another feature of the cloud I forgot to mention is the idea of CDN's being provided for you - some cloud implementations might move your data around the globe to make it more efficient to serve, or just because that's where they've got space. If that's an issue, don't use the cloud.
I cannot answer your question - I'm not experienced in such projects - but I can tell you one thing... both CakePHP and CodeIgniter are designed for PHP4 - in other words: for really old technology. And it seems nothing is going to change in their case. Symfony (especially 2.0 version which is still in heavy beta) is worth considering, but as I said on the very beginning - I can not support this with my own experience.
For designing applications for deployment for the cloud, the main thing to consider if recoverability. If your server is terminated, you may lose all of your data. If you're deploying on Amazon, I'd recommend putting all data that you need persisted onto an Elastic Block Storage (EBS) device. This would be data like user generated content/files, the database files and logs. I also use the EBS snapshot on a 5 day rotation so that's backed up itself. That said, I've had a cloud server up on AWS for over a year without any issues.
As for frameworks, I'm giving Grails a try at the minute and I'm quite enjoying it. Built to be syntactically similar to Rails but runs on the JVM. It means you can take advantage of all the Java goodness, like threading, concurrency and all the great libraries out there to build your web application.
What would you suggest as the best server stack for a dedicated server which needs to host Rails SaaS application (not a lot of traffic but need to keep options open for future).
Regardless of your application, you're probably going to want certain standard components:
nginx/passenger will work for small apps or large apps. You should use it.
Unless you have a specific reason to use something else, you should use MySQL since the vast majority of the Rails community uses it and you will be able to get better support.
You should have memcached running right away, even if you don't use it for much yet. You're going to want to be able to seamlessly add caching as it's needed.
You're going to want to have a process for setting up a new server that is fully automated. That way, if you need to spin up a second server, it's trivial. If you ssh into a box to configure it, this means that if you need another server in a pinch (or the first server gets corrupted), you're going to need to remember all the things you did. Not a good place to be in an emergency.
You should be on the very latest version of Ruby on Rails, and upgrade frequently. Keep an eye on deprecations and changes and make the suggested changes as early as possible. When Rails 3 is released, use it.
Engine Yard, where I work, uses an open source tool called chef to manage our automated deployment solution. That's probably a good option.
As ever with a question that broad, it depends. Some things to think about:
What does the application do?
Does the application use any database vendor-specific SQL?
What are the availability requirements?
What are the performance requirements?
How much data will there be?
Which server stacks do you or the person who will be administering it have experience of?
What is your budget?
One thing I can say with complete certainty is that you don't want to be using Windows because Rails work best on a Linux/UNIX stack.
A lot of it depends on your needs. If the model isn't very complex and/or your traffic is fairly low, you can probably get away with apache, mongrel, and sqlite on some *nix.
If you start seeing performance issues, you can add some memcached into the mix, upgrade (relatively painlessly) to mysql, and use a different server (passenger/nginx).
There are also alternate ruby implementations that have some performance boosting changes. Rubninous and jRuby come to mind.