Recently I configured my instance into a micro environment in EC2 with glassfish and mysql in windows..
I deployed my war and i was able to access my site through http.
I changed my application and redeployed the war and it also worked.
When I was about to redeploy the war for 4th or 5th time, the application got deployed, I saw the message in the log file. But I was unable to access the site through http.
Then I tried the command "asadmin list-applications" and I got the following message.
Error occurred during initialization of VM
Could not reserve enough space for object heap
After that I was not able to connect to my instance through RDP and I had to reboot, I was able to access it again after that. I started the servers again (glassfish mysql), but no luck.
I noticed that the memory usage is around 90% or more. CPU isage is low.
now I can not access my site through http. what shall i do ?
Thanks in advance !
Honestly, there are a couple issues working against you here:
1) Windows requires FAR more RAM than Ubuntu to run at a minimum decent level.
2) GlassFish has a much larger footprint than Tomcat or Jetty.
Is there any particular reason you need Windows? Like is there a specific need that your server run some executables for file processing or something like that outside the JVM? Most would agree that Linux (Ubuntu or other) will give you much better results in performance and stability to run an App Server like GlassFish in any environment.
Related
I am working on an open source project and there is an issue where an upload error only seems to occur on the production side that is running apache and unicorn.
Due to privacy issues and risks, I am unable to mess around in the actual production side (such as creating a temp id for myself with various privileges.
Is there anyway that I can quickly create a VM with such setup in my own computer that would mirror the live site?
The site is running in RoR 4, latest unicorn, and latest stable version of apache.
There are a couple of ways you can be able to accomplish this. You can use vagrant and also with the rising popularity of Docker containers, you can easily model production environment on your computer. Since you indicated a faster way to get the VM up and running, I would recommend using railsbox.io. Its amazing and it saves you a lot of time. According to their webiste -
Fast and easy Ruby on Rails virtual machines. Streamline your
development workflow in no time by creating production-like virtual
machine with your development environment. Try this extremely simple
to use VM configuration tool to create new Ruby on Rails server using
vagrant and ansible.
The app helps you setup a VM with ease.
We had 5 applications over a linode(Ubuntu 10.04 32 bit) of 1G RAM. Recently we moved one of the applications out of that linode to another of 512M. The application is built on Java EE and was working pretty stable on the old server. On the new server however tomcat(Version 6 on both servers) crashes every now and then without any logs. The only difference on the new server is that we are using nginx as the web server against apache2 on the old and the new server uses Ubuntu 12, 64 bit. There is no reason to doubt a memory leak because the application was behaving well on the old server. Are there any tomcat optmizations to be done to prevent such kind of crashes. I doubt if the reason is load due to traffic(since the new server has lower RAM) as well, because even in the middle of the night when there are just about 10 concurrent users, tomcat still crashes. Any insight towards the problem would be appreciated.
I checked the RAM usage and tomcat constantly occupies about 60% of the memory and all of a sudden crashes and goes to 0. I have used a bash script and run it as a cron job every 5 minutes on the new server to check if tomcat is down and restart it automatically. Could that be causing the issue? The script is mentioned below
if [ "$(/etc/init.d/tomcat6 status)" == " * Tomcat servlet engine is not running." ]; then /etc/init.d/tomcat6 start; fi
Please note, I am not an expert at server configuration. I can just about configure a server to install and get required things running.
You moved your app from a 32-bit Hotspot JVM to a 64-bit Openjdk JVM. And on the new server you have less RAM.
First I would try to install the same 32bit Hotspot JVM on the new server,and see if the crashes still occur. If they do, I would start adding more memory, and adjust xmx etc' accordingly.
I upgraded the RAM to 1GB, downgraded to Ubuntu 12, 32 Bit, reinstalled JVM 32 bit and now the server works like a charm. I was unable to zero down on the root cause, but the most possible cause should be either the 64bit OS or the 64 bit JVM eating too much memory. Thanks for your help.
I've run 80,000 http calls thru my grails app running on my desktop PC and it did not fail. On cloudfoundry, the same app runs out of memory after about 3000 http calls. The app uses MySql, Mongodb, and RabbitMQ. On Cloudfoundry, I have increased the memory to 1G using the VMC command. While the test program is running, I can watch the memory usage with the VMC stats command and the memory usage grows up to 1G and the app fails. I'm using Grails 2.0.1 on my local machine.
What could be causing this problem?
Could it be related to this: http://burtbeckwith.com/blog/?p=73 ?
You might get some use out of this post as your test app is likely much like a batch application.
You should use jconsole to monitor your application when running locally and set your local memory allocated to java to the same values as cloudfoundry to see if you can reproduce the error. If jconsole shows that you're never letting go of memory, then you likely have a leak and aren't allowing things to get garbage collected.
For Java web apps you need to also set the Xmx:
vmc set-env APP JAVA_OPTS="-Xmx1024M"
It is the first time I have ever launched live a website (with Grails web framework under Amazon EC2 platform and Cloud Foundry) and I realized quickly that I am not ready for monitoring and maintening correctly my application in production mode (fortunately the website is accessible to a very limited number of users) .
The issues I have faced so far are:
Cannot change my views. I need to redeploy my application
I have no monitoring. I don't know who is connected, when do they sign in / sign out...
Redploying my application (upload WAR + deploy) takes at least 30 minutes.
I don't know how to restart my Tomcat server without a redeploy through Cloud Foundry !
...
So, my question is very simple:
What tools (including grails plugins) and methods can you recommend me for taking me out from my current blindness?
I am not sure how much this will help, however I use the JavaMelody Grails Plugin(http://www.grails.org/plugin/grails-melody) I use it to see if the site is being used before I pull down the service.
Hope that helps.
I tried Cloud Foundry in it's early days and found it a little rough. Sounds like that's still the case with 30+ minute deploys and the inability to restart your tomcat server. Half of your problems could be solved if you just created your own EC2 instance, installed tomcat and managed your own deployments. That'll let you bounce tomcat through shell access:
sudo /etc/init.d/tomcat6 restart
and redeploy your app
sudo /etc/init.d/tomcat6/stop
cp my.war /to/tomcat/dir
sudo /etc/init.d/tomcat6 start
or else you could do it through the tomcat admin console, but I find it to be flaky.
Regarding monitoring, there are a couple of ways you can do that. The easiest is to add CloudWatch monitoring to your system. That'll give you more insight into the performance of the application.
For more detailed monitoring regarding who's connecting to your app, I'd suggest looking at Google Analytics or Mint. If you need to get beyond that (with per user monitoring), you'll likely have to roll your own logging/tracking for what meets your needs. There are also other paid packages out there along the lines of Google Analytics and Mint that you can integrate with, but what fits your needs best, I can't say.
For actual monitoring of deployed system you can also use Hiperic HQ. It's a monitoring solution from Spring Source, who also are owners of Grails Framework.
It can manage, at your case:
tomcat server
database
linux
network
etc
btw redeploying app with changes is ok. it's a very bad practice to modify running app, on the production server.
I don't know if JMX and JConsole can help, but that might be a good way to see what the status of JMX-enabled POJOs is. Spring makes this easy to do.
Yes, you need to re-deploy your app when you change stuff in it, there's no way around that.
Deploy/re-deploy time has been cut significantly recently if using the grails plugin (btw, what version of the plugin do you use?) In some cases the upload time is as short as 15 seconds. Add 2-3 minutes for Amazon to spin up the instances and the deploy time is still pretty manageable. For re-deploy the instances don't have to be started, so it's even less than that.
To restart Tomcat login into your CloudFoundry account, click on the Deployment details, click on the instance that's running your Tomcat, and there will be a button "Restart service" that will do just that - restart Tomcat service.
You don't have to start your own EC2 instances in order to get a shell access. Copy the public DNS name of the instance from DeploymentDetails, and SSH into it using the private key you entered when registering for Cloudfoundry. Example:
ssh -i /path/to/gsg-keypair.pem root#your_instance_DNS_name
Okay, before you guys go nuts -- this is just a small site, temporary setup. Right now I'm having some internal folks remote into the server and use the site through webrick via the dev command: ruby script/server. Not exactly ideal.
I'm just starting Rails dev and I want to know a better way to handle hosting on a Windows Pro box. Again, just temporary so please be gentle :)
As far as I know, mod_rails isn't an alternative.
Mongrel plays very nice on windows, though, so you can set up a few mongrels and have IIS or Apache proxy to them. Or just use Mongrel directly. Before mod_rails, mongrel was the de facto way to deploy on any platform, so it's a very viable choice.
The one time I was forced to deploy on Windows, however, I installed Ubuntu via virtualbox (could also use VMWare or whatever, of course) and deployed on that. Works like a dream, and I got to work with a sensible OS. Phew. SSH and stuff. Can't live without it. Remote desktop isn't exactly my kind of thing.
Your best bet is to setup a Mongrel cluster. Mongrel is an application server which can serve a Rails application on HTTP. But a single Mongrel instance can only handle 1 request at the same time, so typically people run a cluster of Mongrels, i.e. multiple Mongrel instances. These Mongrel instances do not talk to the Internet directly. Instead, they are put behind a load balancer or a web server, which proxies requests to this cluster of Mongrels. If you use Apache on Windows then you can:
1. Setup and start a cluster of Mongrels, each listening on its own port.
2. Setup a virtual host with some mod_proxy_balancer directives, with which you tell mod_proxy_balancer to proxy all requests to the Mongrel cluster. mod_proxy_balancer will automatically distribute the load between the Mongrels.
If usage is really low i.e. likely to be mostly 1 person at a time or your response time is really low then you can get away with a single mongrel and having your users point there browser to the relevent IP address and port.
For some time before I finally got my apps migrated to our corporate Linux/apache "cloud" (which was anything but straightforward, for mostly internal IT-related reasons) I ran two apps on a workstation, using a separate mongrel (different ports) for each. It worked well enough to be useful for almost a year.
These days (well, about three weeks now) I've substituted the immediacy, control - and vulnerability - of local (under my desk) access for the stability of five servers, each with multiple mongrels, staging areas, and deployment annoyances. Swings and roundabouts.