I am attempting to connect my Rails Application running in Opsworks to an Elasticache Redis Layer.
I just can't get it to work.
My current configuration:
1 Stack (2 instances)
Layers
- Rails App Server
- MySQL
The rails app is in the AWS-OpsWorks-Rails-App-Server Security Group.
1 ElasticCache Cluster
The ES cluster is in the default security sg-ff58559a (VPC)(active) Security Group.
I am using the 'Primary Endpoint' to attempt to connect.
This value is visible from the
ElastiCache>Replication Groups
dashboard.
It looks similar to this:
<name>.oveuui.ng.0001.use1.cache.amazonaws.com:6379
In my rails console (after SSH into the rails layer) I try:
>r = Redis.new(:url => 'redis://<name>.oveuui.ng.0001.use1.cache.amazonaws.com:6379')
>r.connected
The results is:
Redis::CannotConnectError: Timed out connecting to Redis on...
If you launched your cluster into an Amazon Virtual Private Cloud (Amazon VPC), you can connect to your ElastiCache cluster only from an Amazon EC2 instance that is running in the same Amazon VPC. In this case, you will need to grant network ingress to the cluster.
To grant network ingress from an Amazon VPC security group to a cluster:
1.Sign in to the AWS Management Console and open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
2.In the left navigation pane, under Network & Security, click Security Groups.
3.In the list of security groups, click the security group for your Amazon VPC. If you are a new ElastiCache user, this security group will be named default.
4.Click Inbound tab, and then do the following:
a. Click Edit.
b. Click Add rule.
c. In the Type column, select Custom TCP rule.
d. In the Port range box, type the port number for your cache cluster node. This number must be the same one that you specified when you launched the cluster. The default ports are as follows:
Memcached: port 11211
Redis: port 6379
e. In the Source box, select Anywhere which has the port range (0.0.0.0/0) so that any Amazon EC2 instance that you launch within your Amazon VPC can connect to your ElastiCache nodes..
f. Click Save.
http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/GettingStarted.AuthorizeAccess.html
Amazon only lets servers in the same security group as your Elasticache server talk to it.
This blog post walks you through the process of adding your Rails Server Layer to the right security group: http://aws.amazon.com/blogs/aws/using-aws-elasticache-for-redis-with-aws-opsworks/. It assumes that when you created your Elasticache cluster you chose the "default" security group, which seems to be the case. If so, go to OpsWorks -> (select the right Stack) -> Layers, and click on Security for your Rails App Server layer. You should see this:
You want to ensure that you've added the "default" security group and then restart your instances. Note that when I did this, it still didn't work. I decided to go look at the details of my instance in the EC2 console (instead of in the OpsWorks console) and found that the new "default" security group that I had added to the layer actually had not propagated to my instance. I don't know why this was the case, so I deleted the instance and created a whole new one, and this new instance had the "AWS-OpsWorks-Rails-App-Server" and "default" security groups applied to it successfully. So, keep that in mind in case things don't work right away and click on the instance to see its settings and confirm that both security groups are displayed.
Let me know if this works for you.
Related
I am working backend server launched on ECS cluster, hosted on an EC2 instance using docker.
the ECS is running great, exposed by IP address and port, but to be used with my ios app it needs to be served over https.
How do serve my ECS container over https? I have read a couple of things regarding using a load balancer, but tutorials are outdated and I can't find one that shows configuration after the ecs cluster has already been created.
Please point me to the right direction so I can get it served over https.
You need to have the following resources:
DNS address
Valid SSL Certificate
Load Balancer
Load balancer security group
Target Group
The target group will mediate between your server and your load balancer.
Also, in the security group define all the rules you currently have in the server security group, and in the server's security group ad a rule that open is open to all traffic in all ports with the security group instead of id.
This guide can help you:https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-create-https-ssl-load-balancer.html
(look at Create an HTTPS/SSL load balancer using the console)
I'm trying to get down and dirty with DevOps and I'm running into a health check request timed out failure. The problem is my Elastic Load Balancer sends a health check to my EC2 instance and gets a network timeout. I'm not sure what I did wrong. I am following this tutorial and I have completed all the steps up to and including "Using a Elastic Load Balancer". My EC2 instance seems to be working fine and I am able to successfully curl localhost on port 9292 from within the EC2 instance.
EC2 instance security group setup:
Elastic Load Balancer setup:
My target group for the ELB routing has port 9292 open via HTTP and here's a screenshot of the target in my target group that is unhealthy.
Health check config:
I have a VPC that my EC2 instance is a part of and my ELB is connected to the same VPC. I do not have Apache installed and I do not have nginx installed. To my understanding, I do not need these. I have a Rails Puma server running and I can send successful curl requests to the server.
My hunch is that my ELB is not allowed to reach my EC2 instance, resulting in a network timeout and a failed health check. I'm unable to find the cause for this. Any ideas? This SO post didn't help much. Are my security groups misconfigured? What else could potentially block a routing request from ELB to my EC2 instance?
Also, is there a way to view network requests / logs for my EC2 instance? I keep seeing VPC flow logging but I feel like there are simpler alternatives.
Here's something I posted in the AWS forums but to no avail.
UPDATE: I can curl the private IP of target just fine from within an EC2 instance. I don't think it's the target instance, I think it's something to do with the security group setup. I am unable to identify why though because I have basically allowed all traffic from the Load Balancer to the EC2 instance.
I made my mistake during the "Setup your VPC" step. I finished creating a subnet for an RDS instance. I proceeded to start an instance and the default subnet that AWS chose when I switched to my VPC was the subnet I made for my RDS, which was NOT a public subnet. Therefore, any attempts, from any EC2 instance or my load balancer, would not be able to reach it because I had only set up my public subnet to take requests.
The solution was to create a new instance and this time, pick the correct public subnet. My original EC2 instance was associated with a private subnet while the load balancer was pointing to the public subnet.
Here's a link to a hand drawn image that helped me pin point my problem, hopefully can help anyone else who's having trouble setting up. I didn't put image here directly because it's bigger than 2MB.
Glad to answer any further questions too!
I was using the AWS free tier to setup my Rails project.
However, I felt confused about why I can't connect RDS via sequel pro.
Here is the VPC security group which RDS was using.
I set the same host, username, pwd and database name in my rails database.yml.
My Rails project on EC2 can connent to RDS perfectly, but it fails to connect from sequel pro.
Lack of connectivity to RDS instance from outside VPC, while access from VPC works fine is usually related to RDS' DNS name not resolving outside VPC. Set Publicly accessible setting of your database to yes to be able to connect from outside.
From AWS docs - Hiding a DB Instance in a VPC from the Internet:
When you launch a DB instance inside a VPC, you can designate whether
the DB instance you create has a DNS that resolves to a public IP
address by using the PubliclyAccessible parameter. This parameter lets
you designate whether there is public access to the DB instance. Note
that access to the DB instance is ultimately controlled by the
security group it uses, and that public access is not permitted if the
security group assigned to the DB instance does not permit it.
You can modify a DB instance to turn on or off public accessibility by
modifying the PubliclyAccessible parameter. This parameter is modified
just like any other DB instance parameter. For more information, see
the modifying section for your DB engine.
NOTE: Opening any port in your RDS' security group for 0.0.0.0/0 CIDR range is a dangerous practice. Be specific and set up only source IPs that should be able to connect to your RDS instance in the security group.
I'm developing an application with MVC that uses a remote database (RDS amazon) to populate the data of the application. Running the application locally everything works fine. The problem occurrs when I deploy the application to an EC2 instance, my online application can't acess the same RDS database. What is the correct procedure to enable the EC2 instance to acess the RDS database ? My security group has at inbound and outbound permissions for all trafic and anywhere enabled.
How do I resolve problems connecting to my Amazon RDS database instance?
Issue
I am unable to connect to my Amazon RDS database instance.
Your Problem:
When attempting to connect from an EC2 instance that is not in a VPC,
the DB instance's security group is not configured to allow access by
the EC2 instance.
Solution:
If attempts to connect from an EC2 classic instance fail, grant user access from the DB instance security group. For more information, check through the following:
If you want to access your DB instance from an Amazon EC2 instance, you must first determine if your EC2 instance and DB instance are in a VPC. If you are using a default VPC, you can assign the same EC2 or VPC security group that you used for your EC2 instance when you create or modify the DB instance that the EC2 instance will access.
If your DB instance and EC2 instance are not in a VPC, you must configure the DB instance's security group with an ingress rule that allows traffic from the Amazon EC2 instance. You would do this by adding the Amazon EC2 security group for the EC2 instance to the DB security group for the DB instance. In this example, you add an ingress rule to a DB security group for an Amazon EC2 security group.
Important
Adding an ingress rule to a DB security group for an Amazon EC2
security group only grants access to your DB instances from Amazon EC2
instances associated with that Amazon EC2 security group.
You can't authorize an Amazon EC2 security group that is in a
different AWS region than your DB instance. You can authorize an IP
range, or specify an Amazon EC2 security group in the same region that
refers to IP address in another region. If you specify an IP range, we
recommend that you use the private IP address of your Amazon EC2
instance, which provides a more direct network route from your Amazon
EC2 instance to your Amazon RDS DB instance, and does not incur
network charges for data sent outside of the Amazon network.
AWS Management Console
To add an EC2 security group to a DB security group
Sign in to the AWS Management Console and open the Amazon RDS console at https://console.aws.amazon.com/rds/.
Select Security Groups from the navigation pane on the left side of the console window.
Select the details icon for the DB security group you want to grant access.
In the details page for your security group, select, select EC2 Security Group from the Connection Type drop-down list, and then select the Amazon EC2 security group you want to use. Then click Authorize.
The status of the ingress rule will be authorizing until the new ingress rule has been applied to all DB instances that are associated with the DB security group that you modified. After the ingress rule has been successfully applied, the status will change to authorized.
CLI
To grant access to an Amazon EC2 security group, use the AWS CLI command authorize-db-security-group-ingress.
Example
For Linux, OS X, or Unix:
aws rds authorize-db-security-group-ingress \
--db-security-group-name default \
--ec2-security-group-name myec2group \
--ec2-security-group-owner-id 987654321021
For Windows:
aws rds authorize-db-security-group-ingress ^
--db-security-group-name default ^
--ec2-security-group-name myec2group ^
--ec2-security-group-owner-id 987654321021
The command should produce output similar to the following:
SECGROUP Name Description
SECGROUP default default
EC2-SECGROUP myec2group 987654321021 authorizing
API
To authorize network access to an Amazon EC2 security group, call that Amazon RDS API function, http://docs.aws.amazon.com//AmazonRDS/latest/APIReference/API_AuthorizeDBSecurityGroupIngress.htmlAuthorizeDBSecurityGroupIngress with the following parameters:
EC2SecurityGroupName = myec2group
EC2SecurityGroupOwnerId = 987654321021
Example
https://rds.amazonaws.com/
?Action=AuthorizeDBSecurityGroupIngress
&EC2SecurityGroupOwnerId=987654321021
&EC2SecurityGroupName=myec2group
&Version=2009-10-16
&SignatureVersion=2
&SignatureMethod=HmacSHA256
&Timestamp=2009-10-22T17%3A10%3A50.274Z
&AWSAccessKeyId=<AWS Access Key ID>
&Signature=<Signature>
Resource Link:
How do I resolve problems connecting to my Amazon RDS database
instance?
Authorizing Network Access to a DB Instance from an Amazon EC2
Instance
Please check if the following are done.
Create a VPC security group (for example, "sg-appsrv1") and define
inbound rules that use as source the IP addresses of the client
application. This security group allows your client application to
connect to EC2 instances in a VPC that uses this security group.
Create an EC2 instance for the application and add the EC2 instance
to the VPC security group ("sg-appsrv1") you created in the previous
step. The EC2 instance in the VPC shares the VPC security group with
the DB instance.
Create a second VPC security group (for example, "sg-dbsrv1") and
create a new rule by specifying the VPC security group you created
in step 1 ("sg-appsrv1") as the source.
Create a new DB instance and add the DB instance to the VPC security
group ("sg-dbsrv1") you created in the previous step. When you
create the instance, use the same port number as the one specified
for the VPC security group ("sg-dbsrv1") rule you created in step 3.
Reference : http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Overview.RDSSecurityGroups.html#Overview.RDSSecurityGroups.Scenarios
By default RDS does not allow any connection that is not specified within the Security Group (SG). You can allow based on CIDR addressing or by Amazon account number which would allow any EC2 under that account to access it.
but also check:
Do the associated Subnets seem properly configured?
Are the Subnets part of a Routing Group that seems properly configured (Internet Gateway specified, etc?)
Does the RDS say it's Publicly Accessible?
And of course check the RDS Security Group and EC2 Security Group
Don't forget your actual source IP may be an internal IP (if accessing internally via a VPC) or an external IP (which may be a router's IP, or an EC2 instance's Instance IP which is distinct from its Load Balancer / Elastic IP) -- to troubleshoot, you may try to allow access to all IPs and ports.
(The routing group was my problem; in creating a new subnet, I neglected to add it to a routing group with a gateway.)
I have a rails 3.2 app running on Heroku which needs to proxy requests to an external API from a static IP address. Since Heroku doesn't offer elastic IPs, and Proximo is too expensive and limiting for the number of requests I need to make, I set up a simple forward proxy on an AWS EC2 micro instance in US-East using mod proxy.
I can proxy requests from my app's local environment just fine. However, requests from heroku time out. My thinking is that, since I can proxy from my local environment, the point of failure must be the connection between Heroku and my Proxy box. I've tried the answer given here: Security settings between ec2 and heroku but it didn't work. I've even tried allowing all inbound access on port 80 (even though that's terrible for the internet).
So, my question is, what are the security settings that I should enable for my ec2 instance in order to allow Heroku to proxy through it?
Heroku dynos are all running on machines within Amazon EC2 us-east-1 data center. They do not have any restrictions/firewalls on outgoing connections.
As long as you have the proper Security Group settings to allow the connections from your dynos to your own EC2 instance, you should be good.
It sounds like you haven't correctly opened up access from within us-east-1 to your instance. Double check your security group.
Information on how to edit the correct security group:
Check what security group you are using for your instance. See value
of Security Groups column in row of your instance. It's important -
I changed rules for default group, but my instance was under
quickstart-1 group when I had similar issue.
Go to Security Groups tab, go to Inbound tab, select HTTP in Create
a new rule combo-box, leave 0.0.0.0/0 in source field and click Add
Rule, then Apply rule changes.