Now you can add proxy to your aks.
https://learn.microsoft.com/en-us/azure/aks/http-proxy
And if you read the article it seems like you are not going to use any other outbound system for your aks.
But when you deploy it using az cli you find that you still have to decide which "--outbound-type" you need. Even if you don't text it it gets its default value.
My question is: proxy? What for? I thought it was an alternative to outbound-type
No matter if I use proxy because I still need other outbound traffic for the whole cluster.
am I wrong?
Related
My backend is a simple dockerized Node.js express app deployed onto elastic beanstalk. It is exposed on port 80. It would be located somewhere like
mybackend.eba-p4e52d.us-east-1.elasticbeanstalk.com
I can call my APIs on the backend
mybackend.eba-p4e52d.us-east-1.elasticbeanstalk.com/hello
mybackend.eba-p4e52d.us-east-1.elasticbeanstalk.com/postSomeDataToMe
and they work! Yay.
The URL is not very user friendly so I was hoping to set up API gateway to allow to me simply forward API requests from
api.myapp.com/apiFamily/ to mybackend.eba-p4e52d.us-east-1.elasticbeanstalk.com
so I can call api.myapp.com/apiFamily/hello or api.myapp.com/apiFamily/postMeSomeData
Unfortunately, I can't figure out (i) if I can do this (ii) how to actually do it.
Can anybody point me to a resource that explains clearly how to do this?
Thanks
Yes, you can do this. For this to happen you need two things:
a custom domain that you own and control, e.g. myapp.com.
a valid, public SSL certificate issued for that domain.
If you don't have them, and want to stay within AWS ecosystem, you can use Route53 to buy and manage your custom domain. For SSL you can use AWS ACM which will provide you with free SSL certificate for the domain.
AWS instructions on how to set it up all is:
Setting up custom domain names for REST APIs
i am (desperately) new to elasticsearch (7.9.0) and i currently have a cluster with two nodes running.
After a lot of effort it is performing as i would like it to.
It is running on docker and also has an nginx in front of it to route the traffic to it since it is being accessed directly from my website (angular 10).
The elasticsearch is being used as well from my laravel backend directly through the docker container name so that is secure (i guess).
My problem now is that i cannot find or understand a way to secure the http access from outside docker (eg the normal website).
Going via Laravel is an option but this is too slow for my purpose.
Is there a way i can securely have http access to the elasticsearch from the web?
Also, is there a way i can restrict the actions to read only actions?
If you need more info to help out please let me know as i am not knowledgable on what is important here and what not.
Thanks
Angular is a front-end and is run in your user's web browser. If Angular can somehow reach your Elasticsearch instance, everyone can do so. No matter what. You can try to obscure it as many as you want, but if there is direct exposure to Elasticsearch, it will be reachable.
So you have to either assume this fact, or go the slow way and proxy the requests to Laravel, so it can verify that the information requested is actually available for the user performing the request.
Preface
I am currently trying to learn how micro-services work and how to implement container replication and API gateways. I've hit a block though.
My Application
I have three main services for my application.
API Gateway
Crawler Manager
User
I will be focusing on the API Gateway and Crawler Manager services for this question.
API Gateway
This is a docker container running a Go server. The communication is all done with GraphQL.
I am using an API Gateway because I expect to have different services in my application each having their own specialized API. This is to unify everything.
All it does is proxy requests to their appropriate service and return a response back to the client.
Crawler Manager
This is another docker container running a Go server. The communication is done with GraphQL.
More or less, this behaves similar to another API gateway. Let me explain.
This service expects the client to send a request like this:
{
# In production 'url' will be encoded in base64
example(url: "https://apple.example/") {
test
}
}
The url can only link to one of these three sites:
https://apple.example/
https://peach.example/
https://mango.example/
Any other site is strictly prohibited.
Once the Crawler Manager service receives a request and the link is one of those three it decides which other service to have the request fulfilled. So in that way, it behaves much like another API gateway, but specialized.
Each URL domain gets its own dedicated service for processing it. Why? Because each site varies quite a bit in markup and each site needs to be crawled for information. Because their markup is varied, I'd like a service for each of them so in case a site is updated the whole Crawler Manager service doesn't go down.
As far as the querying goes, each site will return a response formatted identical to other sites.
Visual Outline
Problem
Now that we have a bit of an idea of how my application works I want to discuss my actual issues here.
Is having a sort of secondary API gateway standard and good practice? Is there a better way?
How can I replicate this system and have multiple Crawler Manager service family instances?
I'm really confused on how I'd actually create this setup. I looked at clusters in Docker Swarm / Kubernetes, but with the way I have it setup it seems like I'd need to make clusters of clusters. That makes me question my design overall. Maybe I need to not think about keeping them so structured?
At a very generic level, if service A calls service B that has multiple replicas B1, B2, B3, ... then it needs to know how to call them. The two basic options are to have some sort of service registry that can return all of the replicas, and then pick one, or to put a load balancer in front of the second service and just directly reach that. Usually setting up the load balancer is a little bit easier: the service call can be a plain HTTP (GraphQL) call, and in a development environment you can just omit the load balancer and directly have one service call the other.
/-> service-1-a
Crawler Manager --> Service 1 LB --> service-1-b
\-> service-1-c
If you're willing to commit to Kubernetes, it essentially has built-in support for this pattern. A Deployment is some number of replicas of identical pods (containers), so it would manage the service-1-a, -b, -c in my diagram. A Service provides the load balancer (its default ClusterIP type provides a load balancer accessible only within the cluster) and also a DNS name. You'd configure your crawler-manager pods with perhaps an environment variable SERVICE_1_URL=http://service-1.default.svc.cluster.local/graphql to connect everything together.
(In your original diagram, each "box" that has multiple replicas of some service would be a Deployment, and the point at the top of the box where inbound connections are received would be a Service.)
In plain Docker you'd have to do a bit more work to replicate this, including manually launching the replicas and load balancers.
Architecturally what you've shown seems fine. The big "if" to me is that you've designed it so that each site you're crawling potentially gets multiple independent crawling containers and a different code base. If that's really justified in your scenario, then splitting up the services this way makes sense, and having a "second routing service" isn't really a problem.
I am facing issues getting Reverse Proxy right. I keep getting "504 Getaway Timeout" when I am using the Reverse Proxy.
I have followed the Microsoft's example to set up the cluster.
IMHO, I think cluster set-up is correct, the only difference is that I've specified port 80 for the proxy and I did not use SSL for test env.
I am trying it out on test environment at the moment, but the production environment is running the same services, just w/o reverse proxy and it is just fine. Also, I have exposed an endpoint for one of the services on test env, tried calling it w/o reverse proxy and it worked.
I've read that it could be caused by the containers, but I am using Windows 2012 RC2 DataCenter. As far as I am aware, it does not utilize windows nat containers. Also, I've read that it could be caused by the 404 error (#case 2 in the example doc) where it tries to reload it and simply times out trying.
These are some of the summed up details that might be important to know
Service Fabric version: 5.5.219.0
OS: Windows
SKU: 2012-R2-Datacenter
Services are using WebListener
All ports are allowed
1 NodeType (stateless)
Services created with ASP.NET Core Web API template
VS 2015 Enterprise
Service endpoints are configured like follows:
Endpoint Protocol="http" Name="ServiceEndpoint" Type="Input"
All services and cluster are healthy.
I have found a cause for this timeout. It was just me not getting the required change in the request url right.
All of my services are holding MVC controllers that are named after service name. So whenever I was calling them without reverse proxy my request url would be something like http://mycluster.westeurope.cloudapp.azure.com:8280/Notifications/TestMethod
and this would be enough as it could find Controller by unique port.
The way I've been trying to call it with reverse proxy was
http://mycluster.westeurope.cloudapp.azure.com/SomeName.API.Services/Notifications/TestMethod
That is not enough as 'Notifications' is parsed as a name of the service and not the controller. So I was calling the service and an action without specifying a controller.
The correct way to call it is to include service name twice, since I've called my controllers the same as service (I might change that).
Here is a correct url I have to use
http://mycluster.westeurope.cloudapp.azure.com/SomeName.API.Services/Notifications/Notifications/TestMethod
I have figured it out by looking up the reverse proxy code sample.
I need to connect to an API with my Heroku/Rails app where I need to have a static IP.
I know about the add-on proximo - https://devcenter.heroku.com/articles/proximo - but it is insanely priced, so that's out of the question.
Most people in my situation ends up deploying to EC2 instead and using a an Elastic IP as their static IP. I've also tried this and it works, but I find the whole flow of playing around with EC2 really cumbersome.
I've read in some answers that it is possible to set up an EC2 server and use it as a proxy for your Heroku/Rails app - Heroku Static IP for SFTP - but it sounds very cryptic for a person who is not very server-savvy.
Can someone give a step-to-step tutorial on how to set up your Heroku/Rails app to use your EC2 instance as a proxy?
In short: I would just switch to AWS. There is a little bit of a learning curve but it's not too difficult. Here is a presentation I gave on it geared toward front-end devs:
https://speakerdeck.com/krunkosaurus/intro-to-scaling-your-web-app-on-the-cloud-with-aws-for-frontend-developers-part-1
Amazon has 9 regions each with 3-5 "Availability Zones". I know that Heroku uses AWS but am not sure if you get to decide (or know) which AWS region and AZ your actual server is hosted.
Whatever it is, you should host your proxy server in the same Region (better yet, even the same Availability Zone). Setting up an EC2 there is easy just use the web console and be sure to give it an Elastic IP so the adress doesn't ever change. Then point your DNS away from your Heroku instance (dyno?) to this EC2 instance. From there you can either install Nginx or HAProxy.
I've just setup QuotaGuard Static as a more robust, better value static IP service that can be used by Heroku apps.
It's hosted on EC2 so you get the low latency benefits without having to start managing servers.
Hello you can use this answer to see how to use NGINX as a reverse proxy.
https://stackoverflow.com/a/27874505/1345865
http://blog.talenox.com/post/107675614745/how-to-setup-static-ip-on-heroku
Fixie is very affordable option for getting static IP address on Heroku. The documentation section has a lot of examples for different languages and frameworks, including Ruby.