I have a Ruby on Rails application with a database that stores sensitive user information (hashed with Devise). I now need to pass some of this sensitive information to another internal service on another server that needs it to make calls to third party APIs, so it needs a way to request that information from the RoR app.
What's the best approach to something like this? My first intuition was to grant an internal API key that would grant access to all sensitive information in the DB (via a private endpoint), the same way developer keys give access to a subset of API endpoints. Is this secure enough as long as I hash the API key? What's the best approach to passing sensitive information around through internal services?
Private APIs
My first intuition was to grant an internal API key that would grant access to all sensitive information in the DB (via a private endpoint), the same way developer keys give access to a subset of API endpoints
Well private endpoints or private APIs don't exist in the sense of only protecting them by using an API key. From a web app you only need to see the html source code to be able to find the API keys. In mobile devices you can see how easy is to reverse engineer API keys in this series of articles about Mobile API Security Techniques. While the articles are in the context of mobile devices, some of the techniques used are also valid in other type of APIs. I hope you can see now how someone could grab the API key and abuse from the API you are trying to secure.
Now even if you don't expose the API key in a mobile app or web app, the API is still discoverable, specially if the endpoint is served by the same API used for the other public endpoints. This is made even easier when you tell in robots.txt that bots should not access some of the endpoints, because this is the first place where hackers look into for trying to enumerate attack vectors into your APIs.
Possible Solutions
Private API Solution
What's the best approach to something like this? My first intuition was to grant an internal API key that would grant access to all sensitive information in the DB (via a private endpoint)
In order to have a private API the server hosting it needs to be protected by a firewall and locked to the other internal server consuming it with certificate pinning and maybe also by IP address. In order to be able to properly secure and lock down the internal server hosting the supposed private API it MUST not support any public requests.
Certificate Pinning:
Pinning effectively removes the "conference of trust". An application which pins a certificate or public key no longer needs to depend on others - such as DNS or CAs - when making security decisions relating to a peer's identity. For those familiar with SSH, you should realize that public key pinning is nearly identical to SSH's StrictHostKeyChecking option. SSH had it right the entire time, and the rest of the world is beginning to realize the virtues of directly identifying a host or service by its public key.
Database Direct Access Solution
What's the best approach to passing sensitive information around through internal services?
Personally I would prefer to access the database directly from the other server and have the database software itself configured to only accept requests from specific internal servers for specific users with the less privileges possible to perform the action they need. Additionally you would employ the firewall locking to and use certificate pinning between the internal servers.
Conclusion
No matter what solution you choose place your database with the sensitive data in server that only hosts that database and is very well lock-down to your internal network.
Anyone needing to access that sensitive data MUST have only read privileges for that specific database table.
Related
We intend to create a REST API that will be implemented on 100+ servers for use by a Centralized Management Portal (CMP). This CMP will itself have full access to the API (for scheduled tasks etc.) and the authorization is done on the CMP itself.
As an added security measure, all the 100+ servers' API can only be accessed from the CMP's IP Address.
In this circumstance, what would be the security advantage, if any, of using OAuth2 rather than a set of API Keys (unique for each server) that is stored as environment variables on the CMP? Upon reading this, it seems that our use case is somewhat different.
Ultimately, we were thinking that we could just open the CMP only to a subset of IP Addresses for those who need to access it, however, this may not be possible later down the track.
I would think about the API from the viewpoint of its clients:
How would a web or mobile client call the API securely?
How would the end user identity flow to the API?
If you don't need to deal with either of these issues then OAuth doesn't provide compelling benefits, other than giving you some improved authorization mechanisms:
Scopes
Claims
Zero Trust
USER v INFRASTRUCTURE SECURITY
I would use OAuth when user level security is involved, rather than for your scenario, which feels more like infrastructure security.
Some systems, such as AWS or Kubernetes, give you built in infrastructure policies, where API hosts could be configured to only allows calls from hosts with a CMP role.
I would prefer this type of option for infrastructure security if possible, rather than writing code to manage API keys.
This is just a general question about microservice architecture. Why do 2 or more internal services still need token auth like oauth2 to communicate with each other if the outside world doesn't have access to them? Couldn't their apis just filter internal IP addresses instead? What are the risks with that approach?
Why do 2 or more internal services still need token auth like oauth2 to communicate with each other if the outside world doesn't have access to them?
You don't need OAuth2 or token authentication, but you should use it. It depends on how much you trust your traffic. Now in the "cloud age" it is common to not own your own datacenters, so there is another part that own your server and network hardware. That part may do a misconfiguration, e.g. traffic from another customer is routed to your server. Or maybe you setup your own infrastructure and do a misconfiguration so that traffic from your test environment is unintendently routed to your production service. There is new practices to handle this new landscape and it is described in Google BeyondCorp and Zero Trust Networks.
Essentially, you should not trust the network traffic. Use authentication (e.g. OAuth2, OpenID Connect, JWT) on all requests, and encrypt all traffic with TLS or mTLS.
Couldn't their apis just filter internal IP addresses instead? What are the risks with that approach?
See above, maybe you should not trust the internal traffic either.
In addition, it is now common that your end-users is authenticated using OpenID Connect (OAuth2 based authentication) - JWT-tokens sent in the Authorization: Bearer header. Most of your system will operate in a user context when handling the request, that is located within the JWT-token, and it is easy to pass that token in requests to all services that are involved in the operation requested by the user.
For internal services it's usually less about verifying the token (which in theory was already done by the external-facing gateway/api), and more about passing through the identifying information on the user. It's very common for even internal services to want to know who the requesting/acting user is for permissions and access control, and sometimes it's just easier to tell every service creator who needs user-scoping to accept a JWT in the Authorization header than it is to say, "look for the user ID in the X-COMPANY-USER-ID header".
You can implement very granular role based access control(RBAC) on the apis exposed by Microservices using Oauth which you can not do using filtering IP address.
Introduction
So in my developer team, we need two server-based applications one located in my company architecture let's call it company server (i.e. resource and authorization server in OAuth2 terminology) and the second one in customer architecture let's call it customer server (i.e client and resource owner). The customer server is loading data from the company server so my company server needs to authenticate it somehow.
My team decides to apply OAuth2 standard with authorization and resource server in a single monolith application, without even thinking of benefits. This would, of course, take more time to implement than a simple constant key stored in the header. So I wonder what are benefits of that solution.
I know that Basic Authentication needs user:password base64-encoded in every request but the customer server is a single user so token would be in fact constant key stored in the header and I will use that terminology in terms of simplicity.
Argument - Microservices
In M2M (machine-to-machine) communication according to this article, customer server should obtain the token by providing client_id and client_secret from authorization server then you can use with multiple resource servers. The first argument I see is that OAuth2 pattern allows us to use multiple resource servers without additionally reimplementing authorization in each of them (because token is JWT or resource server is checking token against authorization) but in our case we have only one monolithic company server that is responsible for being resource and authorization so I see no benefits of that.
Argument - Man-in-the-middle protection
The other argument of using OAuth2 is protection against man-in-the-middle attack if someone intercepts token. The authorization server can invalidate token (directly in storage or in case of signed JWT by short expiry time) and prevent using compromised token. But...
Connection between servers is SSL secured
There's no way to steal token from storage like in a web-based or mobile-based application because key is located on the server-side itself.
Summary
So I can't think of any security benefits using OAuth2 compared to using the constant key in every request in this situation.
Security is mostly a chicken-egg problem. You encrypt secrets with encryption key and then again you think how do we handle the encryption key in a secured way. Don't assume here that TLS/SSL is infallible. But the core objective has always been to reduce the attack surface and make it more difficult for malicious users to break the system.
Even when there is no "Man in the Middle", when you send the password with every request, both the receiving side and the sending side keep the password in memory. It leaves more opportunity for an attacker to get hold of the password. A simple memory dump can expose the password.
In case of tokens, you don't always need the private key in memory to verify the token signature. You can cache the valid tokens at the server end and simply do a string match. Or you can use a public private key pair.
So, it's okay not to use OAuth2, if the security requirements are not stringent enough to justify the development effort required for a more secured solution. But it is better to use proven best practices and solutions.
Since many website will calling service by Web API. In that case, those methods will be exposed for every one. How can I ensure only my web site can calling my Web API service?
One of the easiest way to secure it as a developer is to get the IT people to do it by limiting access down by IP address from site to site. You could also do this in the app by validating the incoming IP addreseses. Sometimes IP addresses change sometimes they don't. Whenever I have done this have used certificates as there is a trust on both sides of the data divide. Have a look at John Petersens article Making web api's secure it has a section on implementing IP security in your app as well as x509 certificates; complete with code examples that I wont reproduce here. You could try and secure it with SSL and create your own key / trust but its easier to maintain (imo) using x509. Perversely it was much easier in the web service world Microsoft had an API to do this much more simple in WSE extension..
I'm developing a simple iPhone app where users register, and sign in with their email/password. These values are stored in a remote database.
I'm using Cloudant to store this information (CouchDB is great), and have granted read-only privileges to a new user (created API key/pass). In order to communicate with Cloudant, you obviously need a URL to access it (eg https://user:pass#db.cloudant.com), which is stored in the app as a string.
Now, while I know this is pretty unsafe, I can't think of any other alternatives in order to keep the db URL safe (specifically the username/password for it). I've seen people talk about using another server to proxy through to obtain the credentials, but it seems a little awkward.
Any help or thoughts would be really appreciated!
Are you trying to make a connection from your iPhone app directly to the database? You shouldn't give your app read access to the whole remote user table / database. Sooner or later someone would find out and would have read access to your data. No matter how you try to obfuscate it, the user/password combination would need to be stored somehow in your app.
What you should do is build a web service that connects to your DB and verifies your users. The database password stays on a server. This proxy-approach is not awkward, it is the only way to keep your database logins away from your users.
One option is to create your own service in the cloud that abstracts away your storage. That also has the benefit of allowing you to change your storage without updating all your devices.
In that model, the service stores the credentials to access the storage and you implement user security in your application layer. I also wouldn't think of it as a proxy layer - that implies that it's a thin pass through. If you develop a service, you should define a web interface (rest, soap) that's agnostic to the storage. In that case, it's a service, not a proxy.
EDIT:
Typically the web service authenticates the user (don't write your own). Basic Auth with SSL is typical. Then, in that services context API, you get access to the username. From there, you do you what you need. Your storage is accessed with the one storage account that has full access to all content.
Another auth option is OAuth which allows them to authenticate with someone like google - you never get the password - just a token from google letting you know they authenticated and they are who they claim to be. For example, that's how stack overflow works.