One of the clients I'm working with is requesting that I encrypt my database. The problem is, none of my other clients need or care about this.
I am wondering two things:
How costly (performance) is it to encrypt my entire database? I.e. how much slower will my read / writes be?
Is there anyway I can provision a small separate DB, or segment my current DB on Heroku and host that one specific client's data (wherein their data is fully encrypted).
Thanks!
Ringo
Related
I'd like know if is possible to have a single Rails App, where i have many different clients using this same App, where every single client have your own PostGreSQL DB in Heroku, so we have the same project to be updated for all this clients when i do the pushs to Heroku.
Do you know if it's possible to be done?
And how to ignore the database.yml file in updates, because every single client has your own DB.
Thanks!
You can but you probably shouldn't!
You can attach any number of Heroku Postgres instances to a Heroku app. You'll see that each instance you create adds a connection string to the list of environment variables - listed under the App's settings tab.
You can map the key string to a customer - via some unique identifier. You would then need to have an interceptor to bind the connection to the relevant database - or chose the relevant connect from a pre-bound list - and add it into the request context for each request.
It's a messy approach and not recommended. What would be slightly less messy is if you created a separate schema per customer instead. This way you bind to a single database instance and retain your database.yml config. But, each customer has their own dedicated schema. However, these are more architectural concerns that Heroku capabilities. From a Heroku perspective, both multi-database and mutli-schema approach is possible.
It should be noted that neither approach would give you any more referential integrity at the application logic level than standard roles and permissions with adequate auth mechanisms... All schemas and/or databases will be visible to the same application regardless of its separation at the database level. So, really there's little tangible benefit to it.
As we all know, I'm one of the thousands of devs who relied on Parse and now forced to find Parse alternative. While transferring Parse-Server to AWS+MongoDB, I've discovered DynamoDB. I'm thinking of just tranferring my whole server side logic to DynamoDB. What are some of the problems that Parse doesn't have that might exist for DynamoDB?
Since Parse includes a web server, you can interact with it via simple HTTP requests. DynamoDB is just a database, so you would need to connect directly through the AWS SDK, or build an API in front of it, possibly using API Gateway and Lambda.
In addition, since Parse is a full-featured Backend as a Service, and DynamoDB is only a database, there are some features in Parse that won't be available if you just use DynamoDB directly from your iOS application. For example user password resets require sending an email to the user. DynamoDB has no "password reset" functionality and can't send emails directly. You would have to build that feature yourself using something like Lambda and SES.
Parse also handles file upload and file hosting, which are features you would no longer have if you just used DynamoDB directly from iOS. You would have to build those features yourself, possibly using S3.
If you are only using Parse as a data store then using DynamoDB directly could certainly work for you, but then again so could MongoDB or any other NoSQL database. You should definitely explore how your database schema would look in DynamoDB before committing to it, because there are certain restrictions on index types and query types that might make it difficult to transition your current schema.
AWS + DynamoDB would be your way to go.
I worked extensively in both, DynamoDB and MongoDB systems and can give you a short summary of an advise.
MongoDB is very easy to work with and has unmatched flexibility in query structure, requires very little thinking ahead of setting up the system.
DynamoDB will provide unmatched scalability, much stricter (very strict) set of rules for creating schemas and requires a lot of planning before you do the setup. However, you don't need to worry about setting up or managing database environment, no worry about master/slave architecture and no concerns of scaling your database.
I go with DynamoDB these days and it's been great.
Just completed a migration from Parse to AWS Dynamo (a few thoughts were posted here: https://www.linkedin.com/pulse/parse-aws-migration-server-less-mobile-backend-mike-kirkwood?trk=prof-post
My experience was that DynamoDD was an acceptable replacement for much of Parse. However, it required some data model changes as DynamoDB doesn't support Pointers or Relationships like Parse did. So, in the app had to adjust some of the writes to add more data to the record in DynamoDB. This did offer some nice benefits in the queries.
DynamoDB also allows you to add indexes to match specific queries.
And, for my use, DynamoDB has proven to be much faster queries than Parse was.
DynamoDB is just a database service, so you can use it to store Parse data but you'll still need a server to process the data and host APIs, etc... On AWS, you could spin up an EC2 instance to run the server, or try to make it run on Lambda.
Parse Server does not natively support either Lambda as a hosting environment or DynamoDB as a storage backend, but fortunately members of the community have recently developed integration for both of these:
https://www.npmjs.com/package/parse-server-dynamodb-adapter
https://github.com/parse-community/parse-server/issues/483
Are there any security best practices for using Rails and Heroku Postgres with pgcrypto's public key PGP.
The naive and direct way seems to be to store the private key and password using Heroku's ENV variables. However this seems like it doesn't add much security since both values are now easily accessed by anyone with access to the environment.
The goal of this would be to encrypt any privacy concerned information such as SIN numbers appropriately.
Scenario:
Let's presume you have some fields that, for some generic regulatory privacy requirement, are required or recommended to be stored encrypted, such as government IDs (SIN numbers for example). What is an appropriate or even common process for encrypting this information using pgcrypto.
If someone has an alternative suggestion for a scenario I'd be glad to include that as well.
This question doesn't have an answer until you define your threat model, but your question suggests that you want to store information in a way that even people with access to the server environment can't read it, in which case there are really only these options:
Don't store the data at all. Depending on what you need the data for you may be able to avoid storing it on your own servers for any reason. For instance, you can ask a user to enter their credit card number and immediately forward it to the credit card processor without saving it (which means you will need to ask them for the number again next time, unless the credit card processor hangs on to it for you.) Some payment processing schemes even send the payment data directly to the processor so that your website doesn't have to touch that data at all. Depending on your requirements this may or may not suit your needs.
Store the data only as a hash, like a password. This is useful if you don't actually need the data, but only need to verify that someone using your site has the data. This is universally the solution for passwords and other "secrets" but is useless for other data.
Store the data with public/private encryption, and don't keep the private key on the server at all. This could work if, for instance, the server has the public key in its environment, with which it saves data to the table, but an administrator has to download the encrypted data and decrypt it locally in order to use it. Again, this has severe limitations so you can only use it if your requirements of what to do with the data are compatible.
Store the data with client-side symmetric encryption, encrypted and decrypted only with a client key. This is how e.g. LastPass works. It means that you, the server admin, can not do anything with the data except return it to the user that submitted it. Once again, this only works if your requirements are very narrow.
If your requirements for what you do with the data are not compatible with any of the above, then you have no recourse. You could encrypt the data with symmetric encryption and keep the key in the server environment as a guard against backups or access to the database, but not the application, falling into the wrong hands, but that does not match the threat model of an attacker with access to the operating environment.
There is no one-size-fits-all "best practice" here because the tradeoffs involved in handling a threat model where the attacker has access to the environment are so great that only applications with very specific, limited requirements can guard against it at all. If the server can read the data, then so can an attacker.
Heroku Env
You get some benefit in storing the key in the Heroku env and encrypting in the database because then an attacker can't get the information from the DB either by direct break in or by SQL injection.
You are still vulnerable to anyone who can break into your app server, your Heroku account or any Heroku or Amazon staff that can obtain access to the server.
Separate Secure Server
If you have a large team and or subcontractors that can access your Heroku account you could have a separate server on a separate account or even separate more secure hosting that is reponsible for saving and loading the secrets with only a few highly trusted people having access to it. It could be kept small and simple and only accept requests from your app servers to minimise the attack surface. The server could be designed to limit the rate at which it provided the confidential data from the database to prevent a complete dump being taken rapidly even in the event that your main App server is broken into. It could access the same DB or a different on and perform the encryption/decryption on every load/save request from the app server before passing the result back.
Doing this will add latency and complexity but that is a trade off you can consider against the security gains. To get the secret information in bulk an attacker would have to either break into/have access to the app server and extract the secret data slowly from the secure server without triggering your intrusion detection systems or obtain direct access to your secure server in some way (which should be harder than the main server as there are less accounts to target for social engineering, password guessing and the direct attack surface area should be smaller.).
I'm doing some research on a new project we want to work on, but before we actually take on the project I have some concerns.
This project involves storing a lot of (text) data somewhere on a server, you can think about it like Instagram (but without photos).
So you can follow people, view profiles, list of activity ...
The question is, how do these apps send all this data securely to a server?
And what kind of server do they use? Something like Amazon AWS?
They are sent over https which is built into the NSURLConnection class. You can have a server (I use linux but you can use anything) have a self signed ssl certificate and your App code can give an exception to that particular URL for self signed certificates if you are wanting to save money. It is better practice and more secure to have a signed certificate though.
As far as storing it, most don't actually store it encrypted. It is sent to the server running a web service encrypted and then is stored in the database / file system unencrypted. The reason for this is there is a lot of processing power required to compress things and the extra over head of storing encrypted things as well as making it harder to search and index depending on how you do it.
Amazon AWS would work although I run my own server at home. It's quite easy to set up.
UPDATED 2010-11-25
A legacy stand-alone application (A1) is being re-created as a web application (A2).
A1 is written in Delphi 7 and uses a MS Access database to store the data. A1 has been distributed to ~1000 active users that we have no control over during the build of A2.
The database has ~50 tables, some which contain user data, some which contain template data (which does not need to be copied); 3-4 of these user tables are larger (<5000 records), the rest is small (<100).
Once A2 is 'live', users of A1 should be able to migrate to A2. I'm looking for a comparison of scenario's to do so.
One option is to develop a stand-alone 'update' tool for these users, and have this update tool talk to the A2 database through webservices.
Another option is to allow users to upload their Access db (~15 MB) database to our server, run some kind of SSIS package (overnight, perhaps) to get this into A2 for that user, and delete the Access db afterward.
Am I missing options? Which option is 'best' (I understand this may be somewhat subjective, but hopefully the pro's and cons for the scenario's can at least be made clear).
I'll gladly make this a community wiki if so demanded.
UPDATE 2010-11-23: it has been suggested that a variant of scenario 1 would be to have the update tool/application talk directly to the production database. Is this feasible?
UPDATE 2011-11: By now, this has been taken into production. Users upload the .zip file the .mdb is in, which is unpacked and placed in a secure location. A nightly SSIS scheduled job comes along and moves the data to staging tables, which are then moved into production through SP's.
I would lean toward uploading the complete database and running the conversion on the server.
In either case you need to write a conversion program. The real questions is how much of the conversion you deploy and run on the customers' computers. I would keep that part as simple as possible, i.e. just the upload. That way if you find any bugs or unexpected data during the conversion you can simply update the server and not need to re-deploy your conversion program.
The total amount of data you are talking about is not too large to upload, and it sounds like the majority of it would need to be uploaded in any case.
If you install a conversion program locally it would need a way to recover from a conversion that stopped part way through. That can be a lot more complicated than simply restarting an upload of the access database.
Also you don't indicate there would be any need for the web services after the conversions are done. The effort to put those services together, and keep them running and secure during the conversions would be far more than a simple upload application or web form.
Another factor is how quickly your customers would convert. If some of them will run the current application for some time period you may need to update your conversion application as the server database changes over time. If you upload the database and run the conversion on the server then only the server conversion program would need to be updated. There would not be any risk of a customer downloading the conversion program but not running it until after the server databases were updated.
We have a similar case where we choose to run the conversion on the server. We built a web page for the user to upload their files. In that case there is nothing to deploy for the new application. The only downside we found is getting the user to select the correct file. If you use a web form for the upload you can't pre-select file name for the user because of security restrictions. In our case we knew where the file was located but the customers did not. We provide directions on the upload page for the users to help them out. You could avoid this by writing a small desktop application to perform the upload for the users.
The only downside I see to writing a server based conversion is some of your template data will be uploaded that is un-needed. That is a small amount of data anyway.
Server Pros:
- No need to re-deploy the conversion due to bugs, unexpected data, or changes to the server database
- Easier to secure (possibly), there is only one access point - the upload. Of course you are accepting customer data in the form of an access database so you still can't trust anything in it.
Server Cons:
- Upload un-needed template data
Desktop Pros:
- ? I'm having trouble coming up with any
Desktop Cons:
- May need multiple versions deployed
As to talking to a server database directly. I have one application that talks to a hosted database directly to avoid creating web services. It works OK, but if given the chance I would not take that route again. The internet is dropped on a regular basis and the SQL Providers do not recover very well. We have trained our clients just to try again when that happens. We did this to avoid creating web services for our desktop application. We just reference the IP address in the server connection string. There is an entire list of security reasons not to take this route - we were comfortable with our security setup and possible risks. In the end the trade off of using the desktop application with no modifications was not worth having an unstable product.
Since a new database server to be likely one the standard database engines in the industry, why not consider linking the access application to this database server? That way you can simply send your data up to sql server that way.
I'm not really sure why you'd consider even suggest using a set of web services to a database engine when access supports an ODBC link to that database engine. So one potential upgrade path would be to simply issue a new application in access that has to be placed in the same directory as to where their current existing data file (and application) is now. Then on startup this application can simply RE link all of its tables to your existing database, plus come with a pre link set of tables to the database server. This is going to be far less work in building up some type of web services approach. I suppose part of this centers around where the database servers going to be hosted, but in most cases perhaps during the migration period, you have the database server running somewhere where everyone can get access to it. And a good many web providers allow external links to their database now.
It's also not clear that on the database server system you're going to create separate databases for each one, or as you suggest in your title it's all going to be placed into one database. Since is going to be placed into one database, then during the upsizing, an additional column that identifies the user location or however you plan to distinguish each database will be added during this upsizing process to distinguish each user set of data.
How easy this type of migration be will depend on the schema and database layout that the developers are using for the new system. Hopefully and obviously it has provisions for each user or location or however you plan to distinguish each individual user of the system. So, I don't suggest web services, but do suggest linking tables from the Access application to the instance of SQL server (or whatever server you run).
How best to do this will depend on the referential integrity and business rules that must be enforced, if there are any. For example, is there the possibility of duplicates when the databases are merged? I gather they are being merged from your somewhat cryptic statement: "And yes, one database for all, aspnet membership for user id's".
If you have no control of the 1000+ users of A1, how are you going to get them all to convert to A2?
Have you considered giving them an SQL Server Express DB to upgrade to, and letting them host the Web App on their own servers?