Background:
I got some data encrypted with AES (ie symmetric crypto) in a database. A server side application, running on a (assumed) secure and isolated Linux box, uses this data. It reads the encrypted data from the DB, and writes back encrypted data, only dealing with the unencrypted data in memory.
So, in order to do this, the app is required to have the key stored in memory.
The question is, is there any good best practices for this? Securing the key in memory.
A few ideas:
Keeping it in unswappable memory (for linux: setting SHM_LOCK with shmctl(2) ?)
Splitting the key over multiple memory locations.
Encrypting the key. With what, and how to keep the...key key.. secure?
Loading the key from file each time its required (slow and if the evildoer can read our memory, he can probably read our files too)
Some scenarios on why the key might leak: evildoer getting hold of mem dump/core dump; bad bounds checking in code leading to information leakage;
The first one seems like a good and pretty simple thing to do, but how about the rest? Other ideas? Any standard specifications/best practices?
Thanks for any input!
All depends on the level of your paranoia and the sensitivity of the key/data. In the extreme cases, as soon as you have an unencrypted key in memory, one can retrieve it using coldboot techniques. There is an interesting development at frozencache to try to defeat that. I merely casually read it, did not try it in practice, but it seems like an interesting approach to try.
With the tinfoil hat off, though - (1), (2), (3) do seem reasonable. (4) won't cut it precisely for the reason you mentioned. (Not only it is slow, but assuming you read into the stack, with different stack depths the key might become visible more than once).
Assuming the decrypted data is worth it, and it would be in the swappable memory, you definitely should encrypt the swap itself as well. Also, the root, /tmp partitions should also be encrypted. This is a fairly standard setup which is readily available in most guides for the OSes.
And then, of course, you want to ensure the high level of physical security for the machine itself and minimize the functions that it performs - the less code runs, the less the exposure is. You also might want to see how you can absolutely minimize the possibilities for the remote access to this machine as well - i.e. use the RSA-keys based ssh, which would be blocked by another ACL controlled from another host. portknocking can be used as one of the additional vectors of authentications before being able to log in to that second host. To ensure that if the host is compromised, it is more difficult to get the data out, ensure this host does not have the direct routable connection to the internet.
In general, the more painful you make it to get to the sensitive data, the less chance someone is going to going to get there, however there this is also going to make the life painful for the regular users - so there needs to be a balance.
In case the application is serious and the amount of things at stake is high, it is best to build the more explicit overall threat model and see what are the possible attack vectors that you can foresee, and verify that your setup effectively handles them. (and don't forget to include the human factor :-)
Update: and indeed, you might use the specialized hardware to deal with the encryption/decryption. Then you don't have to deal with the storage of the keys - See Hamish' answer.
If you are serious about security then you might consider a separate cryptographic subsystem. Preferably one that is FIPS 140-2/3 certified (list of certified modules).
Then the key is held in tamper proof memory (non-extractable) and all cryptographic operations are performed inside the crypto boundary.
Expensive but for some applications necessary.
Also don't forget the threat of core dumps and your memory being swapped out!
On both POSIX (like Linux) and Windows systems, there are techniques to prevent that from happening if you're dealing with C language - see this section from CERT Secure Coding Standards:
MEM06-C. Ensure that sensitive data is not written out to disk
The big problem is the program has to read the key from somewhere. Unless you accept direct keyboard input each time the server reboots, it pretty much has to exist on disk somewhere.
In general you have to assume the evildoer doesn't have access to the root level operating system or hardware as when that's the case they'll eventually manage to get the key even if it's only in RAM.
So you assume the server's OS is secure. But let's say somebody can come and steal the hard drive so starting the server would give them the key. Then let the server ask another server for half of the key, the remote server validates the request (using ip, private/public key pairs) and supplies half the key. Then your server has a complete key, and the remote server never has more than half. That seems to me an improved level of protection.
I'd be looking at what
openssh,
openssl,
GnuPG (see related sub-projects via the project-root dropdown), and
GnuTLS
do when handling keys. They're sufficiently paranoid about such security matters...
Use of "super super user" hardware memory is ideal. All Intel Macs have this SecureEnclave memory area and it also includes an AES decryption in hardware such that the application and operating system never have access to the raw private key. When the machine boots, a password is typed in (optional), and the SecureEnclave decrypts its cold flash memory encrypted version of the key into its RAM area, which is not accessible by the main operating system.
Nice side effect is the hardware accelerated encryption: I benchmarked 600 MB/sec writes to my PCIe storage on a freshly formatted encrypted disk.
In the cloud, Amazon have this AWS Key Management Service (KMS) managed service that makes it easy for you to create and control the encryption keys used to encrypt your data, and uses FIPS 140-2 validated hardware security modules to protect the security of your keys: https://aws.amazon.com/kms/
Related
You know, when an application opens a file and write to it, the system chooses in which cluster will be stored. I want to choose myself ! Let me tell you what I really want to do... In fact, I don't necessarily want to write anything. I have a HDD with a BAD range of clusters in the middle and I want to mark that space as it is occupied by a file, and eventually set it as a hidden-unmoveable-system one (like page file in windows) so that it won't be accessed anymore. Any ideas on how to do that ?
Later Edit:
I think THIS is my last hope. I just found it, but I need to investigate... Maybe a file could be created anywhere and then relocated to the desired cluster. But that requires writing, and the function may fail if that cluster is bad.
I believe the answer to your specific question: "Can I write a file to a specific cluster location" is, in general, "No".
The reason for that is that the architecture of modern operating systems is layered so that the underlying disk store is accessed at a lower level than you can access, and of course disks can be formatted in different ways so there will be different kernel mode drivers that support different formats. Even so, an intelligent disk controller can remap the addresses used by the kernel mode driver anyway. In short there are too many levels of possible redirection for you to be sure that your intervention is happening at the correct level.
If you are talking about Windows - which you haven't stated but which appears to assumed - then you need to be looking at storage drivers in the kernel (see https://learn.microsoft.com/en-us/windows-hardware/drivers/storage/). I think the closest you could reasonably come would be to write your own Installable File System driver (see https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/_ifsk/). This is really a 'filter' as it sits in the IO request chain and can intercept and change IO Request Packets (IRPs). Of course this would run in the kernel, not in userspace, and normally this would be written in C and I note your question is tagged for Delphi.
Your IFS Driver can sit at differnt levels in the request chain. I have used this technique to intercept calls to specific file system locations (paths / file names) and alter the IRP so as to virtualise the request - even calling back to user space from the kernel to resolve how the request should be handled. Using the provided examples implementing basic functionality with an IFS driver is not too involved because it's a filter and not a complete storgae system.
However the very nature of this approach means that another filter can also alter what you are doing in your driver.
You could look at replacing the file system driver that interfaces to the hardware, but I think that's likely to be an excessive task under the circumstances ... and as pointed out already by #fpiette the disk controller hardware can remap your request anyway.
In the days of MSDOS the access to the hardware was simpler and provided by the BIOS which could be hooked to allow the requests to be intercepted. Modern environments aren't that simple anymore. The IFS approach does allow IO to be hooked, but it does not provide the level of control you need.
EDIT regarding suggestion by the OP of using FSCTL_MOVE_FILE
For simple environment this may well do what you want, it is designed to support a defragmentation process.
However I still think there's no guarantee that this actually will do what you want.
You will note from the page you have linked to it states that it is moving one or more virtual clusters of a file from one logical cluster to another within the same volume
This is a code that's passed to the underlying storage drivers which I have referred to above. What the storage layer does is up to the storage layer and will depend on the underlying technology. With more advanced storage there's no guarantee this actually addresses the physical locations which I believe your question is asking about.
However that's entirely dependent on the underlying storage system. For some types of storage relocation by the OS may not be honoured in the same way. As an example consider an enterprise storage array that has a built in data-tiering function. Without the awareness of the OS data will be relocated within the storage based on the tiering algorithms. Also consider that there are technologies which allow data to be directly accessed (like NVMe) and that you are working with 'virtual' and 'logical' clusters, not physical locations.
However, you may well find that in a simple case, with support in the underlying drivers and no remapping done outside the OS and kernel, this does what you need.
Since you problem is to mark bad cluster, you don't need to write any program. Use the command line utility CHKDSK that Windows provides.
I an elevated command prompt (Run as administrator), run the command:
chkdsk /r c:
The check will be done on the next reboot.
Don't forget to read the documentation.
I've noticed several tutorials for most of the major players in social networks have examples where a API key tied to your account is embedded (usually in plan text) in the source code. For example, Google Maps APIs Premium Plan. This key is used to bill your company.
I found a similar question in Is it safe to put private API keys in your .m files when exporting to the appstore?1 - Of note, anyone with a jailbroken phone can see the unencrypted executable.
Is this practice actually safe, and if so, why?
Embedding API keys in an app is not secure and generally not a good practice but does require a substantial work factor to obtain them, it is not trivial. There is no tool to decrypt the executable other than the OS for execution.
RE: "anyone with a jailbroken phone can see the unencrypted executable." is not really true. Just jailbreaking will not decrypt the app binary, it is only decrypted as the binary is loaded in RAM to execute and the key will not be available, it is decrypted in hardware in the DMA path. One needs to add debugging tools and catch the binary after it is loaded into memory for execution.
You need to determine who the attacker is, how much skill and time the attacker will spend and the cost to you.
There is no 100% secure solution, only increasing the work factor.
An alternative is to obtain the API keys on first run at login to a server and then move them to the Keychain. But this is also just an increase in work factor because as above the executable can be examined at run time when it is sent to the service.
As long as the key has to be in the app memory during any part of execution it is vulnerable.
Putting the API keys in the source may meet the security needs.
Since Apple controls the entire hardware/software stack, is it possible to obtain the following (through some type of trusted computing):
the hardware certifies that the software is genuine, non-jail broken iOS
iOS certifies to my server that the app run is an unmodified app
What this achieve is as follows:
when my server sends out data, it is guaranteed that the data can only be used in the way I intend it to be used (since it's running my app unmodified, on an non-jail broken iOS).
This prevents things like a modified app which steals data being transmitted from the server to the client. I realize one could theoretically eavesdrop, but this can be eliminated via encryption.
Thanks!
Briefly, no.
You're talking about Trusted Computing concepts on a platform that does not support TC. IOS does not include anything near Trusted Computing - Remote Attestation. It has no TPM.
The chain of trust established by Apple chip merely tries to stop execution if the signature of the next element in the boot chain is invalid. If one thing fails (jailbroken), their's no real -effective- way of detecting it. It is very similar to Secure Boot introduce by Microsoft but it's very different then Trusted Computing which attest which version of the system it is currently running.
With Trusted Computing, the TPM store the measurements (PCRs) of the system boot (SRTM). At boot, the first thing executed (CRTM - the only thing we really need to trust implicitly) will start the chain by measuring the BIOS, send the measure to the TPM (in a PCR) and pass execution to it (the BIOS). Then the BIOS does the same thing for the next element in the boot chain.
The measurements stored in the PCRs can then be used to encrypt or decrypt information (SEAL/UNSEAL operations) depending on the environment loaded in memory.
The TPM does not take action on the measurements (good or bad). The idea is not to restrain what can be loaded but to being able to know what environment is loaded on the platform. If something has been modified, the TPM will not contain the proper PCRs values and the UNSEAL operation (decrypt using PCRs as the key) will not work.
In the case of Remote Attestation, we're talking about the QUOTE operation. It's basically the same thing then SEAL but uses other keys to make sure the evaluating party can validate the attestation is really coming from a real/compliant TPM.
Sure, a system could use the SEAL operation to protect a secret used to decrypt the operating system and thus produce -in some way- the same effect as secure boot.
For more info, see my other posts.
I have one website mainly composed on javascript. I hosted it on IIS.
This website request for the images from the particular folder on hard disk and display them to end user.
The request of image are very frequent and fast.
Is there any way to reduce this overhead of disk read operation ?
I heard about memory mapping, where portion of hard disk can be mapped and it will be used as the primary memory.
Can somebody tell me if I am wrong or right, if I am right what are steps to do this.
If I am wrong , is there any other solution for this ?
While memory mapping is viable idea, I would suggest using memcached. It runs as a distinct process, permits horizontal scaling and is tried and tested and in active deployment at some of the most demanding website. A well implemented memcached server can reduce disk access significantly.
It also has bindings for many languages including those over the internet. I assume you want a solution for Java (Your most tags relate to that language). Read this article for installation and other admin tasks. It also has code samples (for Java) that you can start of with.
In pseudocode terms, what you need to do, when you receive a request for, lets say, Moon.jpeg is:
String key = md5_hash(Moon.jpeg); /* Or some other key generation mechanism */
IF key IN memcached
SUPPLY FROM memcached /* No disk access */
ELSE
READ Moon.jpeg FROM DISK
STORE IN memcached ASSOCIATED WITH key
SUPPLY
END
This is a very crude algorithm, you can read more about cache algorithms in this Wiki article.
The wrong direction. You want to reduce IO to slow disks (relative). You would want to have the files mapped in physical memory. In simple scenarios the OS will handle this automagically with file cache. You may look if Windows provides any tunable parameters or at least see what perf metric you can gather.
If I remember correctly (years ago) IIS handle static files very efficiently due a kernel routing driver linked to IIS, but only if it doesnt pass through further ISAPI filters etc.. You can probably find some info related to this on Channel9 etc..
Long term wise you should look to move static assets to a CDN such as CloudFront etc..
Like any problem though... are you sure you have a problem?
I want to make a small app that collects data from a device attached to a serial port and passes it across a LAN to another app which stores it in a database.
I have done this already in one app on a single PC, so will effectively be splitting the app in half.
I have zer0 experience of network programming.
I want something "secure" meaning that the data are in some way encrypted and relatively straightforward to implement.
At the moment communication is one way, but I might want so send control data in the return direction later.
Can someone please recommend a protocol (preferably one which is commonly used for this sort of thing) and post a small code snippet of how to send and receive - let's say a data structure with a string for user name and an integer for age.
Or, just point me at a URL.
Thanks in advance.
Answer: I'm going with IpSec and Exchanging Data over the Network using Delphi
You can simply use IPSec between the two computer and the operating system will encrypt the channel without any need of changing the applications.
Other option may be DCOM (if you use the Enterprise version of Delphi Datasnap supports it) that can also encrypt the communication (must be set in the configuration, works best in a domain).
Both options don't need certificates or the like, and the OS will take care of storing the keys safely.