I am hosting a small fileserver, where users can upload documents from all around the world.
Due to problems in encoding (see otherquestion), I am asking myself if I should disallow users to upload (and on the other hand download) files not supported by CP1252 charset?
or otherwise; is it senseful to allow users upload documents with arabian or chinese letters in their filenames?
PS: they download the same file some time later (and it should have the same filename as uploaded)
You should be storing the files on disk using a randomly generated name, or let the file name be based on a hash of the file contents (good for deduplicating storage as well). You can save the original file name as meta data in a database together with all other meta data about the file (who uploaded it and things like that). Then you serve the file again using a PHP script which sets the original file name from the database in an HTTP header. This way you:
don't need to worry about file name sanitisation or duplication
file system encoding issues
storage duplication (if using a hash)
Related
I'm trying to read and analysis iPhone backup files in Library/Application Support/MobileSync/Backup folder.
I use SQLite DB viewer to view the manifest.db:
Files TABLE structure
Files TABLE content
The filename is encrypted. And the file is blob type which I can't decide it's an image or text or documents.
How can I get the details of these backup? (Using Javascript would be best, or Swift).
The numbers and letters under fileID are the file names in the backup as in the folders with the manifest.db, and they correspond to the domain and relative path (duh). In iOS versions newer than 9.3.5, you can find the file itself in the folder named with the 1st 2 characters of the fileID. At the end of the relative path in the database it should have an extension. Use that as a hint to what program to use to open the file. Xcode for .plist etc. Ultimately it depends what you're looking for...
The blob in the blob column is a binary plist. You can parse it using a plist parser e.g. Property List Parsing
The blob appears to be base64 encoded, but I cannot figure out the type of the resulting binary blob. I've checked to see if it can be decrypted with openssl enc -aes-256-cbc -d decoded-string-as-binary-blob.datafile but I get back "bad magic." I've also tries all of the other AES encryption ciphers and block cipher modes, but no change. It could always be XOR'd with a value, but I haven't found an easy to test that theory.
I have a service set up where when the user registers, they are able to download a file to their device. The file is dynamically generated from some local information from our database such custom field information (username, email, web url, etc) and then account specific assets stored on S3 (avatar, icons, background art).
I'm not sure of the best way to handle these S3 files as part of the generation process.
Using a Ruby Tempfile class generates a file that has a unique filename that doesn't match what we are expecting. Using Ruby's File class generates the files we want, but it also litters the filesystem with a bunch of files and I worry won't handle concurrent requests for the same assets properly. We're also using Heroku, and they tend to frown on that from what I read.
What's a best practice/recommended way to handle dynamically generating files based on a mix of local and remote assets and then presenting it to the user?
I think I'm missing something about mediafilesegmenter.
The docs say this about encrypting content:
The media stream segmenter available from Apple provides encryption
and supports three modes for configuring encryption. The first mode
allows you to specify a path to an existing key file on disk. In this
mode the segmenter inserts the URL of the existing key file in the
index file. It encrypts all media files using this key. The second
mode instructs the segmenter to generate a random key file, save it in
a specified location, and reference it in the index file. All media
files are encrypted using this randomly generated key. The third mode
instructs the segmenter to generate a new random key file every n
media segments, save it in a specified location, and reference it in
the index file. This mode is referred to as key rotation. Each group
of n files is encrypted using a different key.
http://developer.apple.com/library/mac/#documentation/NetworkingInternet/Conceptual/StreamingMediaGuide/UsingHTTPLiveStreaming/UsingHTTPLiveStreaming.html#//apple_ref/doc/uid/TP40008332-CH102-SW3
The docs say this about mediafilesegmenter:
The file segmenter performs the same tasks as the stream segmenter, but it takes files as input instead of streams.
http://developer.apple.com/library/mac/#documentation/NetworkingInternet/Conceptual/StreamingMediaGuide/HTTPStreamingArchitecture/HTTPStreamingArchitecture.html#//apple_ref/doc/uid/TP40008332-CH101-DontLinkElementID_7
So, I'm guessing mediafilesegmenter has those three modes, just like mediastreamsegmenter does. I'm trying to use the second mode. Here's my command:
mediafilesegmenter -f encryptedSegmentsDirectory -encrypt-iv=sequence
-encrypt-key encryptedSegmentsDirectory/key.txt media.m4v
The result is .ts files and two m3u8 files, neither of which seem to reference key files. I get this response:
mediafilesegmenter: option `-encrypt-key' is ambiguous
Is there an example that shows how to do this successfully?
Also, HTTP Live Streaming seems to insist on serving key files over http:
You can serve key files using either HTTP or HTTPS. You may also
choose to protect the delivery of the key files using your own
session-based authentication scheme. For details, see “Serving Key
Files Securely Over HTTPS.”
I'd like to serve it on a local file on the iOS file system instead from a server. Is this possible? If not, would there any problem with serving it from a local web server?
Sorry if I missed something obvious - please feel free to just point me to the relevant documentation!
Doh - I got the switch wrong. It's -encrypt-key-file, not -encrypt-key. Sorry, everyone! I just needed to step away for a while.
I have a website that shows galleries. Users can upload their own content from the web (by entering a URL) or by uploading a picture from their computer.
I am storing the URL in the database which works fine for the first use case but I need to figure out where to store the actual images if a user does a upload from their computer.
Is there any recommendation here or best practice on where I should store these?
Should I save them in the appdata or content folders? Should they not be stored with the website at all because it's user content?
You should NOT store the user uploads anywhere they can be directly accessed by a known URL within your site structure. This is a security risk as users could upload .htm file and .js files. Even a file with the correct extension can contain malicious code that can be executed in the context of your site by an authenticated user allowing server-side or client-side attacks.
See for example http://www.acunetix.com/websitesecurity/upload-forms-threat.htm and What security issues appear when users can upload their own files? which mention some of the issues you need to be aware of before you allow users to upload files and then present them for download within your site.
Don't put the files within your normal web site directory structure
Don't use the original file name the user gave you. You can add a content disposition header with the original file name so they can download it again as the same file name but the path and file name on the server shouldn't be something the user can influence.
Don't trust image files - resize them and offer only the resized version for subsequent download
Don't trust mime types or file extensions, open the file and manipulate it to make sure it's what it claims to be.
Limit the upload size and time.
Depending on the resources you have to implement something like this, it is extremely beneficial to store all this stuff in Amazon S3.
Once you get the upload you simply push it over to Amazon and pop the URL in your database as you're doing with the other images. As mentioned above it would probably be wise to open up the image and resize it before sending it over. This both checks it is actually an image and makes sure you don't accidentally present a full camera resolution image to an end user.
Doing this now will make it much, much easier if you ever have to migrate/failover your site and don't want to sync gigabytes of image assets.
One way is to store the image in a database table with a varbinary field.
Another way would be to store the image in the App_Data folder, and create a subfolder for each user (~/App_Data/[userid]/myImage.png).
For both approaches you'd need to create a separate action method that makes it possible to access the images.
While uploading images you need to verify the content of the file before uploading it. The file extension method is not trustable.
Use magic number method to verify the file content which will be an easy way.
See the stackoverflow post and see the list of magic numbers
One way of saving the file is converting it to binary format and save in our database and next method is using App_Data folder.
The storage option is based on your requirement. See this post also
Set upload limit by setting maxRequestLength property to Web.Config like this, where the size of file is specified in KB
<httpRuntime maxRequestLength="51200" executionTimeout="3600" />
You can save your trusted data just in parallel of htdocs/www folder so that any user can not access that folder. Also you can add .htaccess authentication on your trusted data (for .htaccess you should kept your .htpasswd file in parallel of htdocs/www folder) if you are using apache.
I have a upload text file field, and with it I plan to save the file somewhere and then store the location of the file in a database. However, I want to make sure the file they uploaded is a .txt file, and not, say, an image file. I imagine this happens in the validation step. How does one validate such a thing? Also, how do you get the filename of the uploaded file? I could always just check if it said '.txt' but for future reference knowing how to validate without just the filename would be helpful.
Trying to validate the contents of a file based on the filename extension is opening the door for major hackerdom. It's trivial to change the extension and upload the file.
If you are on a Mac/Linux/Unix-based system the OS "file" command is the standard because it looks inside the file for key bytes that flag file types. http://en.wikipedia.org/wiki/File_(Unix) I'm not sure what's available for Windows, but this might help: Determine file type in Ruby
One way of doing it, the simple way really, would be to pass the file through an image loader, preferably one that handles multiple common formats, and see if it throws an error.
The other way is to manually check the file header for common image format headers. For example, .bmp files start with BM. Other formats have their own specific markings you can use.