I have zips in s3 with videos.. some are small (3Mb), but others can be bigger (like 30Mb).. I need to download these zips.. I don't know what is the best way to do it, talking about performance.. Put these videos in a unique zip and download just one zip with all the videos in it or download every file separately.. Joining and compacting the files to a zip in a temp folder isn't expensive in performance or much work for the server?
PS.: The user will choose the videos that he wants to be downloaded and click on a button to download them (I think it would be like 10-100 videos)..
From a strictly performance standpoint, downloading straight from S3 beats everything else hands down as it's no load on the rails box. It doesn't often meet other considerations like security, logging, access control, etc.
Related
please bear with me as I'm not trying to frustrate anyone with inane questions, and I did google search this but I couldn't really find anything recent or helpful.
I am a novice programmer and I am using a classic asp web application. I just enabled the users to upload and download images, but I'm quickly regretting it as it's eating up all of the router bandwidth. I am finding my solution inadequate, so I wanted to start over.
My desire is threefold with this functionality:
Compression. I understand that this is impossible to do BEFORE uploading without some kind of Java/Silverlight/Flash portion of the application to handle uploads, correct? What is the common way most places go about this? Just allow regular file uploads and compress once they are on the server?
Resizing. I want to resize all images before they are uploaded to a reasonable size, instead of just telling users that try and upload huge camera images that they can't upload. I figure I just want to let them upload and have it resize for them before uploading. Does this functionality exist already?
Changing filetype. I want to allow users to upload all image file types but make them .jpg on the server after the upload.
With these three requirements, how hard is it to implement something like this in just pure code and libraries? Would it be better to just use a 3rd party plugin, such as ASPjpeg or ASPupload? Have you encountered something similar, and what was your solution?
Thanks.
Take a look at ASPJpeg and ASPUpload from Persits. We use these components to upload a full size image (can be png even though the library is "ASPJpeg"), resize it to several different sizes we need on our site, then store the resized images on the server in a variety of folders. The ASPUpload component is a little tricky but if you follow their sample code you'll be fine.
I never found a good component for decompressing uploaded zip files and had to write my own, which I've since abandoned. In the end with upload speeds increasing and storage getting so cheap, it started to matter less and less that the files were compressed before being uploaded.
EDIT: Just noticed you mentioned these components in your question. Consider this an endorsement of your idea to use them. :-)
I am encrypting downloaded files and saving them locally in app's documents directory.
To read them you must decrypt those file and store some where temporarily.
My concerns are:
1.if I store them in doc directory for time they are being used, for that time window one can get those files using tools like iExplorer.
2.My idea is to store them in memory for the time they are being used and flush the vault after use.
This option is good for small files but for large files say 50 MB or video of 100 MB, I am afraid that app will receive memory warning in result will terminate abruptly.
I want to know the best approach for doing this.
There is no perfect security storing local files in a safe way. If a person has full access to the device, he can always find a way to decrypt the files, as long as your application is able to decrypt it.
The only question is: How much effort is necessary to decrypt the files?
If your only concern is that a person may use iExplorer to copy and open these files, a simple local symmetric encryption will do the trick.
Just embed a random symmetric key in your application and encrypt the data block by block while you download it.
You can use the comfortable "Security Transforms" framework to do the symmetric encryption. There are some good examples in the Apple Documentation.
When you load the files, you can use the same key to decrypt them while you load them from the file system.
Just to make things clear: This is not a perfect protection of the files. But to decrypt the files, one has access to your app binary. Analyse this binary in a debugger and searching for the decryption part to extract your symmetric key. This is a lot effort necessary just to decrypt the files.
Split your files into smaller sizes before saving them, then decrypt on load.
Later edit: I noticed this is mentioned in the comments. I agree splitting files isn't the easiest thing in the world, but presumably you'll only need this for video. About 100MB is a lot of text or audio. If your PDF weights as much, it's probably scanned text, and you can change it into a series if images.
And yes, splitting better be done server-side, don't want the user waste battery in video processing.
Decrypt them, obfuscate them with a toy algorithm (e. g. XOR with a constant block), and store them in documents. When needed, load and decrypt.
Since the problem has no solution in theory (a determined enough attacker can read your process memory after all), it's as good a solution as any.
It seems info about zip performance on iOS is a bit sparse, so I'm hoping someone can elaborate a bit on the subject (or do some testing since I do not have that option).
I've written a pure C project for use in an iOS application (for iPad 2/3) which uses some pretty large data files (5x ~300 MB) and I know these can be compressed quite well (5x ~90 MB). Currently the application will select one of the 5 unzipped data files and memory map it - this can be changed on the fly.
Because the data files are unzipped this action happens pretty much instantly, but it's quite a large app, so obviously I'm wondering - what if we unzipped the data files on the fly? How long would a switch (delete current data file, unzip new data file) take?
I'm hoping the answer can also be helpful to others who'd like a quick lookup for iOS zip performance.
Anyway you should uncompress these files to the flash drive, not to the memory (since there is not too much memory on iDevice), so your app will require ~1Gb of free storage to run.
So, since this storage can not be used in any other good way, I would recommend to uncompress the data on the first run (and, probably, not deliver data files with the app but to download them from your site).
And, yes, decompressing ~500Mb file would take a while on iDevices.
When uploading a file I know I can access its properties but is it always the same or it varies? I mean, I am writing an app for myself where I can upload songs or videos to my server to watch later, and I'd like to populate the info about said files automatically as much as possible so I was wondering if it's possible to get things like length, quality, name, artists, artwork, or pick a first image like youtube does for its videos?
I'm fairly new to ruby (using rails) so I am unsure as to where to find this or if it's even possible
You can do that using FFMpeg (read the license first).
FFMpeg gives you everything you were asking about and some more.
it's very powerful.
For mp3, check out mp3-info, I haven't used it before but looks promising...
I'm currently learning rails by creating a simple project management app. I've gotten to the point where I would like to be allow a user upload multiple files - pdfs, docs, xls etc. The user only needs to be able to attach one file at a time, but the possibilty to have multiple documents associated with a project is a must.
I've spent quite a lot of time researching my options, and it appears the two main plugins are attachment_fu and paperclip. From what I've read though, these appear to concentrate specifically on the upload and subsequent resizing of images, something I couldn't care less about. Is there a simpler way to achieve what I'm trying?
Thank you all in advance.
You might still consider using attachment_fu or paperclip as those are the "standard" libraries for such tasks. And they work fine for any kind of file.
The multi-upload can't be made without JS or Flash now. You need add some hack in your view to manage it.