When you are running a typical PHP server, you have to set your memory limit higher to allow for larger file uploads. Your memory limit has to be at least as high as the max file size you want to allow.
YouTube allows uploads of I think up to 2GB. Considering how popular YouTube is and how many people upload videos 24/7, how does YouTube handle this? Whatever backend language they are using, I assume they don't have something crazy like 100TB of memory on each server to allow for 100's or 1,000's of concurrent uploads, etc.
Do they have some sort of unique streaming upload type of thing running, where it doesn't require them to load the entire uploaded file into memory during the upload time? What are they doing that allows so many people to upload such massive files at the same time?
Related
I am having a scenario here. I have to build an asp.net core web application from where I have to browse drive and directories (where hosted: windows) and list the available video files. This portion working perfectly.
Now I have to do two things with those video files:
Need to download files (working except the size is < 2 GB)
Need to play the video on the page whatever the format and size is.
Asking for your helps. Thanks in advance.
If your website videos, each video is very small, assuming that the video size in a single page does not exceed 100M, or your internet speed is extremely fast, I think your needs are feasible.
What I said above is only the content contained in a single page. When there are not many video resources, the page loading speed is related to the bandwidth.
It is very simple to read files under a certain path, but I think the network speed and the performance of IIS are the bottleneck for reading and downloading.
Reason and suggestions
1. IIS web server just host web application, and also can store some small static resources, such as website pictures, css and js files.
IIS itself is on windows. When there are not many users, it can run normally by accessing 1G video. It can be loaded slowly. But what if there are many users, such as 100 people or 1000+ people visiting at the same time? I think the pressure on the web server is very high, so your idea is only suitable for a small range of users, and it is recommended that the video resources should not be too large.
2. I suggest you use third-party media services to store and read videos. For example, use Azure Media Service.
The reason is that if the video resource under the path is read through the web server, it must be very slow in terms of video loading, especially when the file is large and more users access it, the phenomenon will be particularly obvious.
Using Azure Media Services, the opening speed and download speed of playing videos will be improved. As a web server, IIS will no longer bear the pressure of this part of the media service.
I want my app to have a bunch of 30-sec mp4 clips. I want to ship these clips with the App and not have the users download them from the cloud
Each of my clip is around 5 MB and I expect to have a lot of them.
Is there a way to compress them to reduce the app download size? ( the 5Mb size is after all the CODEC's etc) I need an iOS solution for this.
MP4 is very compressed already so there isn't a way to compress it more. That's why zipping mp4s barely changes their size.
You have two options:
1) Include whichever ones the user needs first and download the rest, hopefully before they're needed.
2) If you absolutely have to have them all in the app you could reduce the resolution and/or encode at a lower bitrate.
If you go with option 2, you could still download higher quality ones from the cloud in the background and use those if available, but default to the lower quality ones if not.
I'm currently working on a concept for a new iPhone app, the would involve some very basic profile system.
Since all the other data will be stored in iCloud, I was wondering if it would be possible to store pictures in the cloud too.
I see that the storage per app is only 1MB, so this seems extremely low to store pictures.
The 1 MB limit is for Key-Value storage only (and KV storage is inappropriate for images for other reasons, too). If you use the Documents in the Cloud APIs, your app can store a large number of large files, limited only by how much space the user has (or hasn't) paid for. (Though it's wise to use that space judiciously and efficiently, as your potential customers won't be happy if your app alone causes them to blow the storage limits on their account.)
Can anyone tell me what is the maximum size that can be download from the web
and can be stored locally.
The limit is 5 MB.
See also this S.O. post.
You could also have a look at this if you are interested in going beyond that limit.
EDIT: after your comment I see you are not referring to local web storage. Local web storage is a new possibility offered by HTML5 to store key-value pairs. This has got the limitation I was mentioning.
As to your case, I don't think that there is some restrictive limit on the amount of data you can download and store locally (in your Documents directory) in order to access it later. You can check this, where answers range from a minimum of 2GB of flash space to no limit at all. So you can be safe.
If you are referring to the limit in the total amount of data you can download over 3G connections (also called over-the-air download), this is 20MB. You are only allowed to download a larger file when using Wi-Fi.
I was wondering, which way of managing thumbnail images make less impact to web server performance.
This is the scenario:
1) each order can have maximum of 10 images.
2) images does not need to store after order has completed (max period is 2 weeks).
3) potentially, there may have a few thousands of active orders at anytime.
4) orders with images will frequently visit by customers.
IMO, pre-generate thumbnail in hard disk is a better solution as hard disk are cheaper even with RAID.
But what about disk I/O speed, and resource it need to load images? will it take more resource than generate thumbnails at real-time?
It would be most appreciate if you could share your opinion.
I suggest a combination of both - dynamic generation with disk caching. This prevents wasted space from unused images, yet adds absolutely no overhead for repeatedly requested images. SQL and mem caching are not good choices, both require too much RAM. IIS can serve large images from disk while only using 100k of RAM.
While creating http://imageresizing.net, I discovered 29 image resizing pitfalls, and few of them are obvious. I strongly suggest reading the list, even if it's a bit boring. You'll need an HttpModule to be able to pass cached requests off to IIS.
Although - why re-invent the wheel? The ImageResizer library is widely used and well tested.
If the orders are visited frequently by customers, it is better to create the thumbnails ones and store on disk. this way the webserver doesn't need to process the page that long. It will speed up the loading time of your webpages.
It depends on your load. If the resource is being requested multiple times then it makes sense to cache it.
Will there always have to be an image? If not, you can create it on the first request and then cache it either in memory, or more likely a database, for subsequent requests.
However, if you always need the n images to exists per order, and/or you have multiple orders being created regularly, you will be better off passing the thumbnail creation off to a worker thread or some kind of asynchronous page. That way, multiple request's can be stacked up, reducing load on the server.