Unable save large file to S3 using Parse server - ios

I tried to save to S3 bucket using Parse Server, and it can be saved correctly when the file is small, such as 864.2KB. However, when the file is large, say 5MB, it complaints with a message saying: "The data couldn’t be read because it isn’t in the correct format"
I'm using the following code to save the the video file to the S3
func saveVideo(withVideoURL url: URL){
let post = PFObject(className: "Post")
post["caption"] = "Out of the game for 6 months, but back with vengeance. Meet your 2017 AO Men's champion"
do{
let data = try Data(contentsOf: url)
print(data)
post["media"] = PFFile(data: data)
post.saveInBackground { (success, error) in
if success{
print("video saved")
}else{
print("failed")
if error != nil{
print(error!.localizedDescription)
}else{
print("erorr is nil")
}
}
}
}catch let error as NSError{
print("can't read")
print(error.localizedDescription)
}
}
Besides, even when the small video file is indeed being saved to the S3, it contains an extension .bin instead of, for example .mp4. I wonder what's happening here
The url end up looking something like this
https://s3-us-west-1.amazonaws.com/sampleApp/19d5bce20f8b55te1b1b8f370212533e5_file.bin

You need to stipulate the content type. You can do so like this:
post["media"] = PFFile(data: data, contentType: "video/mp4")

The below settings in your parse-server index file will help you:
var bodyParser = require('body-parser');
app.use(bodyParser.json({limit: '20mb'}));
app.use(bodyParser.urlencoded({limit: '20mb', extended: true}));
If you are using elastic beanstalk, you have to have a file named files.config inside the folder .ebextensions, with the below content.
files:
/etc/nginx/conf.d/proxy.conf:
content: |
client_max_body_size 20M;
This fixed the issue for me.

Related

How do you allow very large files to have time to upload to firebase before iOS terminates the task?

I have a video sharing app, and when you save a video to firebase storage it works perfectly for videos that are roughly 1 minute or shorter.
The problem that I am having, is when I try to post a longer video (1 min or greater) it never saves to firebase.
The only thing that I can think of is this error that I am getting, and this error only shows up about 30 seconds after I click the save button:
[BackgroundTask] Background Task 101 ("GTMSessionFetcher-firebasestorage.googleapis.com"), was created over 30 seconds ago. In applications running in the background, this creates a risk of termination. Remember to call UIApplication.endBackgroundTask(_:) for your task in a timely manner to avoid this.
Here is my code to save the video to firebase.
func saveMovie(path: String, file: String, url: URL) {
var backgroundTaskID: UIBackgroundTaskIdentifier?
// Perform the task on a background queue.
DispatchQueue.global().async {
// Request the task asseration and save the ID
backgroundTaskID = UIApplication.shared.beginBackgroundTask(withName: "Finish doing this task", expirationHandler: {
// End the task if time expires
UIApplication.shared.endBackgroundTask(backgroundTaskID!)
backgroundTaskID = UIBackgroundTaskIdentifier.invalid
})
// Send the data synchronously
do {
let movieData = try Data(contentsOf: url)
self.storage.child(path).child("\(file).m4v").putData(movieData)
} catch let error {
fatalError("Error saving movie in saveMovie func. \(error.localizedDescription)")
}
//End the task assertion
UIApplication.shared.endBackgroundTask(backgroundTaskID!)
backgroundTaskID = UIBackgroundTaskIdentifier.invalid
}
}
Any suggestions on how I can allow my video time to upload?
Finally figured this out after a long time...
All you have to do is use .putFile("FileURL") instead of .putdata("Data"). Firebase documentation says you should use putFile() instead of putData() when uploading large files.
But the hard part is for some reason you can't directly upload the movie URL that you get from the didFinishPickingMediaWithInfo function and firebase will just give you an error. So what I did instead was get the data of the movie, save the movie data to a path in the file manager, and use the file manager path URL to upload directly to firebase which worked for me.
//Save movie to Firestore
do {
// Convert movie to Data.
let movieData = try Data(contentsOf: movie)
// Get path so we can save movieData into fileManager and upload to firebase because movie URL does not work, but fileManager url does work.
guard let path = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first?.appendingPathComponent(postId!) else { print("Error saving to file manager in addPost func"); return }
do {
try movieData.write(to: path)
// Save the file manager url file to firebase storage
Storage.storage().reference().child("Videos").child("\(postId!).m4v").putFile(from: path, metadata: nil) { metadata, error in
if let error = error {
print("There was an error \(error.localizedDescription)")
} else {
print("Video successfully uploaded.")
}
// Delete video from filemanager because it would take up too much space to save all videos to file manager.
do {
try FileManager.default.removeItem(atPath: path.path)
} catch let error {
print("Error deleting from file manager in addPost func \(error.localizedDescription)")
}
}
} catch let error {
print("Error writing movieData to firebase \(error.localizedDescription)")
}
} catch let error {
print("There was an error adding video in addPost func \(error.localizedDescription)")
}

GCDWebServer: How do I change file permissions on server for WebDAV operations? (iOS)

I'm currently trying to use GCDWebServer to host a local webpage that's requested by a WKWebView when the app starts. I want to be able to upload external files to the server by grabbing files with a UIDocumentPickerViewController while the app is running. It seems like using a separate GCDWebDAVServer on a different port is a good idea for this.
However, if I try to upload a file to the WebDAV server, I get this data from the response:
<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><title>HTTP Error 500</title></head><body><h1>HTTP Error 500: Failed moving uploaded file to "/arbitrary.txt"</h1><h3>[NSCocoaErrorDomain] “35B890AC-7CD2-4A10-A67F-BAED3D6C34AB-3278-000005593C778876” couldn’t be moved because you don’t have permission to access “www”. (513)</h3></body></html>
Cutting out this bit for readability:
Failed moving uploaded file to "/arbitrary.txt"</h1><h3>[NSCocoaErrorDomain] “35B890AC-7CD2-4A10-A67F-BAED3D6C34AB-3278-000005593C778876” couldn’t be moved because you don’t have permission to access “www”.
www in this context is the local folder I'm serving with GCDWebServer. It has no subfolders, only files.
Here's the code I'm using in viewDidLoad:
let webContentUrl = Bundle.main.path(forResource: "www", ofType: nil)!
httpServer = GCDWebServer()
webDAVURL = "http://localhost:\(WEBDAV_PORT)/"
webDAVServer = GCDWebDAVServer(uploadDirectory: webContentUrl)
httpServer.addGETHandler(forBasePath: "/", directoryPath: webContentUrl, indexFilename: "index.html", cacheAge: 3600, allowRangeRequests: true)
httpServer.start(withPort: HTTP_PORT, bonjourName: nil)
let options: [String: Any] = [
GCDWebServerOption_Port: WEBDAV_PORT
]
do {
try webDAVServer.start(options: options)
} catch let error {
print("Could not start WebDAV server. Reason: \(error.localizedDescription)")
}
let request = URLRequest(url: URL(string: "http://localhost:\(HTTP_PORT)/")!)
webView.load(request)
And the code used for a PUT request to upload a file to the WebDAV server:
let importedFileUrl = urls.first!
do {
let data = try Data(contentsOf: importedFileUrl)
var request = URLRequest(url: URL(string: "http://localhost:\(WEBDAV_PORT)/arbitrary.txt")!)
request.httpMethod = "PUT"
let task = URLSession(configuration: .ephemeral).uploadTask(with: request, from: data) { data, response, error in
print("Data: \(String(describing: data))")
print("Response: \(String(describing: response))")
print("Error: \(String(describing: error))")
print(String(decoding: data!, as: UTF8.self))
}
task.resume()
} catch let error {
createErrorAlert(message: error.localizedDescription)
}
This doesn't have anything to do with iOS 14 local network privacy features. I tried modifying the Info.plist to include the new keys, but nothing changed. It seems like the www folder doesn't have write permissions.
Is there a feature I'm missing that lets you change file permissions in GCDWebServer, or is there a workaround? Right now the only workaround I can think of is to create a local database alongside the web servers.
The reason write permissions were not allowed is because I was serving files directly from the app bundle, which cannot be written to.
My solution was to copy the contents of that directory into the Documents directory and use WebDAV to write to that directory instead.

Swift File Download Issue

I am trying to download a plist file from a remote location and use it in the iOS app I am creating. The file is going to be used for calendar details within the app's calendar. The goal is obviously that I can update the remote file instead of having to push updates to the app itself every time we need to make changes to calendar details.
I started with the code used in this example: Download File From A Remote URL
Here is my modified version:
// Create destination URL
let documentsUrl:URL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first as URL!
let destinationFileUrl = documentsUrl.appendingPathComponent("2017.plist")
//let destinationFileUrl = URL(string: Bundle.main.path(forResource: String(currentYear), ofType: "plist")!)
//Create URL to the source file you want to download
let fileURL = URL(string: "https://drive.google.com/open?id=0BwHDQFwaL9DuLThNYWwtQ1VXblk")
let sessionConfig = URLSessionConfiguration.default
let session = URLSession(configuration: sessionConfig)
let request = URLRequest(url:fileURL!)
let task = session.downloadTask(with: request) { (tempLocalUrl, response, error) in
if let tempLocalUrl = tempLocalUrl, error == nil {
// Success
if let statusCode = (response as? HTTPURLResponse)?.statusCode {
print("Successfully downloaded. Status code: \(statusCode)")
}
do {
try FileManager.default.removeItem(at: destinationFileUrl)
try FileManager.default.moveItem(at: tempLocalUrl, to: destinationFileUrl)
print("File was replaced")
print(NSArray(contentsOf: tempLocalUrl))
//print(tempLocalUrl)
} catch (let writeError) {
print("Error creating a file \(String(describing: destinationFileUrl)) : \(writeError)")
}
} else {
print("Error took place while downloading a file. Error description: %#", error?.localizedDescription as Any);
}
}
task.resume()
I originally tried to overwrite the file that is bundled with the app to being with, that resulted in errors. So I instead tried to just save it in the app's documents folder and that removed that error. I had to make sure and remove any previous version of the file because it was giving me a file already exists error after the first run.
While it says everything is working (The outputs for both successful download and replaced file happen) when I print the contents of the array from the downloaded URL it just gives me nil.
This is my first attempt to use any kind of external resources in an app. Before I have always kept everything internal, so I am sure there is something glaringly obvious I am missing.
Update 1:
I realized I didn't have the correct URL to use to download a file from a Google drive. That line of code has been changed to:
let fileURL = URL(string: "https://drive.google.com/uc?export=download&id=0BwHDQFwaL9DuLThNYWwtQ1VXblk")
So now I actually am downloading the plist like I originally thought I was. Even removing the deletion issue mentioned in the first comment, I still can't get the downloaded file to actually replace the existing one.
Update 2:
I have reduced the actual file manipulation down to the following:
do {
try FileManager.default.replaceItemAt(destinationFileUrl, withItemAt: tempLocalUrl)
print("File was replaced")
print(NSArray(contentsOf: destinationFileUrl))
} catch (let writeError) {
print("Error creating a file \(String(describing: destinationFileUrl)) : \(writeError)")
}
} else {
print("Error took place while downloading a file. Error description: %#", error?.localizedDescription as Any);
}
After the replacement is performed the output of the file shows the correct new contents that were downloaded from the internet.
Later in the code when I try and access the file it seems to be nil in content again.
Look at your download completion code. You:
Delete the file at the destination URL (in case there was one
leftover)
MOVE the temp file to the destination URL (removing it from the temp
URL)
Try to load the file from the temp URL.
What's wrong with this picture?
You are trying to get the contents of the moved file. You already moved the file to destination url and then you are trying to get the contents of the file from temporary location.
For getting file data, Please try the following :
let fileData = try! String(contentsOf: destinationFileUrl, encoding: String.Encoding.utf8)
print(fileData)

How to edit the file in swiftyDropbox?

In my project, I uploaded all my text files to the Dropbox. Here, I need to directly edit the uploaded file in Dropbox using swift. Is it possible to edit files in Dropbox?
(OR) Is it possible to overwrite the same fileName in Dropbox?
Solution moved from #SathishKumarGurunathan's question post.
I found the answer and I will share here.
_ = client?.files.upload(path: path, mode: .overwrite, autorename: false, clientModified: nil, mute: false, input: Description).response { response, error in
if let response = response {
print("The upload response is \(response)")
} else if let error = error {
print("The upload error is \(error)")
}
}
.progress { progressData in
print("The progress Data is \(progressData)")
}

Swift: downloadTaskWithURL sometimes "succeeds" with non-nil location even though file does not exist

The downloadTaskWithURL function sometimes returns a non-nil location for a file that does not exist.
There is no file at http://192.168.0.102:3000/p/1461224691.mp4 in the test environment.
Most of the time, invoking downloadTaskWithURL on this URL results in the expected error message:
Error downloading message during network operation. Download URL:
http://192.168.0.102:3000/p/1461224691.mp4. Location: nil. Error:
Optional(Error Domain=NSURLErrorDomain Code=-1100 "The requested URL
was not found on this server." UserInfo=0x17547b640
{NSErrorFailingURLKey=http://192.168.0.102:3000/p/1461224691.mp4,
NSLocalizedDescription=The requested URL was not found on this
server.,
NSErrorFailingURLStringKey=http://192.168.0.102:3000/p/1461224691.mp4})
Occasionally, and in a non-deterministic way, downloadTaskWithURL believes the file exists and writes something to the location variable. As a result, the guard condition does not fail, and the code continues to execute ... which it should not.
The permanent file created by fileManager.moveItemAtURL(location!, toURL: fileURL) is only 1 byte, confirming that the network file never existed in the first place.
Why does downloadTaskWithURL behave like this?
func download() {
// Verify <networkURL>
guard let downloadURL = NSURL(string: networkURL) where !networkURL.isEmpty else {
print("Error downloading message: invalid download URL. URL: \(networkURL)")
return
}
// Generate filename for storing locally
let suffix = (networkURL as NSString).pathExtension
let localFilename = getUniqueFilename("." + suffix)
// Create download request
let task = NSURLSession.sharedSession().downloadTaskWithURL(downloadURL) { location, response, error in
guard location != nil && error == nil else {
print("Error downloading message during network operation. Download URL: \(downloadURL). Location: \(location). Error: \(error)")
return
}
// If here, no errors so save message to permanent location
let fileManager = NSFileManager.defaultManager()
do {
let documents = try fileManager.URLForDirectory(.DocumentDirectory, inDomain: .UserDomainMask, appropriateForURL: nil, create: false)
let fileURL = documents.URLByAppendingPathComponent(localFilename)
try fileManager.moveItemAtURL(location!, toURL: fileURL)
self.doFileDownloaded(fileURL, localFilename: localFilename)
print("Downloaded file # \(localFilename). Download URL: \(downloadURL)")
} catch {
print("Error downloading message during save operation. Network URL: \(self.networkURL). Filename: \(localFilename). Error: \(error)")
}
}
// Start download
print("Starting download # \(downloadURL)")
task.resume()
}
Just to clarify, the server is returning a 404, but the download task is returning a basically-empty file? And you're certain that the server actually returned an error code (by verifying the server logs)?
Either way, I would suggest checking the status code in the response object. If it isn't a 200, then the download task probably just downloaded the response body of an error page. Or, if the status code is 0, the connection timed out. Either way, treat it as a failure.
You might also try forcing this to all happen on a single thread and see if the nondeterminism goes away.

Resources