I have a program that uploads files to a ftp server. If upload is complete then I delete the files. Occasionally the files are not deleted and because the upload is automated I get an infinite loop where 100 copies of the file reach the server. If I try to manually delete the file from explorer I'm able to do it but for some reason the Deletefile command from my app doesn't work.
I've tried raising last OSError and I get nothing.
The files are stored on a mapped drive. Is there any workaround? If I upload 30 files to a ftp server sometimes one or two files cannot be deleted after loading them to the server.
Is there any way to close the file even if it is opened from another program? Something like unlocker does?
How can i see if my application is locking the file? I haven't implemented any locking mechanism inside the app.
Thanks.
Related
I have a rails app where the user can upload files. The files get uploaded to an external cloud service by a backgroud jobs. It's vital for my app that the files won't get stored in the file system after they've been uploaded. Not right away, in general -- they must not remain in the file system.
Should I delete them on my own? Or will get deleted automatically?
Also, debugging my app, I noticied this for an attachment params:
[2] pry(#<MyController>)> my_params.tempfile.path
"/var/folders/qr/0v5z71xn7x503ykyv1j6lkp00000gn/T/RackMultipart20181007-10937-3ntmgg.png"
That file gets stored not in "/tmp" but in "/var" and that means that it won't get deleted automatically, right?
Note that I'm not using paperclip for this task.
You are right the files won't get deleted automatically.
You have to delete the file explicitly at some point in time.
It depends how you set it up. If you used Tempfile to save it then yes the files will be deleted when the object is garbage collected. If not then it probably won't be deleted.
If the files get stored on an external service it might be worth setting up ActiveStorage which allows you to directly upload to external storage providers without the file ever touching your server.
I have a lotus notes agent which takes some files from the server and does some processing and then deletes those files.
For deleting we have used the Kill command. It was working fine, but now we are getting the error "path/file access". Could anyone please help me on this.
If the files are NSF files and your code is opening them as NotesDatabase objects via the server, the files on disk will not be closed even after you are done using them and the objects are gone. That's because the server maintains a cache of open NSF files. You cannot delete the files until they are out of the cache. (This may or may not be true if you specified "" instead of the server name when you opened the NotesDatabase object. I don't recall, but if the workaround had been as easy as just opening locally using "", I think we would have done that.)
To get around this in the past, what I have done is just leave the files on disk and write another agent that runs once per day to clean them up. It's ugly, but it was the only way to deal with the problem.
I am implementing a resumable upload protocol that uploads in the background on iOS, which means I have to use an NSURLSessionUploadTask with a file. Since it's a resumable upload protocol, the file needs to be truncated based on the data that has already been received by the server, so I need to save a new temporary file to disk that has only the bytes to be uploaded within it.
If I can create that temporary upload file in the tmp/ or /Library/Caches/, can I trust that it will be kept as long as the NSURLSession is running?
EDIT: When an upload fails, the server will be saving the bytes it has already received and communicating that to the client. The client then should only send part of the file, which is why I need to create a smaller temporary file that must not be deleted mid-upload.
Huh? You provide the entire file, and the system takes care of managing the partial upload/download, and notifies you once the transfer is complete. In the case of a download, t hands you a temporary file once the download is complete and you have to save it to a permanent location.
You should not be mucking around with partial files at all.
EDIT:
You don't have access to tmp or /Library/Caches/, except through the sandbox. You can get access to the caches directory with the call
[NSSearchPathForDirectoriesInDomains(
NSCachesDirectory,
NSUserDomainMask, YES) lastObject];
It's my understanding that the caches directory only gets purged on restart, or if the device gets critically low on space, but I seem to remember that the docs are vague on when, exactly, the caches directory gets cleared.
You would probably be better off saving your file to the documents directory, then deleting it once you're done with it.
The answer to your question is no. NSURLSessionUploadTask's description appears to support keeping the source file around but it's misleading:
"In iOS, when you create an upload task for a file in a background session, the system copies that file to a temporary location and streams data from there"
But it says nothing about whether it will keep the original source file in the tmp directory. Specifically for your case where your server supports uploading partial files and you need to restart them after failures. Or in the more common situation where you need to manually restart an entire failed upload, for example from a retry-able server error, or if user killed your app and then restarted it (iOS doesn't continue uploads for user killed apps).
In these cases you can't count on the file still being around if you create it in the apps tmp directory. The file system programming guide tells us this.
https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/FileSystemOverview.html
"Use this directory to write temporary files that do not need to persist between launches of your app. Your app should remove files from this directory when they are no longer needed; however, the system may purge this directory when your app is not running. The contents of this directory are not backed up by iTunes or iCloud."
So any tmp directory files can be deleted by iOS when your app stops running, and I can confirm I've seen this in production releases of our app. If you think you may need the source file for the upload again, you must store it in your own app directory, and manage deleting it yourself when done with it. Sorry, extra work, but I don't know of any way around it.
I’m writing a program which is continously looking for new files in a directory. After it extracts data from each file and makes some treatments with it, the files are moved to another directory containing all scanned files.
Imagine I’m copying a new file in the scanned directory while my program is running. Can a file which has not finished copying be treated (and then produce unforeseen results), or is it locked by the System ?
Now, imagine two instances of the program are running on two different computers, continously scanning the same folder. What can happen if both instances are trying to move the same file ?
Thank you for your help.
I have a project that does much the same thing. Another application is receiving data from a feed and writing files to a folder. My application is processing those files by opening them, acting on them in some way, writing them to another folder, then deleting them.
The strategy I used in the application that does the processing and deleting is to simply open them like this:
TFileStream.Create(AFileName, fmOpenRead OR fmShareDenyWrite);
If the file that is being opened is still being written by another process, the above will fail, and can likely be opened successfully on a subsequent iteration.
I have a website where i upload a zip file and then the serverside decompresses it. i've since moved to amazons S3 service which does not allow such things as decompressing.
I'm wondering, is there a way to check or monitor the status of that zip file- and then run my model/method for pushing to s3? i'd like to run it immediately after it's decompressed- otherwise i'd try a cronjob or something.
The only conclusion i can think of right now is to output the files unzipped in my view. then selecting those files and submitting again to the method for uploading. but this seems cumbersome.
any thoughts on this?