How can I retrieve a file using WKWebView? - ios

There is a file (CSV) that I want to download. It is behind a login screen on a website. I wanted to show a WKWebView to allow the user to log in and then have the app download the file after they had logged in.
I've tried downloading the file outside of WKWebView after the user has logged in to the website, but the session data seems to be sandboxed because it downloads an html document with the login form instead of the desired file.
I've also tried adding a WKUserScript to the WKUserContentController object, but the script doesn't get run when a non-HTML file is loaded.
Is there a way for me to access this file while allowing users to log in via the WKWebView?

Right now, WKWebView instances will ignore any of the default networking storages (NSURLCache, NSHTTPCookieStorage, NSCredentialStorage) and also the standard networking classes you can use to customize the network requests (NSURLProtocol, etc.).
So the cookies of the WKWebView instance are not stored in the standard Cookie storage of your App, and so NSURLSession/NSURLConnection which only uses the standard Cookie storage has no access to the cookies of WKWebView (and exactly this is probably the problem you have: the „login status“ is most likely stored in a cookie, but NSURLSession/NSURLConnection won’t see the cookie).
The same is the case for the cache, for the credentials etc. WKWebView has its own private storages and therefore does not play well with the standard Cocoa networking classes.
You also can’t customize the requests (add your own custom HTTP headers, modify existing headers, etc), use your own custom URL schemes etc, because also NSURLProtocol is not supported by WKWebView.
So right now WKWebView is pretty useless for many Apps, because it does not participate with the standard networking APIs of Cocoa.
I still hope that Apple will change this until iOS 8 gets released, because otherwise WKWebView will be useless for many Apps, and we are probably stick with UIWebView a little bit longer.
So send bug reports to Apple, so Apple gets to know that these issues are serious and needs to be fixed.

Have you checked the response cookies coming back from the request. You could use a delegate method like this.
- (void)webView:(WKWebView *)webView decidePolicyForNavigationResponse:(WKNavigationResponse *)navigationResponse decisionHandler:(void (^)(WKNavigationResponsePolicy))decisionHandler{
NSHTTPURLResponse *response = (NSHTTPURLResponse *)navigationResponse.response;
NSArray *cookies =[NSHTTPCookie cookiesWithResponseHeaderFields:[response allHeaderFields] forURL:response.URL];
for (NSHTTPCookie *cookie in cookies) {
[[NSHTTPCookieStorage sharedHTTPCookieStorage] setCookie:cookie];
}
decisionHandler(WKNavigationResponsePolicyAllow);
}

I've accomplished something similar to what you're trying to do:
use the web view to login as you're already doing
set a navigationDelegate on your webview
implement -webView:decidePolicyForNavigationResponse:decisionHandler in your delegate
when that delegate method is called (after the user logs in), you can inspect navigationResponse.response (cast it to NSHTTPURLResponse*) and look for a Set-Cookie header that contains the session info you'll need for authenticated sessions.
you can then download the CSV by manually specifying the Cookie header in your request with the cookies specified in the response.
Note that the delegate methods are only called for "main frame" requests. Which means that AJAX requests or inner frames will not trigger it. The whole page must refresh for this to work.
If you need to trigger behavior for AJAX requests, iframes etc you'll need to inject some javascript.

You can obtain the cookie via Javascript:
You could then use the obtained cookie to download the file manually:
webView.evaluateJavaScript("(function() { return document.cookie })()", completionHandler: { (response, error) -> Void in
let cookie = response as! String
let request = NSMutableURLRequest(URL: docURL)
request.setValue(cookie, forHTTPHeaderField: "Cookie")
NSURLSession.sharedSession().dataTaskWithRequest(request, completionHandler: { (data, response, error) in
// Your CSV file will be in the response object
}).resume()
})

I do not seem to have the session cookie when I try a request from userContentController didReceiveScriptMessage.
However, when called from decidePolicyForNavigationAction, the following code detects that I'm logged in.
let urlPath: String = "<api endpoint at url at which you are logged in>"
let url = NSURL(string: urlPath)
let session = NSURLSession.sharedSession()
let task = session.dataTaskWithURL(url!, completionHandler: {data, response, error -> Void in
let string1 = NSString(data: data, encoding: NSUTF8StringEncoding)
println(string1)
println(data)
if(error != nil) {
println("Error sending token to server")
// Print any error to the console
println(error.localizedDescription)
}
var err: NSError?
})
task.resume()

Related

iOS Swift URLSession POST request getting duplicated for slow API calls

I have a download task that work by first calling a REST API for which the server needs to generate a fairly large file which takes it several minutes to generate, as it is CPU and disk IO intensive. The client waits for the server to give a JSON response with the URL of the file it generated. The file download then starts after it gets the first result.
For the calls that generate a particularly large file, which causes the server to be very slow to respond, I am seeing duplicate requests that my code is not initiating.
Initially the someone who works on the server side told me about the duplicate requests. Then I set up a way to inspect network traffic. This was done by setting up a Mac connected to a wired network and enabling network sharing and using Proxyman to inspect the traffic from the iPhone to the API server. I see multiple instances of the same API request on the network layer but my code was never notified.
Code looks like this
#objc class OfflineMapDownloadManager : NSObject, URLSessionDelegate, URLSessionDownloadDelegate {
#objc func download(){
let config = URLSessionConfiguration.background(withIdentifier: "OfflineMapDownloadSession")
config.timeoutIntervalForRequest = 500
config.shouldUseExtendedBackgroundIdleMode = true
config.sessionSendsLaunchEvents = true
urlSession = URLSession(configuration: config, delegate: self, delegateQueue: nil)
getMapUrlsFromServer(bounds)
}
func getMapUrlsFromServer(){
var urlString = "http://www.fake.com/DoMakeMap.php"
if let url = URL(string: urlString) {
let request = NSMutableURLRequest(url: url)
//...Real code sets up a JSON body in to params...
request.httpBody = params.data(using: .utf8 )
request.setValue("application/json", forHTTPHeaderField: "Content-Type")
request.httpMethod = "POST"
request.timeoutInterval = 500
urlSession?.configuration.timeoutIntervalForRequest = 500
urlSession?.configuration.timeoutIntervalForResource = 500
request.httpShouldUsePipelining = true
let backgroundTask = urlSession?.downloadTask(with: request as URLRequest)
backgroundTask?.countOfBytesClientExpectsToSend = Int64(params.lengthOfBytes(using: .utf8))
backgroundTask?.countOfBytesClientExpectsToReceive = 1000
backgroundTask?.taskDescription = "Map Url Download"
backgroundTask?.resume()
}
}
func urlSession(_ session: URLSession, downloadTask: URLSessionDownloadTask, didFinishDownloadingTo location: URL) {
if (downloadTask.taskDescription == "CTM1 Url Download") {
do {
let data = try Data(contentsOf: location, options: .mappedIfSafe)
let jsonResult = try JSONSerialization.jsonObject(with: data, options: .mutableLeaves)
if let jsonResult = jsonResult as? Dictionary<String, AnyObject> {
if let ctm1Url = jsonResult["CTM1Url"] as? String {
if let filesize = jsonResult["filesize"] as? Int {
currentDownload?.ctm1Url = URL(string: ctm1Url)
currentDownload?.ctm1FileSize = Int32(filesize)
if (Int32(filesize) == 0) {
postDownloadFailed()
} else {
startCtm1FileDownload(ctm1Url,filesize)
}
}
}
}
} catch {
postDownloadFailed()
}
}
}
There is more to this download class as it will download the actual file once the first api call is done. Since the problem happens before that code would be executed, I did not include it in the sample code.
The log from Proxyman shows that the API call went out at (minutes:seconds) 46:06, 47:13, 48:21, 49:30, 50:44, 52:06, 53:45
It looks like the request gets repeated with intervals that are just over 1 minute.
There is an API field where I can put any value and it will be echoed back to me by the server. I put a timestamp there generated with CACurrentMediaTime() and log in Proxyman shows that indeed its the same API call so there is no way my code is getting called multiple times. It seems as though the iOS networking layer is re-issuing the http request because the server is taking a very long time to respond. This ends up causing problems on the server and the API fails.
Any help would be greatly appreciated.
This sounds a lot like TCP retransmission. If the client sends a TCP segment, and the server does not acknowledge receipt within a short span of time, the client assumes the segment didn't make it to the destination, and it sends the segment again. This is a significantly lower-level mechanism than URLSession.
It's possible the HTTP server application this API is using (think Apache, IIS, LigHTTPd, nginx, etc.) is configured to acknowledge with the response data to save packeting and framing overhead. If so, and if the response data takes longer than the client's TCP retransmission timeout, you will get this behavior.
Do you have a packet capture of the connection? If not, try collecting one with tcpdump and reviewing it in Wireshark. If I'm right, you will see multiple requests, and they will all have the same sequence number.
As for how to fix it if that's the problem, I'm not sure. The server should acknowledge requests as soon as they are received.
I think the problem is in using URLSessionConfiguration.background(withIdentifier:) for this api call.
Use this method to initialize a configuration object suitable for transferring data files while the app runs in the background. A session configured with this object hands control of the transfers over to the system, which handles the transfers in a separate process. In iOS, this configuration makes it possible for transfers to continue even when the app itself is suspended or terminated.
So the problem is that the system is retrying your request unnecessarily because of this wrong API usage.
Here's what I recommend -
Use default session configuration (NOT background).
Do this api call that initiates this long job, do NOT have client wait on this job, from server side return a job_id back to client as soon as this job is initiated.
Client can now poll server every X seconds using that job_id value to know about the status of the job, even can show progress on client side if needed.
When job is completed, and client polls next time, it gets the download URL for this big file.
Download the file (using default / background session configuration as you prefer).

SFAuthenticationSession completion handler not called

I am trying to implement an approach to exchange cookies from Safari and App. I am using SFAuthenticationSession since the cookies sharing was disabled. I read through the topic and it seems this is the best solution to achieve this. There are not too many blogs or repos to use as an example.
I have implemented the changes redirect in the server side as following.
First I store the cookie as https://example.com/?cookie=12345. Then from the app I start an Authentication Session pointing to https://example.com/getcookie which redirects to customapp://dummy/cookies?cookie=12345
Once stated this. The swift implementation is the following (thanks to this):
let callbackUrl = "customapp://dummy/cookies"
let authURL = "https://example.com/getcookie"
self.authSession = SFAuthenticationSession(url: URL(string: authURL)!, callbackURLScheme: callbackUrl, completionHandler: { (callBack:URL?, error:Error? ) in
guard error == nil, let successURL = callBack else {
return
}
let cookie = self.parseQuery(url: (successURL.absoluteString), param: "cookie")
print(cookie!)
})
self.authSession?.start()
You may notice I am not interested on signing in but getting a cookie stored previously.
Can anyone please advice? My problem is that although the site is redirecting, the completion handler is not called, so I can't parse the callback url.
UPDATE
I found out I was missing the protocol in the Info.plist. After adding the custom protocol to it, the handler was called. Nevertheless, the handler was only called the second time I engaged the Authentication Session.
Any clue?

WKWebView, get all cookies

I want obtain all cookies from WKWebView. Why? I have been started a project that use web-based auth. As result, I should intercept cookies to be sure that user is logged in and for some other purposes. Another case - imagine if user logged in, and than he "kill" the app - due to some delay in storing this cookie session will be lost :(.
The problem seems to be that the cookies are cached and not saved out
to a file immediately.
(#Kemenaran from here - p.5 below)
The point where I try to catch them -
webView:decidePolicyForNavigationResponse:decisionHandler:,
func webView(webView: WKWebView, decidePolicyForNavigationResponse navigationResponse: WKNavigationResponse, decisionHandler: (WKNavigationResponsePolicy) -> Void) {
if let httpResponse = navigationResponse.response as? NSHTTPURLResponse {
if let headers = httpResponse.allHeaderFields as? [String: String], url = httpResponse.URL {
let cookies = NSHTTPCookie.cookiesWithResponseHeaderFields(headers, forURL: url {
for cookie in cookies {
NSHTTPCookieStorage.shared.set(cookie)
}
}
}
}
but not all request are navigation, so one cookie (in my case) is skipped, see details below
Few words about other option I tried...
Yes, i Know that starting from iOS 11, we can use WKHTTPCookieStore as mention here. But my project should support iOS 9+
I for 100% sure, that after 5-10 sec from login, required cookie will be saved to NSHttpCookieStorage (at least all my tests during few days confirm that)
I try to use provided observer NSHTTPCookieManagerCookiesChangedNotification, but it provide me callback only for cookies that comes within webView:decidePolicyForNavigationResponse:decisionHandler
I also try to get cookies using some JS like mentioned here and also test all suggestion from here - really great article by the way. Result - negative
I also found this radar bug, and this SO question, and Sample project, but I want to prevent even this case. (described in this post applicable not only for remove but and for save) Also this situation true and when user kill the app, so case when user login, kill app and relaunch, may be present. And preventing this (simply by checking NSHttpCookieStorage for required cookies are also not good idea, because exactly after login required cookie can be stored with some delay, so this approach requires some bool-powered solution, that looks like weird..
I also read few more SO post for some related problem, and the most usefull are
This one
Another one
One more
But still without good solution...
So, is any way exist to obtain or at least force to immediately store cookies?
I ended with simple "force-like" saving Cookie from webpage.
To get all cookie i use
stringByEvaluatingJavaScriptFromString
with JS string like document.cookie();. As result i able to receive all cookies as a string with ; separator. All i need to do - parse string, create cookie and set it to NSHttpSharedStorage

Intercept request with WKWebView

Now i'm using UIWebView and with canInitWithRequest: of NSURLProtocol i can intercept all requests and do with it what I want.
In the new WKWebView this method there isn't, and i not found something similar.
Has someone resolved this problem?
I see that after 5 years this question still generates curiosity, so I describe how I solved it and about some main problems I faced up.
As many who answered here, I have implemented WKURLSchemeHandler and used new schemes.
First of all the URL that wkwebview launches must not be HTTP (or HTTPS) but one of yours new schemes.
Example
mynewscheme://your-server-application.com
In you WKWebViewConfiguration conf, I set the handler:
[conf setURLSchemeHandler:[CustomSchemeHandler new] forURLScheme:#"mynewscheme"];
[conf setURLSchemeHandler:[CustomSchemeHandler new] forURLScheme:#"mynewschemesecure"];
In CustomSchemeHandler I have implemented webView:startURLSchemeTask: and webView:stopURLSchemeTask:.
In my case I check if the request is for a file that I just saved locally, otherwise I change actual protocol ("mynewscheme or "mynewschemesecure") with http (or https) and I make request by myself.
At this point I solved the "interception problem".
In this new way we have the webview "location" (location.href via javascript) with my new scheme and with it new problems started.
First problem is that my applications work mainly with javascript,
and document.cookie has stopped working. I'm using Cordova
framework, so I've develeped a plugin to set and get cookie to
replace document.cookie (I had to do this, because, obviously, I
have also http header set-cookie).
Second problem is that I've got a lot of "cross-origin" problems, then
I changed all my urls in relative url (or with new schemes)
Third problem is that browser automatically handle server port 80
and 443, omitting them, but has now stopped (maybe because of "not
http location"). In my server code I had to handle this.
Writing down these few rows I admit that it seems to was an easy problem to solve, but I ensure that find out a workaround, how to solve it and integrate with the infinite amount of code has been hard. Every step towards the solution corresponded to a new problem.
You can intercept requests on WKWebView since iOS 8.0 by implementing the decidePolicyFor: navigationAction: method for the WKNavigationDelegate
func webView(_ webView: WKWebView, decidePolicyFor
navigationAction: WKNavigationAction,
decisionHandler: #escaping (WKNavigationActionPolicy) -> Swift.Void) {
//link to intercept www.example.com
// navigation types: linkActivated, formSubmitted,
// backForward, reload, formResubmitted, other
if navigationAction.navigationType == .linkActivated {
if navigationAction.request.url!.absoluteString == "http://www.example.com" {
//do stuff
//this tells the webview to cancel the request
decisionHandler(.cancel)
return
}
}
//this tells the webview to allow the request
decisionHandler(.allow)
}
there are many ways to implement intercepter request.
setup a local proxy, use WKNavigationDelegate
-[ViewController webView:decidePolicyForNavigationAction:decisionHandler:]
load new request and forward to local server, and at the local server side you can do something(eg. cache).
private api. use WKBrowsingContextController and custom URLProtocol
Class cls = NSClassFromString(#"WKBrowsingContextController");
SEL sel = NSSelectorFromString(#"registerSchemeForCustomProtocol:");
if ([(id)cls respondsToSelector:sel]) {
// 把 http 和 https 请求交给 NSURLProtocol 处理
[(id)cls performSelector:sel withObject:#"http"];
[(id)cls performSelector:sel withObject:#"https"];
}
use KVO to get system http/https handler.(ios 11, *)
WKWebViewConfiguration *configuration = [[WKWebViewConfiguration alloc] init];
URLSchemeHandler *schemeHandler = [[URLSchemeHandler alloc] init];
[configuration setURLSchemeHandler:schemeHandler forURLScheme:#"test"];
NSMutableDictionary *handlers = [configuration valueForKey:#"_urlSchemeHandlers"];
handlers[#"http"] = schemeHandler;
handlers[#"https"] = schemeHandler;
all the three ways you probably need handle CORS & post bodies to be stripped,you overwrite change browser`s option request(
Preflighted_requests),you can custom http header to replace http body for post bodies to be stripped.
in iOS 11 WKWebView has come up with Custom Scheme Handler called WKURLSchemeHandler, which you can use to intercept the custom events.
for more info check out this project.
https://github.com/BKRApps/KRWebView
I know I am late but I am able to solve this problem. I can intercept each and every request even your http/https call using below trick. I can also trace the call made from html to server calls. I can also use this to render html with offline content.
Download the html of the website that we want to render in offline or online to intercept the request.
Either place the html in document directory of the user or place it inside the archive. But we should know the path of the html file.
Place all your js, cs, woff, font of our website at the same level as our base html. We need to given permission while loading the web view.
Then we have to register our own custom handler scheme with WKWebView. When wkwebview see the pattern "myhandler-webview" then it will give you control and you will get the callback to 'func webView(_ webView: WKWebView, start urlSchemeTask: WKURLSchemeTask)' delegate implementation. You can play around with url in this delegate like mentioned in point 6.
let configuration = WKWebViewConfiguration();
configuration.setURLSchemeHandler(self, forURLScheme: "myhandler-webview");
webView = WKWebView(frame: view.bounds, configuration: configuration);
Convert file scheme to the custom scheme (myhandler-webview) then load it with WKWebView
let htmlPath = Bundle.main.path(forResource: "index", ofType: "html")
var htmlURL = URL(fileURLWithPath: htmlPath!, isDirectory: false)
htmlURL = self.changeURLScheme(newScheme: "myhandler-webview", forURL: htmlURL)
self.webView.load(URLRequest(url: htmlURL))
Implement below methods of WKURLSchemeHandler protocol and handle didReceiveResponse, didReceiveData, didFinish delegate methods of WKURLSchemeTask.
func webView(_ webView: WKWebView, start urlSchemeTask: WKURLSchemeTask) {
print("Function: \(#function), line: \(#line)")
print("==> \(urlSchemeTask.request.url?.absoluteString ?? "")\n")
// You can find the url pattern by using urlSchemeTask.request.url. and create NSData from your local resource and send the data using 3 delegate method like done below.
// You can also call server api from this native code and return the data to the task.
// You can also cache the data coming from server and use it during offline access of this html.
// When you are returning html the the mime type should be 'text/html'. When you are trying to return Json data then we should change the mime type to 'application/json'.
// For returning json data you need to return NSHTTPURLResponse which has base classs of NSURLResponse with status code 200.
// Handle WKURLSchemeTask delegate methods
let url = changeURLScheme(newScheme: "file", forURL: urlSchemeTask.request.url!)
do {
let data = try Data(contentsOf: url)
urlSchemeTask.didReceive(URLResponse(url: urlSchemeTask.request.url!, mimeType: "text/html", expectedContentLength: data.count, textEncodingName: nil))
urlSchemeTask.didReceive(data)
urlSchemeTask.didFinish()
} catch {
print("Unexpected error when get data from URL: \(url)")
}
}
func webView(_ webView: WKWebView, stop urlSchemeTask: WKURLSchemeTask) {
print("Function: \(#function), line: \(#line)")
print("==> \(urlSchemeTask.request.url?.absoluteString ?? "")\n")
}
Let me know if this explanation is not enough.
Objective c example mentioned below
intercepting request with wkwebview
One can use WKURLSchemeHandler to intercept each and every request to be loaded in WKWebView,
Only disadvantage is that you cannot register http or https scheme for interception,
Solution over that is,
Replace your http/https scheme of url with custom scheme url like xyz://
for e.g. https://google.com can be loaded like xyz://google.com
Now you will get a callback in WKURLSchemeHandler there you again replace it back to https and load data programmatically and call urlSchemeTask.didReceive(response)
This way each and every https request will come to your handler.
I am blindly taking guesses since I only have my windows computer with me. By reading the Apple Developer documentation here is information I gathered that might lead to some ideas on how to solve the question.
Based on WKWebView,
Set the delegate property to an object conforming to the WKUIDelegate protocol to track the loading of web content.
Also, I see we can set our navigationDelegate with the,
weak var navigationDelegate: WKNavigationDelegate? { get set }
The methods of the WKNavigationDelegate protocol help you implement custom behaviors that are triggered during a web view's process of accepting, loading, and completing a navigation request.
Then after we create and set our custom WKNavigationDelegate, we would override some methods to intercept something we might be looking for. I found the Responding to Server Actions section of some interest since they receive a WKNavigation as parameter. Moreover, you might want to skim through WKNavigationAction and WKNavigationResponse see if there is perhaps something which might help us achieve our goal.
BTW, I am just giving some ideas on what to try so that we can solve this question, ideas which might be 100% wrong cause I have not tried them myself.

POST Queries in Swift for given website

I am trying to make queries to get the fuel type and consumption of a specified car (the user enters both make and model) for an iOS app written in Swift.
The app is targeted for Spain, and I have found a website that allows the user to enter make and model, and it returns the details for that car (http://coches.idae.es/portal/BaseDatos/MarcaModelo.aspx). I have seen using the tool WireShark, that the query is based on POST instead of GET. But I am not quite sure how I can make the requests within the app I am developing, or how to handle the info that is sent to me back from the sender.
Is there any way to make those requests to the given website? If so, I would really appreciate some help on the subject, I am new in iOS development and am looking forward to learning as much as possible.
Thanks :)
Many people prefer to use AFNetworking for making HTTP requests. However you don't need to do that. You said that its a POST request. Setting that up is easy even without AFNetworking using NSMutableURLRequest. I'm assuming you have a link to the API and not just to the aspx page. My Spanish is pretty weak so I can't look up the API reference for you but here's how you can make the request and receive data from the server. You will have to put the correct values and parse the responses:
let request = NSMutableURLRequest(URL: NSURL(string: "/* Paste URL here */")!)
request.HTTPMethod = "POST"
// Do this as many times are required for filling in the headers.
request.addValue("/* The value for the HTTP header */", forHTTPHeaderField: "/*The header field like Accept-Type, etc..*/")
// If you need an HTTP body too then make the JSONObj as a dictionary or array or whatever and then
let data = NSJSONSerialization.dataWithJSONObject(JSONObj, options: [])
request.HTTPBody = data // This needs to be NSData.
// Now make the request.
let session = NSURLSession.sharedSession()
let task = session.dataTaskWithRequest(request, { (data, response, error) -> Void in
if error == nil
{
assert(data != nil)
let JSON = NSJSONSerialization.JSONObjectWithData(data!, options: []) as? [NSObject: AnyObject]
// If you are using swift 2 this needs to be in a do try catch statement.
// TODO: Use JSON for whatever.
}
else
{
print(error!.localizedDescription)
}
}
task?.resume()
Let me know if you have any other questions or if the API doesn't use JSON or is completely different.

Resources