I'm using turbolinks to simulate a "single-page" app. The app is acceptably quick once pages are cached, however the first clicks are too slow. Is there a way to pre-fill the transition cache by loading some pages in the background?
I'm going to be looking into hacking into the page cache, but wondering if anyone has done this before.
(If this seems unusual, well, just trust that I'm doing this for good reason. Turbolinks gets nearly the performance of a far more complex implementation and overall I'm quite happy with it.)
UPDATE: So it seems like this SHOULD be relatively easy by simply adding entries to the pageCache of Turbolinks, something like:
$.ajax({
url: url,
dataType: 'html',
success: function(data) {
var dom = $(data);
Turbolinks.pageCache[url] = {
...
}
}
});
however it doesn't seem possible to construct a body element in javascript, which is required. Without out that, it doesn't seem like I can't construct the object that is stored in the cache without the browser first rendering it.
Any ideas beyond hacking more into Turbolinks?
UPDATE 2: There was the further problem that pageCache is hidden by a closure, so hacking Turbolinks is necessary. I have a solution that I'm testing that leverages iFrames - seems to be working.
First Hack Turbolinks to allow access to pageCache, the end of your turbolinks.js.coffee should look like this.
#Turbolinks = {
visit,
pagesCached,
pageCache,
enableTransitionCache,
enableProgressBar,
allowLinkExtensions: Link.allowExtensions,
supported: browserSupportsTurbolinks,
EVENTS: clone(EVENTS)
}
Then implement a fetch function. This is what you were thinking about, we can use DOMParser to convert string into DOM object.
function preFetch(url) {
$.ajax({
url: url,
dataType: 'html'
}).done(function(data) {
var key = App.Helper.getCurrentDomain() + url;
var parser = new DOMParser();
var doc = parser.parseFromString(data, "text/html");
Turbolinks.pageCache[key] = {
url: url,
body: doc.body,
title: doc.title,
positionY: window.pageYOffset,
positionX: window.pageXOffset,
cachedAt: new Date().getTime(),
transitionCacheDisabled: doc.querySelector('[data-no-transition-cache]') != null
};
});
};
Usage:
$(function() {
preFetch('/link1'); // fetch link1 into transition cache
preFetch('/link2'); // fetch link2 into transition cache
});
Since the upgrade to iOS 6, we are seeing Safari's web view take the liberty of caching $.ajax calls. This is in the context of a PhoneGap application so it is using the Safari WebView. Our $.ajax calls are POST methods and we have cache set to false {cache:false}, but still this is happening. We tried manually adding a TimeStamp to the headers but it did not help.
We did more research and found that Safari is only returning cached results for web services that have a function signature that is static and does not change from call to call. For instance, imagine a function called something like:
getNewRecordID(intRecordType)
This function receives the same input parameters over and over again, but the data it returns should be different every time.
Must be in Apple's haste to make iOS 6 zip along impressively they got too happy with the cache settings. Has anyone else seen this behavior on iOS 6? If so, what exactly is causing it?
The workaround that we found was to modify the function signature to be something like this:
getNewRecordID(intRecordType, strTimestamp)
and then always pass in a TimeStamp parameter as well, and just discard that value on the server side. This works around the issue.
After a bit of investigation, turns out that Safari on iOS6 will cache POSTs that have either no Cache-Control headers or even "Cache-Control: max-age=0".
The only way I've found of preventing this caching from happening at a global level rather than having to hack random querystrings onto the end of service calls is to set "Cache-Control: no-cache".
So:
No Cache-Control or Expires headers = iOS6 Safari will cache
Cache-Control max-age=0 and an immediate Expires = iOS6 Safari will cache
Cache-Control: no-cache = iOS6 Safari will NOT cache
I suspect that Apple is taking advantage of this from the HTTP spec in section 9.5 about POST:
Responses to this method are not cacheable, unless the response
includes appropriate Cache-Control or Expires header fields. However,
the 303 (See Other) response can be used to direct the user agent to
retrieve a cacheable resource.
So in theory you can cache POST responses...who knew. But no other browser maker has ever thought it would be a good idea until now. But that does NOT account for the caching when no Cache-Control or Expires headers are set, only when there are some set. So it must be a bug.
Below is what I use in the right bit of my Apache config to target the whole of my API because as it happens I don't actually want to cache anything, even gets. What I don't know is how to set this just for POSTs.
Header set Cache-Control "no-cache"
Update: Just noticed that I didn't point out that it is only when the POST is the same, so change any of the POST data or URL and you're fine. So you can as mentioned elsewhere just add some random data to the URL or a bit of POST data.
Update: You can limit the "no-cache" just to POSTs if you wish like this in Apache:
SetEnvIf Request_Method "POST" IS_POST
Header set Cache-Control "no-cache" env=IS_POST
I hope this can be of use to other developers banging their head against the wall on this one. I found that any of the following prevents Safari on iOS 6 from caching the POST response:
adding [cache-control: no-cache] in the request headers
adding a variable URL parameter such as the current time
adding [pragma: no-cache] in the response headers
adding [cache-control: no-cache] in the response headers
My solution was the following in my Javascript (all my AJAX requests are POST).
$.ajaxSetup({
type: 'POST',
headers: { "cache-control": "no-cache" }
});
I also add the [pragma: no-cache] header to many of my server responses.
If you use the above solution be aware that any $.ajax() calls you make that are set to global: false will NOT use the settings specified in $.ajaxSetup(), so you will need to add the headers in again.
Simple solution for all your web service requests, assuming you're using jQuery:
$.ajaxPrefilter(function (options, originalOptions, jqXHR) {
// you can use originalOptions.type || options.type to restrict specific type of requests
options.data = jQuery.param($.extend(originalOptions.data||{}, {
timeStamp: new Date().getTime()
}));
});
Read more about the jQuery prefilter call here.
If you aren't using jQuery, check the docs for your library of choice. They may have similar functionality.
I just had this issue as well in a PhoneGap application. I solved it by using the JavaScript function getTime() in the following manner:
var currentTime = new Date();
var n = currentTime.getTime();
postUrl = "http://www.example.com/test.php?nocache="+n;
$.post(postUrl, callbackFunction);
I wasted a few hours figuring this out. It would have been nice of Apple to notify developers of this caching issue.
I had the same problem with a webapp getting data from ASP.NET webservice
This worked for me:
public WebService()
{
HttpContext.Current.Response.Cache.SetCacheability(HttpCacheability.NoCache);
...
}
Finally, I've a solution to my uploading problem.
In JavaScript:
var xhr = new XMLHttpRequest();
xhr.open("post", 'uploader.php', true);
xhr.setRequestHeader("pragma", "no-cache");
In PHP:
header('cache-control: no-cache');
From my own blog post iOS 6.0 caching Ajax POST requests:
How to fix it: There are various methods to prevent caching of requests. The recommended method is adding a no-cache header. This is how it is done.
jQuery:
Check for iOS 6.0 and set Ajax header like this:
$.ajaxSetup({ cache: false });
ZeptoJS:
Check for iOS 6.0 and set the Ajax header like this:
$.ajax({
type: 'POST',
headers : { "cache-control": "no-cache" },
url : ,
data:,
dataType : 'json',
success : function(responseText) {…}
Server side
Java:
httpResponse.setHeader("Cache-Control", "no-cache, no-store, must-revalidate");
Make sure to add this at the top the page before any data is sent to the client.
.NET
Response.Cache.SetNoStore();
Or
Response.Cache.SetCacheability(System.Web.HttpCacheability.NoCache);
PHP
header('Cache-Control: no-cache, no-store, must-revalidate'); // HTTP 1.1.
header('Pragma: no-cache'); // HTTP 1.0.
This JavaScript snippet works great with jQuery and jQuery Mobile:
$.ajaxSetup({
cache: false,
headers: {
'Cache-Control': 'no-cache'
}
});
Just place it somewhere in your JavaScript code (after jQuery is loaded, and best before you do AJAX requests) and it should help.
You can also fix this issue by modifying the jQuery Ajax function by doing the following (as of 1.7.1) to the top of the Ajax function (function starts at line 7212). This change will activate the built-in anti-cache feature of jQuery for all POST requests.
(The full script is available at http://dl.dropbox.com/u/58016866/jquery-1.7.1.js.)
Insert below line 7221:
if (options.type === "POST") {
options.cache = false;
}
Then modify the following (starting at line ~7497).
if (!s.hasContent) {
// If data is available, append data to URL
if (s.data) {
s.url += (rquery.test(s.url) ? "&" : "?") + s.data;
// #9682: remove data so that it's not used in an eventual retry
delete s.data;
}
// Get ifModifiedKey before adding the anti-cache parameter
ifModifiedKey = s.url;
// Add anti-cache in URL if needed
if (s.cache === false) {
var ts = jQuery.now(),
// Try replacing _= if it is there
ret = s.url.replace(rts, "$1_=" + ts);
// If nothing was replaced, add timestamp to the end.
s.url = ret + ((ret === s.url) ? (rquery.test(s.url) ? "&" : "?") + "_=" + ts : "");
}
}
To:
// More options handling for requests with no content
if (!s.hasContent) {
// If data is available, append data to URL
if (s.data) {
s.url += (rquery.test(s.url) ? "&" : "?") + s.data;
// #9682: remove data so that it's not used in an eventual retry
delete s.data;
}
// Get ifModifiedKey before adding the anti-cache parameter
ifModifiedKey = s.url;
}
// Add anti-cache in URL if needed
if (s.cache === false) {
var ts = jQuery.now(),
// Try replacing _= if it is there
ret = s.url.replace(rts, "$1_=" + ts);
// If nothing was replaced, add timestamp to the end.
s.url = ret + ((ret === s.url) ? (rquery.test(s.url) ? "&" : "?") + "_=" + ts : "");
}
A quick work-around for GWT-RPC services is to add this to all the remote methods:
getThreadLocalResponse().setHeader("Cache-Control", "no-cache");
This is an update of Baz1nga's answer. Since options.data is not an object but a string I just resorted to concatenating the timestamp:
$.ajaxPrefilter(function (options, originalOptions, jqXHR) {
if (originalOptions.type == "post" || options.type == "post") {
if (options.data && options.data.length)
options.data += "&";
else
options.data = "";
options.data += "timeStamp=" + new Date().getTime();
}
});
In order to resolve this issue for WebApps added to the home screen, both of the top voted workarounds need to be followed. Caching needs to be turned off on the webserver to prevent new requests from being cached going forward and some random input needs to be added to every post request in order for requests that have already been cached to go through. Please refer to my post:
iOS6 - Is there a way to clear cached ajax POST requests for webapp added to home screen?
WARNING: to anyone who implemented a workaround by adding a timestamp to their requests without turning off caching on the server. If your app is added to the home screen, EVERY post response will now be cached, clearing safari cache doesn't clear it and it doesn't seem to expire. Unless someone has a way to clear it, this looks like a potential memory leak!
Things that DID NOT WORK for me with an iPad 4/iOS 6:
My request containing: Cache-Control:no-cache
//asp.net's:
HttpContext.Current.Response.Cache.SetCacheability(HttpCacheability.NoCache)
Adding cache: false to my jQuery ajax call
$.ajax(
{
url: postUrl,
type: "POST",
cache: false,
...
Only this did the trick:
var currentTime = new Date();
var n = currentTime.getTime();
postUrl = "http://www.example.com/test.php?nocache="+n;
$.post(postUrl, callbackFunction);
That's the work around for GWT-RPC
class AuthenticatingRequestBuilder extends RpcRequestBuilder
{
#Override
protected RequestBuilder doCreate(String serviceEntryPoint)
{
RequestBuilder requestBuilder = super.doCreate(serviceEntryPoint);
requestBuilder.setHeader("Cache-Control", "no-cache");
return requestBuilder;
}
}
AuthenticatingRequestBuilder builder = new AuthenticatingRequestBuilder();
((ServiceDefTarget)myService).setRpcRequestBuilder(builder);
My workaround in ASP.NET (pagemethods, webservice, etc.)
protected void Application_BeginRequest(object sender, EventArgs e)
{
Response.Cache.SetCacheability(HttpCacheability.NoCache);
}
While adding cache-buster parameters to make the request look different seems like a solid solution, I would advise against it, as it would hurt any application that relies on actual caching taking place. Making the APIs output the correct headers is the best possible solution, even if that's slightly more difficult than adding cache busters to the callers.
For those that use Struts 1, here is how I fixed the issue.
web.xml
<filter>
<filter-name>SetCacheControl</filter-name>
<filter-class>com.example.struts.filters.CacheControlFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>SetCacheControl</filter-name>
<url-pattern>*.do</url-pattern>
<http-method>POST</http-method>
</filter-mapping>
com.example.struts.filters.CacheControlFilter.js
package com.example.struts.filters;
import java.io.IOException;
import java.util.Date;
import javax.servlet.*;
import javax.servlet.http.HttpServletResponse;
public class CacheControlFilter implements Filter {
public void doFilter(ServletRequest request, ServletResponse response,
FilterChain chain) throws IOException, ServletException {
HttpServletResponse resp = (HttpServletResponse) response;
resp.setHeader("Expires", "Mon, 18 Jun 1973 18:00:00 GMT");
resp.setHeader("Last-Modified", new Date().toString());
resp.setHeader("Cache-Control", "no-store, no-cache, must-revalidate, max-age=0, post-check=0, pre-check=0");
resp.setHeader("Pragma", "no-cache");
chain.doFilter(request, response);
}
public void init(FilterConfig filterConfig) throws ServletException {
}
public void destroy() {
}
}
I was able to fix my problem by using a combination of $.ajaxSetup and appending a timestamp to the url of my post (not to the post parameters/body). This based on the recommendations of previous answers
$(document).ready(function(){
$.ajaxSetup({ type:'POST', headers: {"cache-control","no-cache"}});
$('#myForm').submit(function() {
var data = $('#myForm').serialize();
var now = new Date();
var n = now.getTime();
$.ajax({
type: 'POST',
url: 'myendpoint.cfc?method=login&time='+n,
data: data,
success: function(results){
if(results.success) {
window.location = 'app.cfm';
} else {
console.log(results);
alert('login failed');
}
}
});
});
});
I think you have already resolved your issue, but let me share an idea about web caching.
It is true you can add many headers in each language you use, server side, client side, and you can use many other tricks to avoid web caching, but always think that you can never know from where the client are connecting to your server, you never know if he are using a Hotel “Hot-Spot” connection that uses Squid or other caching products.
If the users are using proxy to hide his real position, etc… the real only way to avoid caching is the timestamp in the request also if is unused.
For example:
/ajax_helper.php?ts=3211321456
Then every cache manager you have to pass didnt find the same URL in the cache repository and go re-download the page content.
Depending on the app you can trouble shoot the issue now in iOS 6 using Safari>Advanced>Web Inspector so that is helpful with this situation.
Connect the phone to Safari on a Mac an then use the developer menu to trouble shoot the web app.
Clear the website data on the iPhone after update to iOS6, including specific to the app using a Web View. Only one app had an issue and this solved it during IOS6 Beta testing way back, since then no real problems.
You may need to look at your app as well, check out NSURLCache if in a WebView in a custom app.
https://developer.apple.com/library/ios/#documentation/Cocoa/Reference/Foundation/Classes/NSURLCache_Class/Reference/Reference.html#//apple_ref/doc/uid/TP40003754
I guess depending on the true nature of your problem, implementation, etc. ..
Ref: $.ajax calls
I found one workaround that makes me curious as to why it works. Before reading Tadej's answer concerning ASP.NET web service, I was trying to come up with something that would work.
And I'm not saying that it's a good solution, but I just wanted to document it here.
main page: includes a JavaScript function, checkStatus(). The method calls another method which uses a jQuery AJAX call to update the html content. I used setInterval to call checkStatus(). Of course, I ran into the caching problem.
Solution: use another page to call the update.
On the main page, I set a boolean variable, runUpdate, and added the following to the body tag:
<iframe src="helper.html" style="display: none; visibility: hidden;"></iframe>
In the of helper.html:
<meta http-equiv="refresh" content="5">
<script type="text/javascript">
if (parent.runUpdate) { parent.checkStatus(); }
</script>
So, if checkStatus() is called from the main page, I get the cached content. If I call checkStatus from the child page, I get updated content.
While my login and signup pages works like a charm in Firefox, IE and Chrome... I've been struggling with this issue in Safari for IOS and OSX, few months ago I found a workaround on the SO.
<body onunload="">
OR via javascript
<script type="text/javascript">
window.onunload = function(e){
e.preventDefault();
return;
};
</script>
This is kinda ugly thing but works for a while.
I don't know why, but returning null to the onunload event the page do not get cached in Safari.
We found that older iPhones and iPads, running iOS versions 9 & 10, occasionally return bogus blank AJAX results, perhaps due to Apple's turning down CPU speed. When returning the blank result, iOS does not call the server, as if returning a result from cache. Frequency varies widely, from roughly 10% to 30% of AJAX calls return blank.
The solution is hard to believe. Just wait 1s and call again. In our testing, only one repeat was all that was ever needed, but we wrote the code to call up to 4 times. We're not sure if the 1s wait is required, but we didn't want to risk burdening our server with bursts of repeated calls.
We found the problem happened with two different AJAX calls, calling on different API files with different data. But I'm concerned it could happen on any AJAX call. We just don't know because we don't inspect every AJAX result and we don't test every call multiple times on old devices.
Both problem AJAX calls were using: POST, Asynchronously = true, setRequestHeader = ('Content-Type', 'application/x-www-form-urlencoded')
When the problem happens, there's usually only one AJAX call going on. So it's not due to overlapping AJAX calls. Sometimes the problem happens when the device is busy, but sometimes not, and without DevTools we don't really know what's happening at the time.
iOS 13 doesn't do this, nor Chrome or Firefox. We don't have any test devices running iOS 11 or 12. Perhaps someone else could test those?
I'm noting this here because this question is the top Google result when searching for this problem.
It worked with ASP.NET only after adding the pragma:no-cache header in IIS. Cache-Control: no-cache was not enough.
I suggest a workaround to modify the function signature to be something like this:
getNewRecordID(intRecordType, strTimestamp)
and then always pass in a TimeStamp parameter as well, and just discard that value on the server side. This works around the issue.
I want to prefix URLs which match my patterns. When I open a new tab in Firefox and enter a matching URL the page should not be loaded normally, the URL should first be modified and then loading the page should start.
Is it possible to modify an URL through a Mozilla Firefox Addon before the page starts loading?
Browsing the HTTPS Everywhere add-on suggests the following steps:
Register an observer for the "http-on-modify-request" observer topic with nsIObserverService
Proceed if the subject of your observer notification is an instance of nsIHttpChannel and subject.URI.spec (the URL) matches your criteria
Create a new nsIStandardURL
Create a new nsIHttpChannel
Replace the old channel with the new. The code for doing this in HTTPS Everywhere is quite dense and probably much more than you need. I'd suggest starting with chrome/content/IOUtils.js.
Note that you should register a single "http-on-modify-request" observer for your entire application, which means you should put it in an XPCOM component (see HTTPS Everywhere for an example).
The following articles do not solve your problem directly, but they do contain a lot of sample code that you might find helpful:
https://developer.mozilla.org/en/Setting_HTTP_request_headers
https://developer.mozilla.org/en/XUL_School/Intercepting_Page_Loads
Thanks to Iwburk, I have been able to do this.
We can do this my overriding the nsiHttpChannel with a new one, doing this is slightly complicated but luckily the add-on https-everywhere implements this to force a https connection.
https-everywhere's source code is available here
Most of the code needed for this is in the files
IO Util.js
ChannelReplacement.js
We can work with the above files alone provided we have the basic variables like Cc,Ci set up and the function xpcom_generateQI defined.
var httpRequestObserver =
{
observe: function(subject, topic, data) {
if (topic == "http-on-modify-request") {
var httpChannel = subject.QueryInterface(Components.interfaces.nsIHttpChannel);
var requestURL = subject.URI.spec;
if(isToBeReplaced(requestURL)) {
var newURL = getURL(requestURL);
ChannelReplacement.runWhenPending(subject, function() {
var cr = new ChannelReplacement(subject, ch);
cr.replace(true,null);
cr.open();
});
}
}
},
get observerService() {
return Components.classes["#mozilla.org/observer-service;1"]
.getService(Components.interfaces.nsIObserverService);
},
register: function() {
this.observerService.addObserver(this, "http-on-modify-request", false);
},
unregister: function() {
this.observerService.removeObserver(this, "http-on-modify-request");
}
};
httpRequestObserver.register();
The code will replace the request not redirect.
While I have tested the above code well enough, I am not sure about its implementation. As far I can make out, it copies all the attributes of the requested channel and sets them to the channel to be overridden. After which somehow the output requested by original request is supplied using the new channel.
P.S. I had seen a SO post in which this approach was suggested.
You could listen for the page load event or maybe the DOMContentLoaded event instead. Or you can make an nsIURIContentListener but that's probably more complicated.
Is it possible to modify an URL through a Mozilla Firefox Addon before the page starts loading?
YES it is possible.
Use page-mod of the Addon-SDK by setting contentScriptWhen: "start"
Then after completely preventing the document from getting parsed you can either
fetch a different document from the same domain and inject it in the page.
after some document.URL processing do a location.replace() call
Here is an example of doing 1. https://stackoverflow.com/a/36097573/6085033