I am trying to add some static pages to cache using standard technique in the install event of the service worker:
self.addEventListener('install',function(event) {
event.waitUntil(
caches.open(static_cache).then(function(cache){
return cache.addAll([
'/',
'index.html',
'css/styles.css',
'js/dbhelper.js',
'js/main.js',
'js/restaurant_info.js'
])
})
)
})
but when I look into cached items - I see they're cached under relative paths:
So of course when fetch event comes for the full path - there will be no match in cache. Any idea why this is happening?
I may be mistaken but I think the Dev Tools cache interface just displays the relative paths. But I believe they are cached with the full path.
If you log the cache content you should see the full path.
For example my cache interfaces shows "/", but both of these:
caches.match('https://offline-data-driven-pwa.firebaseapp.com/')
.then(res => console.log(res))
caches.match('/')
.then(res => console.log(res))
log the https://offline-data-driven-pwa.firebaseapp.com/ resource
Related
I have api's that I am caching in my app. I would like to cache the api while the service worker is installing. I came across warming the cache:
import {cacheNames} from 'workbox-core';
self.addEventListener('install', (event) => {
const urls = [/* ... */];
const cacheName = cacheNames.runtime;
event.waitUntil(caches.open(cacheName).then((cache) => cache.addAll(urls)));
});
If you use strategies configured with a custom cache name you can do the same thing; just assign your custom value to cacheName.
1) I am using custom cache names. Would I use an array for multiple cache names? ie const cacheName = [ 'foo-api', 'bar'api']?
2) The url's I use are regexp /foo/. Will those rexexp urls work here?
3) Will I be able to cache the api while the service worker is installing before the browser consumes the api?
You can add as many items to as many caches as you'd like inside of your install handler.
Workbox can use RegExps for routing incoming fetch requests to an appropriate response handler, and I assume that's what you're referring to here. The answer is no, you can't just provide a RegExp if you want to cache URLs in advance—you need to provide a complete list of URLs.
Any caching that you perform inside of an install handler is guaranteed to happen before the service worker activates, and therefore before your fetch handlers start intercepting requests. So yes, this is a way of ensuring that your caches are pre-populated.
A modification of your code could look like:
self.addEventListener('install', (event) => {
const cacheURLs = async () => {
const cache1 = await caches.open('my-first-cache');
await cache1.addAll([
'/url1',
'/url2',
]);
const cache2 = await caches.open('my-second-cache');
await cache2.addAll([
'/url3',
'/url4',
]);
};
event.waitUntil(cacheURLs());
});
I'm currently evaluating whether k6 fits our load testing needs. We have a fairly traditional website architecture that uses Apache webservers with PHP und a MySQL database. Sending simple HTTP requests with k6 looks simple enough and I think we will be able to test all major functionality with it, as we don't rely on JavaScript that much and most pages are static.
However, I'm unsure how to deal with resources (stylesheets, images, etc.) that are referenced in the HTML that is returned in the requests. We need to load them as well, as this sometimes leads to database requests, which must be part of the load test.
Is there some out-of-the-box functionality in k6 that allows you to load all the resources like a browser would? I'm aware that k6 does NOT render the page and I don't need it to. I only need to request all the resources inside the HTML.
You basically have two options, both with their caveats:
Record your session - you can either export har directly from the browser as shown there or use an extension made for your browser here is firefox and chromes. Both should be usable without a k6 cloud account you just need to set them to download the har and it will automatically (and somewhat silently) download them when you hit stop. And then either use the in k6 har converter (which is deprecated, but still works) or the new har-to-k6 one which.
This method is particularly good if you have a lot of pages and/or resources and even works if you have a single page style of application as it just gets what the browser requested as a HAR and then transforms it into a script. And if there were no dynamic things that need to be inputed (username/password) the final script can be used as is most of the time.
The biggest problem with this approach is that if you add a css file you need to redo this whole exercise. This is even more problematic if you css/js file name change on each change or something like that. Which is what the next method is good for:
Use parseHTML and then find the elements you care about and make a request for them.
import http from "k6/http";
import {parseHTML} from "k6/html";
export default function() {
const res = http.get("https://stackoverflow.com");
const doc = parseHTML(res.body);
doc.find("link").toArray().forEach(function (item) {
console.log(item.attr("href"));
// make http gets for it
// or added them to an array and make one batch request
});
}
will produce
NFO[0001] https://cdn.sstatic.net/Sites/stackoverflow/img/favicon.ico?v=4f32ecc8f43d
INFO[0001] https://cdn.sstatic.net/Sites/stackoverflow/img/apple-touch-icon.png?v=c78bd457575a
INFO[0001] https://cdn.sstatic.net/Sites/stackoverflow/img/apple-touch-icon.png?v=c78bd457575a
INFO[0001] /opensearch.xml
INFO[0001] https://cdn.sstatic.net/Shared/stacks.css?v=53507c7c6e93
INFO[0001] https://cdn.sstatic.net/Sites/stackoverflow/primary.css?v=d3fa9a72fd53
INFO[0001] https://cdn.sstatic.net/Shared/Product/product.css?v=c9b2e1772562
INFO[0001] /feeds
INFO[0001] https://cdn.sstatic.net/Shared/Channels/channels.css?v=f9809e9ffa90
As you can see some of the urls are relative and not absolute so you will need to handle this. And in this example only some are css, so probably more filtering is needed.
The problem here is that you need to write the code and if you add a relative link or something else you need to handle it. Luckily k6 is scriptable so you can reuse the code :D.
I've followed Михаил Стойков suggestion and written my own function to load resources. You can set the way resources are loaded (batch or sequential gets with options.concurrentResourceLoading).
/**
* #param {http.RefinedResponse<http.ResponseType>} response
*/
export function getResources(response) {
const resources = [];
response
.html()
.find('*[href]:not(a)')
.each((index, element) => {
resources.push(element.attributes().href.value);
});
response
.html()
.find('*[src]:not(a)')
.each((index, element) => {
resources.push(element.attributes().src.value);
});
if (options.concurrentResourceLoading) {
const responses = http.batch(
resources.map((r) => {
return ['GET', resolveUrl(r, response.url), null, {
headers: createHeader()
}];
})
);
responses.forEach(() => {
check(response, {
'resource returns status 200': (r) => r.status === 200,
});
});
} else {
resources.forEach((r) => {
const res = http.get(resolveUrl(r, response.url), {
headers: createHeader(),
});
!check(res, {
'resource returns status 200': (r) => r.status === 200,
});
});
}
}
Usually whenever I read a blog post about PWA's, the tutorial seems to just precache every single asset. But this seems to go against the app shell pattern a bit, which as I understand is: Cache the bare necessities (only the app shell), and runtime cache as you go. (Please correct me if I understood this incorrectly)
Imagine I have this single page application, it's a simple index.html with a web component: <my-app>. That <my-app> component sets up some routes which looks a little bit like this, I'm using Vaadin router and web components, but I imagine the problem would be the same using React with React Router or something similar.
router.setRoutes([
{
path: '/',
component: 'app-main', // statically loaded
},
{
path: '/posts',
component: 'app-posts',
action: () => { import('./app-posts.js');} // dynamically loaded
},
/* many, many, many more routes */
{
path: '/offline', // redirect here when a resource is not cached and failed to get from network
component: 'app-offline', // also statically loaded
}
]);
My app may have many many routes, and may get very large. I don't want to precache all those resources straight away, but only cache the stuff I absolutely need, so in this case: my index.html, my-app.js, app-main.js, and app-offline.js. I want to cache app-posts.js at runtime, when it's requested.
Setting up runtime caching is simple enough, but my problem arises when my user visits one of the potentially many many routes that is not cached yet (because maybe the user hasn't visited that route before, so the js file may not have loaded/cached yet), and the user has no internet connection.
What I want to happen, in that case (when a route is not cached yet and there is no network), is for the user to be redirected to the /offline route, which is handled by my client side router. I could easily do something like: import('./app-posts.js').catch(() => /* redirect user to /offline */), but I'm wondering if there is a way to achieve this from workbox itself.
So in a nutshell:
When a js file hasn't been cached yet, and the user has no network, and so the request for the file fails: let workbox redirect the page to the /offline route.
Option 1 (not always useful):
As far as I can see and according to this answer, you cannot open a new window or change the URL of the browser from within the service worker. However you can open a new window only if the clients.openWindow() function is called from within the notificationclick event.
Option 2 (hardest):
You could use the WindowClient.navigate method within the activate event of the service worker however is a bit trickier as you still need to check if the file requested exists in the cache or not.
Option 3 (easiest & hackiest):
Otherwise, you could respond with a new Request object to the offline page:
const cacheOnly = new workbox.strategies.CacheOnly();
const networkFirst = new workbox.strategies.NetworkFirst();
workbox.routing.registerRoute(
/\/posts.|\/articles/,
async args => {
const offlineRequest = new Request('/offline.html');
try {
const response = await networkFirst.handle(args);
return response || await cacheOnly.handle({request: offlineRequest});
} catch (error) {
return await cacheOnly.handle({request: offlineRequest})
}
}
);
and then rewrite the URL of the browser in your offline.html file:
<head>
<script>
window.history.replaceState({}, 'You are offline', '/offline');
</script>
</head>
The above logic in Option 3 will respond to the requested URL by using the network first. If the network is not available will fallback to the cache and even if the request is not found in the cache, will fetch the offline.html file instead. Once the offline.html file is parsed, the browser URL will be replaced to /offline.
I followed instructions from the Google PWA tutorial here to make my own app with offline functionality. When I ran Lighthouse check on my localhost:3000, I got a report that said everything is setup fine.
Note that I had only cached my index file and svg image assets only.
self.addEventListener('install', event => {
event.waitUntil(
caches
.open('word-cloud-v1')
.then(cache => {
return cache.addAll([
'/',
'/index.html',
'./images/paper-plane.svg',
'./images/idea.svg',
'./images/desk-lamp.svg',
'./images/stopwatch.svg',
'./images/pie-chart.svg',
])
})
)
})
But when I go offline and try to run my app, I get errors that some files have not loaded.
So I go back and add some more files to cache. Note that this file is not created by me.
.then(cache => {
return cache.addAll([
'/',
'/index.html',
'./static/js/bundle.js',
'./images/paper-plane.svg',
'./images/idea.svg',
'./images/desk-lamp.svg',
'./images/stopwatch.svg',
'./images/pie-chart.svg',
])
})
Although the offline feature works fine now, I'm also seeing a bunch of other randomly generated files that are created in the build folder that I have not explicitly cached yet. So what are the files I should cache inside a service worker so they show up in offline mode?
TLDR; what files should we cache apart from /, /index.html and images so we can have offline functionality?
You can cache anything that you would like to cache. Cache data can be of two types.
1) Static data - You can cache all your html/css/js/images which are not dynamic. When you plan to cache static data, it would be easy to wild card the files like below instead of choosing individual files. Below block (angular ng service worker. Your helper class might look different) will cache everything inside "assets" folder.
{
"name": "assets",
"installMode": "lazy",
"updateMode": "prefetch",
"resources": {
"files": [
"/assets/**"
]
}
}
2) Dynamic data - You can cache dynamic data like API JSON response in IndexDB.
I am working on a PWA and I have installed and activated a service worker on my site. It works perfectly well while testing on local server but when I ship my code live, it fails.
This is my SW:
const cacheName = 'v1';
const cacheFiles = [
'/',
'/css/styles.css',
'/images/test1.png',
'/images/test2.png',
'/js/app.js',
'/js/sw-registration.js'
]
// Install event
self.addEventListener('install', function(event) {
console.log("SW installed");
event.waitUntil(
caches.open(cacheName)
.then(function(cache){
console.log('SW caching cachefiles');
return cache.addAll(cacheFiles);
})
)
});
// Activate event
self.addEventListener('activate', function(event) {
console.log("SW activated");
event.waitUntil(
caches.keys()
.then(function(cacheNames){
return Promise.all(cacheNames.map(function(thisCacheName){
if(thisCacheName !== cacheName){
console.log('SW Removing cached files from', thisCacheName);
return caches.delete(thisCacheName);
}
}))
})
)
});
// Fetch event
self.addEventListener('fetch', function(event) {
console.log("SW fetching", event.request.url);
event.respondWith(
caches.match(event.request)
.then(function(response){
console.log('Fetching new files');
return response || fetch(event.request);
})
);
});
This is the error I'm getting:
Uncaught (in promise) TypeError: Request failed (sw.js:1)
I don't understand why it fails to cache my files online (github pages) when it works locally. Can someone help me understand?
Thank you.
EDIT: I tried to deploy the site via Netlify and it works there. So it has to be something to do with Github pages. I would still like to know what it is, if anyone can shed any light.
As mentioned in Service Worker caches locally but fails online, when deploying to gh-pages, your web app's content will normally be accessed from a subpath, not in the top-level path, for the domain.
For instance, if your files are in the gh-pages branch of https://github.com/<user>/<repo>, then your web content can be accessed from https://<user>.github.io/<repo>/.
All of the URLs in your cacheFiles array are prefixed with /, which isn't what you want, given that all of your content is accessible under /<repo>/. For instance, / is interpreted as https://<user>.github.io/, which is different from https://<user>.github.io/<repo>/.
The solution to your problem, which will lead to a configuration that works regardless of what the base URL is for your hosting environment, is to prepend each of your URLs with ./ rather than /. For instance:
const cacheFiles = [
'./',
'./css/styles.css',
// etc.
];
The ./ means that the URL is relative, with the location of the service worker file used as the base. Your service worker file will be deployed under https://<user>.github.io/<repo>/, so that will end up being the correct base URL to use for the rest of your content as well.