I have been experimenting with workbox service worker toolkit.
The copy here suggests that the workbox-backround-sync QueuePlugin will catch failed requests and later replay them back. So with a bit of a digging I discovered how to get this working on POST method calls.
But the replaying of requests seems a bit hit and miss. Using Chrome dev tools to take the app offline, does indeed place the requests in a queue within IndexedDB. Nothing seemed to be happening and I wanted to debug. I noted that in the demo on github it uses a BroadcastChannel to trigger the replay. So I implemented that. Then from the console in the active tab, I unchecked offline in dev tools and I sent a postMessage. Now the requests fired off happily and the responses were stored for each item in the queue.
I then left the machine running, go into standby etc for a couple of hours. I then noted in the logs that it had happily been sending duplicate requests even though every item had a response.
At this point I try and replicate but can't. I send a BroadcastChannel message and it only fires of requests without responses. So I now go back to the start, and make more offline requests, confirm they are in the DB, restore online and wait. Nothing.
I see that in the constructor of Queue it creates a new RequestManager and in the constructor of that it sets up the sync event listener. So I am guessing that it should be doing something at some point.
I just want to be sure I understand the expected behaviour if someone can help.
Here is the code I have in place.
let bgQueue = new workbox.backgroundSync.QueuePlugin({
callbacks: {
onResponse: async(hash, res) => {
console.log(hash, res);
}
}
});
const requestWrapper = new workbox.runtimeCaching.RequestWrapper({
plugins: [bgQueue]
});
const route = new workbox.routing.RegExpRoute({
regExp: new RegExp('^http://localhost:4200/api/create/note'),
handler: new workbox.runtimeCaching.NetworkOnly({requestWrapper}),
method: 'POST' // Not obvious you need to do this in the docs without digging.
});
const router = new workbox.routing.Router();
router.registerRoute({route});
// Code to manaully trigger a replay.
const replayBroadcastChannel = new BroadcastChannel('replay_channel');
replayBroadcastChannel.onmessage = function() {
bgQueue.replayRequests();
};
Testing in Chrome 58 on macOS Sierra.
I guess flipping offline -> online -> offline via dev tools wont trigger sync for you(neither will "sync" option in this case), so I am afraid you will have to toggle network on->off from your system.
Upon doing so, you should be able to see the requests getting replayed just as you thought.
Also if you are still able to reproduce the duplicate requests(requests which have been responded with 2XX response) please feel free to open a bug.
Related
I'm playing with the service worker API in my computer so I can grasp how can I benefit from it in my real world apps.
I came across a weird situation where I registered a service worker which intercepts fetch event so it can check its cache for requested content before sending a request to the origin.
The problem is that this code has an error which prevented the function from making the request, so my page is left blank; nothing happens.
As the service worker has been registered, the second time I load the page it intercepts the very first request (the one which loads the HTML). Because I have this bug, that fetch event fails, it never requests the HTML and all I see its a blank page.
In this situation, the only way I know to remove the bad service worker script is through chrome://serviceworker-internals/ console.
If this error gets to a live website, which is the best way to solve it?
Thanks!
I wanted to expand on some of the other answers here, and approach this from the point of view of "what strategies can I use when rolling out a service worker to production to ensure that I can make any needed changes"? Those changes might include fixing any minor bugs that you discover in production, or it might (but hopefully doesn't) include neutralizing the service worker due to an insurmountable bug—a so called "kill switch".
For the purposes of this answer, let's assume you call
navigator.serviceWorker.register('service-worker.js');
on your pages, meaning your service worker JavaScript resource is service-worker.js. (See below if you're not sure the exact service worker URL that was used—perhaps because you added a hash or versioning info to the service worker script.)
The question boils down to how you go about resolving the initial issue in your service-worker.js code. If it's a small bug fix, then you can obviously just make the change and redeploy your service-worker.js to your hosting environment. If there's no obvious bug fix, and you don't want to leave your users running the buggy service worker code while you take the time to work out a solution, it's a good idea to keep a simple, no-op service-worker.js handy, like the following:
// A simple, no-op service worker that takes immediate control.
self.addEventListener('install', () => {
// Skip over the "waiting" lifecycle state, to ensure that our
// new service worker is activated immediately, even if there's
// another tab open controlled by our older service worker code.
self.skipWaiting();
});
/*
self.addEventListener('activate', () => {
// Optional: Get a list of all the current open windows/tabs under
// our service worker's control, and force them to reload.
// This can "unbreak" any open windows/tabs as soon as the new
// service worker activates, rather than users having to manually reload.
self.clients.matchAll({type: 'window'}).then(windowClients => {
windowClients.forEach(windowClient => {
windowClient.navigate(windowClient.url);
});
});
});
*/
That should be all your no-op service-worker.js needs to contain. Because there's no fetch handler registered, all navigation and resource requests from controlled pages will end up going directly against the network, effectively giving you the same behavior you'd get without if there were no service worker at all.
Additional steps
It's possible to go further, and forcibly delete everything stored using the Cache Storage API, or to explicitly unregister the service worker entirely. For most common cases, that's probably going to be overkill, and following the above recommendations should be sufficient to get you in a state where your current users get the expected behavior, and you're ready to redeploy updates once you've fixed your bugs. There is some degree of overhead involved with starting up even a no-op service worker, so you can go the route of unregistering the service worker if you have no plans to redeploy meaningful service worker code.
If you're already in a situation in which you're serving service-worker.js with HTTP caching directives giving it a lifetime that's longer than your users can wait for, keep in mind that a Shift + Reload on desktop browsers will force the page to reload outside of service worker control. Not every user will know how to do this, and it's not possible on mobile devices, though. So don't rely on Shift + Reload as a viable rollback plan.
What if you don't know the service worker URL?
The information above assumes that you know what the service worker URL is—service-worker.js, sw.js, or something else that's effectively constant. But what if you included some sort of versioning or hash information in your service worker script, like service-worker.abcd1234.js?
First of all, try to avoid this in the future—it's against best practices. But if you've already deployed a number of versioned service worker URLs already and you need to disable things for all users, regardless of which URL they might have registered, there is a way out.
Every time a browser makes a request for a service worker script, regardless of whether it's an initial registration or an update check, it will set an HTTP request header called Service-Worker:.
Assuming you have full control over your backend HTTP server, you can check incoming requests for the presence of this Service-Worker: header, and always respond with your no-op service worker script response, regardless of what the request URL is.
The specifics of configuring your web server to do this will vary from server to server.
The Clear-Site-Data: response header
A final note: some browsers will automatically clear out specific data and potentially unregister service workers when a special HTTP response header is returned as part of any response: Clear-Site-Data:.
Setting this header can be helpful when trying to recover from a bad service worker deployment, and kill-switch scenarios are included in the feature's specification as an example use case.
It's important to check the browser support story for Clear-Site-Data: before your rely solely on it as a kill-switch. As of July 2019, it's not supported in 100% of the browsers that support service workers, so at the moment, it's safest to use Clear-Site-Data: along with the techniques mentioned above if you're concerned about recovering from a faulty service worker in all browsers.
You can 'unregister' the service worker using javascript.
Here is an example:
if ('serviceWorker' in navigator) {
navigator.serviceWorker.getRegistrations().then(function (registrations) {
//returns installed service workers
if (registrations.length) {
for(let registration of registrations) {
registration.unregister();
}
}
});
}
That's a really nasty situation, that hopefully won't happen to you in production.
In that case, if you don't want to go through the developer tools of the different browsers, chrome://serviceworker-internals/ for blink based browsers, or about:serviceworkers (about:debugging#workers in the future) in Firefox, there are two things that come to my mind:
Use the serviceworker update mechanism. Your user agent will check if there is any change on the worker registered, will fetch it and will go through the activate phase again. So potentially you can change the serviceworker script, fix (purge caches, etc) any weird situation and continue working. The only downside is you will need to wait until the browser updates the worker that could be 1 day.
Add some kind of kill switch to your worker. Having a special url where you can point users to visit that can restore the status of your caches, etc.
I'm not sure if clearing your browser data will remove the worker, so that could be another option.
I haven't tested this, but there is an unregister() and an update() method on the ServiceWorkerRegistration object. you can get this from the navigator.serviceWorker.
navigator.serviceWorker.getRegistration('/').then(function(registration) {
registration.update();
});
update should then immediately check if there is a new serviceworker and if so install it. This bypasses the 24 hour waiting period and will download the serviceworker.js every time this javascript is encountered.
For live situations you need to alter the service worker at byte-level (put a comment on the first line, for instance) and it will be updated in the next 24 hours. You can emulate this with the chrome://serviceworker-internals/ in Chrome by clicking on Update button.
This should work even for situations when the service worker itself got cached as the step 9 of the update algorithm set a flag to bypass the service worker.
We had moved a site from godaddy.com to a regular WordPress install. Client (not us) had a serviceworker file (sw.js) cached into all their browsers which completely messed things up. Our site, a normal WordPress site, has no service workers.
It's like a virus, in that it's on every page, it does not come from our server and there is no way to get rid of it easily.
We made a new empty file called sw.js on the root of the server, then added the following to every page on the site.
<script>
if (navigator && navigator.serviceWorker && navigator.serviceWorker.getRegistration) {
navigator.serviceWorker.getRegistration('/').then(function(registration) {
if (registration) {
registration.update();
registration.unregister();
}
});
}
</script>
In case it helps someone else, I was trying to kill off service workers that were running in browsers that had hit a production site that used to register them.
I solved it by publishing a service-worker.js that contained just this:
self.globalThis.registration.unregister();
I am looking to synchronize an indexed db in the background for offline access. I want to do this while the application is online and have it run in the background where the user doesn't even know it is running
I looked at backgroundSync with service workers but that appears to be for offline usage.
What I am really looking for is something like a cron task in the browser so I can synchronize data from a remote server to a local in-browser database
Here's a different approach, which fetches the json results from the backend's API, store in localStorage and pass results array to a custom function to render it.
If localStorage if not available in the browser, it fetches the results every time the function is called... so it does when the "force" parameter is set to true.
It also uses cookies for storing the last timestamp when the data was retrieved. The given code is set for a duration of 15 minutes (900,000 milliseconds).
It also assumes that in the json result of the api there's a member called .data where's an array of data to be cached / updated.
It requires jquery for the $.ajax, but I'm sure it can be easily refactored for using fetch, axios, or any other approach:
function getTrans(force=false) {
var TRANS=undefined
if(force || window.localStorage===undefined ||
(window.localStorage!==undefined && localStorage.getItem("TRANS") === null) ||
$.cookie('L_getTrans')===undefined ||
($.cookie('L_getTrans')!==undefined && Date.now()-$.cookie('L_getTrans')>900000)) {
$.ajax({
url: 'api/',type: 'post',
data: {'OP':'getTrans'},
success: function (result) {
TRANS = result.data ?? []
if(window.localStorage!==undefined) {
localStorage.setItem('TRANS', JSON.stringify(TRANS))
$.cookie('L_getTrans',Date.now())
}
renderTransactionList(TRANS)
},
error: function (error) { console.error(error) }
});
} else {
TRANS = JSON.parse(localStorage.getItem('TRANS'))
renderTransactionList(TRANS)
}
}
Hope it helps some of you, or even amuse.
For your purpose you probably need webworker instead of service worker.
While service worker acts as a proxy for connections, web worker can be more generic.
https://www.quora.com/Whats-the-difference-between-service-workers-and-web-workers-in-JavaScript
Is has some limitation interacting with browser objects but http connections and indexed db are allowed.
Pay particular attention to browser’s cache during development: even cltr + F5 does not reload web worker code.
Force reload/prevent cache of Web Workers
I believe what you're going for is a Progressive Web Application (PWA).
To build on Claudio's answer, performing background fetches are best done with a web worker. Web workers are typically stateless, and you would need to adapt your project to note what data was loaded last. However, using History API and (lazy) loading other page contents via JavaScript means that the user wouldn't have to exit the page.
A service worker would be able to monitor when your application is online or offline, and can call methods to pause or continue downloads to the indexed db.
As a side note, it is advisable to load only what is needed by your users, as excessive background loading may offend some users.
Further Reading.
Mozilla's PWA Documentation
An example of Ajax loading and the History API from Mozilla
I looked at backgroundSync with service workers but that appears to be
for offline usage.
No it is not just for offline usage! Also Answer about PWA and service workers also right!
A solution in your case:
You can use navigator.onLine to check the internet connection like that
window.addEventListener('offline', function() {
alert('You have lost internet access!');
});
If the user loses their internet connection, they'll see an alert. Next we'll add an event listener for watching for the user to come back online.
window.addEventListener('online', function() {
if(!navigator.serviceWorker && !window.SyncManager) {
fetchData().then(function(response) {
if(response.length > 0) {
return sendData();
}
});
}
});
A good pattern to detection
if(registration.sync) {
registration.sync.register('example-sync')
.catch(function(err) {
return err;
})
} else {
if(navigator.onLine) {
requestSync();
} else {
alert("You are offline! When your internet returns, we'll finish up your request.");
}
}
Note: Maybe you need to limit your application activity while offline
I'm trying to send (push to db) data with fetch api below, but here things like HTML5 Apex or Socket or CacheAPI can also be used.
requestSync() {
navigator.serviceWorker.ready.then(swRegistration => swRegistration.sync.register('todo_updated'));
}
Maybe if you try with a socket(PHP) and a setTimeout(JS)
Let me explain to you:
When you enter your page, it uses the indexDB, and at the same moment starts a setTimeout, for exemple every 30 sec., and also tries to comunicate with the socket. If this action is successful, the web page synchronizes with the indexDB.
I don't know if you understand me.
My English is very bad. I'm sorry.
Service Worker Push Notifications
Given your reference to cron, it sounds like you want to sync a remote client at times when they are not necessarily visiting your site. Service Workers are the correct answer for running an operation in ~1 context irregardless of how many tabs the client might have open, and more specifically service workers with Push Notifications are necessary if the client might have no tabs open to the origin site at the time the sync should occur.
Resources to setup web push notifications:
There are many guides for setting up push notifications, i.e.:
https://developers.google.com/web/fundamentals/codelabs/push-notifications/
which link to the test push services like:
https://web-push-codelab.glitch.me
To test sending the push before you have configured your own server to send pushes.
Once you have your test service worker for a basic push notification, you can modify the service worker's push handler to call back to your site and do the necessary DB sync.
Example that triggers a sync into an IndexedDB via a push
Here is a pouchdb example from a few years back that was used to show pouchdb could use its plugins for http and IndexedDB from within the push handler:
self.addEventListener('push', (event) => {
if (event.data) {
let pushData = event.data.json();
if (pushData.type === 'couchDBChange') {
logDebug(`Got couchDBChange pushed, attempting to sync to: ${pushData.seq}`);
event.waitUntil(
remoteSync(pushData.seq).then((resp) => {
logDebug(`Response from sync ${JSON.stringify(resp, null, 2)}`);
})
);
} else {
logDebug('Unknown push event has data and here it is: ', pushData);
}
}
});
Here:
Pouchdb inside a service worker is receiving only a reference to a sync point in the push itself.
It now has a background browser context with an origin of the server that registered the service worker.
Therefore, it can use its http sync wrapper to contact the db provided over http by the origin server.
This is used to sync its contents which are stored in IndexedDB via its wrapper for IndexedDB
So:
Awaking on a push and contacting the server over http to sync to a reference can be done by a service worker to update an IndexedDB implementation as long as the client agreed to receive pushes and has a browser with internet connectivity.
A client that does not agree to pushes, but has service workers can also centralize/background sync operations on the service worker when tabs are visiting the site either with messages or chosen URLs to represent a cached sync results. (For a single tab SPA visitor, performance benefits as far as backgrounding are similar to web workers DB performance) but pushes, messages and fetches from the origin are all being brought together in one context which can then provide synchronization to prevent repeated work.
Safari appears to have fixed their IndexedDB in service workers bug, but some browsers may still be a bit unreliable or have unfortunate edge cases. (For example, hitting DB quotas should be tested carefully as they can cause problems when a quota is hit while in an unexpected context.)
Still, this is the only reliable way to get multiple browsers to call back and perform a sync to your server without building custom extensions, etc for each vendor separately.
document.ononline is an event available in the browser. Is there an equivalent event supported by service worker code, which does not have DOM access?
All the sample code I have seen checks network status in the course of handling a request. It would be desirable to respond to network availability immediately for the purposes of committing local updates to the server or cloud.
The best I could find in terms of documentation was https://developer.mozilla.org/en-US/docs/Web/API/ServiceWorkerGlobalScope and it lists only these events:
onactivate
onfetch
oninstall
onmessage
onnotificationclick
onnotificationclose
onpush
onpushsubscriptionchange
onsync
Of these, sync seems most like what I seek, but it depends on use of a SyncManager, and the documentation for that is fraught with warnings against use in production code.
I'm not sure whether there's an event available in the SW for that. Someone should confirm this.
You could work around this problem by having the ononline event handler logic in your page's JS which could inform the SW of the connectivity changes. This would also be an appropriate place to handle any notifications in the UI for the user.
Spesifically:
SW registers an onmessage handler
Client/page JS registers a handler for the ononline event
When connection changes, the handler notifies the SW via postMessage API
SW receives the connection change msg from the client and acts accordingly
The postMessage API is very useful and can be used to pass basically any data between the page and the SW. Note that there isn't any spesific message for my proposal, just pass something like { "new_status": "online" } etc.
This is an old question but one I'm currently interested in. What I've found is that while MDN currently says it's supported in the (shared) WorkerGlobalScope, I'm sitting on a breakpoint right now in my service worker using Chrome DevTools and the ServiceWorkerGlobalScope does not have an "ononline" property. The "navigator.onLine" indicator appears to work however. You can at least check that and return cached responses immediately.
I added event listeners for online and offline events and neither triggered, so it appears to be unsupported. I can think of plausible reasons for not supporting it, but it would be interesting to hear the real ones.
I'm developing a chat web app, so of course I need to use some kind of a "push" method to post something if an event happens in the background. I've decided to stick with long-polling for the moment, because I'm kinda new to web development and I don't have a lot of time to learn a good way to push new information to the view, and it seems to work just fine. So, now to my question: In the server side I have a method in which I have a never ending while cycle, with no sleep in it, so if anything happens (e.g. the user gets a new message) I can post it to the view in real time. With a few users, it works fine, but what will happen to the server if a lot of users start to use it? Will it crash?
Code sample:
def update() {
boolean stayInWhile = true
while(stayInWhile) {
//check for updates
if(/*update available*/) {
stayInWhile = false
//set up a response
}
}
//return response
}
As you suspect, I think the approach you're using could end up starving the container of request threads if lots of people are simultaneously using your application.
A better way to do this might be to run a javascript timer on the client browser that submits an ajax request to your update() method every few seconds. That way your request thread will be returned to the pool after each 'check'. Take a look at the javascript setTimeout method. And also a couple of stackoverflow questions about updating a progress bar using setTimeout.
Alternatively, you could set up an ajax 'push' (from server -> client) using the excellent Grails Atmosphere plugin.
I'm aware of the Chris Fulstow project log4net.signalr, it is a great idea if you want a non production log since it logs all messages from all requests. I would like to have something that discriminates log messages by the request originating them and sed back to the proper browser.
Here what I've done in the appender:
public class SignalRHubAppender:AppenderSkeleton
{
protected override void Append(log4net.Core.LoggingEvent loggingEvent)
{
if (HttpContext.Current != null)
{
var cookie = HttpContext.Current.Request.Cookies["log-id"];
if (null != cookie)
{
var formattedEvent = RenderLoggingEvent(loggingEvent);
var context = GlobalHost.ConnectionManager.GetHubContext<Log4NetHub>();
context.Clients[cookie.Value].onLog(new { Message = formattedEvent, Event = loggingEvent });
}
}
}
}
I'm trying to attach the session id to a cookie, but this does not work on the same machine because the cookie is overwritten.
here is the code I use on the client to attach the event:
//start hubs
$.connection.hub.start()
.done(function () {
console.log("hub subsystem running...");
console.log("hub connection id=" + $.connection.hub.id);
$.cookie("log-id", $.connection.hub.id);
log4netHub.listen();
});
As a result, just the last page connected shows the log messages. I would like to know if there is some strategies to have the current connection id from the browser which originate the current request, if there is any.
Also I'm interested to know if there is better design to achieve a per browser logging.
EDIT
I could made a convention name based cookie ( like log-id-someguid ), but I wonder if there is something smarter.
BOUNTY
I decided to start a bounty on that question, and I would additionally ask about the architecture, in order to see if my strategy makes sense or not.
My doubt is, I'm using the hub in a single "direction" from server to client, and I use it to log activities not originating from calls to the hub, but from other requests ( potentially requests raised on other hubs ), is that a correct approach, having as a goal a browser visible log4net appender?
The idea about how to correctly target the right browser instance/tab, even when multiple tabs are open on the same SPA, is to differentiate them through the Url. One possible way to implement that is to redirect them at the first access from http://foo.com to http://foo.com/hhd83hd8hd8dh3, randomly generated each time. That url rewriting could be done in other ways too, but it's just a way to illustrate the problem. This way the appender will be able to inspect the originating Url, and from the Url through some mapping you keep server side you can identify the right SignalR ConnectionId. The implementation details may vary, but the basic idea is this one. Tracking some more info available in the HttpContext since the first connection you could also put in place additional strategies in order to prevent any hijacking.
About your architecture, I can tell you that this is exactly the way I used it in ElmahR. I have messages originating from outside the notification hub (errors posted from other web apps), and I do a broadcast to all clients connected to that hub (and subscribing certain groups): it works fine.
I'm not an authoritative source, but I also guess that such an architecture is ok, even with multiple hubs, because hubs at the end of the day are just an abstraction over a (one) persistent connection which allows you to group messaging by contexts. Behind the scenes (I'm simplifying) you have just a persistent connection with messages going back and forth, so whatever hub structure you define on top of it (which is there just to help you organizing things) you still insist on that connection, so you cannot do any harm.
SignalR is good on doing 2 things: massive broadcast (Clients), and one-to-one communication (Caller). As long as you do not try to do weird things like building keeping server-side references to specific callers, you should be ok, whatever number of Hubs, and interactions among them, you have.
These are my conclusions, coming from the field. Maybe you can twit #dfowler about this question and see if he has (much) more authoritative guidelines.