I have some Web Hooks setup to push data to my custom endpoint during certain events. If my service is unavailable for some reason the web hook will go offline and I need to manually go to the web page inside of TFS and re-enable it. I wasn't able to find a way to queue items or auto-retry later.
Assuming it doesn't have an option for queuing and re-trying, is there a way I can automate:
1. Check each of the web hooks to determine if they are enabled
2. Re-Enable if after a period of time
I'm using on-prem TFS V15 (from help about)
There is not any configure to never auto-disable serviceHooks when encounter errors. You may have to re-enable the disable web hooks in this case.
You could try to enable web hooks through this REST API: Update a subscription.
Put https://[account].visualstudio.com/_apis/hooks/subscriptions/[subscription id]?api-version=1.0
Body (Content-Type:application/json)
{
"publisherId": "tfs",
"eventType": "build.complete",
"resourceVersion": "1.0-preview.1",
"consumerId": "webHooks",
"consumerActionId": "httpRequest",
"scope":1,
"status":0,
"publisherInputs": {
"buildStatus": "",
"definitionName":"ClassTestVNext",
"projectId": "578ca584-4268-4ba2-b579-7aaee499c306"
},
"consumerInputs":{"url":"http://XXXX/"}
}
You could also use F12 in chrome, choose network to capture the request info when you do the re-enable action.
Related
I am serving my service worker from /worker.js and want it to intercept fetches to /localzip/*, but the fetch event is never fired.
I register it like this:
navigator.serviceWorker.register(
"worker.js",
{ scope: "/localzip/" }
);
And I claim all clients when it activates, so that I can start intercepting fetches from the current page immediately. I am sure that the service worker is activating and that clients.claim() is succeeding.
self.addEventListener("activate", (e) => {
// Important! Start processing fetches for all clients immediately.
//
// MDN: "When a service worker is initially registered, pages won't use it
// until they next load. The claim() method causes those pages to be
// controlled immediately."
e.waitUntil(clients.claim());
});
Chrome seems happy with it and the scope appears correct:
My fetch event handler is very simple:
self.addEventListener("fetch", (e) => {
console.log("Trying to make fetch happen!");
});
From my application, after the worker is active, I try to make a request, e.g.,
const response = await fetch("/localzip/lol.jpg");
The fetch does not appear to trigger the above event handler, and the browser instead tries to make a request directly to the server and logs GET http://localhost:3000/localzip/lol.jpg 404 (Not Found).
I have tried:
Making sure the latest version of my worker code is running.
Disabling / clearing caches to make sure the fetch isn't being handled by the browser's cache.
Hosting from an HTTPS server. (Chrome is supposed to support service workers on plaintext localhost for development.)
What more does it want from me?
Live demo: https://rgov.github.io/service-worker-zip-experiment/
Note that the scope is slightly different, and the fetch is performed by creating an <img> tag.
First, let's confirm you are not using hard-reload while testing your code. If you use hard-reload, all requests will not go through the service worker.
See https://web.dev/service-worker-lifecycle/#shift-reload
I also checked chrome://serviceworker-internals/ in Chrome, and your service worker has fetch handler.
Then, let's check the codes in detail.
After trying your demo page, I found a network request is handled by the service worker after clicking "Display image from zip archive" button since I can see this log:
Service Worker: Serving archive/maeby.jpg from zip archive
Then, the error is thrown:
Failed to load ‘https://rgov.github.io/localzip/maeby.jpg’. A ServiceWorker passed a promise to FetchEvent.respondWith() that rejected with ‘TypeError: db is undefined’.
This is caused by db object is not initialized properly. It would be worth confirming whether you see the DB related issue as I see in your demo. If not, my following statement might be incorrect.
I try to explain some service worker mechanism alongside my understanding of your code:
Timing of install handler
Your DB open code happens in the install handler only. This means DB object will be assigned only when the install handler is executed.
Please notice the install handler will be executed only when it's necessary. If a service worker exists already and does not need to update, the install handler won't be called. Hence, the db object in your code might not be always available.
Stop/Start Status
When the service worker does not handle events for a while (how long it would be is by browser's design), the service worker will go to stop/idle state.
When the service worker is stopped/idle (you can check the state in the devtools) and started again, the global db object will be undefined.
In my understanding, this is why I see the error TypeError: db is undefined’.
Whenever the service worker wakes up, the whole worker script will be executed. However, the execution of event handlers will depend on whether the events are coming.
How to prevent stop/idle for debugging?
Open your devtools for the page then the browser will keep it being alive.
Once you close the devtool, the service worker might go to "stop" soon.
Why does the service worker stops?
The service worker is designed for handling requests. If no request/event should be handled by a service worker, the service worker thread is not necessary to run for saving resources.
Please notice both fetch and message events (but not limited to) will awake the service worker.
See Prevent Service Worker from automatically stopping
Solution for the demo page
If the error is from the DB, this means the getFromZip function should open the DB when db is unavailable.
By the way, even without any change, your demo code works well in the following steps:
As a user, I open the demo page at the first time. (This is for ensuring that the install handler is called.)
I open the devtools ASAP once I see the page content. (This is for preventing the service worker goes to "stop" state)
I click "Download zip archive to IndexedDB" button.
I click "Display image from zip archive" button.
Then I can see the image is shown properly.
Jake Archibald pointed out that I was confused about the meaning of the service worker's scope, explained here:
We call pages, workers, and shared workers clients. Your service worker can only control clients that are in-scope. Once a client is "controlled", its fetches go through the in-scope service worker.
In other words:
The scope affects which pages (clients) get placed under the service worker's control and not which fetched URLs get intercepted.
I'm playing with the service worker API in my computer so I can grasp how can I benefit from it in my real world apps.
I came across a weird situation where I registered a service worker which intercepts fetch event so it can check its cache for requested content before sending a request to the origin.
The problem is that this code has an error which prevented the function from making the request, so my page is left blank; nothing happens.
As the service worker has been registered, the second time I load the page it intercepts the very first request (the one which loads the HTML). Because I have this bug, that fetch event fails, it never requests the HTML and all I see its a blank page.
In this situation, the only way I know to remove the bad service worker script is through chrome://serviceworker-internals/ console.
If this error gets to a live website, which is the best way to solve it?
Thanks!
I wanted to expand on some of the other answers here, and approach this from the point of view of "what strategies can I use when rolling out a service worker to production to ensure that I can make any needed changes"? Those changes might include fixing any minor bugs that you discover in production, or it might (but hopefully doesn't) include neutralizing the service worker due to an insurmountable bug—a so called "kill switch".
For the purposes of this answer, let's assume you call
navigator.serviceWorker.register('service-worker.js');
on your pages, meaning your service worker JavaScript resource is service-worker.js. (See below if you're not sure the exact service worker URL that was used—perhaps because you added a hash or versioning info to the service worker script.)
The question boils down to how you go about resolving the initial issue in your service-worker.js code. If it's a small bug fix, then you can obviously just make the change and redeploy your service-worker.js to your hosting environment. If there's no obvious bug fix, and you don't want to leave your users running the buggy service worker code while you take the time to work out a solution, it's a good idea to keep a simple, no-op service-worker.js handy, like the following:
// A simple, no-op service worker that takes immediate control.
self.addEventListener('install', () => {
// Skip over the "waiting" lifecycle state, to ensure that our
// new service worker is activated immediately, even if there's
// another tab open controlled by our older service worker code.
self.skipWaiting();
});
/*
self.addEventListener('activate', () => {
// Optional: Get a list of all the current open windows/tabs under
// our service worker's control, and force them to reload.
// This can "unbreak" any open windows/tabs as soon as the new
// service worker activates, rather than users having to manually reload.
self.clients.matchAll({type: 'window'}).then(windowClients => {
windowClients.forEach(windowClient => {
windowClient.navigate(windowClient.url);
});
});
});
*/
That should be all your no-op service-worker.js needs to contain. Because there's no fetch handler registered, all navigation and resource requests from controlled pages will end up going directly against the network, effectively giving you the same behavior you'd get without if there were no service worker at all.
Additional steps
It's possible to go further, and forcibly delete everything stored using the Cache Storage API, or to explicitly unregister the service worker entirely. For most common cases, that's probably going to be overkill, and following the above recommendations should be sufficient to get you in a state where your current users get the expected behavior, and you're ready to redeploy updates once you've fixed your bugs. There is some degree of overhead involved with starting up even a no-op service worker, so you can go the route of unregistering the service worker if you have no plans to redeploy meaningful service worker code.
If you're already in a situation in which you're serving service-worker.js with HTTP caching directives giving it a lifetime that's longer than your users can wait for, keep in mind that a Shift + Reload on desktop browsers will force the page to reload outside of service worker control. Not every user will know how to do this, and it's not possible on mobile devices, though. So don't rely on Shift + Reload as a viable rollback plan.
What if you don't know the service worker URL?
The information above assumes that you know what the service worker URL is—service-worker.js, sw.js, or something else that's effectively constant. But what if you included some sort of versioning or hash information in your service worker script, like service-worker.abcd1234.js?
First of all, try to avoid this in the future—it's against best practices. But if you've already deployed a number of versioned service worker URLs already and you need to disable things for all users, regardless of which URL they might have registered, there is a way out.
Every time a browser makes a request for a service worker script, regardless of whether it's an initial registration or an update check, it will set an HTTP request header called Service-Worker:.
Assuming you have full control over your backend HTTP server, you can check incoming requests for the presence of this Service-Worker: header, and always respond with your no-op service worker script response, regardless of what the request URL is.
The specifics of configuring your web server to do this will vary from server to server.
The Clear-Site-Data: response header
A final note: some browsers will automatically clear out specific data and potentially unregister service workers when a special HTTP response header is returned as part of any response: Clear-Site-Data:.
Setting this header can be helpful when trying to recover from a bad service worker deployment, and kill-switch scenarios are included in the feature's specification as an example use case.
It's important to check the browser support story for Clear-Site-Data: before your rely solely on it as a kill-switch. As of July 2019, it's not supported in 100% of the browsers that support service workers, so at the moment, it's safest to use Clear-Site-Data: along with the techniques mentioned above if you're concerned about recovering from a faulty service worker in all browsers.
You can 'unregister' the service worker using javascript.
Here is an example:
if ('serviceWorker' in navigator) {
navigator.serviceWorker.getRegistrations().then(function (registrations) {
//returns installed service workers
if (registrations.length) {
for(let registration of registrations) {
registration.unregister();
}
}
});
}
That's a really nasty situation, that hopefully won't happen to you in production.
In that case, if you don't want to go through the developer tools of the different browsers, chrome://serviceworker-internals/ for blink based browsers, or about:serviceworkers (about:debugging#workers in the future) in Firefox, there are two things that come to my mind:
Use the serviceworker update mechanism. Your user agent will check if there is any change on the worker registered, will fetch it and will go through the activate phase again. So potentially you can change the serviceworker script, fix (purge caches, etc) any weird situation and continue working. The only downside is you will need to wait until the browser updates the worker that could be 1 day.
Add some kind of kill switch to your worker. Having a special url where you can point users to visit that can restore the status of your caches, etc.
I'm not sure if clearing your browser data will remove the worker, so that could be another option.
I haven't tested this, but there is an unregister() and an update() method on the ServiceWorkerRegistration object. you can get this from the navigator.serviceWorker.
navigator.serviceWorker.getRegistration('/').then(function(registration) {
registration.update();
});
update should then immediately check if there is a new serviceworker and if so install it. This bypasses the 24 hour waiting period and will download the serviceworker.js every time this javascript is encountered.
For live situations you need to alter the service worker at byte-level (put a comment on the first line, for instance) and it will be updated in the next 24 hours. You can emulate this with the chrome://serviceworker-internals/ in Chrome by clicking on Update button.
This should work even for situations when the service worker itself got cached as the step 9 of the update algorithm set a flag to bypass the service worker.
We had moved a site from godaddy.com to a regular WordPress install. Client (not us) had a serviceworker file (sw.js) cached into all their browsers which completely messed things up. Our site, a normal WordPress site, has no service workers.
It's like a virus, in that it's on every page, it does not come from our server and there is no way to get rid of it easily.
We made a new empty file called sw.js on the root of the server, then added the following to every page on the site.
<script>
if (navigator && navigator.serviceWorker && navigator.serviceWorker.getRegistration) {
navigator.serviceWorker.getRegistration('/').then(function(registration) {
if (registration) {
registration.update();
registration.unregister();
}
});
}
</script>
In case it helps someone else, I was trying to kill off service workers that were running in browsers that had hit a production site that used to register them.
I solved it by publishing a service-worker.js that contained just this:
self.globalThis.registration.unregister();
I am looking to synchronize an indexed db in the background for offline access. I want to do this while the application is online and have it run in the background where the user doesn't even know it is running
I looked at backgroundSync with service workers but that appears to be for offline usage.
What I am really looking for is something like a cron task in the browser so I can synchronize data from a remote server to a local in-browser database
Here's a different approach, which fetches the json results from the backend's API, store in localStorage and pass results array to a custom function to render it.
If localStorage if not available in the browser, it fetches the results every time the function is called... so it does when the "force" parameter is set to true.
It also uses cookies for storing the last timestamp when the data was retrieved. The given code is set for a duration of 15 minutes (900,000 milliseconds).
It also assumes that in the json result of the api there's a member called .data where's an array of data to be cached / updated.
It requires jquery for the $.ajax, but I'm sure it can be easily refactored for using fetch, axios, or any other approach:
function getTrans(force=false) {
var TRANS=undefined
if(force || window.localStorage===undefined ||
(window.localStorage!==undefined && localStorage.getItem("TRANS") === null) ||
$.cookie('L_getTrans')===undefined ||
($.cookie('L_getTrans')!==undefined && Date.now()-$.cookie('L_getTrans')>900000)) {
$.ajax({
url: 'api/',type: 'post',
data: {'OP':'getTrans'},
success: function (result) {
TRANS = result.data ?? []
if(window.localStorage!==undefined) {
localStorage.setItem('TRANS', JSON.stringify(TRANS))
$.cookie('L_getTrans',Date.now())
}
renderTransactionList(TRANS)
},
error: function (error) { console.error(error) }
});
} else {
TRANS = JSON.parse(localStorage.getItem('TRANS'))
renderTransactionList(TRANS)
}
}
Hope it helps some of you, or even amuse.
For your purpose you probably need webworker instead of service worker.
While service worker acts as a proxy for connections, web worker can be more generic.
https://www.quora.com/Whats-the-difference-between-service-workers-and-web-workers-in-JavaScript
Is has some limitation interacting with browser objects but http connections and indexed db are allowed.
Pay particular attention to browser’s cache during development: even cltr + F5 does not reload web worker code.
Force reload/prevent cache of Web Workers
I believe what you're going for is a Progressive Web Application (PWA).
To build on Claudio's answer, performing background fetches are best done with a web worker. Web workers are typically stateless, and you would need to adapt your project to note what data was loaded last. However, using History API and (lazy) loading other page contents via JavaScript means that the user wouldn't have to exit the page.
A service worker would be able to monitor when your application is online or offline, and can call methods to pause or continue downloads to the indexed db.
As a side note, it is advisable to load only what is needed by your users, as excessive background loading may offend some users.
Further Reading.
Mozilla's PWA Documentation
An example of Ajax loading and the History API from Mozilla
I looked at backgroundSync with service workers but that appears to be
for offline usage.
No it is not just for offline usage! Also Answer about PWA and service workers also right!
A solution in your case:
You can use navigator.onLine to check the internet connection like that
window.addEventListener('offline', function() {
alert('You have lost internet access!');
});
If the user loses their internet connection, they'll see an alert. Next we'll add an event listener for watching for the user to come back online.
window.addEventListener('online', function() {
if(!navigator.serviceWorker && !window.SyncManager) {
fetchData().then(function(response) {
if(response.length > 0) {
return sendData();
}
});
}
});
A good pattern to detection
if(registration.sync) {
registration.sync.register('example-sync')
.catch(function(err) {
return err;
})
} else {
if(navigator.onLine) {
requestSync();
} else {
alert("You are offline! When your internet returns, we'll finish up your request.");
}
}
Note: Maybe you need to limit your application activity while offline
I'm trying to send (push to db) data with fetch api below, but here things like HTML5 Apex or Socket or CacheAPI can also be used.
requestSync() {
navigator.serviceWorker.ready.then(swRegistration => swRegistration.sync.register('todo_updated'));
}
Maybe if you try with a socket(PHP) and a setTimeout(JS)
Let me explain to you:
When you enter your page, it uses the indexDB, and at the same moment starts a setTimeout, for exemple every 30 sec., and also tries to comunicate with the socket. If this action is successful, the web page synchronizes with the indexDB.
I don't know if you understand me.
My English is very bad. I'm sorry.
Service Worker Push Notifications
Given your reference to cron, it sounds like you want to sync a remote client at times when they are not necessarily visiting your site. Service Workers are the correct answer for running an operation in ~1 context irregardless of how many tabs the client might have open, and more specifically service workers with Push Notifications are necessary if the client might have no tabs open to the origin site at the time the sync should occur.
Resources to setup web push notifications:
There are many guides for setting up push notifications, i.e.:
https://developers.google.com/web/fundamentals/codelabs/push-notifications/
which link to the test push services like:
https://web-push-codelab.glitch.me
To test sending the push before you have configured your own server to send pushes.
Once you have your test service worker for a basic push notification, you can modify the service worker's push handler to call back to your site and do the necessary DB sync.
Example that triggers a sync into an IndexedDB via a push
Here is a pouchdb example from a few years back that was used to show pouchdb could use its plugins for http and IndexedDB from within the push handler:
self.addEventListener('push', (event) => {
if (event.data) {
let pushData = event.data.json();
if (pushData.type === 'couchDBChange') {
logDebug(`Got couchDBChange pushed, attempting to sync to: ${pushData.seq}`);
event.waitUntil(
remoteSync(pushData.seq).then((resp) => {
logDebug(`Response from sync ${JSON.stringify(resp, null, 2)}`);
})
);
} else {
logDebug('Unknown push event has data and here it is: ', pushData);
}
}
});
Here:
Pouchdb inside a service worker is receiving only a reference to a sync point in the push itself.
It now has a background browser context with an origin of the server that registered the service worker.
Therefore, it can use its http sync wrapper to contact the db provided over http by the origin server.
This is used to sync its contents which are stored in IndexedDB via its wrapper for IndexedDB
So:
Awaking on a push and contacting the server over http to sync to a reference can be done by a service worker to update an IndexedDB implementation as long as the client agreed to receive pushes and has a browser with internet connectivity.
A client that does not agree to pushes, but has service workers can also centralize/background sync operations on the service worker when tabs are visiting the site either with messages or chosen URLs to represent a cached sync results. (For a single tab SPA visitor, performance benefits as far as backgrounding are similar to web workers DB performance) but pushes, messages and fetches from the origin are all being brought together in one context which can then provide synchronization to prevent repeated work.
Safari appears to have fixed their IndexedDB in service workers bug, but some browsers may still be a bit unreliable or have unfortunate edge cases. (For example, hitting DB quotas should be tested carefully as they can cause problems when a quota is hit while in an unexpected context.)
Still, this is the only reliable way to get multiple browsers to call back and perform a sync to your server without building custom extensions, etc for each vendor separately.
I have a function app with a durable task running on Azure. What is the best way of changing the 'MaxConcurrentActivityFunctions' binding (i.e. in host.json) once it's been deployed/published?
"extensions": {
"durableTask": {
"MaxConcurrentActivityFunctions": 4, // ensures scale-out for mappers
}
}
I want to change it to 1 or 4 depending on the type of service plan I use.
I haven't been able to find a way to do it with the Azure SDK, or with a rest api. The only way I can think of getting it to work is to stop the function app, download the host.json file, modify it, upload it, and restart the function app.
Seems like overkill to me. Am I missing something?
host.json uses the aspnet core config system.
You can set an app setting (environment variable) AzureFunctionsJobHost__extensions__durableTask__MaxConcurrentActivityFunctions=1 through the portal (or Azure ARM REST APIs) and it should take precedence over the value in host.json
Testing Twilio Autopilot behavior and features, I noticed the following # task_creation:
1-Action->redirect_url(fetch JSON from app Server)
I could not find a way to configure any timeout
The action redirect_url (CallbackURL) is "blocking"
assistant is in between states
cannot SAY nor LISTEN
Assistant goes to Fallback_Task after as much as 20+ seconds
Meanwhile, dialog is blank (no SAY, no LISTEN... no action)
2-Action->SAY->then->redirect_url (fetch more JSON from app Server)
The SAY action, which precedes the redirect_url action, will not "speak" until the Task has the complete picture (JSON), some of which is fetched with redirect_url
Hence, I cannot use SAY to announce: "Please wait while I lookup some... with our Systems..." because the SAY is performed ONLY after app Server responded with the JSON needed to "complete" the actions_description
Meanwhile, same as previous point #1. Blocking, no SAY, no LISTEN
My question: Is there any mechanism in place to prevent Dialog to "hang" in the event that the app Server is "hanging"? I am thinking of some sort of decoupled architecture where the Assistant configuration is self-reliant on its own and cannot be thrown-off by app Server malfunction/lag/. i.e. : some sort of timeout control on the Autopilot side.
My tests:
Test 1 Created simple Task that fetches JSON from app Server using redirect_url (HTTP CALLBACK URL) option rather than ACTIONS BIN # Task creation.
Server side, I forced a long sleep(30sec), so server keeps socket connected but lags responding with JSON that would define the actions for this task
Meanwhile, Dialog is hanging and Assistant is not responsive (nor LISTENing)
After what seems to be 20 seconds, the assistant proceeds with default fallback task
{
"actions": [
{
"redirect": "https://www.myExampleAppServer.com/assistant_endpoint/"
}
]}
Test 2 Same as above but added SAY before redirect_url.
The SAY will not "speak" until the response is received from app Server
I lowered the sleep(10sec) server side to confirm the behavior, and it works as described in my question introduction. Dialog hangs for 10 seconds, then proceeds with the hard-coded SAY following with actions (JSON) received from app Server
{
"actions": [
{
"say": "Please Wait while I check with the central offices."
},
{
"redirect": "https://www.myExampleAppServer.com/assistant_endpoint/"
}
]}
I also tested different combinations using LISTEN, which is also Blocking, and could not help to un-block the assistant. i.e. : While hanging/waiting for app Server, could not find any structure where the Caller can ask : "Anybody there?" and trigger any new routing.
Any ideas would be greatly appreciated. I know Autopilot is BETA, but thought my question could still be pertinent while in BETA.
Thanks!