I'm using staleWhileRevalidate Workbox v3 strategy for all of my API Get calls. However, sometimes I want to remove a particular request from the cache at runtime.
Example: I have a yes/no state that can be updated by the client. Initially, the UI displays the "no" state (taken from the cache) while executing a request that also returns a "no" result. The result of the request is cached by Workbox.
Now, the user switches it to "yes", the system updates the server and sends another query after some time to get the latest value. But the cache still stores the "no" value which is displayed in the UI while the new request returns "yes" which is now stored in the cache.
What I need here is to remove the "no" result from the cache right when I send the update request from the server. How do I do that? How can I access the Workbox's cache from the page code?
Workbox uses the browser's Cache Storage API for its caching. You can access the same caches directly, either via the window context or inside of a service worker, from your own code.
It's easier to do this if you configure your Workbox strategy to use a specific cache name, rather than the default cache name, since you'd otherwise need to reverse-engineer some of the Workbox logic to create the default cache name inside of the window context. Assuming you know the cache name and the URL to delete, you can call the following function from within the window context to use the Cache Storage API to delete the entry:
async function deleteFromCache(cacheName, urlOrPath) {
const cache = await caches.open(cacheName);
await cache.delete(urlOrPath);
}
// Later:
await deleteFromCache('runtime-cache', '/path/to/entry');
Related
In my C# code with the Graph SDK, I am testing "download large file" with a method generously provided here: Download large files from OneDrive using Microsoft Graph SDK
However, even though I explicitly request the DriveItem from the service just prior to calling this method (which then uses the downloadUrl from the AdditionalData in the DriveItem), I am getting a response indicating JWT Token Expired.
I assume this token is embedded in the pre-authenticated downloadUrl. Further, I theorize that this token is either single-use or it expires relatively quickly...both reasons for me to explicitly request the DriveItem just before attempting to utilize that URL.
How do I ensure that the service is sending me back a "fresh" downloadUrl? Are there some no-cache headers or something I should explicitly include in my DriveItem request?
As mentioned, even though I have the DriveItem object local, I am explicitly re-fetching it from the service to ensure I have the latest properties, and a new downloadUrl. (But this is not working.) Using the Request object without any extra Headers (or other odata parameters) is apparently not enough to trigger the generation of a new URL.
This does not happen every test. I believe it happens when re-running a test with the same DriveItem in a short time-window. Again, I'm not sure if the JWT token is single-use or time-expired, but if it's expired in any case, I think the service should automatically generate a new one. If that's not practical, just understanding how to explicitly ask for a new one is certainly effective too!
Continuing to debug and try different things, I believe I have found the answer to my question...
Sending a known-false ETag in the request for the DriveItem seems to force the service to send me a new copy. Example code as follows:
//we need a "recent" copy of this object, because the downloadURL is only good for a limited time...
IList<HeaderOption> opts = new List<HeaderOption>();
opts.Add(new HeaderOption("Cache-Control", "no-cache, no-store"));
opts.Add(new HeaderOption("if-none-match", "xyz")); //never match
DriveItem item = await client.Me.Drive.Items[Id].Request(opts).GetAsync();
if (item == null)
{
log.Warn("Could not fetch existing driveItem? " + Id);
return null;
}
object downloadUrl = null;
item.AdditionalData?.TryGetValue(#"#microsoft.graph.downloadUrl", out downloadUrl);
I am still testing this, but initial tests show it to work. I will update this answer when my testing is reasonably conclusive.
Still "bonus points" for anyone who can identify the specific expiration of these URLs (time, number of uses, etc.). Then we can actually check locally whether our object is "stale" and only re-fetch when necessary.
How will the json data sent over from external Api be cached into the Falcor-Model? Also, how to specify in the Model to hit the external Api again if data not present in the cache?
My doubt was partially answered in one of the posts:
How does Falcor cache data in the server side?
So now I understand that Falcor-Model cache works only at the client side, which is fine. But how will the Model work if the data is not present in the cache?
var model = new falcor.Model({source: new falcor.HttpDataSource('http://localhost/rating.json') });
model.
get("rating").
then(function(response) {
document.getElementById('filmRating').innerText = JSON.stringify(response.json.rating,null, 4);
});
Here the response is a json object, which can be put into a Falcor-Model cache and stored globally in the client side. But how can the Model be triggered again if data not present in cache?
The main advantage of using Falcor is that you should not have to care whether data is present in cache or fetched from the server, because model.get() which fetch all the data missing in the cache from the server by making an HTTP request.
So, the first model.get(path) query will fetch from server, and put response in cache. If you call model.get(path) again, it will be served from the cache.
I have an API code, which loads a data necessary for my application.
It's as simple as:
- (void) getDataForKey:(NSString*) key onSuccess:(id (^)())completionBlock
I cache data returned from server, so next calls of that functions should not do network request, until there is some data missing for given key, then I need to load it again from server side.
Everything was okey as long as I had one request per screen, but right now I have a case where I need to do that for every cell on one screen.
Problem is my caching doesn't work because before the response comes in from the first one, 5-6 more are created at the same time.
What could be a solution here to not create multiple network request and make other calls waiting for the first one ?
You can try to make a RequestManager class. Use dictionary to cache the requesting request.
If the next request is the same type as first one, don't make a new request but return the first one. If you choose this solution, you need to manager a completionBlock list then you will be able to send result to all requesters.
If the next request is the same type as first one, waiting in another thread until the first one done. Then make a new request, you API will read cache automatically. Your must make sure your codes are thread-safe.
Or you can use operation queues to do this. Some documents:
Apple: Operation Queues
Soheil Azarpour: How To Use NSOperations and NSOperationQueues
May be there will be so many time consuming solutions for this. I have a trick. Create a BOOL in AppDelegate, its default is FALSE. When you receive first response, then set it TRUE. So when you go to other screen and before making request just check value of your BOOL variable in if condition. If its TRUE means response received so go for it otherwise in else don't do anything.
i have this code:
[AcceptVerbs(HttpVerbs.Get)]
[OutputCache(Duration = 86400, VaryByParam = "*")]
public ActionResult GetData(MyParams myParams)
{
return GetDataEx(myParams);
}
to cache a bunch of Json on my web server.
i see on this page, it says
There is no guarantee that content will be cached for the amount of time that you specify. When memory resources become low, the cache starts evicting content automatically.
is there any way to get a specific callback when this item gets evicted from teh cache because of memory resources getting low. I don't see the cache working properly but my guess is that i am running out of memory but if i had this callback, i could tell for sure.
With response output caching there is no way to be notified when the contents expires. And to be honest you shouldn't really care about, if the contents is expired the controller action will simply be hit and the new contents cached again. Also notice that depending on the location where you configured this output cache to be stored, if it is downstream on intermediate proxy servers or client browsers, the cache expiration is totally out of your control.
If you are caching objects manually into the ASP.NET cache you could register a CacheItemRemovedCallback which will be executed when the item is removed but this doesn't apply to response output caching which is what you are using here.
i am devloping a site using .net MVC
i have a data access layer which basically consists of static list objects that are created from data within my database.
The method that rebuilds this data first clears all the list objects. Once they are empty it then add the data. Here is an example of one of the lists im using. its a method which generates all the UK postcodes. there are about 50 methods similar to this in my application that return all sorts of information, such as towns, regions, members, emails etc.
public static List<PostCode> AllPostCodes = new List<PostCode>();
when the rebuild method is called it first clears the list.
ListPostCodes.AllPostCodes.Clear();
next it re-bulilds the data, by calling the GetAllPostCodes() method
/// <summary>
/// static method that returns all the UK postcodes
/// </summary>
public static void GetAllPostCodes()
{
using (fab_dataContextDataContext db = new fab_dataContextDataContext())
{
IQueryable AllPostcodeData = from data in db.PostCodeTables select data;
IDbCommand cmd = db.GetCommand(AllPostcodeData);
SqlDataAdapter adapter = new SqlDataAdapter();
adapter.SelectCommand = (SqlCommand)cmd;
DataSet dataSet = new DataSet();
cmd.Connection.Open();
adapter.FillSchema(dataSet, SchemaType.Source);
adapter.Fill(dataSet);
cmd.Connection.Close();
// crete the objects
foreach (DataRow row in dataSet.Tables[0].Rows)
{
PostCode postcode = new PostCode();
postcode.ID = Convert.ToInt32(row["PostcodeID"]);
postcode.Outcode = row["OutCode"].ToString();
postcode.Latitude = Convert.ToDouble(row["Latitude"]);
postcode.Longitude = Convert.ToDouble(row["Longitude"]);
postcode.TownID = Convert.ToInt32(row["TownID"]);
AllPostCodes.Add(postcode);
postcode = null;
}
}
}
The rebuild occurs every 1 hour. this ensures that every 1 hour the site will have fresh set of cached data.
the issue ive got is that occasionally if during a rebuild, the server will be hit by a request and an exception is thrown. The exception is "Index was outside the bounds of the array." it is due to when a list is being cleared.
ListPostCodes.AllPostCodes.Clear(); - // throws exception - although its not always in regard to this list.
Once this exception is thrown application dies, All users are affected. I have to restart the server to fix it.
i have 2 questions...
If i utilise caching instead of static objects would this help ?
Is there any way i can say "while the rebuild is taking place, wait for it to complete until accepting requests"
any help is most appricaiated ;)
truegilly
1 If i utilise caching instead of
static objects would this help ?
Yes, all the things you do are easier done by the caching functionality that is build into ASP.NET
Is there any way i can say "while the
rebuild is taking place, wait for it
to complete until accepting requests"
The common pattern goes like this:
You request data from the Data layer
If the Datlayer sees that there is data in the cache, then it serves the data from cache
If no data is in the cache the data is requested from the db and put into cache. After that it is served to the client
There are rules (CacheDependency and Timeout) when the cache is to be cleared.
The easiest solution would be you stick to this pattern: This way the first request would hit the database and other requests get served from the cache. You trigger the refresh by implementing an SQLCacheDependency
You have to make sure that your list is not modified by one thread while other threads are trying to use it. This would be a problem even if you used the ASP.NET cache since collections are just not thread-safe. One way you can do this is by using a SynchronizedCollection instead of a List. Then make sure to use code like the following when you access the collection:
lock (synchronizedCollection.SyncRoot) {
synchronizedCollection.Clear();
etc...
}
You will also have to use locking when you read the collection. If you are enumerating over it, you should probably make a copy before doing so as you don't want to lock for a long time. For example:
List<whatever> tempCollection;
lock (synchrnonizedCollection.SyncRoot) {
tempCollection = new List<whatever>(synchronizedCollection);
}
//use temp collection to access cached data
The other option would be to create a ThreadSafeList class that uses locking internally to make the list object itself thread-safe.
I agree with Tom, you will have to do synchronization to make this work. One thing that would improve the performance is not clearing the list until you actually receive the new values from the database:
// Modify your function to return a new list instead of filling the existing one.
public static List<PostCode> GetAllPostCodes()
{
List<PostCode> temp = new List<PostCode>();
...
return temp;
}
And when you rebuild the data:
List<PostCode> temp = GetAllPostCodes();
AllPostCodes = temp;
This makes sure that your cached list is still valid while GetAllPostCodes() is executing. It also has the advantage that you can use a read-only list which makes the synchronization a bit easier.
In your case you need to refresh the data every one hour.
1) IT should use cache with absolute expiration set to 1 hour, so it expires after every 1 hour. Check the Cache before using it, by doing a NULL check.If its NULL get the data from DB and populate the Cache.
2) With above approach the disadvantage is that data can be stale by 1 hour. So if u want most updated data at all times, use SQLCacheDependency (PUSH). so whenever there is a change in the select command u r using, cache will be refreshed from the database with updated data.