I'd like to write a Workbox plugin that intercepts a particular API call, does some logic (omitted from the code below) that determines other URL's that can be cached, then adds them to the runtime cache.
export class PreCachePlugin implements WorkboxPlugin {
fetchDidSucceed: WorkboxPlugin["fetchDidSucceed"] = async ({
response,
}) => {
if (response.ok) {
const clonedResponse = await response.clone();
const json = await clonedResponse.json();
const urls: string[] = getUrlsToCache(json); // Omitting this logic.
await Promise.all(urls.map(url => {
const response = await fetch(url);
// TODO: How do I fetch these URL's and put them in the cache configured
// below while still respecting the expiration.maxEntries setting below?
}));
}
}
}
I am using GenerateSW with workbox-webpack-plugin and adding my plugin:
new GenerateSW({
include: [/\.js$/, /\.json$/, /\.html$/],
runtimeCaching: [
{
urlPattern: ({ url }) => url.toString().endsWith("/foo"),
handler: "StaleWhileRevalidate",
options: {
cacheName: "Foo",
expiration: { maxEntries: 100 },
plugins: [
new PreCachePlugin(),
],
},
},
],
swDest: "./sw.js",
}
How do I fetch the above new URL's and put them in the configured runtime cache while still respecting the expiration.maxEntries settings?
Related
I am using Capacitor v3, NextJS static export, and a Django backend to build out an iOS app based on a production website.
The current backend authentication scheme uses Django sessions via cookies as well as setting the CSRF token via cookies. The CSRF token can be bypassed pretty easily for the app and not worried about disabling that but forking our authentication scheme would be somewhat of a hassle. The capacitor-community/http claims to allow Cookies but I haven't been able to configure that correctly.
Capacitor Config:
import { CapacitorConfig } from '#capacitor/cli';
const config: CapacitorConfig = {
appId: 'com.nextwebapp.app',
appName: 'nextwebapp',
webDir: 'out',
bundledWebRuntime: false
};
export default config;
Note that I have tried setting server.hostname to myapp.com as well.
Based on the comments at the bottom of the capacitor http readme I set the following Info.plist values.
App/Info.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
....
<key>WKAppBoundDomains</key>
<array>
<string>staging.myapp.com</string>
<string>myapp.com</string>
</array>
</dict>
</plist>
The web app uses a react hooks wrapper package for axios so in order to keep changes minimal I made a hook that mimics the state returned from that package.
hooks/useNativeRequest.ts
import { useEffect, useState } from "react";
import { Http } from "#capacitor-community/http";
import {
BASE_URL,
DEFAULT_HEADERS,
HOST_NAME,
ERROR_MESSAGE,
Refetch,
RequestOptions,
ResponseValues,
RequestConfig,
} from "#utils/http";
import { handleResponseToast } from "#utils/toast";
const makeUrl = (url): string => `${BASE_URL}${url}`;
const getCSRFToken = async () =>
await Http.getCookie({ key: "csrftoken", url: HOST_NAME });
const combineHeaders = async (headers: any) => {
const newHeaders = Object.assign(DEFAULT_HEADERS, headers);
const csrfHeader = await getCSRFToken();
if (csrfHeader.value) {
newHeaders["X-CSRFToken"] = csrfHeader.value;
}
return newHeaders;
};
function useNativeRequest<T>(
config?: RequestConfig,
options?: RequestOptions
): [ResponseValues<T>, Refetch<T>] {
const [responseState, setResponseState] = useState({
data: null,
error: null,
loading: false,
});
let method = "get";
let url = config;
let headers = {};
let params = undefined;
let data = undefined;
if (config && typeof config !== "string") {
url = config.url;
method = config.method?.toLowerCase() ?? method;
headers = config.headers;
params = config.params;
data = config.data;
}
const requestMethod = Http[method];
const makeRequest = async () => {
setResponseState({ error: null, data: null, loading: true });
try {
const reqHeaders = await combineHeaders(headers);
console.log({
url,
reqHeaders,
params,
data
})
const response = await requestMethod({
url: makeUrl(url),
headers: reqHeaders,
params,
data,
});
if (response?.status === 200) {
setResponseState({ error: null, data: response.data, loading: false });
handleResponseToast(response?.data?.detail);
} else {
const errorMessage = response?.data?.detail || ERROR_MESSAGE;
handleResponseToast(errorMessage);
setResponseState({
error: errorMessage,
data: response.data,
loading: false,
});
}
return response;
} catch {
setResponseState({
error: ERROR_MESSAGE,
data: null,
loading: false,
});
return Promise.reject(ERROR_MESSAGE);
}
};
useEffect(() => {
if (!options?.manual) {
makeRequest();
}
}, [options?.manual]);
return [responseState, makeRequest];
}
export { useNativeRequest };
The console.log above never includes the additional csrf cookie and in the getter logs it doesn't contain a value.
Backend Django
MIDDLEWARE = [
...
'myapp_webapp.middle.CustomCSRFMiddleWare',
]
CORS_ALLOWED_ORIGINS = [
...
"capacitor://localhost",
]
REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': (
'rest_framework.authentication.BasicAuthentication',
'rest_framework.authentication.TokenAuthentication',
'rest_framework.authentication.SessionAuthentication',
),
}
middleware
class CustomCSRFMiddleWare(CsrfViewMiddleware):
def process_request(self, request):
# Special Processing for API Requests
if "/api/v1" in request.path:
try:
requested_with = request.headers['X-Requested-With']
myapp_request = request.headers['X-Myapp-Request']
# Check Custom Headers
if not (requested_with == 'XMLHttpRequest' and myapp_request == '1'):
raise PermissionDenied()
return None
except KeyError:
# All API Requests should include the above headers
raise PermissionDenied()
# Call original CSRF Middleware
return super(CustomCSRFMiddleWare, self).process_request(request)
Occasionally the backend will also show that X-Requested-With is not being sent but it is included in the DEFAULT_HEADERS constant I have in the UI and appears in the console.log.
Is anything above preventing me from being able to read and send cookies from Capacitor on iOS? Does Cookie based authentication even work with capacitor?
Here is my updated react hook that combine's my above question and thread mentioned in the comments as well as some manual cookie setting.
The below client side code worked without changes to existing Django Session authentication.
The changes from my code above
Added credentials: "include" to webFetchExtra
Added "Content-Type": "application/json" to headers
Handle override of the initial config for manual request & refetch
Set Session Cookie After Response
Based on the docs this shouldn't be necessary but I am keeping in my code for now.
import { useCallback, useEffect, useState } from "react";
import { AxiosRequestConfig } from "axios";
import { Http } from "#capacitor-community/http";
const DEFAULT_HEADERS = {
"X-Requested-With": "XMLHttpRequest",
"X-MyApp-Request": "1",
"Content-Type": "application/json",
};
const makeUrl = (url): string => `${BASE_URL}${url}`;
const getCSRFToken = async () =>
await Http.getCookie({ key: "csrftoken", url: HOST_NAME });
const setSessionCookie = async () => {
const sessionId = await Http.getCookie({ key: "sessionid", url: HOST_NAME });
if (sessionId.value) {
await Http.setCookie({
key: "sessionid",
value: sessionId.value,
url: HOST_NAME,
});
}
};
const combineHeaders = async (headers: any) => {
const newHeaders = Object.assign(DEFAULT_HEADERS, headers);
const csrfHeader = await getCSRFToken();
if (csrfHeader.value) {
newHeaders["X-CSRFToken"] = csrfHeader.value;
}
return newHeaders;
};
const parseConfig = (config: RequestConfig, configOverride?: RequestConfig) => {
let method = "get";
let url = config;
let headers = {};
let params = undefined;
let data = undefined;
if (config && typeof config !== "string") {
url = config.url;
method = config.method ?? method;
headers = config.headers;
params = config.params;
data = config.data;
}
return {
url,
method,
headers,
params,
data,
...(configOverride as AxiosRequestConfig),
};
};
function useNativeRequest<T>(
config?: RequestConfig,
options?: RequestOptions
): [ResponseValues<T>, Refetch<T>] {
const [responseState, setResponseState] = useState({
data: null,
error: null,
loading: false,
});
const makeRequest = useCallback(
async (configOverride) => {
setResponseState({ error: null, data: null, loading: true });
const { url, method, headers, params, data } = parseConfig(
config,
configOverride
);
try {
const reqHeaders = await combineHeaders(headers);
const response = await Http.request({
url: makeUrl(url),
headers: reqHeaders,
method,
params,
data,
webFetchExtra: {
credentials: "include",
},
});
if (response?.status === 200) {
setResponseState({
error: null,
data: response.data,
loading: false,
});
await setSessionCookie();
} else {
setResponseState({
error: errorMessage,
data: response.data,
loading: false,
});
}
return response;
} catch {
setResponseState({
error: ERROR_MESSAGE,
data: null,
loading: false,
});
return Promise.reject(ERROR_MESSAGE);
}
},
[config]
);
useEffect(() => {
if (!options?.manual) {
makeRequest(config);
}
}, [options?.manual]);
return [responseState, makeRequest];
}
export { useNativeRequest };
I'm learning how to write code of service worker and stuck with the error "ReferenceError: document is not defined" in my app.js file. I'm using workbox library with InjectManifest mode. I think the problem in the webpack.config.js, because when I delete InjectManifest in webpack.config.js the error disappears.
My webpack.config.js
const path = require('path');
const webpack = require('webpack');
const HtmlWebPackPlugin = require('html-webpack-plugin');
const MiniCssExtractPlugin = require('mini-css-extract-plugin');
const CssMinimizerPlugin = require('css-minimizer-webpack-plugin');
const {InjectManifest} = require('workbox-webpack-plugin');
module.exports = {
entry: './src/index.js',
output: {
path: path.resolve(__dirname, 'dist'),
},
module: {
rules: [
{
test: /\.js$/,
exclude: /node_modules/,
use: {
loader: 'babel-loader',
},
},
{
test: /\.html$/i,
use: [
{
loader: 'html-loader',
},
],
},
{
test: /\.css$/,
use: [
MiniCssExtractPlugin.loader, 'css-loader',
],
},
{
test: /\.(png|jpg|gif)$/i,
use: [
{
loader: 'url-loader',
options: {
limit: 8192,
},
},
],
},
],
},
optimization: {
minimize: true,
minimizer: [
new CssMinimizerPlugin(),
],
},
plugins: [
new HtmlWebPackPlugin({
template: './src/index.html',
filename: './index.html',
}),
new MiniCssExtractPlugin({
filename: '[name].css',
chunkFilename: '[id].css',
}),
new InjectManifest({
swSrc: './src/js/service.worker.js',
swDest: 'service.worker.js',
}),
],
};
My service.worker.js file:
import { precacheAndRoute } from 'workbox-precaching/precacheAndRoute';
import { cinemaNews } from './cinemaNews';
import { url } from './app';
precacheAndRoute(self.__WB_MANIFEST);
const CACHE_NAME = 'v1';
const responseCache = new Response(JSON.stringify(cinemaNews));
self.addEventListener('install', (evt) => {
console.log('install')
evt.waitUntil((async () => {
console.log('install waitUntil')
const cache = await caches.open(CACHE_NAME);
await cache.put(url, responseCache);
await self.skipWaiting();
})());
});
self.addEventListener('activate', (evt) => {
console.log('activate')
evt.waitUntil(self.clients.claim());
});
self.addEventListener('fetch', (evt) => {
console.log('sw fetch')
const requestUrl = new URL(evt.request.url);
if (!requestUrl.pathname.startsWith('/news')) return;
evt.respondWith((async () => {
console.log('respondWith')
const cache = await caches.open(CACHE_NAME);
const cachedResponse = await cache.match(evt.request);
return cachedResponse;
})());
evt.waitUntil((async () => {
console.log('waitUntil');
const response = await fetch(evt.request.url);
const client = await clients.get(evt.clientId);
let json = await response.json();
client.postMessage(json);
})());
});
This statement:
import { url } from './app';
appears to be triggering the issue, as there must be code inside of your app.js that is executed via that import, and which assumes that document will be defined. (It's not defined inside of the ServiceWorkerGlobalScope.)
Based on how you're using the export, I'm assuming that it's just a string constant containing a shared URL that you want to use from both your main web app and your service worker. Assuming that's the case, the easiest thing to do would be to refactor your modules that there's a constants.js (or some similar name) module that only exports your string constants, and doesn't try to run any code that references document. You can then import the constant from either your web app or the service worker without issue.
// constants.js
export const url = '/path/to/url';
// service-worker.js
import {url} from './constants';
// do something with url
// app.js
import {url} from './constants';
// do something with url
In my project, I am using NextJS+KOA+Apollo. My nextJS app is inside client in root directory. I am using next-offline to convert it to PWA.
Nextjs app is inside client directory. koa server is inside server directory.
when i am building the app via below command:
next build client && tsc --project tsconfig.server.json
it creates a build directory inside client for nextjs and dist directory at the top level for koa server.
i run the code in production via below command
NODE_ENV=production node dist/server/index.js
ISSUE
Service worker is getting registered properly. But I am getting below error:
PrecacheController.mjs:194
Uncaught (in promise) bad-precaching-response: bad-precaching-response :: [{"url":"https://my-domain/_next/bo.svg?__WB_REVISION__=e02afe0476bb357aebde18136fda06e0","status":404}]
at l.o (https://storage.googleapis.com/workbox-cdn/releases/4.3.1/workbox-precaching.prod.js:1:1749)
at async Promise.all (index 0)
at async l.install (https://storage.googleapis.com/workbox-cdn/releases/4.3.1/workbox-precaching.prod.js:1:1221)
Below is my build file that gets generated:
tsconfig.server.json
{
"extends": "./tsconfig.json",
"compilerOptions": {
"module": "commonjs",
"outDir": "dist",
"target": "es2017",
"isolatedModules": false,
"noEmit": false
},
"include": ["server/**/*.ts"]
}
Below is my next.config.js (inside client direcotry)
/* eslint-disable #typescript-eslint/no-var-requires */
const withPlugins = require("next-compose-plugins");
const offline = require("next-offline");
const pino = require("next-pino");
const withTypescript = require("#zeit/next-typescript");
const withCSS = require("#zeit/next-css");
const withLess = require("#zeit/next-less");
const Dotenv = require("dotenv-webpack");
const path = require("path");
const _ = require("lodash");
const nextConfig = {
distDir: "build",
webpack(config) {
config.module.rules.push({
test: /\.(eot|woff|woff2|ttf|svg|png|jpg|gif)$/,
use: {
loader: "url-loader",
options: {
limit: 100000,
name: "[name].[ext]",
},
},
});
config.plugins.push(
new Dotenv({
path: path.resolve(process.cwd(), ".env"),
systemvars: true,
}),
);
return config;
},
// overwrites values given in the .env file with the current
// process.env value
env: _.omitBy(
{
GRAPHQL_SERVER: process.env.GRAPHQL_SERVER,
},
_.isUndefined,
),
workboxOpts: {
globPatterns: ["static/**/*"],
globDirectory: "client",
runtimeCaching: [
{
urlPattern: /^https?.*/,
handler: "NetworkFirst",
options: {
cacheName: "offlineCache",
expiration: {
maxEntries: 200,
},
},
},
],
},
};
const cssConfig = {
cssModules: true,
cssLoaderOptions: {
importLoaders: 1,
localIdentName: "[local]",
},
};
const lessConfig = cssConfig;
module.exports = withPlugins(
[
[offline],
[pino],
[withTypescript],
[withCSS, cssConfig],
[withLess, lessConfig],
],
nextConfig,
);
And below is my file to start koa server
import Router from "koa-router";
const server = new Koa();
const dev = !["production", "staging"].includes(process.env.NODE_ENV || "");
const app = next({ dir: "./client", dev });
const publicRouter = new Router();
const handle = app.getRequestHandler();
publicRouter.get("/service-worker.js", async ctx => {
const pathname = await join(
__dirname,
"../../../client/build",
"service-worker.js",
);
ctx.body = await app.serveStatic(ctx.req, ctx.res, pathname);
ctx.respond = false;
});
publicRouter.get("*", async ctx => {
if (!ctx.path.match(/graphql/)) {
await handle(ctx.req, ctx.res);
ctx.respond = false;
}
});
server.use(async (ctx, next) => {
ctx.res.statusCode = 200;
await next();
});
server.use(publicRouter.routes()).use(publicRouter.allowedMethods());
server.listen({ port: 3000 });
================================================================
I have done a dirty fix for now. I am not sure how to handle it properly. I will really appreciate if anyone can put forth their view on this.
As bo.svg, firfox.svg, all these static files are throwing 404,
Ex - (/_next/bo.svg?WB_REVISION=e02afe0476bb357aebde18136fda06e0)
in my file to start koa server, added a condition to check this URL and serve static file from build directory like below:
publicRouter.get("*", async ctx => {
if (ctx.path.match(/\_next/) && ctx.path.match(/\.svg/)) {
const pathname = await join(
__dirname,
"../../../client/build",
ctx.path.replace("_next/", ""),
);
ctx.body = await app.serveStatic(ctx.req, ctx.res, pathname);
ctx.respond = false;
} else if (!ctx.path.match(/graphql/)) {
await handle(ctx.req, ctx.res);
ctx.respond = false;
}
});
It served my prupose for now, but not sure how to handle this properly.
I have implemented this class in workbox 2, now I have upgraded to version 3 but workbox.runtimeCaching.Handler is deprecated.
Can someone help me on how to develop it in workbox 3?*
importScripts('workbox-sw.prod.v2.1.2.js');
importScripts('workbox-runtime-caching.prod.v2.0.3.js');
importScripts('workbox-cache-expiration.prod.v2.0.3.js');
const workboxSW = new self.WorkboxSW();
class AlwaysNetworkWithCacheUpdateHandler extends workbox.runtimeCaching.Handler{
setCacheOptions(cacheOptions){
this.cacheOptions = cacheOptions;
}
handle({event}){
let requestWrapper = new workbox.runtimeCaching.RequestWrapper({
cacheName: this.cacheOptions.cacheName,
plugins:[
new workbox.cacheExpiration.CacheExpirationPlugin(this.cacheOptions.expirationOptions)
]
});
return (
requestWrapper
.fetchAndCache({
request: event.request,
waitOnCache: true
})
);
}
}
I'm not sure what you are going to reach, however, I used runtimeCaching for third-party requests (CDN), so now it is handling by regular way:
https://developers.google.com/web/tools/workbox/guides/handle-third-party-requests
Strategies now do the job of RequestWrapper, choose one and use like so:
const strategy = workbox.strategies.networkFirst({
cacheName,
plugins: [
new workbox.expiration.Plugin({
maxEntries: 100,
maxAgeSeconds: 60 * 60 * 24 *7,
})
],
});
const handler = async ({event}) => {
const request = new Request(event.request, {
mode: 'cors',
credentials: 'omit',
});
const cachedResponse = await caches.match(request, {
cacheName,
});
return cachedResponse || strategy.makeRequest({
event,
request,
});
}
router.registerRoute(
({ url }) => url.origin === 'http://example.com',
handler,
)
Example comes directly from this issue
While using strongloop loopback I want check the database for accesstoken and username existence in every request.
So i am making a middleware code:
module.exports = function() {
return function xAuth(req, res, next) {
console.log(req);
};
};
I have added it to :initial" middleware json
"initial": {
"compression": {},
"cors": {
"params": {
"origin": true,
"credentials": true,
"maxAge": 86400
}
},
"./middleware/trumptAuth": {},
"helmet#xssFilter": {},
"helmet#frameguard": {
"params": [
"deny"
]
},
"helmet#hsts": {
"params": {
"maxAge": 0,
"includeSubdomains": true
}
},
"helmet#hidePoweredBy": {},
"helmet#ieNoOpen": {},
"helmet#noSniff": {},
"helmet#noCache": {
"enabled": false
}
}
i want to execute an sql query here but i have no idea on how can i do that, I probably just need "app" variable access or directly "dataSource" access.
Any help is appreciated.
From the docs, "Using variables in middleware" (https://docs.strongloop.com/display/LB/Defining+middleware#Definingmiddleware-Usingvariablesinvalues), it looks like you can pass any part of the app object using this syntax: ${var}. The datasource property exists under the core app object, so you should be able to pass it in that way.
To access "app" variable from middleware use
app.use(function(req, res, next) {
var app = req.app;
...
});
To access "dataSource" variable from middleware use
app.use(function(req, res, next) {
var dataSource = app.datasources.db;
...
});
To access an specific model from middleware use
app.use(function(req, res, next) {
var app = req.app;
var modelName = app.model.modelName;
...
});
for more methods/sources use this documentation link
https://docs.strongloop.com/display/public/LB/Working+with+LoopBack+objects