In order to use MDX v2 with remark and rehype plugins in react applications created with Create React App v5 I have to use CRACO (v7) and especially its ability to handle a configuration that is returned as a promise/async function. The reason is that CRACO uses CJS module-based configuration, while the remark and rehype plugins are ES modules. Now I want to use MDXv2 with remark/rehype plugins in React Styleguidist too.
This is my craco.config.js:
const { addAfterLoader, loaderByName } = require('#craco/craco')
const mdxplagues = require('./mdxplugins.js')
// https://github.com/facebook/create-react-app/pull/11886#issuecomment-1055054685
const ForkTsCheckerWebpackPlugin =
process.env.TSC_COMPILE_ON_ERROR === 'true'
? require('react-dev-utils/ForkTsCheckerWarningWebpackPlugin')
: require('react-dev-utils/ForkTsCheckerWebpackPlugin');
module.exports = async (env) => {
const mdxplagueConfig = await mdxplagues()
return {
webpack: {
configure: (webpackConfig) => {
addAfterLoader(webpackConfig, loaderByName('babel-loader'), {
test: /\.(md|mdx)$/,
loader: require.resolve('#mdx-js/loader'),
/** #type {import('#mdx-js/loader').Options} */
options: mdxplagueConfig,
})
// https://github.com/facebook/create-react-app/pull/11886#issuecomment-1055054685
webpackConfig.plugins.forEach((plugin) => {
if (plugin instanceof ForkTsCheckerWebpackPlugin) {
plugin.options.issue.exclude.push({ file: '**/src/**/?(*.){spec,test,cy}.*' });
}
})
return webpackConfig
}
}
}
}
And this is my mdxplugins.js that returns an async function returing the needed plugin configurations:
const textrTypoEnDashes = (input) => {
return input
.replace(/ -- /gim, '–')
}
module.exports = async () => {
const remarkGfm = (await import('remark-gfm')).default
const remarkImages = (await import('remark-images')).default
const remarkTextr = (await import('remark-textr')).default
const remarkAccessibleEmoji = (await import('#fec/remark-a11y-emoji')).default
const rehypeSlug = (await import('rehype-slug')).default
const textrTypoApos = (await import('typographic-apostrophes')).default
const textrTypoQuotes = (await import('typographic-quotes')).default
const textrTypoPossPluralsApos = (await import('typographic-apostrophes-for-possessive-plurals')).default
const textrTypoEllipses = (await import('typographic-ellipses')).default
//const textrTypoEmDashes = (await import('typographic-em-dashes')).default
const textrTypoNumberEnDashes = (await import('typographic-en-dashes')).default
return {
remarkPlugins: [
remarkGfm,
remarkImages,
remarkAccessibleEmoji,
[remarkTextr, {
plugins: [
textrTypoApos,
textrTypoQuotes,
textrTypoPossPluralsApos,
textrTypoEllipses,
// textrTypoEmDashes,
textrTypoNumberEnDashes,
textrTypoEnDashes,
],
options: {
locale: 'en-us'
}
}],
],
rehypePlugins: [
rehypeSlug,
],
}
}
My problem is now that I need to apply this webpack/MDXv2 plugin configuration to styleguide.config.js (v13.0.0). I tried...
module.exports = async () => {
return {
...
}
}
...but this doesn't seem to be supported. A look into React Styleguidist's sources, namely scripts/config.ts seems to confirm this.
How can I synchronously resolve my MDX plugin configuration that I return via an async function from my mdxplugin.js CJS module within Styleguidist's CJS configuration module?
I'd like to write a Workbox plugin that intercepts a particular API call, does some logic (omitted from the code below) that determines other URL's that can be cached, then adds them to the runtime cache.
export class PreCachePlugin implements WorkboxPlugin {
fetchDidSucceed: WorkboxPlugin["fetchDidSucceed"] = async ({
response,
}) => {
if (response.ok) {
const clonedResponse = await response.clone();
const json = await clonedResponse.json();
const urls: string[] = getUrlsToCache(json); // Omitting this logic.
await Promise.all(urls.map(url => {
const response = await fetch(url);
// TODO: How do I fetch these URL's and put them in the cache configured
// below while still respecting the expiration.maxEntries setting below?
}));
}
}
}
I am using GenerateSW with workbox-webpack-plugin and adding my plugin:
new GenerateSW({
include: [/\.js$/, /\.json$/, /\.html$/],
runtimeCaching: [
{
urlPattern: ({ url }) => url.toString().endsWith("/foo"),
handler: "StaleWhileRevalidate",
options: {
cacheName: "Foo",
expiration: { maxEntries: 100 },
plugins: [
new PreCachePlugin(),
],
},
},
],
swDest: "./sw.js",
}
How do I fetch the above new URL's and put them in the configured runtime cache while still respecting the expiration.maxEntries settings?
I want to send parallel POST requests in puppeteer. I have to change the payload with every request (URL remains the same).
I tried using puppeteer cluster, but how do I change payload with every request when I queue the same request?
Using normal puppeteer
(async() => {
const browser = await puppeteer.launch({
args: [
"--no-sandbox",
"--disable-setuid-sandbox",
"--disable-web-security",
],
executablePath: 'C:/Program Files/..',
headless: false,
});
for(const id of Ids) {
const page = await browser.newPage();
await page.setDefaultNavigationTimeout(60000);
await page.evaluateOnNewDocument(() => {
// Some code
})
await page.setRequestInterception(true);
// Request intercept handler... will be triggered with
// each page.goto() statement
page.on('request', interceptedRequest => {
// Here, is where you change the request method and
// add your post data
var data = {
'method': 'POST',
'postData': JSON.stringify({
....
"similarMaterialId": `${id}`,
}),
'headers': {
.....
},
};
// Request modified... finish sending!
interceptedRequest.continue(data);
});
const response = await page.goto('https://.../getProductInfo');
const responseBody = await response.json();
try {
let title = responseBody.description;
let price = responseBody.price;
fs.appendFile('temp.tsv', `${title}\t${price}\n`, function (err) {
if (err) throw err;
})
}
catch {
console.log(id)
}
await page.close();
}
console.log("Code ended!!")
await browser.close();
})();
I want to create many pages in parallel on a single browser.
I am using Capacitor v3, NextJS static export, and a Django backend to build out an iOS app based on a production website.
The current backend authentication scheme uses Django sessions via cookies as well as setting the CSRF token via cookies. The CSRF token can be bypassed pretty easily for the app and not worried about disabling that but forking our authentication scheme would be somewhat of a hassle. The capacitor-community/http claims to allow Cookies but I haven't been able to configure that correctly.
Capacitor Config:
import { CapacitorConfig } from '#capacitor/cli';
const config: CapacitorConfig = {
appId: 'com.nextwebapp.app',
appName: 'nextwebapp',
webDir: 'out',
bundledWebRuntime: false
};
export default config;
Note that I have tried setting server.hostname to myapp.com as well.
Based on the comments at the bottom of the capacitor http readme I set the following Info.plist values.
App/Info.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
....
<key>WKAppBoundDomains</key>
<array>
<string>staging.myapp.com</string>
<string>myapp.com</string>
</array>
</dict>
</plist>
The web app uses a react hooks wrapper package for axios so in order to keep changes minimal I made a hook that mimics the state returned from that package.
hooks/useNativeRequest.ts
import { useEffect, useState } from "react";
import { Http } from "#capacitor-community/http";
import {
BASE_URL,
DEFAULT_HEADERS,
HOST_NAME,
ERROR_MESSAGE,
Refetch,
RequestOptions,
ResponseValues,
RequestConfig,
} from "#utils/http";
import { handleResponseToast } from "#utils/toast";
const makeUrl = (url): string => `${BASE_URL}${url}`;
const getCSRFToken = async () =>
await Http.getCookie({ key: "csrftoken", url: HOST_NAME });
const combineHeaders = async (headers: any) => {
const newHeaders = Object.assign(DEFAULT_HEADERS, headers);
const csrfHeader = await getCSRFToken();
if (csrfHeader.value) {
newHeaders["X-CSRFToken"] = csrfHeader.value;
}
return newHeaders;
};
function useNativeRequest<T>(
config?: RequestConfig,
options?: RequestOptions
): [ResponseValues<T>, Refetch<T>] {
const [responseState, setResponseState] = useState({
data: null,
error: null,
loading: false,
});
let method = "get";
let url = config;
let headers = {};
let params = undefined;
let data = undefined;
if (config && typeof config !== "string") {
url = config.url;
method = config.method?.toLowerCase() ?? method;
headers = config.headers;
params = config.params;
data = config.data;
}
const requestMethod = Http[method];
const makeRequest = async () => {
setResponseState({ error: null, data: null, loading: true });
try {
const reqHeaders = await combineHeaders(headers);
console.log({
url,
reqHeaders,
params,
data
})
const response = await requestMethod({
url: makeUrl(url),
headers: reqHeaders,
params,
data,
});
if (response?.status === 200) {
setResponseState({ error: null, data: response.data, loading: false });
handleResponseToast(response?.data?.detail);
} else {
const errorMessage = response?.data?.detail || ERROR_MESSAGE;
handleResponseToast(errorMessage);
setResponseState({
error: errorMessage,
data: response.data,
loading: false,
});
}
return response;
} catch {
setResponseState({
error: ERROR_MESSAGE,
data: null,
loading: false,
});
return Promise.reject(ERROR_MESSAGE);
}
};
useEffect(() => {
if (!options?.manual) {
makeRequest();
}
}, [options?.manual]);
return [responseState, makeRequest];
}
export { useNativeRequest };
The console.log above never includes the additional csrf cookie and in the getter logs it doesn't contain a value.
Backend Django
MIDDLEWARE = [
...
'myapp_webapp.middle.CustomCSRFMiddleWare',
]
CORS_ALLOWED_ORIGINS = [
...
"capacitor://localhost",
]
REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': (
'rest_framework.authentication.BasicAuthentication',
'rest_framework.authentication.TokenAuthentication',
'rest_framework.authentication.SessionAuthentication',
),
}
middleware
class CustomCSRFMiddleWare(CsrfViewMiddleware):
def process_request(self, request):
# Special Processing for API Requests
if "/api/v1" in request.path:
try:
requested_with = request.headers['X-Requested-With']
myapp_request = request.headers['X-Myapp-Request']
# Check Custom Headers
if not (requested_with == 'XMLHttpRequest' and myapp_request == '1'):
raise PermissionDenied()
return None
except KeyError:
# All API Requests should include the above headers
raise PermissionDenied()
# Call original CSRF Middleware
return super(CustomCSRFMiddleWare, self).process_request(request)
Occasionally the backend will also show that X-Requested-With is not being sent but it is included in the DEFAULT_HEADERS constant I have in the UI and appears in the console.log.
Is anything above preventing me from being able to read and send cookies from Capacitor on iOS? Does Cookie based authentication even work with capacitor?
Here is my updated react hook that combine's my above question and thread mentioned in the comments as well as some manual cookie setting.
The below client side code worked without changes to existing Django Session authentication.
The changes from my code above
Added credentials: "include" to webFetchExtra
Added "Content-Type": "application/json" to headers
Handle override of the initial config for manual request & refetch
Set Session Cookie After Response
Based on the docs this shouldn't be necessary but I am keeping in my code for now.
import { useCallback, useEffect, useState } from "react";
import { AxiosRequestConfig } from "axios";
import { Http } from "#capacitor-community/http";
const DEFAULT_HEADERS = {
"X-Requested-With": "XMLHttpRequest",
"X-MyApp-Request": "1",
"Content-Type": "application/json",
};
const makeUrl = (url): string => `${BASE_URL}${url}`;
const getCSRFToken = async () =>
await Http.getCookie({ key: "csrftoken", url: HOST_NAME });
const setSessionCookie = async () => {
const sessionId = await Http.getCookie({ key: "sessionid", url: HOST_NAME });
if (sessionId.value) {
await Http.setCookie({
key: "sessionid",
value: sessionId.value,
url: HOST_NAME,
});
}
};
const combineHeaders = async (headers: any) => {
const newHeaders = Object.assign(DEFAULT_HEADERS, headers);
const csrfHeader = await getCSRFToken();
if (csrfHeader.value) {
newHeaders["X-CSRFToken"] = csrfHeader.value;
}
return newHeaders;
};
const parseConfig = (config: RequestConfig, configOverride?: RequestConfig) => {
let method = "get";
let url = config;
let headers = {};
let params = undefined;
let data = undefined;
if (config && typeof config !== "string") {
url = config.url;
method = config.method ?? method;
headers = config.headers;
params = config.params;
data = config.data;
}
return {
url,
method,
headers,
params,
data,
...(configOverride as AxiosRequestConfig),
};
};
function useNativeRequest<T>(
config?: RequestConfig,
options?: RequestOptions
): [ResponseValues<T>, Refetch<T>] {
const [responseState, setResponseState] = useState({
data: null,
error: null,
loading: false,
});
const makeRequest = useCallback(
async (configOverride) => {
setResponseState({ error: null, data: null, loading: true });
const { url, method, headers, params, data } = parseConfig(
config,
configOverride
);
try {
const reqHeaders = await combineHeaders(headers);
const response = await Http.request({
url: makeUrl(url),
headers: reqHeaders,
method,
params,
data,
webFetchExtra: {
credentials: "include",
},
});
if (response?.status === 200) {
setResponseState({
error: null,
data: response.data,
loading: false,
});
await setSessionCookie();
} else {
setResponseState({
error: errorMessage,
data: response.data,
loading: false,
});
}
return response;
} catch {
setResponseState({
error: ERROR_MESSAGE,
data: null,
loading: false,
});
return Promise.reject(ERROR_MESSAGE);
}
},
[config]
);
useEffect(() => {
if (!options?.manual) {
makeRequest(config);
}
}, [options?.manual]);
return [responseState, makeRequest];
}
export { useNativeRequest };
Is it possible to define a browser with Javascript disabled to simulate how a crawler would view a page? There should be an option that can be passed.
You can pass javaScriptEnabled in the BrowserContext options:
const playwright = require("playwright");
(async () => {
const browser = await playwright.chromium.launch();
const context = await browser.newContext({
javaScriptEnabled: false
});
const page = await context.newPage();
// ...
await browser.close();
})();
In the case of #playwright/test (where you don't define browser.newContext yourself) you can set javaScriptEnabled in testOptions.
(1) In the playwright.config.js file:
const config = {
use: {
headless: false,
javaScriptEnabled: false
},
};
module.exports = config;
(2) or with test.use:
const { test, expect } = require('#playwright/test');
test.use({ javaScriptEnabled: false });
test('website without JavaScript', async ({ page }) => {
// ...
});