Google Lighthouse Speed Index 100% visually completed frame definition - lighthouse

Question

I'm looking for a way to optimize our website's Speed Index metric on Lighthouse
I found this helpful article describe Speed Index metric very well, and help me understand how Speed Index is calculated.
https://calendar.perfplanet.com/2016/speed-index-tips-and-tricks/
But there is a key concept not being described clear on the article, and I search for a lot of other Speed Index related blogs still can't found the answer.
What is the 100% visual completeness frame?
We all know the First Frame is 0% VC because it's blank, but the VC keep increasing during the page load process, so what frame will be consider as 100% visual completeness?
The definition of 100% VC frame is important because it's the baseline for calculate all other frame's visual completeness.
If I have a page that simply print from 1 to 100 with interval 100ms and just enough to fill in the viewport, will the 100% VC frame be the frame that number 100 is printed?

Answer 1

Lighthouse
According to Google's description of the Lighthouse "Speed Index" audit:
Lighthouse uses a node module called Speedline to generate the Speed Index score.
sends Speedline
Speedline's Github readme says
The Speed Index, introduced by WebpageTest.org, aims to solve this issue. It measures how fast the page content is visually displayed. The current implementation is based on the Visual Progress from Video Capture calculation method described on the Speed Index page. The visual progress is calculated by comparing the distance between the histogram of the current frame and the final frame.
(Italics mine.)
a timeline of paints
The Speed Index page goes into painful detail about how visual progress is calculated. Here's a snippet:
In the case of Webkit-based browsers, we collect the timeline data which includes paint rects as well as other useful events.
I believe "timeline data" refers to a JSON object retrieved via the Performance Timeline API.
It seems Lighthouse passes the JSON timeline to Speedline, which then extracts an array of "frames," describing the page load's paint events:
/**
* #param {string|Array<TraceEvent>|{traceEvents: Array<TraceEvent>}} timeline
* #param {Options} opts
*/
function extractFramesFromTimeline(timeline, opts) {
which calculates histograms
Speedline converts the image data from each paint event to an image histogram, interestingly excluding pixels that are "close enough" to pass as white:
/**
* #param {number} i
* #param {number} j
* #param {ImageData} img
*/
function isWhitePixel(i, j, img) {
return getPixel(i, j, 0, img.width, img.data) >= 249 &&
getPixel(i, j, 1, img.width, img.data) >= 249 &&
getPixel(i, j, 2, img.width, img.data) >= 249;
}
A lot of math goes into calculating and comparing histograms. The project maintainer is the right person to ask about that. But this is where the eventual determination of the "visually complete" happens:
// find visually complete
for (let i = 0; i < frames.length && !visuallyCompleteTs; i++) {
if (frames[i][progressToUse]() >= 100) {
visuallyCompleteTs = frames[i].getTimeStamp();
}
}
and infers "progress",
The "progress" of a given frame seems to be calculated by this function:
/**
* #param {Frame} current
* #param {Frame} initial
* #param {Frame} target
*/
function calculateFrameProgress(current, initial, target) {
let total = 0;
let match = 0;
const currentHist = current.getHistogram();
const initialHist = initial.getHistogram();
const targetHist = target.getHistogram();
for (let channel = 0; channel < 3; channel++) {
for (let pixelVal = 0; pixelVal < 256; pixelVal++) {
const currentCount = currentHist[channel][pixelVal];
const initialCount = initialHist[channel][pixelVal];
const targetCount = targetHist[channel][pixelVal];
const currentDiff = Math.abs(currentCount - initialCount);
const targetDiff = Math.abs(targetCount - initialCount);
match += Math.min(currentDiff, targetDiff);
total += targetDiff;
}
}
let progress;
if (match === 0 && total === 0) { // All images are the same
progress = 100;
} else { // When images differs
progress = Math.floor(match / total * 100);
}
return progress;
}
and "visually complete" is the first frame with 100% progress.
Without fully auditing the code, my interpretation is that the "visually complete frame" is the first frame calculated to have the same total difference from the initial frame as the final frame (which is determined by which frames Lighthouse chooses to send to Speedline).
Or, in other words, it's complicated.

Answer 2

Visually complete is when the page in the viewport stops changing. I.e. the visuals are not changing.
It is calculated by taking screenshots throughout the load and comparing them to each other and to the final end state. So yes in your example when all numbers 1-100 are printed and the page stops changing you are “visually complete”.
So if a page loads the data in view quickly but renders “below the fold” content (e.g. off screen images) more slowly then you will get a quick visually complete, even if the page overall load time is still long.
Similarly if most of the on screen content is drawn early on but one small part is drawn later (perhaps a “click to chat” option) you will get mostly visually complete early on and so a good speed index, even if not as good as the above example.
On the other hand if you load fonts, or perhaps a large hero image, last and it redraws large parts of the page in view you will get a slow visual complete time and also a slow speed index score.
More details here: https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/metrics/speed-index

Answer 3

I just got the answer from Lighthouse repo contributor, pls check this link guys.
https://github.com/GoogleChrome/lighthouse/issues/8148