Why can't I get a manually modified MPEG-4 extended box (chunk) size to work? - parsing

Question

Overview
As part of a project to write an MPEG-4 (MP4) file parser, I need to understand how an extended box (or chunk) size is processed within an MP4 file. When I tried to manually simulate an MP4 file with an extended box size, media players report that the file is invalid.
Technical Information
Paraphrasing the MPEG-4 specification:
An MP4 file is formed as a series of objects called 'boxes'. All data is contained in boxes, there is no other data within the file.
Here is a screen capture of Section 4.2: Object Structure, which describes the box header and its size and type fields:
Most MP4 box headers contain two fields: a 32-bit compact box size and a 32-bit box type. The compact box size supports a box's data up to 4 GB. Occasionally an MP4 box may have more data than that (e.g., a large video file). In this case, the compact box size is set to 1, and eight (8) octets are added immediately following the box type. This 64-bit number is known as the 'extended box size', and supports a box's size up to 2^64.
To understand the extended box size better, I took a simple MP4 file and wanted to modify the moov/trak/mdia box to use the extended box size, rather than the compact size.
Here is what the MP4 file looks like before modifying it. The three box headers are highlighted in RED:
My plan was as follows:
Modify the moov/trak/mdia box
In the moov/trak/mdia, insert eight (8) octets immediately following the box type ('mdia'). This will eventually be our extended box size.
Copy the compact box size to the newly-inserted extended box size, adding 8 to the size to compensate for the newly inserted octets. The size is inserted in big-endian order.
Set the compact size to 1.
Modify the moov/trak box
Add 8 to the existing compact box size (to compensate for the eight octets added to mdia).
Modify the moov box
Add 8 to the existing compact box size (again, to compensate for the eight octets in mdia)
Here's what the MP4 file looks like now, with the modified octets are in RED:
What have we done?
We have told the MP4 parser/player to take the moov/trak/mdia box size from the extended field rather than the compact size field, and have increased all parent boxes by eight (8) to compensate for the newly-inserted extended box size in the mdia box.
What's the problem?
When I attempt to play the modified MP4 file I receive error messages from different media players:
Why do the media players see the modified file as invalid MP4?
Did I need to alter any other fields?
Does the extended box size have to be greater than 2^32?
Can it be that only specific box types support extended box size (e.g., Media Data)?

Answer 1

A tip of the hat to #Alan Birtles for pointing out that the chunk offsets would also need to be modified. Indeed, the stco (sample table chunk offset?) box contains absolute file offsets to the data chunks in the mdat box (rather than relative offsets within a box). This can be seen in the specification document:
The chunk offsets need to be increased by the number of octets we added to the file before the mdat box. In our case, this is the eight (8) octet extended box size inserted in the mdia box.
All that remained was to manually change the chunk offsets found in the two stco boxes (both video and audio tracks), adding eight (8) to each chunk offset. Here are the stco boxes before adding 8 to their chunk offsets:
Now the file passes validity tests of both the ffmpeg and ffprobe tools. Interestingly, although VLC succeeds in playing the modified file, other media players (e.g., Windows Media Player, MS Photos, MS Movies & TV, MS MovieMaker) report the file as corrupted. It is not clear why they fail to play the file. Unverified possibilities include:
Not supporting the extended box size for any box other than mdat
Balking if the extended box size is less than 2^32
In summary, if any fields are added to boxes (e.g., extended box size), the stco chunk offsets need to be incremented by the number of octets inserted in the MP4 file preceding each stco box.