Why does bios read at partition's boot record at 0000:7c00 ? What is special about that address ? what ':' doing in referencing an address ?
The simple answer is that 7C00h is 1k (512 bytes for the boot sector plus an additional 512 bytes for possible boot sector use) from the bottom of the original 32k installed memory.
The happy answer is that org 7C00h has become synonymous with boot sector - boot loader programming.
The ":" is a holdover from segmented memory days, when PCs ran in real mode and could only do 64K at a time. The number to the left of the ":" is your segment, the number to the right is your address.
The windows debug command accepts this notation if you want to poke around in memory yourself:
C:\Users\Seth> debug
-d0000:7c00
0000:7C00 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0000:7C10 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0000:7C20 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0000:7C30 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0000:7C40 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0000:7C50 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0000:7C60 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0000:7C70 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
With regard to this particular address, it's just an address that was picked to load the MBR, See: https://web.archive.org/web/20140701052540/http://www.ata-atapi.com/hiwmbr.html
"If an MBR is found it is read into memory at location 0000:7c00 and INT 19 jumps to memory location 0000:7c00"
In the original IBM PC it was thought inconceivable to have more than 32K RAM. In segmented addressing terms this is 0000:8000 where 8000 hex is 32768 decimal. The fashion of the time was for the BIOS POST conclude by loading the Boot Sector of the floppy in A: or the Master Boot Record of the hard drive in C: at the location 512 bytes below the top of memory which means subtract 0200 hex from 8000 hex to get 7C00. So the boot sequence loaded the first valid 512 byte first sector into, and then set the Instruction Pointer to 0000:7C00 to execute it. I used to write the code for these first sectors to load the operating system.
Read this article:
http://en.wikibooks.org/wiki/X86_Assembly/Bootloaders
From the above URL, BIOS (which is effectively PC hardware) will make the jump to memory at 0000:7c00 to continue execution in 16-bit mode.
And to quote from above:
A bootloader runs under certain conditions that the programmer must appreciate in order to make a successful bootloader. The following
pertains to bootloaders initiated by the PC BIOS:
The first sector of
a drive contains its boot loader.
One sector is 512 bytes — the last
two bytes of which must be 0xAA55 (i.e. 0x55 followed by 0xAA), or
else the BIOS will treat the drive as unbootable.
If everything is in
order, said first sector will be placed at RAM address 0000:7C00, and
the BIOS's role is over as it transfers control to 0000:7C00. (I.e. it
JMPs to that address)
So from bootup, if u want the CPU to start executing your code, it has to be located in memory at 0000:7c00. And this part of the code is loaded from the first sector the harddisk - also done by hardware. And it is only the first sector which is loaded, the remaining of other parts of the code then have to be loaded by this initial "bootloader".
More information on harddisk's first sector and the 7c00 design:
http://www.ata-atapi.com/hiwdos.html
http://www.ata-atapi.com/hiwmbr.html
Please don't confuse with the starting up mode of the CPU - the first instruction it will fetch and execute is at physical address 0xfffffff0 (see page 9-5):
http://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-vol-3a-part-1-manual.pdf
and at this stage it is executing non-volatile (meaning you cannot reprogram it easily, and thus not part of bootloader's responsibility) BIOS code.
Related
I am using the MusicSequenceFileCreate method to generate a MIDI file from a beat-based MusicSequence. In iOS 16.0.2 devices, the file that is created has a Sysex MIDI message added (not by me) to the file at time 0:
f0 2a 11 67 40 40 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00 00 00 f7
Sysex messages are manufacturer dependent, a file with this Sysex message can't be read into apps like Nanostudio, Ableton, Zenbeats, which is a problem. It can only be read by GarageBand. Is there a way for MusicSequenceFileCreate to not add this Sysex MIDI message and basically work like it worked before iOS 16?
I have already written a bit on this in https://superuser.com/questions/1389657/backup-access-sms-on-nokia-3310-3g-2017-from-linux-pc ; basically, I'm trying to back up SMS messages on a Nokia 3310 3G onto a Ubuntu 18.04 PC; note that hardware system-on-chip and OS differs by version for a Nokia 3310 (2017):
System on chip / Operating system:
MediaTek MT6260 / Nokia Series 30+ (2G)
Spreadtrum SC7701B / Java-powered Smart Feature OS (3G)
Spreadtrum SC9820A / Yun OS (4G, CMCC)
I have the 3G, so I have a "Smart Feature OS" which apparently is a version of KaiOS (Is there any difference between KaiOS and 'Smart Feature OS'? : KaiOS), which apparently (KaiOS – A Smartphone Operating System | Hacker News) is a fork of Firefox OS.
When you hit Menu > Storage > Create backup (https://www.nokia.com/phones/en_int/support/nokia-3310-3g-user-guide/create-a-backup) on this phone, it generates a folder with files in it, named like this:
$ tree All-backup_01-01-2019_20-18-54
All-backup_01-01-2019_20-18-54
├── ibphone_head.in
├── phonebook.ib
└── sms.ib
I've never heard of .ib files before, and I was hoping someone here knew what they are. A quick look suggests IB File Extension - Open .IB File (InterBase Database), however I tried using these .ib files with http://fbexport.sourceforge.net/fbexport.php "Tool for exporting and importing data with Firebird and InterBase databases", and I get:
Engine Code : 335544323
Engine Message :
file ./All-backup_01-01-2019_20-18-54/phonebook.ib is not a valid database
So, that's not it.
Here is a hexdump of ibphone_head.in - looks like there is no personal identifying info here:
$ hexdump -C All-backup_01-01-2019_20-18-54/ibphone_head.in
00000000 00 00 00 00 41 00 6c 00 6c 00 2d 00 62 00 61 00 |....A.l.l.-.b.a.|
00000010 63 00 6b 00 75 00 70 00 5f 00 30 00 31 00 2d 00 |c.k.u.p._.0.1.-.|
00000020 30 00 31 00 2d 00 32 00 30 00 31 00 39 00 5f 00 |0.1.-.2.0.1.9._.|
00000030 32 00 30 00 2d 00 31 00 38 00 2d 00 35 00 34 00 |2.0.-.1.8.-.5.4.|
00000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00000100 00 00 00 00 45 00 3a 00 5c 00 42 00 61 00 63 00 |....E.:.\.B.a.c.|
00000110 6b 00 75 00 70 00 73 00 5c 00 41 00 6c 00 6c 00 |k.u.p.s.\.A.l.l.|
00000120 2d 00 62 00 61 00 63 00 6b 00 75 00 70 00 5f 00 |-.b.a.c.k.u.p._.|
00000130 30 00 31 00 2d 00 30 00 31 00 2d 00 32 00 30 00 |0.1.-.0.1.-.2.0.|
00000140 31 00 39 00 5f 00 32 00 30 00 2d 00 31 00 38 00 |1.9._.2.0.-.1.8.|
00000150 2d 00 35 00 34 00 00 00 00 00 00 00 00 00 00 00 |-.5.4...........|
00000160 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00000200 00 00 00 00 30 30 30 31 2e 30 30 30 30 33 00 00 |....0001.00003..|
00000210 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00000230 00 00 00 00 00 00 6d 6d 69 6b 65 79 62 61 63 6b |......mmikeyback|
00000240 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000250 00 00 00 00 00 00 03 00 00 00 00 00 b0 cd 09 00 |................|
00000260 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000270 00 00 00 00 00 00 00 00 f3 dd 00 00 7e 2f 00 00 |............~/..|
00000280 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
000002c4
So, it seems that strings are encoded with 2 bytes, "wide char"; and for the most part, ibphone_head.in seems to just encode the name of its containing/parent folder, All-backup_01-01-2019_20-18-54.
Here is a hexdump of phone book, where I've anonymized names to AAAAAAAAA and BBB:
$ hexdump -C -n 1900 All-backup_01-01-2019_20-18-54/phonebook.ib
00000000 70 00 68 00 6f 00 6e 00 65 00 62 00 6f 00 6f 00 |p.h.o.n.e.b.o.o.|
00000010 6b 00 2e 00 69 00 62 00 00 00 00 00 00 00 00 00 |k...i.b.........|
00000020 00 00 00 00 01 00 00 00 30 f8 04 00 62 01 00 00 |........0...b...|
00000030 62 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |b...............|
00000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00000240 00 00 00 00 98 03 00 00 01 00 00 00 ff ff ff ff |................|
00000250 ff ff ff ff 01 00 01 00 00 00 00 00 00 00 00 00 |................|
00000260 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00000360 d9 d4 37 46 00 00 00 00 00 00 01 02 00 00 04 01 |..7F............|
00000370 07 12 80 88 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000380 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
000003b0 09 00 41 00 41 00 41 00 41 00 41 00 41 00 41 00 |..A.A.A.A.A.A.A.|
000003c0 41 00 41 00 00 00 00 00 00 00 00 00 00 00 00 00 |A.A.............|
000003d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
000005e0 00 00 00 00 00 00 00 00 00 00 00 00 ff ff ff ff |................|
000005f0 ff ff ff ff 98 03 00 00 01 00 00 00 ff ff ff ff |................|
00000600 ff ff ff ff 04 00 01 00 00 00 00 00 00 00 00 00 |................|
00000610 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00000710 d9 d4 37 46 00 00 00 00 00 00 01 02 00 00 06 11 |..7F............|
00000720 83 29 23 13 58 f9 00 00 00 00 00 00 00 00 00 00 |.)#.X...........|
00000730 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00000760 03 00 42 00 42 00 42 00 00 00 00 00 |..B.B.B.....|
0000076c
Here it seems d9 d4 37 46 is a delimiter for entries, and entries seem to be 0x3b0 = 944 bytes apart; cannot tell where the actual phone number is stored, though.
I won't be pasting the hexdump contents of sms.ib, because I fear to reveal more personal info than I intend to.
However, maybe what has been posted already, will help someone see if this is an already established file format? In any case, I'd like to convert the contents of these files to plain text ...
The phonebook.ib is a proprietary file format. I did some analysis and got to the point where I am able to extract names and phones numbers from it.
Header is 580 bytes and doesn't seem to contain anything interesting.
Entries appear to begin with the entry length, encoded as 3 4-bit decimal nibbles followed by another digit (version number maybe?).
In my sample file all entries were 940 bytes and therefore had 94 03 as their first two bytes. Other field entries I identified at different offsets:
entry+0x12a [1 byte]: Number of bytes representing phone number.
entry+0x12b [1 byte]: Seems like a two-digit decimal coded flag. If high nibble is set (e.g. 10) then phone number begins with +.
entry+0x12c: Phone number, decimal coded. For example, 123456 would appear as 21 43 65. Special digits:
a is *
b is #
f is ignored (seen as a last digit when the number of digits is not even).
entry+0x16c [1 byte]: Name length.
entry+0x16e: Name in UTF-16 (i.e. 2 bytes per char).
Taking your snippet as an example:
0x36e is the phone length, 4 bytes.
0x36f is the extra flag, high nibble is 0 so no + prefix.
0x370 is where the phone number begins, 07 12 80 88 translates to 70210888.
A simple reference parser can be found at my repo: https://github.com/yossigo/phonebook_ib_export
I was about to make the opposite script than Yossi's to convert .vcf files back into the .is format when I found a workaround to backup and restore all contacts to and from the vcf file format with the Nokia 3310 3G :
BACKUP CONTACTS TO VCF FILE: in your contacts, click the left button to share, select all of them using the option in the left button menu, send via Bluetooth to any device, you can then retrieve the vcf file from the other Bluetooth device OR from the /vCard/ folder at the root of your phone (sd card if one is plugged)
RESTORE CONTACTS FROM VCF FILE: copy the file anywhere on the phone or sd card, unplug the phone and use the embedded file browser to find the file, then follow Nokia’s engineer logic by not using the Open button but instead click on the left button and use the option Save vCard
We are developing a system with a custom processor, Microblaze and some peripherals in VC709 FPGA using Xilinx Vivado. We are using two 'PCIe : BARs' in 'AXI Bridge for PCI express'.
Initially the command 'lspci -vv' used to show memory regions in the Ubuntu teminal.
$ lspci -vv
0a:00.0 Memory controller: Xilinx Corporation Device 7038 | 0a:00.0 Memory controller: Xilinx Corporation Device 7018
Subsystem: Xilinx Corporation Device 0007 | Subsystem: Xilinx Corporation Device 0008
Physical Slot: 3 | Physical Slot: 3
Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- Pa| Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- Pa
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-| Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
Interrupt: pin A routed to IRQ 16 | Interrupt: pin A routed to IRQ 16
Region 0: Memory at fbff0000 (32-bit, non-prefetchable) [size=| ----------------------------------------------------------------------
Region 1: Memory at fb800000 (32-bit, non-prefetchable) [size=| ----------------------------------------------------------------------
Capabilities: <access denied> | Capabilities: <access denied>
Kernel modules: riffa
I have created a copy of the project and edited something in the design (which I don't remember unfortunately) and now the result of 'lspci -vv' is as follows. Please note that Region 0 and Region 1 are missing now.
$ lspci -vv
0a:00.0 Memory controller: Xilinx Corporation Device 7018
Subsystem: Xilinx Corporation Device 0008
Physical Slot: 3
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin A routed to IRQ 16
Capabilities: <access denied>
Kernel modules: riffa
Q: What could be the reason?
Notes:
The block design and connections are exactly same for both the projects
The options for 'AXI bridge for PCI express Gen3.0 subsystem' are the same in both the projects
Thanks :)
Extra information required by the community
$ sudo lspci -vv
0a:00.0 Memory controller: Xilinx Corporation Device 7018
Subsystem: Xilinx Corporation Device 0008
Physical Slot: 3
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin A routed to IRQ 16
Capabilities: [80] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [90] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [c0] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal+ Fatal+ Unsupported-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 4096 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x8, ASPM unknown, Latency L0 unlimited, L1 unlimited
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range B, TimeoutDis+
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
UESvrt: DLP- SDES+ TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
Kernel modules: riffa
$ sudo od -tx1z -Ax /proc/bus/pci/0a/00.0
000000 ee 10 38 70 43 00 10 00 00 00 80 05 10 00 00 00 >..8pC...........<
000010 00 00 ff fb 00 00 00 00 00 00 00 00 00 00 00 00 >................<
000020 00 00 00 00 00 00 00 00 00 00 00 00 ee 10 07 00 >................<
000030 00 00 00 00 80 00 00 00 00 00 00 00 0b 01 00 00 >................<
000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 >................<
*
000080 01 90 03 00 08 00 00 00 00 00 00 00 00 00 00 00 >................<
000090 05 c0 80 00 00 00 00 00 00 00 00 00 00 00 00 00 >................<
0000a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 >................<
0000b0 11 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 >................<
0000c0 10 00 02 00 02 80 00 00 16 58 09 00 83 f0 43 00 >.........X....C.<
0000d0 40 00 42 10 00 00 00 00 00 00 00 00 00 00 00 00 >#.B.............<
0000e0 00 00 00 00 12 00 00 00 00 00 00 00 0e 00 00 00 >................<
0000f0 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 >................<
000100 01 00 02 30 00 00 10 00 00 00 10 00 20 00 00 00 >...0........ ...<
000110 00 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 >. ..............<
000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 >................<
*
000150 03 00 01 30 00 00 00 00 00 00 00 00 00 00 00 00 >...0............<
000160 04 00 41 27 00 00 00 00 f0 80 0b 00 00 00 00 00 >..A'............<
000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 >................<
*
0001b0 00 00 00 00 00 00 00 00 18 00 01 30 00 00 00 00 >...........0....<
0001c0 16 00 01 30 07 00 00 00 00 00 00 00 00 01 00 00 >...0............<
0001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 >................<
*
000270 00 00 00 00 17 00 01 30 05 00 00 00 00 00 00 00 >.......0........<
000280 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 >................<
*
000300 19 00 01 00 00 00 00 00 00 00 00 00 7f 7f 7f 7f >................<
000310 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 00 00 00 00 >................<
000320 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 >................<
*
0003c0 02 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 >................<
0003d0 00 00 00 00 01 00 00 80 00 00 00 00 00 00 00 00 >................<
0003e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 >................<
*
001000
The memory space is not enabled in your device (I did not notice this in the first place).
In the device listing on the left, in the Control line, you have "Mem+" but on the right you have "Mem-" (ditto for I/O access). In the PCI spec, the Mem flag is bit 1 of the Command register.
Typically, in linux, a driver calls pci_enable_device function, which brings the device online and enables memory and/or I/O access as directed by the driver (that is, it writes a 1 into either or both bits). See also Documentation/PCI/pci.txt for guidance on writing drivers.
My guess is that lspci just isn't showing the BAR because the device's memory space isn't enabled; it appears that the same address -- 0xfbff0000 -- has been assigned to BAR 0 at least [see offset 0x10 in the /proc/bus/pci dump]. There is no address programmed in the BAR 1 address slot at offset 0x14.
I can see that this is a non-prefetchable 32-bit BAR
I found a similar problem when using an Altera FPGA. If I made the BARs 32-bit, they did not appear when I did lspci. When I changed to 64-bit prefetchable BARs, I saw them as expected when I run lspci.
I am trying to display the video my (PAUL) drone is sending to my over a UDP connection.
Frames look like (hexdump):
00 00 01 a1 00 1d 00 03
90 1a 00 00 a0 8a dc 0c
00 00 00 00 03 00 00 00
d0 02 40 02 00 04 00 00
90 1a 00 00 19 00 ......
Frames always start (trough my observation) with:
00 00 01 a
Are these NAL units?
I am want to display this in VLC player but don't know how to stream it to the VLC media player.
It looks a lot like a valid h.264 bitstream, but its not. 00 00 01 does look like a start code, But after a start code the first bit MUST be 0. a in binary is 1010 hence not valid.
My project is swift-only (for the code I wrote, at least). At the start of the app, I download some json to show content. I deserialize this content with swift 4 Coder protocol. This has worked for some time, but just now I got an unexpected stack-buffer-overflow error:
==44088==ERROR: AddressSanitizer: stack-buffer-overflow
while deserializing one of the objects, in one of the background threads.
Based on this, I have 2 questions:
How can I ensure this doesn't happen again?
Is there a way to reproduce it?
More info:
I have this summary, but I'm not sure what to make of it:
SUMMARY: AddressSanitizer: stack-buffer-overflow JsonClass.swift in _T06MyApp11JsonClassVACs7Decoder_p4from_tKcfC
Shadow bytes around the buggy address:
0x100026a904d0: 00 02 f2 f2 f2 f2 f2 f2 f2 f2 00 00 00 00 00 00
0x100026a904e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x100026a904f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x100026a90500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x100026a90510: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x100026a90520: 00 00 00 00 00 00 00 00 00 00 00 00 00 00[f2]f2
0x100026a90530: f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 00 00
0x100026a90540: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x100026a90550: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x100026a90560: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x100026a90570: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
EDIT:
It reproduced every time (in the simulator). Then I cleaned the build and deleted the derived data folder and it didn't happen since. I'd still like to know if I need to worry for a bug in production...
First I'll briefly answer your questions
How can I ensure this doesn't happen again?
This specific defect you can fix and write unit tests for to prevent having a regression. In general, however, mistakes happen; you cannot prevent them, only mitigate them. Use tools and warnings to try and identify them early on. You are already doing a good thing by employing Address Sanitizer (you should also check out Undefined Behavior Sanitizer and Thread Sanitizer).
Is there a way to reproduce it?
Address Sanitizer will report it 100% of the time it happens. Unfortunately, it sounds like it depends on what data you are working with. Maybe try some malformed data to try and coax it into happening. You can write Unit Tests (be sure to build your tests with Asan enabled) with various types of data. Can't say more without seeing code.
What is the problem?
You definitely have a bug. Asan does not report false positives. It sounds like it might only happen for bad data, but you should never trust you will always have good data.
It is not easy to help you fix your problem without seeing code. Asan reporting a stack buffer overflow is usually one of these things (in my experience):
Allocating a variable on the stack and keeping a reference to it. When the stack frame it was allocated in is destroyed the reference is no longer valid.
Having an array on the stack and going out of bounds.
(rarely) An improper cast of a stack object to an object larger size and accessing a "member" that is beyond the stack top.
If you are running though Xcode it should break when Asan aborts. you should be able to see the stack trace and find where in your code this is happening.
Additional tools to help identify the problem
Compiler warnings
Turn these on and up as much as possible. Often when you do things like take a reference to something with automatic storage duration the compiler can warn you about it.
Clang Static Analyzer
This is also built into Xcode and can be ran by the Product -> Analyize menu. This should populate issues it find in the "issue navigator" on the left column of Xcode. Clang is pretty good at identifying memory errors
31==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffcfa87e770
at pc 0x000000344049 bp 0x7ffcfa87e6b0 sp 0x7ffcfa87e6a8
READ of size 1 at 0x7ffcfa87e770 thread T0
#3 0x7f35b846e0b2 (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
Address 0x7ffcfa87e770 is located in stack of thread T0 at offset 80 in frame
This frame has 3 object(s):
[32, 33) 'ref.tmp'
[48, 80) 'agg.tmp' <== Memory access at offset 80 overflows this variable
[112, 144) 'agg.tmp8'
HINT: this may be a false positive if your program uses some custom stack, unwind mechanism, swapcontext or vfork.
(longjmp and C++ exceptions *are* supported)
Shadow bytes around the buggy address:
0x10001f507c90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10001f507ca0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10001f507cb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10001f507cc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10001f507cd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x10001f507ce0: 00 00 00 00 f1 f1 f1 f1 01 f2 00 00 00 00[f2]f2
0x10001f507cf0: f2 f2 00 00 00 00 f3 f3 f3 f3 f3 f3 00 00 00 00
0x10001f507d00: 00 00 00 00 00 00 00 00 00 00 00 00 f1 f1 f1 f1
0x10001f507d10: 04 f2 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10001f507d20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10001f507d30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==31==ABORTING
i am also suffering from the same problem in leet code.