I am developing a VPN (iOS Network Extension), and using C/C++ to read file-descriptor directly (instead of Swift), currently it successfully captures device's request Packets, but I don't know how to parse iOS's packets, I could not even find what network layer or protocol the packets are formatted in.
I converted Packet's binary into Hex to be able to decode with online tools; below are samples of what I need to parse:
000000024500003B5461000040110C390A07000208080808FA2D0035002739B4DE790100000100000000000003777777056170706C6503636F6D0000010001
000000024500003CBAE200004011A5B60A07000208080808E48A0035002892DAE43B01000001000000000000037777770669636C6F756403636F6D0000010001
00000002450000375324000040110D7A0A07000208080808DD7F003500232BBA841801000001000000000000056170706C6503636F6D0000010001
But when tried parsing with online decoder, they fail saying invalid packet.
What network layer or protocol is above?
Note that above are 3 packet samples (not one splitted by me).
It's tun-layer protocol with 4 bytes prefix:
1. Once we use C/C++ to read file-descriptor, in NEPacketTunnelProvider like:
let tunFd = self.packetFlow.value(forKeyPath: "socket.fileDescriptor") as! Int32
// ... pass above to C/C++ backend
Instead of using Swift like:
self.packetFlow.readPackets { [weak self] (packets: [Data], protocols: [NSNumber]) in
// Handle packets here...
}
2. There are 4 additional bytes prefixed to the tun-layer packet (e.g. 00 00 00 02), each time we read packets.
3. To allow most online-tools to understand the packet, remove those starting 4 bytes, and instead prefix it with Mac-Header-Hex like:
01 00 5E 00 00 09 C2 01 17 23 00 00 08 00
Note that by doing above, we convert initial tun-layer packet to a tap-layer packet.
Also, remember to prefix again those 4 bytes, once writing a packet into the file-descriptor (after removing Mac-Header).
Update 2021; Apple discourages accessing file-descriptor directly (and it may be removed in future iOS releases).
Related
Here is the poc piece of code I had:
let text = "Hello, World"
let data = text.data(using: .utf8)! as CFData
let newData = Data(data as NSData)
print(newData.base64EncodedString())
return true
When I run this in a device running iOS 13 or 14, I get the following output:
SGVsbG8sIFdvcmxk
which is the correct base64 for "Hello, World". However, when I run the exact same code in iOS 12, I get the following:
SAAAAAAAAAAMAAAA
which is quite meaningless in text (utf-8) format, but the hex for this is:
48 00 00 00 00 00 00 00 0c 00 00 00
So I'm not sure what is happening here, and even if it was just filling in zeroes, why is it just not zeroes all the way through? While I get that what I'm doing is wrong, and the following works perfect on all versions:
print((data as Data).base64EncodedString())
I'm just curious as to why is it acting differently on the versions. The documentation does not seem to mention any particular behavioral caveat in an older version.
Please stop dealing with CFData and NSData
In Swift the recommended API to convert a String to Data is
let text = "Hello, World"
let data = Data(text.utf8)
print(data.base64EncodedString())
It works reliably in all versions and it avoids the optional.
The problem is the unnecessary CF - NS bridging. It seems that a zero-terminator is inserted after the H somewhere.
This is not a programming related question. It's a question about where to find the right documentation.
If this is not the right place to ask, please let me know and I'll move the question
I'm working on a C++-Deamon for parsing recieved BLE Advertising Packages (like hcidump from bleuz). For this I'm focusing on parsing only LE Advertising Report events (Bluetooth Core Specification 7.7.65.2, S. 2382 ff)
E. g: I need to extract the local name and the 16-bit UUID. This information can be found (if broadcasted) in the field "data".
I can extract the data part from the Advertising Event Packet (since it's described in the Core Specification how these packages are arranged) but I can't parse the data. I wasn't able to find a description of the codes used in this data frame.
I also went through the Core Specification Supplement, but I couldn’t find the corresponding codes. I only found some examples:
See Site 26: Apparently the byte 0x09 indicates that the following n bytes are the broadcasted name of the BLE-Device.
Or Site 25: 0x03 seems to indicate that a "Complete list of 16-bit Service UUIDs" is following.
So far I gained my knowledge about this by just going throught these (incomplete) list of examples I found and from looking at the source code of hcidump.c.
Now the question is: Where can I find the official complete list to those bytes/codes?
Just an example of the package content I revieve:
Respone: 04 3E 1C 02 01 04 01 55 19 19 E4 8D FE 10 0F 09 54 65 73 74
20 49 50 53 50 20 6E 6F 64 65 C5
It should be able to extract the following data
HCI Event: LE Meta Event (0x3e) plen 28
LE Advertising Report
SCAN_RSP - Scan Response (4)
bdaddr FE:8D:E4:19:19:55 (Random)
Complete local name: 'Test IPSP node'
RSSI: -61
After going throught the whole Bluetooth SIG, I finally found the site where these codes are described:
https://www.bluetooth.com/specifications/assigned-numbers/generic-access-profile/
I'd like to parse a proprietary message with the following format. The messages are of variable length from 8bytes to 16 bytes. If the message is more than 8 bytes, it is put into multi packet message.
Byte0 Byte1 Byte2 Byte3 Byte4 Byte5 Byte6 Byte7-14 Byte15
Header Header Flags SlaveID Sequence Command Command Data Chksum
I send these messages to a device and expect a response from said device with similar format. I know what the messages are supposed to say, but I'm really struggling to find a away to parse the data in real time and display the information in somesort of GUI framework. Currently, I'm planning on dumping the data into a .txt/.csv file and post-processing with matlab.
I'm a hardware guy. This is not my strong suit.
Example:
Command, Reponse
0xAC 04 01 01 00 8C 01 01 chksum
0x04 AC 83 01 00 8C 01 01 0B D9 chksum
0x8c01 command to test power supply 0x01
Device echos the command and power supply it was checking and also returns the bit-shifted value of the power supply voltage 0x0BD9
I am new to the Bluetooth system and I am trying to understand the data used for the new Apple's technology : iBeacon.
There is already some nice answers which explain how it works and I have been reading everything I could find (especially the Bluetooth Specification). Still, I am missing some points and I will go for an example first : (I am using the Set Advertising Data Command, it misses here the hcitool cmd before the OGF)
0x08 0x0008 1E 02 01 1A 1A FF 4C 00 02 15 E2 C5 6D B5 DF FB 48 D2 B0 60 D0 F5 A7 10 96 E0 00 00 00 00 C5 00
I will list here what I didn't understand or find information about.
Is there any information anywhere about the OGF (here it is 0x08)? I know it stands for OpCode Group Field, but contratry to the OCF which follows the OGF, I didn't find anything.
What does the 02 01 1A 1A bytes line stand for? I know that the first byte, 1E, tells the length of the rest of the data and after that line, starting with FF, you get the manufacturer specific data. But I couldn't find anything about those 4 bytes.
How does the power byte work? Here it is C5. I know that what I get is the dBm value when ranging my iBeacon (on my iPhone for instance). And I know that the higher the value (on that power byte), the higher the power which means more accuracy but also more energy consumption. But how do you use that byte? What are the min and max values you can set? Or is there any kind of formula there? Do you get a set dBm value (at one meter from your iBeacon) for a set value on the byte?
Thank you.
Answers to the first two questions can be found in the gigantic Bluetooth 4.0 Core spec.
The OGF of 0x08 groups OCF commands for LE Controllers:
For the LE Controller Commands, the OGF code is defined as 0x08.
(Bluetooth Specification Version 4.0 [Vol 2], page 1114)
Because the 0x0008 OCF command is a controller command, you have to use the 0x08 OGF code with it. Confused? Forget it. Just know you use 0x08 0x0008 to set the advertising data using hcitool.
The byte sequence starting the advertisement is as follows:
1E Number of bytes that follow in the advertisement
02 Number of bytes that follow in first AD structure
01 Flags AD type
1A Flags value 0x1A = 000011010
bit 0 (OFF) LE Limited Discoverable Mode
bit 1 (ON) LE General Discoverable Mode
bit 2 (OFF) BR/EDR Not Supported
bit 3 (ON) Simultaneous LE and BR/EDR to Same Device Capable (controller)
bit 4 (ON) Simultaneous LE and BR/EDR to Same Device Capable (Host)
1A Number of bytes that follow in second (and last) AD structure
FF Manufacturer specific data AD type
4C Company identifier code LSB
00 Company identifier code MSB (0x004C == Apple)
02 Byte 0 of iBeacon advertisement indicator
15 Byte 1 of iBeacon advertisement indicator
-- Bluetooth Specification Version 4.0 [Vol 3], "ADVERTISING AND SCAN RESPONSE DATA FORMAT" p. 375
-- Bluetooth Specification Version 4.0 [Vol 3], Appendix C (NORMATIVE): EIR AND AD FORMATS", p. 401
The power field is simply a one byte two's complement number representing the "measured power" in RSSI at one meeter away. In simpler terms
Here is how that works:
Hold an iBeacon scanner (like Locate for iBeacon for iOS) one meter away from your iBeacon transmitter.
Read its signal strength in RSSI. It will be a number between -1 and -127. (-1 is very strong, -127 is very weak)
Convert this number into a hex using two's complement. (Add 256 then convert to hex)
Note: The power field can be 80-FF. If it is 00, iOS will not do a distance calculation at all. You can read more on how this is used here.
Recently I had chance to work with two devices that are streaming the H264 through RTSP.
And I've ran into some problem trying to decompress this stream using FFmpeg library.
Every time the "avcodec_decode_video2" is called - FFmpeg just says something like:
[h264 # 00339220] no frame!
My raw H264 stream I frame data starts like this: "65 88 84 21 3F F8 F8 0D..."
(as far as I understand this 0x65 indicates that it's a IDR frame?)
Other frames for one device starts like: "41 9A 22 07 F3 4E 48 CC...."
and for other device - like this: "61 9A 25 C1 1C 45 62 39...."
Am I missing some frame data here?
Does FFmpeg needs to have some extra parameters set up?
I was expecting at least "00 00 00 01" bytes at the start for the frame data... but this is what I've got..
Ok, managed to make things working.
I needed to include the sequence (SPS) and picture parameter sets
(PPS) for my frame data before sending frame to the FFmpeg.
I needed to add 4 extra bytes "00 00 00 01" after SPS and PPS data.
Here is a little picture showing what I mean:
Bytes "65 88..." is where my original frame data begins.
This SPS and PPS information was not included in RTP packet. I'm using Live555 library for RTSP streaming, so I've used subsessions "fmtp_spropparametersets" function to get what I need. This information was Base64 encoded. (Sample: Something like this "Z0KAKNoC0EkQ,aM48gA==") Note that there are two "parameters" SPS and PPS seperated by "," and those parameters doesn't have a "00 00 00 01" included, so you need to add them.
Some code sample (I'm using Qt library here):
QByteArray ba = pSubSession->fmtp_spropparametersets();
QList<QByteArray> recordsList = ba.split(',');
for (int i = 0; i < recordsList.size(); i++)
{
mExtraData.append(char(0x00));
mExtraData.append(char(0x00));
mExtraData.append(char(0x00));
mExtraData.append(char(0x01));
mExtraData += QByteArray::fromBase64(recordsList.at(i));
}
Now for every frame I do something like this:
QByteArray ba = QByteArray(4, 0); // Prepare the "00 00 00 01"
ba[3] = 0x01;
mpTrackVideo->buffer.insert(0, mExtraData);
mpTrackVideo->buffer.insert(mExtraData.size(), ba);
Year ago I thought I had H264 stream support integrated in my project till I've had chance to test it with some other devices...
So you need to keep in mind that some devices might send SPS and PPS data for every I frame... and some might not!