I see a few implementations, but I decided to look at exactly how the specification calls out the FCS for encoding.
So say my input is as follows:
dst: 0xAA AA AA AA AA AA
src: 0x55 55 55 55 55 55
len: 0x00 04
msg: 0xDE AD BE EF
concatenating this in the order that seems to be specified in the format (and the order expressed in the spec later on) seems to indicate my input is:
M(x) = 0xAA AA AA AA AA AA 55 55 55 55 55 55 00 04 DE AD BE EF
a) "The first 32 bits of the frame are complemented."
complemented first 32 MSB of M(x): 0x55 55 55 55 AA AA 55 55 55 55 55 55 00 04 DE AD BE EF
b) "The n bits of the protected fields are then considered to be the coefficients of a polynomial M(x) of
degree n – 1. (The first bit of the Destination Address field corresponds to the x(n–1) term and the last
bit of the MAC Client Data field (or Pad field if present) corresponds to the x0 term.)"
I did this in previous. see M(x)
c) "M(x) is multiplied by x^32 and divided by G(x), producing a remainder R(x) of degree <=31."
Some options online seem to ignore the 33rd bit to represent x^32. I am going to ignore those simplified shortcuts for this exercise since it doesn't seem to specify that in the spec.
It says to multiply M(x) by x^32. So this is just padding with 32 zeroes on the LSBs. (i.e. if m(x) = 1x^3 + 1, then m(x) * x^2 = 1x^5 + 1x^2 + 0)
padded: 0x55 55 55 55 AA AA 55 55 55 55 55 55 00 04 DE AD BE EF 00 00 00 00
Next step is to divide. I am dividing the whole M(x) / G(x). Can you use XOR shifting directly? I see some binary division examples have the dividened as 101 and the divisior as 110 and the remainder is 11. Other examples explain that by converting to decimal, you cannot divide. Which one is it for terms for this standard?
My remainder result is for option 1 (using XOR without carry bit consideration, shifting, no padding) was:
0x15 30 B0 FE
d) "The coefficients of R(x) are considered to be a 32-bit sequence."
e) "The bit sequence is complemented and the result is the CRC."
CRC = 0xEA CF 4F 01
so my entire Ethernet Frame should be:
0xAA AA AA AA AA AA 55 55 55 55 55 55 00 04 DE AD BE EF EA CF 4F 01
In which my dst address is its original value.
When I check my work with an online CRC32 BZIP2 calculator, I see this result: 0xCACF4F01
Is there another option or online tool to calculate the Ethernet FCS field? (not just one of many CRC32 calculators)
What steps am I missing? Should I have padded the M(x)? Should I have complemented the 32 LSBs instead?
Update
There was an error in my CRC output in my software. It was a minor issue with copying a vector. My latest result for CRC is (before post-complement) 35 30 B0 FE.
The post-complement is: CA CF 4F 01 (matching most online CRC32 BZIP2 versions).
So my ethernet according to my programming is currently:
0xAA AA AA AA AA AA 55 55 55 55 55 55 00 04 DE AD BE EF CA CF 4F 01
The CRC you need is commonly available in zlib and other libraries as the standard PKZip CRC-32. It is stored in the message in little-endian order. So your frame with the CRC would be:
AA AA AA AA AA AA 55 55 55 55 55 55 00 04 DE AD BE EF B0 5C 5D 85
Here is an online calculator, where the first result listed is the usual CRC-32, 0x855D5CB0.
Here is simple example code in C for calculating that CRC (calling with NULL for mem gives the initial CRC):
unsigned long crc32iso_hdlc(unsigned long crc, void const *mem, size_t len) {
unsigned char const *data = mem;
if (data == NULL)
return 0;
crc ^= 0xffffffff;
while (len--) {
crc ^= *data++;
for (unsigned k = 0; k < 8; k++)
crc = crc & 1 ? (crc >> 1) ^ 0xedb88320 : crc >> 1;
}
return crc ^ 0xffffffff;
}
The 0xedb88320 constant is 0x04c11db7 reflected.
The actual code used in libraries is more complex and faster.
Here is the calculation of that same CRC (in Mathematica), using the approach described in the IEEE 802.3 document with polynomials, so you can see the correct resulting powers of x used for the remainder calculation:
If you click on the image, it will embiggen to make it easier to read the powers.
The confusing factor here is the 802.3 spec. It mentions that first bit is LSB (least significant bit == bit 0) only in one place, in section 3.2.3-b, and it mentions that for CRC, "the first bit of the Destination Address field corresponds to the x^(n-1) term", so each byte input to the CRC calculation is bit reflected.
Using this online calculator:
http://www.sunshine2k.de/coding/javascript/crc/crc_js.html
Select CRC-32 | CRC32, click on custom, input reflected on, result reflected off. With this data:
AA AA AA AA AA AA 55 55 55 55 55 55 00 04 DE AD BE EF
per the spec, the calculated CRC is 0x0D3ABAA1, stored and transmitted as shown:
bit 0 first | bit 7 first
AA AA AA AA AA AA 55 55 55 55 55 55 00 04 DE AD BE EF | 0D 3A BA A1
To simplify the output to always transmit bit 0 first, bit reflect the CRC bytes:
bit 0 first | bit 0 first
AA AA AA AA AA AA 55 55 55 55 55 55 00 04 DE AD BE EF | B0 5C 5D 85
Note that the bit 0 always first method results in transmitted bits identical to the spec.
Change result setting for the CRC calculator, input reflected on, result reflected on. With this data:
AA AA AA AA AA AA 55 55 55 55 55 55 00 04 DE AD BE EF
the calculated CRC is 0x855D5CB0, stored least significant byte first and transmitted as shown:
bit 0 first | bit 0 first
AA AA AA AA AA AA 55 55 55 55 55 55 00 04 DE AD BE EF | B0 5C 5D 85
For verifying received data, rather than compare a calculated CRC of received data versus the received CRC, the process can calculate a CRC on received data and CRC. Assuming the alternative setup where all bytes are received bit 0 first, then with this received frame or any frame without error
bit 0 first | bit 0 first
AA AA AA AA AA AA 55 55 55 55 55 55 00 04 DE AD BE EF | B0 5C 5D 85
the calculated CRC will always be 0x2144DF1C. In the case of a hardware implementation, the post complement of the CRC is usually performed one bit at a time as bits are shifted out, outside of the logic used to calculate the CRC, and in this case, after receiving a frame without error, the CRC register will always contain 0xDEBB20E3 (0x2144DF1C ^ 0xFFFFFFFF). So verification is done by computing CRC on a received frame and comparing the CRC to a 32 bit constant (0x2144DF1C or 0xDEBB20E3).
Related
My friend and I are trying to communicate with a drone (APM 2.8 flight controller - ArduCopter V3.2.1 (36b405fb)) with MavLink. After several hours, we managed to handle messages and make ByteBuffers a little more meaningful but we're stuck on parsing one response that seems to didn’t fit with the documentation.
When performing a PARAM_REQUEST_LIST request, the APM sends back (a lot of) PARAM_VALUE (#22) responses, but the format of what we're receiving is really not what we’ve expected
The payload
<Buffer cd cc 4c 3f 32 01 36 00 45 4b 46 5f 43 48 45 43 4b 5f 54 48 52 45 53 48 09>
The expected object (doc)
PARAM_VALUE
{
param_id char[16]
param_value float
param_type uint8_t
param_count uint16_t
param_index uint16_t
}
So param_id = cd cc 4c 3f 32 01 36 00 45 4b 46 5f 43 48 45 43 = ÍÌL?2(soh)6(nul)EKF_CHEC
Although the 8 last characters are OK, it seems that the first 8 characters are absolutely everything but not the beginning of a char[16] - by the way the 8th is always (nul) … And it’s the same for a param_count = 17747 when param_index = 18441 (?!), the different param_count values in every response, or param_type=82 when this value must be 0 < x <= 10 (doc)
We already managed to parse other requests without any problem so we thought we’d understood the way MavLink works but I'm doubting right now … Did we miss something obvious ? Or is there a problem with the drone ?
For those who encountered the same problem, this page is the answer. According to this, my object looks more like that when reordered
PARAM_VALUE
{
param_value float
param_count uint16_t
param_index uint16_t
param_id char[16]
param_type uint8_t
}
So param_id = 45 4b 46 5f 43 48 45 43 4b 5f 54 48 52 45 53 48 = EKF_CHECK_THRESH
I want to generate random bytes in Ruby, but I also want to insert some constant values into specific positions of the random bytes.
random_hex_string = SecureRandom.hex (length)
random_hex_string.insert(0,"0102")
random_hex_string.insert(30*1,"36")
So I generate random hex bytes and insert my hex values there. The problem is that I have now a string not a byte array. So when I print it:
File.open("file.txt", 'w+b') do |f|
f.write(random_hex_string)
It - not surprisingly - converts the hex string into binary then writes it. So my hex values are not kept. To be more clear, when I write my hex string to the file, and then I want to see the same hex values when I hex dump the file. How can I do this?
You can turn it into a single element array and pack it as hex. For instance, when I run your code:
require 'securerandom'
length = 2 ** 6
random_hex_string = SecureRandom.hex (length)
random_hex_string.insert(0,"0102")
random_hex_string.insert(30*1,"36")
puts random_hex_string
File.open("file.txt", 'w+b') do |file|
file.write([random_hex_string].pack('H*')) # see also 'h'
end
I get the output
010299e84e9e4541d08cb800462b6f36a87ff118d6291368e96e8907598a2dfd4090658fea1dab6ed460ab512ddc54522329f6b4ddd287e4302ef603ce60e85e631591
and then running
$ hexdump file.txt
0000000 01 02 99 e8 4e 9e 45 41 d0 8c b8 00 46 2b 6f 36
0000010 a8 7f f1 18 d6 29 13 68 e9 6e 89 07 59 8a 2d fd
0000020 40 90 65 8f ea 1d ab 6e d4 60 ab 51 2d dc 54 52
0000030 23 29 f6 b4 dd d2 87 e4 30 2e f6 03 ce 60 e8 5e
0000040 63 15 91
0000043
Unless, I'm mistaken, it matches up perfectly with the output from the script.
I've never messed with Array#pack before, and haven't done much with hex, but this seems to be giving the results you require, so should be a step in the right direction at least.
Why ask for hex when you don't actually want hex? Just do this:
random_bytes = SecureRandom.random_bytes(length)
random_bytes.insert(0, "\x01\x02")
random_bytes.insert(15, "\x36")
I'm trying to add a filter to a field in Wireshak.
My dissector name is: "basic".
it has 3 fields - field1, field2, field3.
each field can have a value of string.
I want that on Wireshark i'll be able to filter by a particular field, for example: basic.field1. (just the same as you look for tcp.len)
How can i do this?
You must declare the fields, assign them to your protocol and add them to the tree when appropriate. There are currently 2 different types of strings supported by Lua, those of type ftypes.STRING, which is used for strings of a known, fixed length, and type ftypes.STRINGZ, which is a NULL (zero)-terminated string, so how you declare the fields will depend upon which of the 2 types they are.
Unfortunately, despite the documentation listing ftypes.UINT_STRING as a supported type, it isn't as can be seen in the source code for wslua_proto_field.c. This type of string is applicable when a length field precedes the string to indicate the length of the string in bytes. In any case, it isn't currently available for Lua-based dissectors.
So, as an example, let's suppose your protocol uses UDP/33333 as its transport and port number, and its 3 fields consist of each of the 3 types of strings described above, namely:
field1: a fixed-length string of 12 bytes.
field2: a NULL-terminated string of arbitrary length.
field3: a counted string preceded by a 2 byte length field in big-endian (network) byte order.
Given these assumptions, the following will dissect the packets:
-- Protocol
local p_basic = Proto("basic", "Basic Protocol")
-- Fields
local f_basic_field1 = ProtoField.string("basic.field1", "Field1")
local f_basic_field2 = ProtoField.stringz("basic.field2", "Field2")
local f_basic_field3 = ProtoField.string("basic.field3", "Field3")
p_basic.fields = { f_basic_field1, f_basic_field2, f_basic_field3 }
-- Dissection
function p_basic.dissector(buf, pinfo, tree)
local basic_tree = tree:add(p_basic, buf(0,-1))
pinfo.cols.protocol:set("BASIC")
basic_tree:add(f_basic_field1, buf(0, 12))
local strz = buf(12):stringz()
local field2_len = string.len(strz) + 1
basic_tree:add(f_basic_field2, buf(12, field2_len))
local field3_len = buf:range(12 + field2_len, 2):uint()
basic_tree:add(f_basic_field3, buf(12 + field2_len + 2, field3_len))
end
-- Registration
local udp_table = DissectorTable.get("udp.port")
udp_table:add(33333, p_basic)
If you want to test this, first save the above lua code to a file such as basic.lua in your personal plugins directory (Found via Help -> About Wireshark -> Folders -> Personal Plugins). You can then use the following hex bytes to test it:
0000 00 0e b6 00 00 02 00 0e b6 00 00 01 08 00 45 00
0010 00 37 00 00 40 00 40 11 b5 ea c0 00 02 65 c0 00
0020 02 66 82 35 82 35 00 23 22 32 48 65 6c 6c 6f 20
0030 77 6f 72 6c 64 21 48 69 20 74 68 65 72 65 00 00
0040 04 42 79 65 21
Save these bytes into a text file, e.g., basic.txt. Start Wireshark and import the file via File -> Import from Hex Dump... -> Filename:basic.txt -> OK. You should see the 3 fields dissected as part of the "Basic Protocol".
For further help with Lua dissectors, you might want to refer to one or more of the following:
Wireshark Lua (wiki page)
Wireshark Lua Examples (wiki page)
Wireshark Lua Dissectors (wiki page)
Wireshark Contributed scripts, macros, colouring rules and plugins (wiki page)
Chapter 10. Lua Support in Wireshark (Wireshark Developer's Guide)
https://www.wireshark.org/docs/wsdg_html_chunked/wsluarm_modules.html (Wireshark Developer's Guide)
Wireshark Lua API Reference Manual Addendum (wiki page)
Lua Directory
Sharkfest '15 presentation and accompanying YouTube video, by Graham Bloice
I have a check printer that I want to connect in Delphi 7 by COM port and operate.
I have a command that I extracted with Serial Port Monitor:
STX "PIRI(781" FS NULL ETX "0B" wich is 02 50 49 52 49 28 37 38 31 1c 00 03 30 42 hex
The manual says the following:
CRC (which is the last two digits after the ETX) - packet checksum. It
is calculated by the following algorithm: executing XOR for every
byte of the block including ETX by excluding STX. The data of the
checksum take up two bytes and are a symbolic representation of the
numeric in a hexadecimal calculation system.
I tried to an ONLINE CRC calculator and return a 1B result and a 27 numeric.
How to do it? For "PIRI(781" FS NULL ETX it should be 0B
The documentation incorrectly identifies the check value as a CRC. It is not. It is simply the exclusive-or of the noted bytes. The exclusive-or of 50 49 52 49 28 37 38 31 1c 00 03 is 0b. You then convert the 0b to hex (with an upper case B, i.e. 0B), and get 30 42.
I have a problem trying to parse a midi file. I am trying to parse the notes files used by the frets on fire game (it just uses midi files so i don't think this is relevent) if any of you are familiar with it, the problem i am having is a general midi problem. I have a file with a track called guitar part, the hex, as viewed in a hex editor is as follows:
4D 54 72 6B 00 00 1E 74 00 FF 03 0B 50 41 52 54 20 47 55 49 54 41 52 A9 20 90 61 40 9A 20 61 00 83 60 63 40 BC
My program parses this fine as follows:
4D M
54 T
72 R
6B K
00 < --
00 size of
1E track part
74 -- >
00 time of this event
FF event type (this is meta)
03 meta event type
0B length of data
50 "P"
41 "A"
52 "R"
54 "T"
20 " "
47 "G"
55 "U"
49 "I"
54 "T"
41 "A"
52 "R"
A9 time of event (variable length) 10101001
20 time of event (variable length) 00100000
90 event,channel (non-meta) 1001=note on,channel=0000
61 note on has 2 params this is the first
40 this is the second
9A variable time 10011010
20 variable time 00100000
This is where my problem lies, there is no event that has event type 0x6, since 0x61 is 01100001 we have to assume it's non meta, therefore the event type should be 6 (0110) and the channel is (0001) but the midi specification contains no identification for this event.. I've added a few of the bytes after this incase they are somehow relevent but obviously at the moment my program hits the next byte, doesn't recognise the event and bombs out.
61
00
83
60
63
40
BC
If anyone thinks they could shed any light on where my parsing logic has gone wrong i'd be most appreciative, sorry for the formatting, i couldn't think of a better way to illustrate my problem.
I have been using this site: http://www.sonicspot.com/guide/midifiles.html as a reference and it hasn't led me wrong so far. I figured this might be something directly relating to frets on fire but it doesn't seem to be as i downloaded another notes file for the game and that file did not contain this event.
Thanks in advance.
It's called running status. If an event is of the same type as the previous event, the MIDI status byte can be eliminated. So if the first byte after the timing info is < $80, use the previous status. In the case of your $61 byte, the previous status was $90, so it's Note On, channel 0. Which makes sense since the previous event was note number $61 velocity $40. This event is note number $61 velocity 0 (releasing the previously played note). The next event is note number $63 velocity $40.