I see a few implementations, but I decided to look at exactly how the specification calls out the FCS for encoding.
So say my input is as follows:
dst: 0xAA AA AA AA AA AA
src: 0x55 55 55 55 55 55
len: 0x00 04
msg: 0xDE AD BE EF
concatenating this in the order that seems to be specified in the format (and the order expressed in the spec later on) seems to indicate my input is:
M(x) = 0xAA AA AA AA AA AA 55 55 55 55 55 55 00 04 DE AD BE EF
a) "The first 32 bits of the frame are complemented."
complemented first 32 MSB of M(x): 0x55 55 55 55 AA AA 55 55 55 55 55 55 00 04 DE AD BE EF
b) "The n bits of the protected fields are then considered to be the coefficients of a polynomial M(x) of
degree n – 1. (The first bit of the Destination Address field corresponds to the x(n–1) term and the last
bit of the MAC Client Data field (or Pad field if present) corresponds to the x0 term.)"
I did this in previous. see M(x)
c) "M(x) is multiplied by x^32 and divided by G(x), producing a remainder R(x) of degree <=31."
Some options online seem to ignore the 33rd bit to represent x^32. I am going to ignore those simplified shortcuts for this exercise since it doesn't seem to specify that in the spec.
It says to multiply M(x) by x^32. So this is just padding with 32 zeroes on the LSBs. (i.e. if m(x) = 1x^3 + 1, then m(x) * x^2 = 1x^5 + 1x^2 + 0)
padded: 0x55 55 55 55 AA AA 55 55 55 55 55 55 00 04 DE AD BE EF 00 00 00 00
Next step is to divide. I am dividing the whole M(x) / G(x). Can you use XOR shifting directly? I see some binary division examples have the dividened as 101 and the divisior as 110 and the remainder is 11. Other examples explain that by converting to decimal, you cannot divide. Which one is it for terms for this standard?
My remainder result is for option 1 (using XOR without carry bit consideration, shifting, no padding) was:
0x15 30 B0 FE
d) "The coefficients of R(x) are considered to be a 32-bit sequence."
e) "The bit sequence is complemented and the result is the CRC."
CRC = 0xEA CF 4F 01
so my entire Ethernet Frame should be:
0xAA AA AA AA AA AA 55 55 55 55 55 55 00 04 DE AD BE EF EA CF 4F 01
In which my dst address is its original value.
When I check my work with an online CRC32 BZIP2 calculator, I see this result: 0xCACF4F01
Is there another option or online tool to calculate the Ethernet FCS field? (not just one of many CRC32 calculators)
What steps am I missing? Should I have padded the M(x)? Should I have complemented the 32 LSBs instead?
Update
There was an error in my CRC output in my software. It was a minor issue with copying a vector. My latest result for CRC is (before post-complement) 35 30 B0 FE.
The post-complement is: CA CF 4F 01 (matching most online CRC32 BZIP2 versions).
So my ethernet according to my programming is currently:
0xAA AA AA AA AA AA 55 55 55 55 55 55 00 04 DE AD BE EF CA CF 4F 01
The CRC you need is commonly available in zlib and other libraries as the standard PKZip CRC-32. It is stored in the message in little-endian order. So your frame with the CRC would be:
AA AA AA AA AA AA 55 55 55 55 55 55 00 04 DE AD BE EF B0 5C 5D 85
Here is an online calculator, where the first result listed is the usual CRC-32, 0x855D5CB0.
Here is simple example code in C for calculating that CRC (calling with NULL for mem gives the initial CRC):
unsigned long crc32iso_hdlc(unsigned long crc, void const *mem, size_t len) {
unsigned char const *data = mem;
if (data == NULL)
return 0;
crc ^= 0xffffffff;
while (len--) {
crc ^= *data++;
for (unsigned k = 0; k < 8; k++)
crc = crc & 1 ? (crc >> 1) ^ 0xedb88320 : crc >> 1;
}
return crc ^ 0xffffffff;
}
The 0xedb88320 constant is 0x04c11db7 reflected.
The actual code used in libraries is more complex and faster.
Here is the calculation of that same CRC (in Mathematica), using the approach described in the IEEE 802.3 document with polynomials, so you can see the correct resulting powers of x used for the remainder calculation:
If you click on the image, it will embiggen to make it easier to read the powers.
The confusing factor here is the 802.3 spec. It mentions that first bit is LSB (least significant bit == bit 0) only in one place, in section 3.2.3-b, and it mentions that for CRC, "the first bit of the Destination Address field corresponds to the x^(n-1) term", so each byte input to the CRC calculation is bit reflected.
Using this online calculator:
http://www.sunshine2k.de/coding/javascript/crc/crc_js.html
Select CRC-32 | CRC32, click on custom, input reflected on, result reflected off. With this data:
AA AA AA AA AA AA 55 55 55 55 55 55 00 04 DE AD BE EF
per the spec, the calculated CRC is 0x0D3ABAA1, stored and transmitted as shown:
bit 0 first | bit 7 first
AA AA AA AA AA AA 55 55 55 55 55 55 00 04 DE AD BE EF | 0D 3A BA A1
To simplify the output to always transmit bit 0 first, bit reflect the CRC bytes:
bit 0 first | bit 0 first
AA AA AA AA AA AA 55 55 55 55 55 55 00 04 DE AD BE EF | B0 5C 5D 85
Note that the bit 0 always first method results in transmitted bits identical to the spec.
Change result setting for the CRC calculator, input reflected on, result reflected on. With this data:
AA AA AA AA AA AA 55 55 55 55 55 55 00 04 DE AD BE EF
the calculated CRC is 0x855D5CB0, stored least significant byte first and transmitted as shown:
bit 0 first | bit 0 first
AA AA AA AA AA AA 55 55 55 55 55 55 00 04 DE AD BE EF | B0 5C 5D 85
For verifying received data, rather than compare a calculated CRC of received data versus the received CRC, the process can calculate a CRC on received data and CRC. Assuming the alternative setup where all bytes are received bit 0 first, then with this received frame or any frame without error
bit 0 first | bit 0 first
AA AA AA AA AA AA 55 55 55 55 55 55 00 04 DE AD BE EF | B0 5C 5D 85
the calculated CRC will always be 0x2144DF1C. In the case of a hardware implementation, the post complement of the CRC is usually performed one bit at a time as bits are shifted out, outside of the logic used to calculate the CRC, and in this case, after receiving a frame without error, the CRC register will always contain 0xDEBB20E3 (0x2144DF1C ^ 0xFFFFFFFF). So verification is done by computing CRC on a received frame and comparing the CRC to a 32 bit constant (0x2144DF1C or 0xDEBB20E3).
I have an alarm system that I have configured to send SMS messages to my phone as well as over Ethernet.
Here a few of the SMSes I receive:
5522 18 1137 00 003 1C76
5522 18 3137 00 003 3278
5522 18 1130 00 002 E36E
5522 18 1401 00 001 ED6E
5522 18 1302 00 003 ED70
5522 18 1302 00 004 EE71
5522 18 1302 00 009 F376
5522 18 3147 00 009 417F
5522 18 1137 00 004 1D77
5522 18 3137 00 009 3379
5522 18 1602 00 000 0870
The first 4 bytes are the account number, the next 2 are always 18, the next 4 are event codes, 2 group bytes and 3 zone numbers. At the end there are 4 bytes which I suspect is some kind of checksum.
This is some kind of Ademco Contact ID format. However, I do not recognize the checksum.
It's not a time stamp as the last message (0870) is sent periodically and is always the same.
When sending via DTMF 0 should have value 10, but I do not know if that is the case with SMSes. Most likely not.
The checksum formula in Ademco's Contact ID is calculated using the formula:
S= HEX Checksum which is one digit.
(Sum of all message digits + S) MOD 15 = 0
and if the value is equal to 10 the checksum digit is 0.
The official Contact ID specification is here:http://li0r.files.wordpress.com/2012/07/sia-dc-05-1999-09_contact_id.pdf
So, using 5522 18 1602 00 000 0870 as an example:
LET C = checksum
5+5+2+2+1+8+1+6+2=32
(32+S) modulo 15 is congruent to 0
We then need the closest multiple of 15 going higher than 32 which would be 45.
45-32=13
Lets test that.
45 modulo 15 is congruent to 0
It is correct however, as Contact ID is 16 digits and you have 19 I would suspect that your panel is using a different proprietary implementation of Contact ID. If you post the make/model of the panel that this came from I may be able to explain things further.
I hope this answers your question!
-Alex
P.S.: To calculate mod use a percent sign in Google
P.P.S: The document that describes Contact ID is actually: DC-05-1999.09 the document you referenced is actually the computer interface communication protocol specifications.
I just want to correct AdemcoGuy's calculation as it seems to be incorrect:
So, the example was 5522 18 1602 00 000 0870
We need to replace each 0 by 10.
So:
5+5+2+2+1+8+1+6+10+2+10+10+10+10+10 = 92
than 100-92 = 8
So the cheksum is 8
Anyway in the question the checksum seems missing and what is the last 4 digit only knows who manufactured the panel which has been send it :)
#ACCT MT QXYZ GG CCC where:
ACCT: 4 Digit Account number (0-9, B-F)
MT: Message Type - Always 18
Q: Event qualifier, which gives specific event information:
1: New Event or Opening
3: New Restore or Closing
6: Previously reported condition still present (Status report)
XYZ: Event code (3 Hex digits 0-9,B-F)
GG: Group or Partition number (2 Hex digits 0-9, B-F). Use 00 to indicate that no specific group or partition information applies.
CCC: Zone number (Event reports) or User (Open / Close reports) (3 Hexdigits 0-9,B-F ). Use 000 to indicate that no specific zone or user information applies.
To look up event codes see this document (pdf).
I came across this post while trying to figure out the checksums of my own alarm system (Woonveilig/Egardia) that seems to be using the same format. I found a forum post on the german alarm forum that contains a snippet of C code to calculate CRCs for the LUPUS alarm system. This CRC calculation method seems to match both my own and Lasse's SMS based system. Here's the C code converted to a simple calculation tool:
#include <stdio.h>
#include <string.h>
// Code from: https://www.alarmforum.de/showthread.php?tid=12037&pid=75893
/**
* Fletcher Checksum.(LUPUS version, 16-bit)
*/
static unsigned int fletcher_sum(char* data, int len) {
unsigned int sum1 = 0x0, sum2 = 0x0;
while (len) {
unsigned int tlen = (len > 256) ? 256 : len;
len -= tlen;
do {
sum1 += *data++;
sum1 = (sum1 & 0xff);
sum2 += sum1;
sum2 = (sum2 & 0xff);
} while (--tlen);
}
return sum2 << 8 | sum1;
}
int main() {
char input[50];
int sum;
printf("Enter input: ");
fgets(input, sizeof(input), stdin);
sum = fletcher_sum(input, strlen(input)-1);
printf("%x\n", sum);
return 0;
}
Example (first SMS from the question post):
# cc checksum.c
# ./a.out
Enter input: 5522 18 1137 00 003
1c76
This is an Erlang question.
I have run into some unexpected behavior by io:fread.
I was wondering if someone could check whether there is something wrong with the way I use io:fread or whether there is a bug in io:fread.
I have a text file which contains a "triangle of numbers"as follows:
59
73 41
52 40 09
26 53 06 34
10 51 87 86 81
61 95 66 57 25 68
90 81 80 38 92 67 73
30 28 51 76 81 18 75 44
...
There is a single space between each pair of numbers and each line ends with a carriage-return new-line pair.
I use the following Erlang program to read this file into a list.
-module(euler67).
-author('Cayle Spandon').
-export([solve/0]).
solve() ->
{ok, File} = file:open("triangle.txt", [read]),
Data = read_file(File),
ok = file:close(File),
Data.
read_file(File) ->
read_file(File, []).
read_file(File, Data) ->
case io:fread(File, "", "~d") of
{ok, [N]} ->
read_file(File, [N | Data]);
eof ->
lists:reverse(Data)
end.
The output of this program is:
(erlide#cayle-spandons-computer.local)30> euler67:solve().
[59,73,41,52,40,9,26,53,6,3410,51,87,86,8161,95,66,57,25,
6890,81,80,38,92,67,7330,28,51,76,81|...]
Note how the last number of the fourth line (34) and the first number of the fifth line (10) have been merged into a single number 3410.
When I dump the text file using "od" there is nothing special about those lines; they end with cr-nl just like any other line:
> od -t a triangle.txt
0000000 5 9 cr nl 7 3 sp 4 1 cr nl 5 2 sp 4 0
0000020 sp 0 9 cr nl 2 6 sp 5 3 sp 0 6 sp 3 4
0000040 cr nl 1 0 sp 5 1 sp 8 7 sp 8 6 sp 8 1
0000060 cr nl 6 1 sp 9 5 sp 6 6 sp 5 7 sp 2 5
0000100 sp 6 8 cr nl 9 0 sp 8 1 sp 8 0 sp 3 8
0000120 sp 9 2 sp 6 7 sp 7 3 cr nl 3 0 sp 2 8
0000140 sp 5 1 sp 7 6 sp 8 1 sp 1 8 sp 7 5 sp
0000160 4 4 cr nl 8 4 sp 1 4 sp 9 5 sp 8 7 sp
One interesting observation is that some of the numbers for which the problem occurs happen to be on 16-byte boundary in the text file (but not all, for example 6890).
I'm going to go with it being a bug in Erlang, too, and a weird one. Changing the format string to "~2s" gives equally weird results:
["59","73","4","15","2","40","0","92","6","53","0","6","34",
"10","5","1","87","8","6","81","61","9","5","66","5","7",
"25","6",
[...]|...]
So it appears that it's counting a newline character as a regular character for the purposes of counting, but not when it comes to producing the output. Loopy as all hell.
A week of Erlang programming, and I'm already delving into the source. That might be a new record for me...
EDIT
A bit more investigation has confirmed for me that this is a bug. Calling one of the internal methods that's used in fread:
> io_lib_fread:fread([], "12 13\n14 15 16\n17 18 19 20\n", "~d").
{done,{ok,"\f"}," 1314 15 16\n17 18 19 20\n"}
Basically, if there's multiple values to be read, then a newline, the first newline gets eaten in the "still to be read" part of the string. Other testing suggests that if you prepend a space it's OK, and if you lead the string with a newline it asks for more.
I'm going to get to the bottom of this, gosh-darn-it... (grin) There's not that much code to go through, and not much of it deals specifically with newlines, so it shouldn't take too long to narrow it down and fix it.
EDIT^2
HA HA! Got the little blighter.
Here's the patch to the stdlib that you want (remember to recompile and drop the new beam file over the top of the old one):
--- ../erlang/erlang-12.b.3-dfsg/lib/stdlib/src/io_lib_fread.erl
+++ ./io_lib_fread.erl
## -35,9 +35,9 ##
fread_collect(MoreChars, [], Rest, RestFormat, N, Inputs).
fread_collect([$\r|More], Stack, Rest, RestFormat, N, Inputs) ->
- fread(RestFormat, Rest ++ reverse(Stack), N, Inputs, More);
+ fread(RestFormat, Rest ++ reverse(Stack), N, Inputs, [$\r|More]);
fread_collect([$\n|More], Stack, Rest, RestFormat, N, Inputs) ->
- fread(RestFormat, Rest ++ reverse(Stack), N, Inputs, More);
+ fread(RestFormat, Rest ++ reverse(Stack), N, Inputs, [$\n|More]);
fread_collect([C|More], Stack, Rest, RestFormat, N, Inputs) ->
fread_collect(More, [C|Stack], Rest, RestFormat, N, Inputs);
fread_collect([], Stack, Rest, RestFormat, N, Inputs) ->
## -55,8 +55,8 ##
eof ->
fread(RestFormat,eof,N,Inputs,eof);
_ ->
- %% Don't forget to count the newline.
- {more,{More,RestFormat,N+1,Inputs}}
+ %% Don't forget to strip and count the newline.
+ {more,{tl(More),RestFormat,N+1,Inputs}}
end;
Other -> %An error has occurred
{done,Other,More}
Now to submit my patch to erlang-patches, and reap the resulting fame and glory...
Besides the fact that it seems to be a bug in one of the erlang libs I think you could (very) easily circumvent the problem.
Given the fact your file is line-oriented I think best practice is that you process it line-by-line as well.
Consider the following construction. It works nicely on an unpatched erlang and because it uses lazy evaluation it can handle files of arbitrary length without having to read all of it into memory first. The module contains an example of a function to apply to each line - turning a line of text-representations of integers into a list of integers.
-module(liner).
-author("Harro Verkouter").
-export([liner/2, integerize/0, lazyfile/1]).
% Applies a function to all lines of the file
% before reducing (foldl).
liner(File, Fun) ->
lists:foldl(fun(X, Acc) -> Acc++Fun(X) end, [], lazyfile(File)).
% Reads the lines of a file in a lazy fashion
lazyfile(File) ->
{ok, Fd} = file:open(File, [read]),
lazylines(Fd).
% Actually, this one does the lazy read ;)
lazylines(Fd) ->
case io:get_line(Fd, "") of
eof -> file:close(Fd), [];
{error, Reason} ->
file:close(Fd), exit(Reason);
L ->
[L|lazylines(Fd)]
end.
% Take a line of space separated integers (string) and transform
% them into a list of integers
integerize() ->
fun(X) ->
lists:map(fun(Y) -> list_to_integer(Y) end,
string:tokens(X, " \n")) end.
Example usage:
Eshell V5.6.5 (abort with ^G)
1> c(liner).
{ok,liner}
2> liner:liner("triangle.txt", liner:integerize()).
[59,73,41,52,40,9,26,53,6,34,10,51,87,86,81,61,95,66,57,25,
68,90,81,80,38,92,67,73,30|...]
And as a bonus, you can easily fold over the lines of any (lineoriented) file w/o running out of memory :)
6> lists:foldl( fun(X, Acc) ->
6> io:format("~.2w: ~s", [Acc,X]), Acc+1
6> end,
6> 1,
6> liner:lazyfile("triangle.txt")).
1: 59
2: 73 41
3: 52 40 09
4: 26 53 06 34
5: 10 51 87 86 81
6: 61 95 66 57 25 68
7: 90 81 80 38 92 67 73
8: 30 28 51 76 81 18 75 44
Cheers,
h.
I noticed that there are multiple instances where two numbers are merged, and it appears to be at the line boundaries on every line starting at the fourth line and beyond.
I found that if you add a whitespace character to the beginning of every line starting at the fifth, that is:
59
73 41
52 40 09
26 53 06 34
10 51 87 86 81
61 95 66 57 25 68
90 81 80 38 92 67 73
30 28 51 76 81 18 75 44
...
The numbers get parsed properly:
39> euler67:solve().
[59,73,41,52,40,9,26,53,6,34,10,51,87,86,81,61,95,66,57,25,
68,90,81,80,38,92,67,73,30|...]
It also works if you add the whitespace to the beginning of the first four lines, as well.
It's more of a workaround than an actual solution, but it works. I'd like to figure out how to set up the format string for io:fread such that we wouldn't have to do this.
UPDATE
Here's a workaround that won't force you to change the file. This assumes that all digits are two characters (< 100):
read_file(File, Data) ->
case io:fread(File, "", "~d") of
{ok, [N] } ->
if
N > 100 ->
First = N div 100,
Second = N - (First * 100),
read_file(File, [First , Second | Data]);
true ->
read_file(File, [N | Data])
end;
eof ->
lists:reverse(Data)
end.
Basically, the code catches any of the numbers which are the concatenation of two across a newline and splits them into two.
Again, it's a kludge that implies a possible bug in io:fread, but that should do it.
UPDATE AGAIN The above will only work for two-digit inputs, but since the example packs all digits (even those < 10) into a two-digit format, that will work for this example.