Converting a string to a list of integers in Erlang - erlang

I am trying to convert a string to a list of integers.
String = "08 02 22 97 38 15 00 40 00 75 04 05 07 78 52 12 50 77 91 08".
But
lists:map(fun(X) -> string:to_integer(X) end, string:tokens(String, " ")).
just gives me...
[{8,[]}, {2,[]}, {22,[]}, {97,[]}, ... , {91,[]}, {8,[]}]
Can someone perhaps tell me what a good/nice way would be to get?
[8,2,22,97...91,8]
(Or do I need a helper function?)


This works:
String = "08 02 22 97 38 15 00 40 00 75 04 05 07 78 52 12 50 77 91 08".
lists:map(fun(X) -> {Int, _} = string:to_integer(X),
Int end,
string:tokens(String, " ")).
> [8,2,22,97,38,15,0,40,0,75,4,5,7,78,52,12,50,77,91,8]
See, string:to_integer returns not a single integer, but a tuple:
to_integer(String) -> {Int,Rest} | {error,Reason}
... so you have to extract the first element from this tuple to get the actual number.


Also, you might use list generator syntax:
[begin {Int,_}=string:to_integer(Token), Int end|| Token<-string:tokens(String," ")].
From shell:
1> String = "08 02 22 97 38 15 00 40 00 75 04 05 07 78 52 12 50 77 91 08".
"08 02 22 97 38 15 00 40 00 75 04 05 07 78 52 12 50 77 91 08"
2>
2> [begin {Int,_}=string:to_integer(Token), Int end|| Token<-string:tokens(String," ")].
[8,2,22,97,38,15,0,40,0,75,4,5,7,78,52,12,50,77,91,8]


[list_to_integer(X)|| X<-string:tokens(String," ")].

Related

Physical storage of COBOL PACKED-DECIMAL field on Little Endian machines

I understand the shape of the bytes behind a PACKED-DECIMAL number when on a Big-Endian machine. What I have not been able to discover is what shape would those bytes take on a Little-Endian machine.
Note: The reason I believe there does exist a separate format for such is because the IBM MQ Encoding field has the following values:-
MQENC_DECIMAL_NORMAL
MQENC_DECIMAL_REVERSED
Of MQENC_DECIMAL_REVERSED it simply says:-
Packed-decimal integers are represented in the same way as MQENC_DECIMAL_NORMAL, but with the bytes arranged in reverse order. The bits within each byte are arranged in the same way as MQENC_DECIMAL_NORMAL.
Can anyone corroborate or refute this description?
For example, the number +123.45 stored in a PIC 9(3)V99 COMP-3 would have the following bytes:-
12 34 5C
The above quote suggests to me that if in reversed, the same packed decimal number would be represented on a Little-Endian machine as:-
5C 34 12
N.B. I have not tagged this question with ibm-mq, as I don't really think the crux of my question has anything to do with IBM MQ. It is simply the reason I am asking.

It looks like packed data is stored in "normal" byte order.
The following program was compiled with OpenCOBOL 1.1.0 on Ubuntu 16.04 on a Lenovo ThinkPad...
ID Division.
Program-ID. testcmp3.
Environment Division.
Input-Output Section.
File-Control.
Select OUTPUT01 Assign To './output01'
Status OUTPUT01-STATUS.
Data Division.
File Section.
FD OUTPUT01
Record 16
Block 0
Recording F.
01 OUTPUT01-REC PIC X(016).
Working-Storage Section.
01 CONSTANTS.
05 MYNAME PIC X(008) VALUE 'testcmp3'.
01 WS-OUTPUT01-REC.
05 WS-OUT-S94COMP PIC S9(004) COMP VALUE +0.
05 FILLER PIC X(001) VALUE HIGH-VALUES.
05 WS-OUT-S95COMP3 PIC S9(005) COMP-3 VALUE +0.
05 FILLER PIC X(008) VALUE SPACES.
01 WORK-AREAS.
05 OUTPUT01-STATUS PIC X(002) VALUE ZEROES.
Procedure Division.
OPEN OUTPUT OUTPUT01
IF OUTPUT01-STATUS NOT = '00'
DISPLAY
MYNAME
' open of OUTPUT01 status = '
OUTPUT01-STATUS
END-IF
PERFORM 100-WRITE 100 TIMES
CLOSE OUTPUT01
STOP RUN.
100-WRITE.
WRITE OUTPUT01-REC FROM WS-OUTPUT01-REC
ADD 1 TO WS-OUT-S94COMP
ADD 1 TO WS-OUT-S95COMP3
.
...with the option binary-byteorder set to native.
The resulting file, displayed with the hexdump program, follows.
00000000 00 00 ff 00 00 0c 20 20 20 20 20 20 20 20 20 20 |...... |
00000010 01 00 ff 00 00 1c 20 20 20 20 20 20 20 20 20 20 |...... |
00000020 02 00 ff 00 00 2c 20 20 20 20 20 20 20 20 20 20 |....., |
00000030 03 00 ff 00 00 3c 20 20 20 20 20 20 20 20 20 20 |.....< |
00000040 04 00 ff 00 00 4c 20 20 20 20 20 20 20 20 20 20 |.....L |
00000050 05 00 ff 00 00 5c 20 20 20 20 20 20 20 20 20 20 |.....\ |
00000060 06 00 ff 00 00 6c 20 20 20 20 20 20 20 20 20 20 |.....l |
00000070 07 00 ff 00 00 7c 20 20 20 20 20 20 20 20 20 20 |.....| |
00000080 08 00 ff 00 00 8c 20 20 20 20 20 20 20 20 20 20 |...... |
00000090 09 00 ff 00 00 9c 20 20 20 20 20 20 20 20 20 20 |...... |
000000a0 0a 00 ff 00 01 0c 20 20 20 20 20 20 20 20 20 20 |...... |
000000b0 0b 00 ff 00 01 1c 20 20 20 20 20 20 20 20 20 20 |...... |
000000c0 0c 00 ff 00 01 2c 20 20 20 20 20 20 20 20 20 20 |....., |
000000d0 0d 00 ff 00 01 3c 20 20 20 20 20 20 20 20 20 20 |.....< |
000000e0 0e 00 ff 00 01 4c 20 20 20 20 20 20 20 20 20 20 |.....L |
000000f0 0f 00 ff 00 01 5c 20 20 20 20 20 20 20 20 20 20 |.....\ |
00000100 10 00 ff 00 01 6c 20 20 20 20 20 20 20 20 20 20 |.....l |
00000110 11 00 ff 00 01 7c 20 20 20 20 20 20 20 20 20 20 |.....| |
00000120 12 00 ff 00 01 8c 20 20 20 20 20 20 20 20 20 20 |...... |
00000130 13 00 ff 00 01 9c 20 20 20 20 20 20 20 20 20 20 |...... |
00000140 14 00 ff 00 02 0c 20 20 20 20 20 20 20 20 20 20 |...... |
00000150 15 00 ff 00 02 1c 20 20 20 20 20 20 20 20 20 20 |...... |
00000160 16 00 ff 00 02 2c 20 20 20 20 20 20 20 20 20 20 |....., |
00000170 17 00 ff 00 02 3c 20 20 20 20 20 20 20 20 20 20 |.....< |
00000180 18 00 ff 00 02 4c 20 20 20 20 20 20 20 20 20 20 |.....L |
00000190 19 00 ff 00 02 5c 20 20 20 20 20 20 20 20 20 20 |.....\ |
000001a0 1a 00 ff 00 02 6c 20 20 20 20 20 20 20 20 20 20 |.....l |
000001b0 1b 00 ff 00 02 7c 20 20 20 20 20 20 20 20 20 20 |.....| |
000001c0 1c 00 ff 00 02 8c 20 20 20 20 20 20 20 20 20 20 |...... |
000001d0 1d 00 ff 00 02 9c 20 20 20 20 20 20 20 20 20 20 |...... |
000001e0 1e 00 ff 00 03 0c 20 20 20 20 20 20 20 20 20 20 |...... |
000001f0 1f 00 ff 00 03 1c 20 20 20 20 20 20 20 20 20 20 |...... |
00000200 20 00 ff 00 03 2c 20 20 20 20 20 20 20 20 20 20 | ...., |
00000210 21 00 ff 00 03 3c 20 20 20 20 20 20 20 20 20 20 |!....< |
00000220 22 00 ff 00 03 4c 20 20 20 20 20 20 20 20 20 20 |"....L |
00000230 23 00 ff 00 03 5c 20 20 20 20 20 20 20 20 20 20 |#....\ |
00000240 24 00 ff 00 03 6c 20 20 20 20 20 20 20 20 20 20 |$....l |
00000250 25 00 ff 00 03 7c 20 20 20 20 20 20 20 20 20 20 |%....| |
00000260 26 00 ff 00 03 8c 20 20 20 20 20 20 20 20 20 20 |&..... |
00000270 27 00 ff 00 03 9c 20 20 20 20 20 20 20 20 20 20 |'..... |
00000280 28 00 ff 00 04 0c 20 20 20 20 20 20 20 20 20 20 |(..... |
00000290 29 00 ff 00 04 1c 20 20 20 20 20 20 20 20 20 20 |)..... |
000002a0 2a 00 ff 00 04 2c 20 20 20 20 20 20 20 20 20 20 |*...., |
000002b0 2b 00 ff 00 04 3c 20 20 20 20 20 20 20 20 20 20 |+....< |
000002c0 2c 00 ff 00 04 4c 20 20 20 20 20 20 20 20 20 20 |,....L |
000002d0 2d 00 ff 00 04 5c 20 20 20 20 20 20 20 20 20 20 |-....\ |
000002e0 2e 00 ff 00 04 6c 20 20 20 20 20 20 20 20 20 20 |.....l |
000002f0 2f 00 ff 00 04 7c 20 20 20 20 20 20 20 20 20 20 |/....| |
00000300 30 00 ff 00 04 8c 20 20 20 20 20 20 20 20 20 20 |0..... |
00000310 31 00 ff 00 04 9c 20 20 20 20 20 20 20 20 20 20 |1..... |
00000320 32 00 ff 00 05 0c 20 20 20 20 20 20 20 20 20 20 |2..... |
00000330 33 00 ff 00 05 1c 20 20 20 20 20 20 20 20 20 20 |3..... |
00000340 34 00 ff 00 05 2c 20 20 20 20 20 20 20 20 20 20 |4...., |
00000350 35 00 ff 00 05 3c 20 20 20 20 20 20 20 20 20 20 |5....< |
00000360 36 00 ff 00 05 4c 20 20 20 20 20 20 20 20 20 20 |6....L |
00000370 37 00 ff 00 05 5c 20 20 20 20 20 20 20 20 20 20 |7....\ |
00000380 38 00 ff 00 05 6c 20 20 20 20 20 20 20 20 20 20 |8....l |
00000390 39 00 ff 00 05 7c 20 20 20 20 20 20 20 20 20 20 |9....| |
000003a0 3a 00 ff 00 05 8c 20 20 20 20 20 20 20 20 20 20 |:..... |
000003b0 3b 00 ff 00 05 9c 20 20 20 20 20 20 20 20 20 20 |;..... |
000003c0 3c 00 ff 00 06 0c 20 20 20 20 20 20 20 20 20 20 |<..... |
000003d0 3d 00 ff 00 06 1c 20 20 20 20 20 20 20 20 20 20 |=..... |
000003e0 3e 00 ff 00 06 2c 20 20 20 20 20 20 20 20 20 20 |>...., |
000003f0 3f 00 ff 00 06 3c 20 20 20 20 20 20 20 20 20 20 |?....< |
00000400 40 00 ff 00 06 4c 20 20 20 20 20 20 20 20 20 20 |#....L |
00000410 41 00 ff 00 06 5c 20 20 20 20 20 20 20 20 20 20 |A....\ |
00000420 42 00 ff 00 06 6c 20 20 20 20 20 20 20 20 20 20 |B....l |
00000430 43 00 ff 00 06 7c 20 20 20 20 20 20 20 20 20 20 |C....| |
00000440 44 00 ff 00 06 8c 20 20 20 20 20 20 20 20 20 20 |D..... |
00000450 45 00 ff 00 06 9c 20 20 20 20 20 20 20 20 20 20 |E..... |
00000460 46 00 ff 00 07 0c 20 20 20 20 20 20 20 20 20 20 |F..... |
00000470 47 00 ff 00 07 1c 20 20 20 20 20 20 20 20 20 20 |G..... |
00000480 48 00 ff 00 07 2c 20 20 20 20 20 20 20 20 20 20 |H...., |
00000490 49 00 ff 00 07 3c 20 20 20 20 20 20 20 20 20 20 |I....< |
000004a0 4a 00 ff 00 07 4c 20 20 20 20 20 20 20 20 20 20 |J....L |
000004b0 4b 00 ff 00 07 5c 20 20 20 20 20 20 20 20 20 20 |K....\ |
000004c0 4c 00 ff 00 07 6c 20 20 20 20 20 20 20 20 20 20 |L....l |
000004d0 4d 00 ff 00 07 7c 20 20 20 20 20 20 20 20 20 20 |M....| |
000004e0 4e 00 ff 00 07 8c 20 20 20 20 20 20 20 20 20 20 |N..... |
000004f0 4f 00 ff 00 07 9c 20 20 20 20 20 20 20 20 20 20 |O..... |
00000500 50 00 ff 00 08 0c 20 20 20 20 20 20 20 20 20 20 |P..... |
00000510 51 00 ff 00 08 1c 20 20 20 20 20 20 20 20 20 20 |Q..... |
00000520 52 00 ff 00 08 2c 20 20 20 20 20 20 20 20 20 20 |R...., |
00000530 53 00 ff 00 08 3c 20 20 20 20 20 20 20 20 20 20 |S....< |
00000540 54 00 ff 00 08 4c 20 20 20 20 20 20 20 20 20 20 |T....L |
00000550 55 00 ff 00 08 5c 20 20 20 20 20 20 20 20 20 20 |U....\ |
00000560 56 00 ff 00 08 6c 20 20 20 20 20 20 20 20 20 20 |V....l |
00000570 57 00 ff 00 08 7c 20 20 20 20 20 20 20 20 20 20 |W....| |
00000580 58 00 ff 00 08 8c 20 20 20 20 20 20 20 20 20 20 |X..... |
00000590 59 00 ff 00 08 9c 20 20 20 20 20 20 20 20 20 20 |Y..... |
000005a0 5a 00 ff 00 09 0c 20 20 20 20 20 20 20 20 20 20 |Z..... |
000005b0 5b 00 ff 00 09 1c 20 20 20 20 20 20 20 20 20 20 |[..... |
000005c0 5c 00 ff 00 09 2c 20 20 20 20 20 20 20 20 20 20 |\...., |
000005d0 5d 00 ff 00 09 3c 20 20 20 20 20 20 20 20 20 20 |]....< |
000005e0 5e 00 ff 00 09 4c 20 20 20 20 20 20 20 20 20 20 |^....L |
000005f0 5f 00 ff 00 09 5c 20 20 20 20 20 20 20 20 20 20 |_....\ |
00000600 60 00 ff 00 09 6c 20 20 20 20 20 20 20 20 20 20 |`....l |
00000610 61 00 ff 00 09 7c 20 20 20 20 20 20 20 20 20 20 |a....| |
00000620 62 00 ff 00 09 8c 20 20 20 20 20 20 20 20 20 20 |b..... |
00000630 63 00 ff 00 09 9c 20 20 20 20 20 20 20 20 20 20 |c..... |
00000640
The draft standard document ISO/IEC CD2.1 1989:202x says of the USAGE PACKED-DECIMAL phrase...
The USAGE PACKED-DECIMAL clause specifies that a radix of 10 is used
to represent a numeric item in the storage of the computer.
Furthermore, this clause specifies that each digit position shall
occupy the minimum possible configuration in computer storage. Each
implementor specifies the precise effect of the USAGE PACKED-DECIMAL
clause upon the alignment and representation of the data item in the
storage of the computer, including the representation of any algebraic
sign. Sufficient computer storage shall be allocated by the
implementor to contain the maximum range of values implied by the
associated decimal picture character-string. If the WITH NO SIGN
phrase is specified the representation of the data item in the storage
of the computer reserves no storage for representing any sign value.
The PICTURE character string of the data item shall not contain the
symbol ‘S’; the data item is always considered to have a zero, or
positive value.

Indy odd/strang behavior with TLS version

I sometimes get TLSv1 even when I set the version explicit to TLSv1.2.
I get this exception:
First chance exception at $75D917D2. Exception class
EIdOSSLConnectError with message 'Error connecting with SSL. EOF was
observed that violates the protocol'.
My Delphi version:
Embarcadero® RAD Studio 10 Seattle Version 23.0.21418.4207
I am on Windows 10:
Microsoft Windows [Version 10.0.17134.345]
My Indy Version: 10.6.2.5311 - That's what's coming with Seattle
This is a trace when it works:
This is a trace when it does NOT work:
As you can see it is TLSv1. Why?
I had a look at following questions:
Using Indy 10 IdHTTP with TLS 1.2
EIdOSSLConnectError Error connecting with SSL - EOF was observed
Here is my example code:
procedure TOpenWeatherFr.SetIcon(const IconCode: String);
var
HTTPS: TSSLHTTP;
MS : TMemoryStream;
PNG: TPngImage;
URL: string;
begin
HTTPS := TSSLHTTP.Create(nil);
MS := TMemoryStream.Create;
PNG := TPngImage.Create;
URL := 'https://openweathermap.org/img/w/' + IconCode + '.png';
try
(HTTPS.IOHandler as TIdSSLIOHandlerSocketOpenSSL).SSLOptions.SSLVersions := [sslvTLSv1_2];
HTTPS.Get(URL, MS);
MS.Seek(0, soFromBeginning);
PNG.LoadFromStream(MS);
Image1.Picture.Assign(PNG);
finally
HTTPS.Free;
PNG.Free;
MS.Free;
end;
Image1.Visible := true;
end;
For the sake of completeness here is the code of TSSLHTTP. The purpose is an old Indy version without SNI Support.
unit SSLHTTP;
interface
uses
Classes, IdHTTP, IdSSLOpenSSLHeaders, IdCTypes;
type
TSSLHTTP = class(TIdHTTP)
private
procedure OnStatusInfoEx(ASender: TObject; const AsslSocket: PSSL; const AWhere, Aret: TIdC_INT; const AType, AMsg: String);
public
constructor Create(AOwner: TComponent);
end;
implementation
uses
IdSSLOpenSSL;
{ TSSLHTTP }
constructor TSSLHTTP.Create(AOwner: TComponent);
begin
inherited Create(AOwner);
IOHandler := TIdSSLIOHandlerSocketOpenSSL.Create(self);
(IOHandler as TIdSSLIOHandlerSocketOpenSSL).OnStatusInfoEx := OnStatusInfoEx;
end;
procedure TSSLHTTP.OnStatusInfoEx(ASender: TObject; const AsslSocket: PSSL;
const AWhere, Aret: TIdC_INT; const AType, AMsg: String);
begin
SSL_set_tlsext_host_name(AsslSocket, Request.Host);
end;
end.
Where is my mistake? What am I doing wrong? This is the URL of the image I download:
https://openweathermap.org/img/w/04d.png
EDIT: Adding paylod as hex dumb
Working payload - Client Hello:
0000 16 03 01 01 49 01 00 01 45 03 03 8b 3d e5 54 59
0010 f8 59 44 82 f7 14 92 45 50 e1 a3 86 68 3a c2 94
0020 76 be ea 8b 54 98 f3 27 69 50 af 00 00 94 c0 30
0030 c0 2c c0 28 c0 24 c0 14 c0 0a 00 a3 00 9f 00 6b
0040 00 6a 00 39 00 38 c0 32 c0 2e c0 2a c0 26 c0 0f
0050 c0 05 00 9d 00 3d 00 35 00 88 00 87 00 84 c0 2f
0060 c0 2b c0 27 c0 23 c0 13 c0 09 00 a2 00 9e 00 67
0070 00 40 00 33 00 32 c0 31 c0 2d c0 29 c0 25 c0 0e
0080 c0 04 00 9c 00 3c 00 2f 00 9a 00 99 00 45 00 44
0090 00 96 00 41 00 07 c0 11 c0 07 c0 0c c0 02 00 05
00a0 00 04 c0 12 c0 08 00 16 00 13 c0 0d c0 03 00 0a
00b0 00 15 00 12 00 09 00 14 00 11 00 08 00 06 00 03
00c0 00 ff 01 00 00 88 00 00 00 17 00 15 00 00 12 6f
00d0 70 65 6e 77 65 61 74 68 65 72 6d 61 70 2e 6f 72
00e0 67 00 0b 00 04 03 00 01 02 00 0a 00 34 00 32 00
00f0 0e 00 0d 00 19 00 0b 00 0c 00 18 00 09 00 0a 00
0100 16 00 17 00 08 00 06 00 07 00 14 00 15 00 04 00
0110 05 00 12 00 13 00 01 00 02 00 03 00 0f 00 10 00
0120 11 00 23 00 00 00 0d 00 20 00 1e 06 01 06 02 06
0130 03 05 01 05 02 05 03 04 01 04 02 04 03 03 01 03
0140 02 03 03 02 01 02 02 02 03 00 0f 00 01 01
Not working payload - Client hello:
0000 16 03 01 01 49 01 00 01 45 03 03 f6 7b de 81 5d
0010 76 74 a7 31 99 36 8d 17 4c 07 5e 73 5d f7 b8 a1
0020 4f 06 5e 91 e5 f0 4b 37 0d 65 e7 00 00 94 c0 30
0030 c0 2c c0 28 c0 24 c0 14 c0 0a 00 a3 00 9f 00 6b
0040 00 6a 00 39 00 38 c0 32 c0 2e c0 2a c0 26 c0 0f
0050 c0 05 00 9d 00 3d 00 35 00 88 00 87 00 84 c0 2f
0060 c0 2b c0 27 c0 23 c0 13 c0 09 00 a2 00 9e 00 67
0070 00 40 00 33 00 32 c0 31 c0 2d c0 29 c0 25 c0 0e
0080 c0 04 00 9c 00 3c 00 2f 00 9a 00 99 00 45 00 44
0090 00 96 00 41 00 07 c0 11 c0 07 c0 0c c0 02 00 05
00a0 00 04 c0 12 c0 08 00 16 00 13 c0 0d c0 03 00 0a
00b0 00 15 00 12 00 09 00 14 00 11 00 08 00 06 00 03
00c0 00 ff 01 00 00 88 00 00 00 17 00 15 00 00 12 6f
00d0 70 65 6e 77 65 61 74 68 65 72 6d 61 70 2e 6f 72
00e0 67 00 0b 00 04 03 00 01 02 00 0a 00 34 00 32 00
00f0 0e 00 0d 00 19 00 0b 00 0c 00 18 00 09 00 0a 00
0100 16 00 17 00 08 00 06 00 07 00 14 00 15 00 04 00
0110 05 00 12 00 13 00 01 00 02 00 03 00 0f 00 10 00
0120 11 00 23 00 00 00 0d 00 20 00 1e 06 01 06 02 06
0130 03 05 01 05 02 05 03 04 01 04 02 04 03 03 01 03
0140 02 03 03 02 01 02 02 02 03 00 0f 00 01 01

How to split strings into substring start with some characters in ios

I am stuck in finding the solution of splitting the string into substrings. I need some proper logic in which I need to read string starting with some character and it should contain at least 17 characters.
e.g. "43 01 08 43 13 01 18 FD 48 6B D1 43 01 22 02 01 02 02 F1 48 6B D1 43 02 03 03 51 03 52 75 48 6B D1 43 03 53 11 08 15 52 9D 48 6B D1 43 16 14 16 32 00 00"
For Above string , I need to split in into
"43 01 08 43 13 01 18"
"43 01 22 02 01 02 02"
"43 02 03 03 51 03 52"
"43 03 53 11 08 15 52"
"43 16 14 16 32 00 00"
So On... I mean I need to get the position of "43" ,then from its index i Need to read 20 characters (Including space) and reject the characters until the next "43" appears (BOLD characters as mentioned above). Thanks in advance .. :)

I would just use rangeOfString: to locate the next 43, take the 20 characters from there on with substringWithRange:, and start over with the remaining substring (substringFromIndex:).
NSString *str = #"43 01 08 43 13 01 18 FD 48 6B D1 43 01 22 02 01 02 02 F1 48 6B D1 43 02 03 03 51 03 52 75 48 6B D1 43 03 53 11 08 15 52 9D 48 6B D1 43 16 14 16 32 00 00";
NSMutableArray *substrings = [NSMutableArray array];
NSRange range = { .length = 20 };
while ((range.location = [str rangeOfString:#"43"].location) != NSNotFound
&& str.length >= range.length + range.location) {
[substrings addObject:[str substringWithRange:range]];
str = [str substringFromIndex:range.location + range.length];
}

Try this:
NSMutableArray *output = [[NSMutableArray alloc] init];
NSString *allString = #"43 01 08 43 13 01 18 FD 48 6B D1 43 01 22 02 01 02 02 F1 48 6B D1 43 02 03 03 51 03 52 75 48 6B D1 43 03 53 11 08 15 52 9D 48 6B D1 43 16 14 16 32 00 00";
while (allString.length>0) {
[output addObject:[allString substringToIndex:20]];
allString = [allString substringFromIndex:20];
NSArray *arr = [allString componentsSeparatedByString:#"43"];
if (arr.count>1){
allString =#"43";
for (int i=1;i<arr.count;i++){
allString = [allString stringByAppendingString:[arr objectAtIndex:i]];
}
}else{
allString=#"";
}
}

Assuming your sequence always starts with "43" and is seven items long.
Split the string and walk down the array creating sub arrays.
-(NSArray *)splitString:(NSString *)masterString
{
NSArray *values = [masterString componentsSeparatedByString:#" "];
NSMutableArray *outputarray = [#[] mutableCopy];
NSUInteger marker = 0;
NSString *startValue = #"43";
NSUInteger subarraycount = 7;
BOOL foundFlag = YES;
while (foundFlag) {
NSRange checkRange = NSMakeRange(marker, (values.count - marker));
NSUInteger next = [values indexOfObject:startValue inRange:checkRange];
foundFlag = (next != NSNotFound) && ((next + subarraycount) < values.count);
if (foundFlag) {
NSRange subrange = NSMakeRange(next, subarraycount);
NSArray *subarray = [values subarrayWithRange:subrange];
[outputarray addObject:[subarray componentsJoinedByString:#" "]];
marker = next + subarraycount;
}
}
return [outputarray copy];
}

Parsing AMF3 objects

I'm trying to decode an AMF0 message with AMF3 included .. All is well
except making sense of the U29 format and the way string are encoded
in AMF4
channel=3 size=134 pkttype=0x11 time=1973
00000: 00 02 00 14 73 65 6E 64 55 6E 69 76 65 72 73 61 6C 4D 65 73 ....sendUniversalMes
00020: 73 61 67 65 00 00 00 00 00 00 00 00 00 05 02 00 11 6D 62 5F sage.............mb_
00040: 31 32 32 31 30 5F 37 35 39 32 33 33 38 30 05 00 40 58 C0 00 12210_75923380..#X..
00060: 00 00 00 00 11 0A 0B 01 09 68 62 69 64 04 81 CF 5E 09 73 72 .........hbid...^.sr
00080: 63 65 06 49 63 37 62 39 32 33 65 65 2D 30 61 30 38 2D 34 62 ce.Ic7b923ee-0a08-4b
00100: 61 32 2D 38 65 37 63 2D 63 38 32 61 39 33 64 37 37 31 34 32 a2-8e7c-c82a93d77142
00120: 09 68 62 64 6C 04 00 09 74 65 78 74 01 01 .hbdl...text..
first byte I skip
02 = string
00 14 = length of string ( 20 characters , sendUniversalMessage )
00 00 00 00 00 00 00 00 00 = number = 0
05 = null
02 = string
00 11 = length of string ( 17 characters , mb_12210_75923380 )
05 = null
00 40 58 C0 00 00 00 00 00 = number = 99
11 = AMV+
here is where I have problems
0A = AMF3 object
now I need to do a readU29 which starts with
0B = what does this mean
01 = what does this mean
09 = what does this mean
where is the length of the string 'hbid' ?

in U29, if the first byte is worth less than 128, it fits on the first byte. So you have to read this as 3 different u29s, worth 0B, 01, 09.
0B: details about the type of object. It seems you're not too interested in that byte and it's complicated, so pass.
01: The LSB is the bit that says that this isn't a string reference. Then 01 >> 1 = 0 is the length of the class name. This means empty string, which means this is an anonymous (untyped) object.
09: = 00001001. The LSB is also the bit that says this isn't a string reference. Then 1001 >> 1 = 0100 = 4 is the string length.
hope that helps

How to get raw jpeg data (but no metatags / proprietary markers)

I want to get raw jpeg data - no metadata.
I'm very confused looking at the jpeg "standards".
How correct is my understanding of the marker "tree"?
0xFFD8 - Identifies the file as an image
0xFFE? - EXIF, JFIF, SPIFF, ICC, etc
0x???? - the length of the tag
0xFFD8 - Start of Image
0xFFE0 - should follow SOI as per spec, but often preceded by comments ???
0x???? - Matrices, tags, random data ???
There are never other markers in-between these markers?
Or these include the matrices and such?
0xFFDA - Start of Stream - This is what I want, yes?
0xXXXX - length of stream
0xFFD9 - End of Stream (EOI)
0x???? - Comment tags, extra exif, jfif info???
0xFFD9 - End of Image
0xFF00 - escaped 0xFF, not to be confused with a marker
This has been my reading material:
http://en.wikipedia.org/wiki/JPEG
https://ExifTool.org/TagNames/JPEG.html
http://www.media.mit.edu/pia/Research/deepview/exif.html
http://www.faqs.org/faqs/jpeg-faq/part1/section-15.html

Besides the SOS (0xFFDA), you may also want the following:
0xFFDB DQT Quantization Table
0xFFC4 DHT Define Huffman Table

There's a fairly reasonable "Anatomy of a JPEG" here which shows you how to use grep to extract JPEG markers and sections from JPEG files, e.g. for SOF marker:
xxd -c16 -g1 -u testimg.jpg | grep --color=always -C2 "FF C0"
0000090: 32 32 32 32 32 32 32 32 32 32 32 32 32 32 FF C0 22222222222222..
00000a0: 00 11 08 00 95 00 E3 03 01 22 00 02 11 01 03 11 ........."......
00000b0: 01 FF C4 00 1F 00 00 01 05 01 01 01 01 01 01 00 ................
Explanation
0xFF, 0xC0, // SOF0 segement
0x00, 0x11, // length of segment depends on the number of components
0x08, // bits per pixel
0x00, 0x95, // image height
0x00, 0xE3, // image width
0x03, // number of components (should be 1 or 3)
0x01, 0x22, 0x00, // 0x01=Y component, 0x22=sampling factor, quantization table number
0x02, 0x11, 0x01, // 0x02=Cb component, ...
0x03, 0x11, 0x01 // 0x03=Cr component, ...
And there's a complete, highly detailed and thorough series of videos here on YouTube that tell you more than you could possibly ever want to know about JPEG.
Note also that you can extract and examine the JPEG markers in a file with exiftool, like this:
exiftool -v -v -v image.jpg
Sample Output
ExifToolVersion = 12.00
FileName = iphone.jpg
Directory = .
FileSize = 2219100
FileModifyDate = 1596126733
FileAccessDate = 1610564128
FileInodeChangeDate = 1596126733
FilePermissions = 33188
FileType = JPEG
FileTypeExtension = JPG
MIMEType = image/jpeg
JPEG APP1 (12284 bytes):
0006: 45 78 69 66 00 00 4d 4d 00 2a 00 00 00 08 00 0b [Exif..MM.*......]
0016: 01 0f 00 02 00 00 00 06 00 00 00 92 01 10 00 02 [................]
0026: 00 00 00 09 00 00 00 98 01 12 00 03 00 00 00 01 [................]
0036: 00 01 00 00 01 1a 00 05 00 00 00 01 00 00 00 a2 [................]
0046: 01 1b 00 05 00 00 00 01 00 00 00 aa 01 28 00 03 [.............(..]
0056: 00 00 00 01 00 02 00 00 01 31 00 02 00 00 00 06 [.........1......]
0066: 00 00 00 b2 01 32 00 02 00 00 00 14 00 00 00 b8 [.....2..........]
[snip 12172 bytes]
ExifByteOrder = MM
+ [IFD0 directory with 11 entries]
| 0) Make = Apple
| - Tag 0x010f (6 bytes, string[6]):
| 009e: 41 70 70 6c 65 00 [Apple.]
| 1) Model = iPhone 4
| - Tag 0x0110 (9 bytes, string[9]):
| 00a4: 69 50 68 6f 6e 65 20 34 00 [iPhone 4.]
| 2) Orientation = 1
| - Tag 0x0112 (2 bytes, int16u[1]):
| 0036: 00 01 [..]
| 3) XResolution = 72 (72/1)
| - Tag 0x011a (8 bytes, rational64u[1]):
| 00ae: 00 00 00 48 00 00 00 01 [...H....]
| 4) YResolution = 72 (72/1)
| - Tag 0x011b (8 bytes, rational64u[1]):
| 00b6: 00 00 00 48 00 00 00 01 [...H....]
| 5) ResolutionUnit = 2
| - Tag 0x0128 (2 bytes, int16u[1]):
| 005a: 00 02 [..]
| 6) Software = 6.1.2
| - Tag 0x0131 (6 bytes, string[6]):
| 00be: 36 2e 31 2e 32 00 [6.1.2.]
| 7) ModifyDate = 2013:03:09 08:59:50
| - Tag 0x0132 (20 bytes, string[20]):
| 00c4: 32 30 31 33 3a 30 33 3a 30 39 20 30 38 3a 35 39 [2013:03:09 08:59]
| 00d4: 3a 35 30 00 [:50.]
| 8) YCbCrPositioning = 1
| - Tag 0x0213 (2 bytes, int16u[1]):
| 007e: 00 01 [..]
| 9) ExifOffset (SubDirectory) -->
| - Tag 0x8769 (4 bytes, int32u[1]):
| 008a: 00 00 00 cc [....]
| + [ExifIFD directory with 24 entries]
| | 0) ExposureTime = 0.001094091904 (1/914)
| | - Tag 0x829a (8 bytes, rational64u[1]):
| | 01fe: 00 00 00 01 00 00 03 92 [........]
| | 1) FNumber = 2.8 (14/5)
| | - Tag 0x829d (8 bytes, rational64u[1]):
| | 0206: 00 00 00 0e 00 00 00 05 [........]
| | 2) ExposureProgram = 2
| | - Tag 0x8822 (2 bytes, int16u[1]):
| | 00fa: 00 02 [..]
| | 3) ISO = 80
| | - Tag 0x8827 (2 bytes, int16u[1]):
| | 0106: 00 50 [.P]
| | 4) ExifVersion = 0221
| | - Tag 0x9000 (4 bytes, undef[4]):
| | 0112: 30 32 32 31 [0221]
| | 5) DateTimeOriginal = 2013:03:09 08:59:50
| | - Tag 0x9003 (20 bytes, string[20]):
| | 020e: 32 30 31 33 3a 30 33 3a 30 39 20 30 38 3a 35 39 [2013:03:09 08:59]
| | 021e: 3a 35 30 00 [:50.]
| | 6) CreateDate = 2013:03:09 08:59:50
| | - Tag 0x9004 (20 bytes, string[20]):
| | 0222: 32 30 31 33 3a 30 33 3a 30 39 20 30 38 3a 35 39 [2013:03:09 08:59]
| | 0232: 3a 35 30 00 [:50.]
| | 7) ComponentsConfiguration = 1 2 3 0
| | - Tag 0x9101 (4 bytes, undef[4] read as int8u[4]):
| | 0136: 01 02 03 00 [....]
| | 8) ShutterSpeedValue = 9.83619211 (11469/1166)
| | - Tag 0x9201 (8 bytes, rational64s[1]):
| | 0236: 00 00 2c cd 00 00 04 8e [..,.....]
| | 9) ApertureValue = 2.970853574 (4281/1441)
| | - Tag 0x9202 (8 bytes, rational64u[1]):
| | 023e: 00 00 10 b9 00 00 05 a1 [........]
| | 10) BrightnessValue = 8.880310458 (21739/2448)
| | - Tag 0x9203 (8 bytes, rational64s[1]):
| | 0246: 00 00 54 eb 00 00 09 90 [..T.....]
| | 11) MeteringMode = 5
| | - Tag 0x9207 (2 bytes, int16u[1]):
| | 0166: 00 05 [..]
| | 12) Flash = 0
| | - Tag 0x9209 (2 bytes, int16u[1]):
| | 0172: 00 00 [..]
| | 13) FocalLength = 3.85 (77/20)
| | - Tag 0x920a (8 bytes, rational64u[1]):
| | 024e: 00 00 00 4d 00 00 00 14 [...M....]
| | 14) FlashpixVersion = 0100
| | - Tag 0xa000 (4 bytes, undef[4]):
| | 018a: 30 31 30 30 [0100]
| | 15) ColorSpace = 1
| | - Tag 0xa001 (2 bytes, int16u[1]):
| | 0196: 00 01 [..]
| | 16) ExifImageWidth = 2592
| | - Tag 0xa002 (4 bytes, int32u[1]):
| | 01a2: 00 00 0a 20 [... ]
| | 17) ExifImageHeight = 1936
| | - Tag 0xa003 (4 bytes, int32u[1]):
| | 01ae: 00 00 07 90 [....]
| | 18) SensingMethod = 2
| | - Tag 0xa217 (2 bytes, int16u[1]):
| | 01ba: 00 02 [..]
| | 19) CustomRendered = 2
| | - Tag 0xa401 (2 bytes, int16u[1]):
| | 01c6: 00 02 [..]
| | 20) ExposureMode = 0
| | - Tag 0xa402 (2 bytes, int16u[1]):
| | 01d2: 00 00 [..]
| | 21) WhiteBalance = 0
| | - Tag 0xa403 (2 bytes, int16u[1]):
| | 01de: 00 00 [..]
| | 22) FocalLengthIn35mmFormat = 35
| | - Tag 0xa405 (2 bytes, int16u[1]):
| | 01ea: 00 23 [.#]
| | 23) SceneCaptureType = 0
| | - Tag 0xa406 (2 bytes, int16u[1]):
| | 01f6: 00 00 [..]
| 10) GPSInfo (SubDirectory) -->
| - Tag 0x8825 (4 bytes, int32u[1]):
| 0096: 00 00 02 4a [...J]
| + [GPS directory with 9 entries]
| | 0) GPSLatitudeRef = N
| | - Tag 0x0001 (2 bytes, string[2]):
| | 0260: 4e 00 [N.]
| | 1) GPSLatitude = 20 50.66 0 (20/1 5066/100 0/1)
| | - Tag 0x0002 (24 bytes, rational64u[3]):
| | 02c8: 00 00 00 14 00 00 00 01 00 00 13 ca 00 00 00 64 [...............d]
| | 02d8: 00 00 00 00 00 00 00 01 [........]
| | 2) GPSLongitudeRef = E
| | - Tag 0x0003 (2 bytes, string[2]):
| | 0278: 45 00 [E.]
| | 3) GPSLongitude = 107 5.46 0 (107/1 546/100 0/1)
| | - Tag 0x0004 (24 bytes, rational64u[3]):
| | 02e0: 00 00 00 6b 00 00 00 01 00 00 02 22 00 00 00 64 [...k......."...d]
| | 02f0: 00 00 00 00 00 00 00 01 [........]
| | 4) GPSAltitudeRef = 0
| | - Tag 0x0005 (1 bytes, int8u[1]):
| | 0290: 00 [.]
| | 5) GPSAltitude = 1.12868102 (4561/4041)
| | - Tag 0x0006 (8 bytes, rational64u[1]):
| | 02f8: 00 00 11 d1 00 00 0f c9 [........]
| | 6) GPSTimeStamp = 1 59 48.81 (1/1 59/1 4881/100)
| | - Tag 0x0007 (24 bytes, rational64u[3]):
| | 0300: 00 00 00 01 00 00 00 01 00 00 00 3b 00 00 00 01 [...........;....]
| | 0310: 00 00 13 11 00 00 00 64 [.......d]
| | 7) GPSImgDirectionRef = T
| | - Tag 0x0010 (2 bytes, string[2]):
| | 02b4: 54 00 [T.]
| | 8) GPSImgDirection = 324.4435484 (40231/124)
| | - Tag 0x0011 (8 bytes, rational64u[1]):
| | 0318: 00 00 9d 27 00 00 00 7c [...'...|]
+ [IFD1 directory with 6 entries]
| 0) Compression = 6
| - Tag 0x0103 (2 bytes, int16u[1]):
| 032a: 00 06 [..]
| 1) XResolution = 72 (72/1)
| - Tag 0x011a (8 bytes, rational64u[1]):
| 036e: 00 00 00 48 00 00 00 01 [...H....]
| 2) YResolution = 72 (72/1)
| - Tag 0x011b (8 bytes, rational64u[1]):
| 0376: 00 00 00 48 00 00 00 01 [...H....]
| 3) ResolutionUnit = 2
| - Tag 0x0128 (2 bytes, int16u[1]):
| 034e: 00 02 [..]
| 4) ThumbnailOffset = 882
| - Tag 0x0201 (4 bytes, int32u[1]):
| 035a: 00 00 03 72 [...r]
| 5) ThumbnailLength = 7847
| - Tag 0x0202 (4 bytes, int32u[1]):
| 0366: 00 00 1e a7 [....]
JPEG DQT (130 bytes):
3006: 00 01 01 01 01 01 01 01 01 01 01 01 01 01 01 02 [................]
3016: 04 02 02 02 02 02 04 03 03 02 04 05 05 06 06 05 [................]
3026: 05 05 05 06 07 09 07 06 06 08 06 05 05 08 0a 08 [................]
3036: 08 09 09 0a 0a 0a 06 07 0b 0c 0b 0a 0c 09 0a 0a [................]
3046: 09 01 01 01 01 02 02 02 04 02 02 04 09 06 05 06 [................]
3056: 09 09 09 09 09 09 09 09 09 09 09 09 09 09 09 09 [................]
3066: 09 09 09 09 09 09 09 09 09 09 09 09 09 09 09 09 [................]
[snip 18 bytes]
JPEG SOF0 (15 bytes):
308c: 08 07 90 0a 20 03 01 22 00 02 11 01 03 11 01 [.... ..".......]
ImageWidth = 2592
ImageHeight = 1936
EncodingProcess = 0
BitsPerSample = 8
ColorComponents = 3
YCbCrSubSampling = 2 2
JPEG DHT (416 bytes):
309f: 00 00 01 05 01 01 01 01 01 01 00 00 00 00 00 00 [................]
30af: 00 00 01 02 03 04 05 06 07 08 09 0a 0b 10 00 02 [................]
30bf: 01 03 03 02 04 03 05 05 04 04 00 00 01 7d 01 02 [.............}..]
30cf: 03 00 04 11 05 12 21 31 41 06 13 51 61 07 22 71 [......!1A..Qa."q]
30df: 14 32 81 91 a1 08 23 42 b1 c1 15 52 d1 f0 24 33 [.2....#B...R..$3]
30ef: 62 72 82 09 0a 16 17 18 19 1a 25 26 27 28 29 2a [br........%&'()*]
30ff: 34 35 36 37 38 39 3a 43 44 45 46 47 48 49 4a 53 [456789:CDEFGHIJS]
[snip 304 bytes]
JPEG SOS
JPEG EOI
Keywords: JPEG, marker, anatomy, SOF, DQT, DHT, prime.

Resources