Error when reading txt-file using dart.io - dart

I'm trying to read a file of multiple lines with two digit numbers on each line representing a binary tree structure. I'm using dart and the standard io library. This is the code I'm using:
String contents = new File('./assets/triangle.txt').readAsStringSync();
Problem is I get the following error:
max_path_sum/assets/triangle.txt:1:1: Error: Expected a declaration, but got '75'.
75
^^
max_path_sum/assets/triangle.txt:2:1: Error: Expected a declaration, but got '95'.
95 64
^^
max_path_sum/assets/triangle.txt:2:4: Error: Expected a declaration, but got '64'.
95 64
^^
max_path_sum/assets/triangle.txt:3:1: Error: Expected a declaration, but got '17'.
17 47 82
^^
max_path_sum/assets/triangle.txt:3:4: Error: Expected a declaration, but got '47'.
17 47 82
^^
max_path_sum/assets/triangle.txt:3:7: Error: Expected a declaration, but got '82'.
17 47 82
^^
max_path_sum/assets/triangle.txt:4:1: Error: Expected a declaration, but got '18'.
18 35 87 10
^^
max_path_sum/assets/triangle.txt:4:4: Error: Expected a declaration, but got '35'.
18 35 87 10
^^
max_path_sum/assets/triangle.txt:4:7: Error: Expected a declaration, but got '87'.
18 35 87 10
^^
max_path_sum/assets/triangle.txt:4:10: Error: Expected a declaration, but got '10'.
18 35 87 10
^^
This is the file I'm trying to read:
75
95 64
17 47 82
18 35 87 10
20 04 82 47 65
19 01 23 75 03 34
88 02 77 73 07 63 67
99 65 04 28 06 16 70 92
41 41 26 56 83 40 80 70 33
41 48 72 33 47 32 37 16 94 29
53 71 44 65 25 43 91 52 97 51 14
70 11 33 28 77 73 17 78 39 68 17 57
91 71 52 38 17 14 91 43 58 50 27 29 48
63 66 04 68 89 53 67 30 73 16 69 87 40 31
04 62 98 27 23 09 70 98 73 93 38 53 60 04 23
I've tried different variations of reading the file. Asynchronously, synchronously, line by line, the whole file in one string but keep getting the same error. It obviously reads the lines but seem to expect some other form of input and is not happy with just giving me the data it reads... All examples I find on the web indicate that the code is correct and it should work. So my guess is that I might have to do something with the input file in order for it to be accepted as a valid input but it would seem unpractical since it is able to read the data just fine as it is.
I found the issue. The following formatting of the file works for the first line:
var a = "75";
95 64
17 47 82
18 35 87 10
20 04 82 47 65
19 01 23 75 03 34
88 02 77 73 07 63 67
99 65 04 28 06 16 70 92
41 41 26 56 83 40 80 70 33
41 48 72 33 47 32 37 16 94 29
53 71 44 65 25 43 91 52 97 51 14
70 11 33 28 77 73 17 78 39 68 17 57
91 71 52 38 17 14 91 43 58 50 27 29 48
63 66 04 68 89 53 67 30 73 16 69 87 40 31
04 62 98 27 23 09 70 98 73 93 38 53 60 04 23
So I guess if I make the contents of the file into a string variable it will be fine. But why is this? Isn't it possible to read an ordinary text file in dart without having to change the contents first?

Problem was that I imported the text file using import statement which is probably why dart expected it to be code.

Related

FFmpeg stream stops after a certain time

We have a little Nodejs app, which starts a stream process, with a child_process.spawn. On the client-side, we have an HTML5-canvas element, which records the video data new MediaRecorder(canvas.captureStream(30), config), then this client sends its data to our Nodejs server over a WebSocket connection. We using FFmpeg for video encoding and decoding, then we send the data to our 3-rd party service (MUX), which accepts the stream and broadcasts them. Sadly the process continuously loses its fps, and after in general 1 minute, stops with an interesting error code. (when we save the video result locally instead of streaming via rtmps, it works perfectly.
*The whole system is in docker.
The error:
stderr: [tls # 0x7f998e7bca40] Error in the pull function.
Our_app_logs: | av_interleaved_write_frame(): I/O error
Our_app_logs: | [flv # 0x7f998eeb1680] Failed to update header with correct duration.
Our_app_logs: | [flv # 0x7f998eeb1680] Failed to update header with correct filesize.
Our_app_logs: | Error writing trailer of rtmps://global-live.mux.com/app/94e85197-78a3-f092-3437-03d93aba74e0: I/O error
Our_app_logs: | <Buffer 5b 74 6c 73 20 40 20 30 78 37 66 39 39 38 65 37 62 63 61 34 30 5d 20 45 72 72 6f 72 20 69 6e 20 74 68 65 20 70 75 6c 6c 20 66 75 6e 63 74 69 6f 6e 2e ... >
Our_app_logs: | stderr: frame= 1478 fps= 25 q=23.0 Lsize=
402kB time=00:01:02.89 bitrate= 52.4kbits/s speed=1.05x
Our_app_logs: | video:369kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 9.034639%
Our_app_logs: | <Buffer 66 72 61 6d 65 3d 20 31 34 37 38 20 66 70 73 3d 20 32 35 20 71 3d 32 33 2e 30 20 4c 73 69 7a 65 3d 20 20 20 20 20 34 30 32 6b 42 20 74 69 6d 65 3d 30 ... >
Our_app_logs: | stderr: [tls # 0x7f998e7bca40] <Buffer 5b
74 6c 73 20 40 20 30 78 37 66 39 39 38 65 37 62 63 61 34 30 5d 20>
Our_app_logs: | stderr: The specified session has been invalidated for some reason.
Our_app_logs: | <Buffer 54 68 65 20 73 70 65 63 69 66 69 65 64 20 73 65 73 73 69 6f 6e 20 68 61 73 20 62 65 65 6e 20 69 6e 76 61 6c 69 64 61 74 65 64 20 66 6f 72 20 73 6f 6d ... >
Our_app_logs: | stderr: Last message repeated 1 times
Our_app_logs: | <Buffer 20 20 20 20 4c 61 73 74 20 6d 65 73 73 61 67 65 20 72 65 70 65 61 74 65 64 20 31 20 74 69 6d 65 73 0a>
Our_app_logs: | stderr: [libx264 # 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: frame I:13 Avg QP: 5.39 size:
2478
Our_app_logs: | <Buffer 66 72 61 6d 65 20 49 3a 31 33 20 20 20 20 41 76 67 20 51 50 3a 20 35 2e 33 39 20 20 73 69 7a 65 3a 20 20 32 34 37 38 0a>
Our_app_logs: | stderr: [libx264 # 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: frame P:1465 Avg QP:13.51 size:
235
Our_app_logs: | <Buffer 66 72 61 6d 65 20 50 3a 31 34 36 35 20 20 41 76 67 20 51 50 3a 31 33 2e 35 31 20 20 73 69 7a 65 3a 20 20 20 32 33 35 0a>
Our_app_logs: | stderr: [libx264 # 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: mb I I16..4: 99.2% 0.1% 0.7%
Our_app_logs: | <Buffer 6d 62 20 49 20 20 49 31 36 2e 2e 34 3a 20 39 39 2e 32 25 20 20 30 2e 31 25 20 20 30 2e 37 25 0a>
Our_app_logs: | stderr: [libx264 # 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: mb P I16..4: 0.3% 0.0% 0.0% P16..4: 0.1% 0.0% 0.0% 0.0% 0.0% skip:99.6%
Our_app_logs: | <Buffer 6d 62 20 50 20 20 49 31 36 2e 2e 34 3a 20 20 30 2e 33 25 20 20 30 2e 30 25 20 20 30 2e 30 25 20 20 50 31 36 2e 2e 34 3a 20 20 30 2e 31 25 20 20 30 2e ... >
Our_app_logs: | stderr: [libx264 # 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: 8x8 transform intra:0.3% inter:17.3%
Our_app_logs: | <Buffer 38 78 38 20 74 72 61 6e 73 66 6f 72 6d 20 69 6e 74 72 61 3a 30 2e 33 25 20 69 6e 74 65 72 3a 31 37 2e 33 25 0a>
Our_app_logs: | stderr: [libx264 # 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: coded y,uvDC,uvAC intra: 1.4% 6.9%
4.7% inter: 0.0% 0.0% 0.0%
Our_app_logs: | <Buffer 63 6f 64 65 64 20 79 2c 75 76 44 43 2c 75 76 41 43 20 69 6e 74 72 61 3a 20 31 2e 34 25 20 36 2e 39 25 20 34 2e 37 25 20 69 6e 74 65 72 3a 20 30 2e 30 ... >
Our_app_logs: | stderr: [libx264 # 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: i16 v,h,dc,p: 90% 5% 5% 0%
Our_app_logs: | <Buffer 69 31 36 20 76 2c 68 2c 64 63 2c 70 3a 20 39 30 25 20 20 35 25 20 20 35 25 20 20 30 25 0a>
Our_app_logs: | stderr: [libx264 # 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 23%
18% 51% 6% 0% 0% 0% 0% 3%
Our_app_logs: | <Buffer 69 38 20 76 2c 68 2c 64 63 2c 64 64 6c 2c 64 64 72 2c 76 72 2c 68 64 2c 76 6c 2c 68 75 3a 20 32 33 25 20 31 38 25 20 35 31 25 20 20 36 25 20 20 30 25 ... >
Our_app_logs: | stderr: [libx264 # 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 33%
25% 40% 0% 0% 0% 0% 0% 0%
Our_app_logs: | <Buffer 69 34 20 76 2c 68 2c 64 63 2c 64 64 6c 2c 64 64 72 2c 76 72 2c 68 64 2c 76 6c 2c 68 75 3a 20 33 33 25 20 32 35 25 20 34 30 25 20 20 30 25 20 20 30 25 ... >
Our_app_logs: | stderr: [libx264 # 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: i8c dc,h,v,p: 86% 7% 6% 0%
Our_app_logs: | <Buffer 69 38 63 20 64 63 2c 68 2c 76 2c 70 3a 20 38 36 25 20 20 37 25 20 20 36 25 20 20 30 25 0a>
Our_app_logs: | stderr: [libx264 # 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: Weighted P-Frames: Y:0.1% UV:0.1%
Our_app_logs: | <Buffer 57 65 69 67 68 74 65 64 20 50 2d 46 72 61 6d 65 73 3a 20 59 3a 30 2e 31 25 20 55 56 3a 30 2e 31 25 0a>
Our_app_logs: | stderr: [libx264 # 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: kb/s:2041.23
Our_app_logs: | <Buffer 6b 62 2f 73 3a 32 30 34 31 2e 32 33 0a>
Our_app_logs: | stderr: Conversion failed!
Our_app_logs: | <Buffer 43 6f 6e 76 65 72 73 69 6f 6e 20 66 61 69 6c 65 64 21 0a>
Our_app_logs: | close, code: 1, signal: null
Our_app_logs: | from react application: 14203
Our_app_logs: | Status ok...
Our_app_logs: | Data ok...
Our_app_logs: | FFmpeg ok...
Our_app_logs: | Writeable ok... <Buffer c4 81 0e 11 00 00 00 00 01 61 c7 80 5b 00 b6 72 03 bc 00 b7 03 de 59 7f 3c 27 80 01 b3 87 bc b2 e6 84 d0 f0 02 2d c0 00 00 00 01 61 00 08 70 c7 80 5b ... > undefined
Our_app_logs: | stderr: ffmpeg version 4.2.4 Copyright (c)
2000-2020 the FFmpeg developers
Our_app_logs: | built with gcc 9.2.0 (Alpine 9.2.0)
Our_app_logs: | configuration: --prefix=/usr --enable-avresample --enable-avfilter --enable-gnutls --enable-gpl --enable-libass --enable-libmp3lame --enable-libvorbis --enable-libvpx --enable-libxvid --enable-libx264 --enable-libx265 --enable-libtheora --enable-libv4l2 --enable-postproc --enable-pic --enable-pthreads --enable-shared --enable-libxcb --disable-stripping --disable-static --disable-librtmp --enable-vaapi --enable-vdpau --enable-libopus --disable-debug
Our_app_logs: | <Buffer 66 66 6d 70 65 67 20 76 65 72 73 69 6f 6e 20 34 2e 32 2e 34 20 43 6f 70 79 72 69 67 68 74 20 28 63 29 20 32 30 30 30 2d 32 30 32 30 20 74 68 65 20 46 ... >
Our_app_logs: | stderr: libavutil 56. 31.100 / 56. 31.100
Our_app_logs: | libavcodec 58. 54.100 / 58. 54.100
Our_app_logs: | libavformat 58. 29.100 / 58. 29.100
Our_app_logs: | libavdevice 58. 8.100 / 58. 8.100
Our_app_logs: | libavfilter 7. 57.100 / 7. 57.100
Our_app_logs: | libavresample 4. 0. 0 / 4. 0. 0
Our_app_logs: | libswscale 5. 5.100 / 5. 5.100
Our_app_logs: | libswresample 3. 5.100 / 3. 5.100
Our_app_logs: | libpostproc 55. 5.100 / 55. 5.100
Our_app_logs: | <Buffer 20 20 6c 69 62 61 76 75 74 69 6c 20 20 20 20 20 20 35 36 2e 20 33 31 2e 31 30 30 20 2f 20 35 36 2e 20 33 31 2e 31 30 30 0a 20 20 6c 69 62 61 76 63 6f ... >
Our_app_logs: | stderr: [h264 # 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | Last message repeated 4 times
Our_app_logs: | [h264 # 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 # 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 # 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 # 0x7f299f44f600] no frame!
Our_app_logs: | [h264 # 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 # 0x7f299f44f600] decode_slice_header error
Our_app_logs: | <Buffer 5b 68 32 36 34 20 40 20 30 78 37 66 32 39 39 66 34 34 66 36 30 30 5d 20 6e 6f 6e 2d 65 78 69 73 74 69 6e 67 20 50 50 53 20 31 34 20 72 65 66 65 72 65 ... >
Our_app_logs: | stderr: [h264 # 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 # 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 # 0x7f299f44f600] no frame!
Our_app_logs: | [h264 # 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 # 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 # 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 # 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 # 0x7f299f44f600] no frame!
Our_app_logs: | [h264 # 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 # 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 # 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 # 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 # 0x7f299f44f600] no frame!
Our_app_logs: | [h264 # 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | Last message repeated 5 times
Our_app_logs: | [h264 # 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 # 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 # 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 # 0x7f299f44f600] no frame!
Our_app_logs: | [h264 # 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 # 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 # 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 # 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 # 0x7f299f44f600] no frame!
Our_app_logs: | [h264 # 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 # 0x7f299f44f600] decode_slice_headerOur_app_logs: | [h264 # 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 # 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 # 0x7f299f44f600] no frame!
Our_app_logs: | [h264 # 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 # 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 # 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 # 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 # 0x7f299f44f600] no frame!
Our_app_logs: | [h264 # 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 # 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 # 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 # 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 # 0x7f299f44f600] no frame!
The FFmpeg config:
const FFMPEG_CONFIG = [
'-i',
'-',
// video codec config: low latency, adaptive bitrate
// '-vcodec',
// 'copy',
'-c:v',
'libx264',
'-preset',
'veryfast',
'-tune',
'zerolatency',
// audio codec config: sampling frequency (11025, 22050, 44100), bitrate 64 kbits
'-c:a',
'aac',
'-ar',
'44100',
'-b:a',
'64k',
//force to overwrite
'-y',
// used for audio sync
'-use_wallclock_as_timestamps',
'1',
'-async',
'1',
//'-filter_complex', 'aresample=44100', // resample audio to 44100Hz, needed if input is not 44100
//'-strict', 'experimental',
'-bufsize',
'1000',
'-f',
'flv',
];
The process:
const process = child_process.spawn('ffmpeg', [
...FFMPEG_CONFIG,
// 'local.bin',
url,
]);
process.stderr.on('data', data => {
console.log(`stderr: ${data}`, data);
});
process.stdin.on('error', e => {
console.log('FFmpeg STDIN Error', e);
});
process.on('error', err => console.log(err));
process.on('close', (code, signal) => {
console.log(`close, code: ${code}, signal: ${signal}`);
});
The writing:
if (!Buffer.isBuffer(data)) return;
if (!process.stdin.writable) return;
process.stdin.write(data);
Im found another FFmpeg config that works perfectly.
-f lavfi -re -i anullsrc -f h264 -thread_queue_size 1024 -framerate 10 -probesize 100 -i - -vcodec copy -acodec aac -g 20 -f flv
Update: Another issue with this, if you use docker, the built-in docker network is limited. So you have to build a network manually between the services. Why? The data transfer between the services is huge and with this limitation, the FFmpeg not receives enough data.

Export dataset to LaTeX

I am trying to export summary statistics that are saved as variables in the main dataset from Stata to LaTeX using the community-contributed command esttab. Here is the code:
sysuse auto, clear
collapse (sum) price mpg, by(make)
estpost tabstat price mpg, by(make)
esttab
The estpost tabstat command generates exactly the table I want to create in LaTeX but esttab only generates an empty table.
I also posted this question on Statalist.
The following works for me:
sysuse auto, clear
collapse (sum) price mpg, by(make)
estpost tabstat price mpg, by(make)
matrix A = e(price)', e(mpg)'
esttab matrix(A), title("Summary statistics: mean") nomtitle
Summary statistics: mean
--------------------------------------
price mpg
--------------------------------------
1 4099 22
2 4749 17
3 3799 22
4 9690 17
5 6295 23
6 9735 25
7 4816 20
8 7827 15
9 5788 18
10 4453 26
11 5189 20
12 10372 16
13 4082 19
14 11385 14
15 14500 14
16 15906 21
17 3299 29
18 5705 16
19 4504 22
20 5104 22
21 3667 24
22 3955 19
23 6229 23
24 4589 35
25 5079 24
26 8129 21
27 3984 30
28 4010 18
29 5886 16
30 6342 17
31 4296 21
32 4389 28
33 4187 21
34 5799 25
35 4499 28
36 11497 12
37 13594 12
38 13466 14
39 3995 30
40 3829 22
41 5379 14
42 6165 15
43 4516 18
44 6303 14
45 3291 20
46 8814 21
47 5172 19
48 4733 19
49 4890 18
50 4181 19
51 4195 24
52 10371 16
53 12990 14
54 4647 28
55 4425 34
56 4482 25
57 6486 26
58 4060 18
59 5798 18
60 4934 18
61 5222 19
62 4723 19
63 4424 19
64 4172 24
65 3895 26
66 3798 35
67 5899 18
68 3748 31
69 5719 18
70 7140 23
71 5397 41
72 4697 25
73 6850 25
74 11995 17
Total 6165.257 21.2973
--------------------------------------

Neo4j bolt driver ClientException

I have loaded about 20000 odd nodes from the consumer complaints csv file available in the neo4j load csv example. I am using Neo4j bolt driver in my java class to query .
Driver driver = GraphDatabase.driver( "bolt://localhost/trialschema", AuthTokens.basic( "neo4j", "neo" ) );
Session session = driver.session();
StatementResult result = session.run( "MATCH(n)-[r]-(m) return n,type(r),m limit 10" );
Map<String,Object> resultOfQuery = processResult(result);
session.close();
driver.close();
return returnMap ;
But when I am trying to loop through this StatementResult, inside processResult Method, I get the below exception
There was an unexpected error (type=Internal Server Error,
status=500).
org.springframework.web.util.NestedServletException: Request processing failed; nested exception is
org.neo4j.driver.v1.exceptions.ClientException: Trying to read message
complete ending '00 00' while there are more data left in the message
content unread: buffer [41 49 4e 53 54 b3 4e c9 77 af 91 89 43 6f 6d
70 6c 61 69 6e 74 a1 82 69 64 ca 00 1d 13 40 00 00 00 46 b1 71 93 b3
4e 07 91 87 43 6f 6d 70 61 6e 79 a1 84 6e 61 6d 65 8f 42 41 4e 4b 20
4f 46 20 41 4d 45 52 49 43 41 87 41 47 41 49 4e 53 54 b3 4e c9 77 9c
91 89 43 6f 6d 70 6c 61 69 6e 74 a1 82 69 64 ca 00 1b d4 06 00 00 00
46 b1 71 93 b3 4e 07 91 87 43 6f 6d 70 61 6e 79 a1 84 6e 61 6d 65 8f
42 41 4e 4b 20 4f 46 20 41 4d 45 52 49 43 41 87 41 47 41 49 4e 53 54
b3 4e c9 77 a4 91 89 43 6f 6d 70 6c 61 69 6e 74 a1 82 69 64 ca 00 1d
3c 54 00 00 00 46 b1 71 93 b3 4e 07 91 87 43 6f 6d 70 61 6e 79 a1 84
6e 61 6d 65 8f 42 41 4e 4b 20 4f 46 20 41 4d 45 52 49 43 41 87 41 47
41 49 4e 53 54 b3 4e c9 77 95 91 89 43 6f 6d 70 6c 61 69 6e 74 a1 82
69 64 ca 00 0b 56 ca 00 00 00 46 b1 71 93 b3 4e 07 91 87 43 6f 6d 70
61 6e 79 a1 84 6e 61 6d 65 8f 42 41 4e 4b 20 4f 46 20 41 4d 45 52 49
43 41 --------------unread chunk size 11075
I don't get this issue if I am trying to fetch only a small result by limiting the result to 50 or 100 . Can someone help ?

Making .bin File

This may be a stupid question, but I want a simpler way to replicate this technique used by Jon Erikson in the art of exploitation. In it he fills a .bin file using a bunch of complicated stuff grepping for source code and so forth.
$ for i in $(head exploit_notesearch.c | grep "^\"" | cut -d\" -f2)
do
echo -en $i
done > shellcode.bin
This generates a file like so:
hexdump -C shellcode.bin
00000000 48 31 f6 48 31 d2 48 bb 2f 62 69 6e 2f 73 68 11 |H1.H1.H./bin/sh.|
00000010 48 c1 e3 08 48 c1 eb 08 53 48 c7 c0 3b 11 11 11 |H...H...SH..;...|
00000020 48 89 e7 48 c1 e0 38 48 c1 e8 38 0f 05 |H..H..8H..8..|
0000002d
cat shellcode.bin
H1�H1�H�/bin/shH�H�SH��;H��H��8H��8
I don't know how to replicate this witchcraft, how can I make an exact replica using different commands? I tried
cat > new.bin
x48x31xf6x48x31xd2x48xbbx2fx62x69x6ex2fx73x68x11x48xc1xe3x08x48xc1xe3x08x53x48xc7xc0x3bx11x11x11x48x89xe7x48xc1xe0x38x48xc1xe8x38x0fx05
cat new.bin
x48x31xf6x48x31xd2x48xbbx2fx62x69x6ex2fx73x68x11x48xc1xe3x08x48xc1xe3x08x53x48xc7xc0x3bx11x11x11x48x89xe7x48xc1xe0x38x48xc1xe8x38x0fx05
hexdump -C new.bin
00000000 78 34 38 78 33 31 78 66 36 78 34 38 78 33 31 78 |x48x31xf6x48x31x|
00000010 64 32 78 34 38 78 62 62 78 32 66 78 36 32 78 36 |d2x48xbbx2fx62x6|
00000020 39 78 36 65 78 32 66 78 37 33 78 36 38 78 31 31 |9x6ex2fx73x68x11|
00000030 78 34 38 78 63 31 78 65 33 78 30 38 78 34 38 78 |x48xc1xe3x08x48x|
00000040 63 31 78 65 33 78 30 38 78 35 33 78 34 38 78 63 |c1xe3x08x53x48xc|
00000050 37 78 63 30 78 33 62 78 31 31 78 31 31 78 31 31 |7xc0x3bx11x11x11|
00000060 78 34 38 78 38 39 78 65 37 78 34 38 78 63 31 78 |x48x89xe7x48xc1x|
00000070 65 30 78 33 38 78 34 38 78 63 31 78 65 38 78 33 |e0x38x48xc1xe8x3|
00000080 38 78 30 66 78 30 35 0a |8x0fx05.|
00000088
I hardly knew how to ask this question and am not aware of anywhere else to look. I am grateful for any help. Thanks.
The tool you are looking for is xxd, which is normally used to get a hex dump of a binary file, but can also be used to perform a reverse hex dump, which is what you are trying to do.
Put your ascii hex into a file called Input.txt, without any of the x's.
4831f64831d248bb2f62696e2f73681148c1e30848c1e3085348c7c03b1111114889e748c1e03848c1e8380f05
Next, run xxd with the flag -p for plain, and -r for reverse.
xxd -r -p Input.txt Output.bin
Now, the desired output should be in Output.bin.
hexdump -C Output.bin
00000000 48 31 f6 48 31 d2 48 bb 2f 62 69 6e 2f 73 68 11 |H1.H1.H./bin/sh.|
00000010 48 c1 e3 08 48 c1 e3 08 53 48 c7 c0 3b 11 11 11 |H...H...SH..;...|
00000020 48 89 e7 48 c1 e0 38 48 c1 e8 38 0f 05 |H..H..8H..8..|
0000002d

SSL/TLS connection monitoring/analysis

I want to show the client that my client/server app is using a secure connection with mutual authentication.
My server app is implemented using C#, and the client is using C with OpenSSL. They can talk to each other and work fine. I'm trying do this demonstration with the two solutions below, but I'm not satisfied with the results.
Monitoring the connection using SSLDump:
The output is listed below, seems to be nothing wrong with it, but not sure about that. Please help me with this.
Monitoring the connection using Wireshark, and Unsniff,
A network analysis tool, that can give me some kind of high level analysis result, could be very helpful with my demonstration. But when I watch the connection with these two tools, they all show me that the protocol is TCP, and I want they show me that protocol of the connection is SSL/TLS.
Any ideas?
Output from SSLDump:
fan#ubuntu:~/Desktop$ sudo ssldump -i eth0 port 9527 and host fan
New TCP connection #1: 192.168.181.144(60992) <-> fan.local(9527)
1 1 0.0044 (0.0044) C>S Handshake
ClientHello
Version 3.1
cipher suites
Unknown value 0xc014
Unknown value 0xc00a
TLS_DHE_RSA_WITH_AES_256_CBC_SHA
TLS_DHE_DSS_WITH_AES_256_CBC_SHA
Unknown value 0x88
Unknown value 0x87
Unknown value 0xc00f
Unknown value 0xc005
TLS_RSA_WITH_AES_256_CBC_SHA
Unknown value 0x84
Unknown value 0xc012
Unknown value 0xc008
TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA
TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA
Unknown value 0xc00d
Unknown value 0xc003
TLS_RSA_WITH_3DES_EDE_CBC_SHA
Unknown value 0xc013
Unknown value 0xc009
TLS_DHE_RSA_WITH_AES_128_CBC_SHA
TLS_DHE_DSS_WITH_AES_128_CBC_SHA
Unknown value 0x9a
Unknown value 0x99
Unknown value 0x45
Unknown value 0x44
Unknown value 0xc00e
Unknown value 0xc004
TLS_RSA_WITH_AES_128_CBC_SHA
Unknown value 0x96
Unknown value 0x41
Unknown value 0xc011
Unknown value 0xc007
Unknown value 0xc00c
Unknown value 0xc002
TLS_RSA_WITH_RC4_128_SHA
TLS_RSA_WITH_RC4_128_MD5
TLS_DHE_RSA_WITH_DES_CBC_SHA
TLS_DHE_DSS_WITH_DES_CBC_SHA
TLS_RSA_WITH_DES_CBC_SHA
TLS_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA
TLS_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA
TLS_RSA_EXPORT_WITH_DES40_CBC_SHA
TLS_RSA_EXPORT_WITH_RC2_CBC_40_MD5
TLS_RSA_EXPORT_WITH_RC4_40_MD5
Unknown value 0xff
compression methods
unknown value
NULL
1 2 0.2572 (0.2528) S>C Handshake
ServerHello
Version 3.1
session_id[32]=
5e 08 00 00 82 35 0d a1 2a 91 c2 ac cb 62 1e f5
d0 88 3a d3 1a 64 58 a3 11 be 56 ef c4 fe 73 b7
cipherSuite TLS_RSA_WITH_RC4_128_MD5
compressionMethod NULL
Certificate
CertificateRequest
certificate_types rsa_sign
certificate_types dss_sign
certificate_authority
30 4f 31 15 30 13 06 0a 09 92 26 89 93 f2 2c 64
01 19 16 05 6c 6f 63 61 6c 31 19 30 17 06 0a 09
92 26 89 93 f2 2c 64 01 19 16 09 4f 62 6a 65 63
74 69 76 61 31 1b 30 19 06 03 55 04 03 13 12 4f
62 6a 65 63 74 69 76 61 2d 42 4a 50 44 43 2d 43
41
certificate_authority
30 6f 31 0b 30 09 06 03 55 04 06 13 02 53 45 31
14 30 12 06 03 55 04 0a 13 0b 41 64 64 54 72 75
73 74 20 41 42 31 26 30 24 06 03 55 04 0b 13 1d
41 64 64 54 72 75 73 74 20 45 78 74 65 72 6e 61
6c 20 54 54 50 20 4e 65 74 77 6f 72 6b 31 22 30
20 06 03 55 04 03 13 19 41 64 64 54 72 75 73 74
20 45 78 74 65 72 6e 61 6c 20 43 41 20 52 6f 6f
74
certificate_authority
30 81 c1 31 0b 30 09 06 03 55 04 06 13 02 55 53
31 17 30 15 06 03 55 04 0a 13 0e 56 65 72 69 53
69 67 6e 2c 20 49 6e 63 2e 31 3c 30 3a 06 03 55
04 0b 13 33 43 6c 61 73 73 20 31 20 50 75 62 6c
69 63 20 50 72 69 6d 61 72 79 20 43 65 72 74 69
66 69 63 61 74 69 6f 6e 20 41 75 74 68 6f 72 69
74 79 20 2d 20 47 32 31 3a 30 38 06 03 55 04 0b
13 31 28 63 29 20 31 39 39 38 20 56 65 72 69 53
69 67 6e 2c 20 49 6e 63 2e 20 2d 20 46 6f 72 20
61 75 74 68 6f 72 69 7a 65 64 20 75 73 65 20 6f
6e 6c 79 31 1f 30 1d 06 03 55 04 0b 13 16 56 65
72 69 53 69 67 6e 20 54 72 75 73 74 20 4e 65 74
77 6f 72 6b
certificate_authority
...
certificate_authority
30 52 31 0b 30 09 06 03 55 04 06 13 02 55 53 31
0b 30 09 06 03 55 04 08 13 02 43 41 31 12 30 10
06 03 55 04 07 13 09 50 61 6c 6f 20 41 6c 74 6f
31 10 30 0e 06 03 55 04 0a 13 07 42 6f 78 2e 6e
65 74 31 10 30 0e 06 03 55 04 03 13 07 62 6f 78
2e 6e 65 74
certificate_authority
30 16 31 14 30 12 06 03 55 04 03 13 0b 58 59 5a
20 43 6f 6d 70 61 6e 79
ServerHelloDone
1 3 0.3889 (0.1316) C>S Handshake
Certificate
1 4 0.3889 (0.0000) C>S Handshake
ClientKeyExchange
1 5 0.3889 (0.0000) C>S Handshake
CertificateVerify
Signature[256]=
02 fb a2 32 cd 1f 43 6e e7 1c b6 d8 8e a0 cc 49
6e 04 17 fa 8d 86 b0 a5 98 23 b0 19 ec f2 a5 8d
65 2d 31 81 73 96 43 89 19 81 ea 60 c8 12 4a 86
99 a5 b1 7b b5 29 ee 57 46 39 32 b4 f4 df 49 e0
97 35 c8 a2 e1 12 98 21 fa 75 87 9a 84 17 82 ba
72 a1 60 0a 44 3b 72 97 88 0c 44 0b 7c 14 f5 01
1b 47 90 fb c0 0e dc ae 91 c3 a4 38 c9 b7 c5 37
52 d6 4e a1 fb d5 87 35 df a3 cb 28 ab 73 f6 c3
b5 11 48 fc db 9b 84 a2 35 b7 c8 42 df b0 7a 20
b3 20 52 f0 6c 29 ae 96 4c 32 2e ba af ea 2e ad
2d ee 2e ed da 49 f7 55 38 29 7e 90 62 a7 03 4f
cd 76 14 36 b2 e0 a6 73 f2 7c c3 04 7f c1 a7 ca
db 5b 97 84 a7 df c3 e6 a5 15 0b f1 d6 bf e0 8b
7c 62 55 c9 2b 24 2d ac 8c 7b c8 72 70 9c ef 77
c4 5b d5 32 a8 30 6f e7 43 46 f9 47 05 c6 b9 4a
9d 98 6b f4 b6 bd 82 14 ec 65 99 42 f0 a0 9b 18
1 6 0.3889 (0.0000) C>S ChangeCipherSpec
1 7 0.3889 (0.0000) C>S Handshake
1 8 0.5480 (0.1591) S>C ChangeCipherSpec
1 9 0.5480 (0.0000) S>C Handshake
1 10 0.5502 (0.0022) C>S application_data
1 11 0.5513 (0.0011) C>S application_data
1 12 0.5517 (0.0004) C>S application_data
1 13 0.5521 (0.0004) C>S application_data
1 14 0.6444 (0.0923) S>C application_data
1 15 9.8598 (9.2153) C>S application_data
1 16 10.2293 (0.3694) C>S application_data
1 12.3329 (2.1035) C>S TCP FIN
1 12.3401 (0.0072) S>C TCP FIN
This looks like a normal SSL/TLS negotiation with a Client and Server certificate exchange. However at least the client and maybe the server look like they are running TLS 1.0. The use of TLS 1.2 and a more secure block cipher could be good depending on your security requirements.

Resources