We have a Docker Swarm with manager and several workers (in OpenStack) and encounter a problem that services cannot send emails using SMTP-SSL. Indeed from bash in-container this gets stuck:
# openssl s_client -debug -connect smtp.gmail.com:465 -crlf
CONNECTED(00000003)
write to 0x15bd2e0 [0x15bd360] (305 bytes => 305 (0x131))
0000 - 16 03 01 01 2c 01 00 01-28 03 03 50 0f 63 7c a4 ....,...(..P.c|.
0010 - 3f 96 d0 e4 e1 be 9a 0c-2b de 29 16 c9 54 b4 9d ?.......+.)..T..
0020 - 56 8d d7 76 f5 18 10 93-96 90 87 00 00 aa c0 30 V..v...........0
0030 - c0 2c c0 28 c0 24 c0 14-c0 0a 00 a5 00 a3 00 a1 .,.(.$..........
0040 - 00 9f 00 6b 00 6a 00 69-00 68 00 39 00 38 00 37 ...k.j.i.h.9.8.7
0050 - 00 36 00 88 00 87 00 86-00 85 c0 32 c0 2e c0 2a .6.........2...*
0060 - c0 26 c0 0f c0 05 00 9d-00 3d 00 35 00 84 c0 2f .&.......=.5.../
0070 - c0 2b c0 27 c0 23 c0 13-c0 09 00 a4 00 a2 00 a0 .+.'.#..........
0080 - 00 9e 00 67 00 40 00 3f-00 3e 00 33 00 32 00 31 ...g.#.?.>.3.2.1
0090 - 00 30 00 9a 00 99 00 98-00 97 00 45 00 44 00 43 .0.........E.D.C
00a0 - 00 42 c0 31 c0 2d c0 29-c0 25 c0 0e c0 04 00 9c .B.1.-.).%......
00b0 - 00 3c 00 2f 00 96 00 41-c0 11 c0 07 c0 0c c0 02 .<./...A........
00c0 - 00 05 00 04 c0 12 c0 08-00 16 00 13 00 10 00 0d ................
00d0 - c0 0d c0 03 00 0a 00 ff-01 00 00 55 00 0b 00 04 ...........U....
00e0 - 03 00 01 02 00 0a 00 1c-00 1a 00 17 00 19 00 1c ................
00f0 - 00 1b 00 18 00 1a 00 16-00 0e 00 0d 00 0b 00 0c ................
0100 - 00 09 00 0a 00 23 00 00-00 0d 00 20 00 1e 06 01 .....#..... ....
0110 - 06 02 06 03 05 01 05 02-05 03 04 01 04 02 04 03 ................
0120 - 03 01 03 02 03 03 02 01-02 02 02 03 00 0f 00 01 ................
0130 - 01 .
^C
Moreover, apt update/install nor pip install works:
# apt update
Hit:1 http://archive.ubuntu.com/ubuntu xenial InRelease
0% [Waiting for headers] [Waiting for headers]
pip install httpie
^CERROR: Operation cancelled by user
^CTraceback (most recent call last):
File "/usr/local/bin/pip", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.8/site-packages/pip/_internal/cli/main.py", line 75, in main
return command.main(cmd_args)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 105, in main
return self._main(args)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 224, in _main
self.handle_pip_version_check(options)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 149, in handle_pip_version_check
pip_self_version_check(session, options)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/self_outdated_check.py", line 207, in pip_self_version_check
best_candidate = finder.find_best_candidate("pip").best_candidate
File "/usr/local/lib/python3.8/site-packages/pip/_internal/index/package_finder.py", line 881, in find_best_candidate
candidates = self.find_all_candidates(project_name)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/index/package_finder.py", line 825, in find_all_candidates
package_links = self.process_project_url(
File "/usr/local/lib/python3.8/site-packages/pip/_internal/index/package_finder.py", line 790, in process_project_url
html_page = self._link_collector.fetch_page(project_url)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/index/collector.py", line 497, in fetch_page
return _get_html_page(location, session=self.session)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/index/collector.py", line 337, in _get_html_page
resp = _get_html_response(url, session=session)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/index/collector.py", line 126, in _get_html_response
resp = session.get(
File "/usr/local/lib/python3.8/site-packages/pip/_vendor/requests/sessions.py", line 546, in get
return self.request('GET', url, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/network/session.py", line 405, in request
return super(PipSession, self).request(method, url, *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pip/_vendor/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.8/site-packages/pip/_vendor/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pip/_vendor/cachecontrol/adapter.py", line 53, in send
resp = super(CacheControlAdapter, self).send(request, **kw)
File "/usr/local/lib/python3.8/site-packages/pip/_vendor/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/usr/local/lib/python3.8/site-packages/pip/_vendor/urllib3/connectionpool.py", line 665, in urlopen
httplib_response = self._make_request(
File "/usr/local/lib/python3.8/site-packages/pip/_vendor/urllib3/connectionpool.py", line 376, in _make_request
self._validate_conn(conn)
File "/usr/local/lib/python3.8/site-packages/pip/_vendor/urllib3/connectionpool.py", line 994, in _validate_conn
conn.connect()
File "/usr/local/lib/python3.8/site-packages/pip/_vendor/urllib3/connection.py", line 386, in connect
self.sock = ssl_wrap_socket(
File "/usr/local/lib/python3.8/site-packages/pip/_vendor/urllib3/util/ssl_.py", line 370, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/local/lib/python3.8/ssl.py", line 500, in wrap_socket
return self.sslsocket_class._create(
File "/usr/local/lib/python3.8/ssl.py", line 1040, in _create
self.do_handshake()
File "/usr/local/lib/python3.8/ssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
KeyboardInterrupt
(Terminated after significant wait time.)
Of course, everything works well on hosts (manager as well as workers)... For wget I have strange problem, that works for some and for some not:
# wget seznam.cz
--2020-04-30 06:14:34-- http://seznam.cz/
Resolving seznam.cz (seznam.cz)... 77.75.75.172, 77.75.75.176, 2a02:598:4444:1::1, ...
Connecting to seznam.cz (seznam.cz)|77.75.75.172|:80... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: https://www.seznam.cz/ [following]
--2020-04-30 06:14:34-- https://www.seznam.cz/
Resolving www.seznam.cz (www.seznam.cz)... 77.75.75.172, 77.75.74.172, 77.75.75.176, ...
Connecting to www.seznam.cz (www.seznam.cz)|77.75.75.172|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: 'index.html.1'
# wget google.com
--2020-04-30 06:15:10-- http://google.com/
Resolving google.com (google.com)... 216.58.201.110, 2a00:1450:4014:801::200e
Connecting to google.com (google.com)|216.58.201.110|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://www.google.com/ [following]
--2020-04-30 06:15:10-- http://www.google.com/
Resolving www.google.com (www.google.com)... 172.217.23.196, 2a00:1450:4014:80c::2004
Connecting to www.google.com (www.google.com)|172.217.23.196|:80... connected.
HTTP request sent, awaiting response...
Any ideas how to identify and fix the problem please? I am kinda out of ideas.
UPDATE: Now I suspect that the problem could be MTU (Docker default 1500 but ens3 has 1442 on all nodes) - need to figure out how to change it everywhere... But surprising I still DID NOT SOLVE this - adding --mtu, adjusting daemon.json, using --opt when creating networks, etc. still results in MTU 1500!
The problem was caused by MTU mismatch (1500 vs. 1442 used in OpenStack cloud).
I had to change /etc/docker/daemon.json, "break" my swarm, customize ingress network (docs) and docker_gwbridge (on each node before forming swarm again) (docs), remove the interfaces, restart Docker (it created interfaces again), and finally also change MTU on interfaces:
sudo ifconfig docker0 mtu 1442
sudo ifconfig docker_gwbridge mtu 1442
Because it seems that /etc/docker/daemon.json affects only interfaces in containers and not those on host.
If anyone knows a better solution, please let me know...
You can add a service to modify the mtu value After the docker.service
Create a script setdockermtu.sh
#!/bin/bash
sudo ifconfig docker0 mtu 9000
Create a service to run after docker.service
[Unit]
Description=Change docker0 default mtu
# When systemd stops or restarts the docker.service, the action is propagated to this unit
PartOf=docker.service
# Start this unit after the docker.service start
After=docker.service
[Service]
# The program will exit after running the script
Type=oneshot
# Execute the shell script
ExecStart=/usr/local/bin/setdockermtu.sh
# This service shall be considered active after start
RemainAfterExit=yes
[Install]
# This unit should start when docker.service is starting
WantedBy=docker.service
Copy it to /etc/systemd/system/setdockermtu.service
Make sure to add proper execute permission to both script and service
chmod a+x /usr/local/bin/setdockermtu.sh
chmod a+x /etc/systemd/system/setdockermtu.service
Reload changes and start service
sudo systemctl daemon-reload
sudo systemctl start setdockermtu.service
sudo systemctl enable setdockermtu.service
I have a list of messages from a control unit that i'm trying to replicate.
I have the body of the message correct, however, i can't seem to work out what CRC or checksum is being utilised.
00 FE 0F 32 A8 80 84 90
00 FE 0F 32 A8 80 84 54
00 FE FF 31 A8 80 84 38
00 FE 0F 32 A8 80 84 DC
00 FE 0F 32 A8 80 84 90
00 FE 0F 32 A8 80 84 54
00 FE 0F 32 A8 80 84 18
00 FE 0F 32 A8 80 84 DC
00 FE 0F 32 A8 80 84 90
00 FE 1F 32 A8 80 84 44
00 FE 0F 32 A8 80 84 18
00 FE 0F 32 A8 80 84 DC
00 FE AF 31 A8 80 84 4C
00 FE BF 31 A8 80 84 F0
00 FE CF 31 A8 80 84 A4
I know the checksum is the last 4 bits in the last byte of the message. the other 4 bits in the last byte are a counter, that counts from 0 to 3 and then wraps back around.
I know the whole message is little endian as well.
I have run this in reveng (probably done it wrong) and it does not return any results.
I was hoping someone smarter than me would be able to assist with identifying this.
I found the answer: its not CRC, but rather: 16 - (Sum of Byte 0 to 7 % 16)
I have a binary file as an output of an analytical device. I know it contains all the data I need. I'm trying to extract them from the file.
With the help of this question: How to view files in binary in the terminal?
I opened the file with Vim, and switched to binary editing. I can now browse the binary file. Some parts seem pretty readable:
00000340: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000350: 0000 0000 0000 0000 0000 1a4c 0061 0062 ...........L.a.b
00000360: 0072 0061 0073 006f 006c 0020 0037 0030 .r.a.s.o.l. .7.0
00000370: 0067 004c 0020 0066 006c 0075 006f 0072 .g.L. .f.l.u.o.r
00000380: 0065 0073 0063 0065 0069 006e 0065 0000 .e.s.c.e.i.n.e..
00000390: 0000 0000 0000 0000 0000 0000 0000 0000 ................
But some don't:
00001000: 4300 ea00 4b00 0000 d80e 401f 2800 5100 C...K.....#.(.Q.
00001010: 0400 0000 0000 6e03 36fe eaff b000 9cff ......n.6.......
00001020: 71ff e500 0eff f9ff 4aff 1200 2cff c400 q.......J...,...
00001030: 6f00 6bff 0d00 c4ff f1ff fdff d9ff 6b00 o.k...........k.
00001040: f8ff 1c00 5400 34ff a600 deff feff beff ....T.4.........
00001050: 1600 acff f5ff ffff 7600 39ff 5e00 9700 ........v.9.^...
00001060: 2a00 92ff 3300 94ff 5200 a2ff 6100 afff *...3...R...a...
00001070: b9ff 3500 a1ff 2300 f6ff a000 f9fe ef00 ..5...#.........
00001080: c5ff 6000 2100 53ff 9200 8cff 9200 a0ff ..`.!.S.........
00001090: 5d00 b0ff 8eff 8b00 30ff 0d01 adff 0300 ].......0.......
000010a0: 26ff ae00 cfff c000 6900 a2fe cc00 dfff &.......i.......
000010b0: fdff 4fff b900 f0ff ba00 cdfe 2a00 3400 ..O.........*.4.
000010c0: 7cff f800 56ff c7ff 8100 3300 f7fe 6cff |...V.....3...l.
000010d0: c500 3a00 0600 0500 8600 3800 56ff 1bff ..:.......8.V...
I would like to know if I can extract the data in a structured and clear way. So I have several questions, I don't really know where to start:
If I can read clearly some text, will I be able to read the other data in the file ?
How do I parse the test into an usable way ? I
I know my question is a bit unclear. I mainly need a starting point.
I'm comfortable in using Python and Bash for this task.
Here is the start of the file:
00000000: 0331 3331 0000 0000 0000 0000 0000 0000 .131............
00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000050: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000060: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000070: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000080: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000090: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000f0: 0000 0000 0000 0005 0000 0083 0001 0005 ................
00000100: 0001 0001 0010 2232 0000 0009 0000 0000 ......"2........
00000110: 0000 0000 0000 0000 1195 0000 0000 0000 ................
00000120: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000130: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000140: 1000 0000 0001 0331 0033 0031 0000 0000 .......1.3.1....
00000150: 0000 0000 0000 0001 0000 000c 4c00 4300 ............L.C.
00000160: 2000 4400 4100 5400 4100 2000 4600 4900 .D.A.T.A. .F.I.
00000170: 4c00 4500 0000 0000 0000 0000 0000 0000 L.E.............
00000180: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000190: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000001a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000001b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
EDIT:
Ok, now I know more about binary files. I'll try to improve my question.
Now, I know binary files are encoded with a structure (a certain formatting). So, to decode it, you must know the structure of the data. What I don't understand is that, for a certain portion of the binary file, the characters seem erratic. But the beginning of the file is completely readable. Why ? How can you try to discover the structure if you can't read the file correctly ?
The question has probably lost its relevance to its owner during this period. But still, needs a little touch to give an idea to find solutions in similar situations.
Any kind of binary file can be parsed into meaningful data, as long as one knows the structure used in the creation of the file. Then what is needed is to use a parser in any language known to read file content to get data. If there is no suitable kind of parser then it is needed to learn how to parse that data using file tools. if needed, seek for the words like "offset" and "seek" in file operations.
If the structure is not known, but the program is available to the user, it can be used to create new data files with small changes in the data itself, such as changing a character in a name or increasing a value by 1. Then these new files can be compared (as binary) to find which bytes have been changed. In these cases, it is not needed to map all data blocks. Instead, it is enough to know essential blocks only. After that, the rest is the same as described in above paragraph.
An example file structure:
Let say you have written your name, your age, and your weight into a file. If I write it to a file as text data content will be "YILMAZ4078.5" or "YILMAZ( B" in binary mode. In text mode, it is easy to construct a structure, yet binary file needs more elegant touch, which is a long topic on its own. it should suffice to say counting the bytes are needed to if the block is an integer or a real number etc.
I'm looking for method about to convert little endian.pfm file from imageMagick.
As I know we can get a pfm file like this.
convert input.bmp output.pfm
This output file is made by Big endian. But I want to convert as Little endian.
So is there any method to convert to Little endian from Big endian on ImageMagick?
Thanks
Endian can be controlled with -endian option.
Example. Create a 2x2 red PFM image with little endian, and write to hexdump.
$ convert -size 2x2 xc:red -endian LSB PFM:- | hexdump
0000000 50 46 0a 32 20 32 0a 2d 31 2e 30 0a 00 00 80 3f
0000010 00 00 00 00 00 00 00 00 00 00 80 3f 00 00 00 00
0000020 00 00 00 00 00 00 80 3f 00 00 00 00 00 00 00 00
0000030 00 00 80 3f 00 00 00 00 00 00 00 00
000003c
You can confirm the little endian by translating the header.
50 46 0a 32 20 32 0a 2d 31 2e 30 0a 00 00 80 3f
| | | |
little endian -------| "-1.0" | | LSM data|
Repeat above with big endian.
$ convert -size 2x2 xc:red -endian MSB PFM:- | hexdump
0000000 50 46 0a 32 20 32 0a 31 2e 30 0a 3f 80 00 00 00
0000010 00 00 00 00 00 00 00 3f 80 00 00 00 00 00 00 00
0000020 00 00 00 3f 80 00 00 00 00 00 00 00 00 00 00 3f
0000030 80 00 00 00 00 00 00 00 00 00 00
000003b
and observe...
50 46 0a 32 20 32 0a 31 2e 30 0a 3f 80 00 00 00
| | | |
big endian ----------| "1.0"| | MSB data|
I've been trying to inject sctp packets in linux environment with no success,
This is what I have done, I've created an association between two points(one terminal with sctp server and another terminal with sctp client).
Then I sent a char from client to the server and it's been delivered successfully, then I copied the packets from wireshark and put in my source code and incremented the TCN and stream sequence number and send it but i didn't receive it on the sctp server. as you can see below that my packets are exactly the same,
my packet is this:
IP Part
0000 45 00 00 34 00 00 40 00 40 84 3c 44 7f 00 00 01
0010 7f 00 00 01
SCTP Part
0000 0b 5e 0b 59 c2 e5 f8 00 00 00 00 00 00 03 00 13
0010 fe aa 43 3e 00 00 00 0c 00 00 00 00 64 0a 00 00
then after my packet failed to be delivered to the sctp server i sent with the sctp client just to compare it with my packet and it arrived to the sctp server,
IP Part
0000 45 02 00 34 00 00 40 00 40 84 3c 42 7f 00 00 01
0010 7f 00 00 01
SCTP Part
0000 0b 5e 0b 59 c2 e5 f8 00 00 00 00 00 00 03 00 13
0010 fe aa 43 3e 00 00 00 0c 00 00 00 00 64 0a 00 00
Regards,
devbag
Your SCTP Message seems badly formatted see RFC4960.
Regards
0b 5e :src port
0b 59 :dst port
c2 e5 f8 00 :verification tag
00 00 00 00 : checksum - WRONG
00 : chunk type (payload data)
03 : chunk flags (beginning and end fragment)
00 13 : chunk length
00 10 fe aa : TSN
43 3e : STREAM
00 00 : STREAM Sequence
00 0c 00 00 : PPID
00 00 64 0a 00 00 : User Data