I need to use a private key to do RSA Signing.
The company gave me the private key like this:
3082025C02010002818100AB13EC000380B4E489F717440D42473BD1C0F0B4F36D765171B3868ADF1CCAA782B48C71560C48342DFAEA01C6DBAF7A36F885B58BB24D7934352AA304941B1EB8373B561C4FBF11181C2E75ED3356CAE5B0DC9759A42CE997F5E5321AA5A67C5A2AE923F4705E61C2C7C8C2441CDCE6DE8638AB9294DA7A9A5B59E2E31C8A9B0203010001028180037ECDB4965DBBD46B8933DD7D13DC96B94B62DF9F959DF43E0977F74065BB323EF667642D68E4D4C417BB4E3BFCE311F12B94B7C7D9E5C15332BEE343C5AEE4223BF3ADE524C2726A685E62938C6B62ADA73529C762A61ABF707E936CFAC2233AD2C7DB0D8764194A7648C16A85FA54E0EBA32BFAB616CBE0009E5E3B8B5349024100D463E0012A09AA1399B5AD6BDDB47A7418F35109786899DDE1913647D3864A7897747D001122E3430CA58F4E94391208E9059606AEA8389E045B31273EBD2C75024100CE347F1CF65ACEFB6B21B758D7AB6B850F4BA1ECFC7DA1B0FC52538AF5D5280393ADB06D0A0762E66526C5755ABC6F81C22A6463E8B0E27D69BDEFFA8F3C38CF024100A10054FE8CEF668E1527339F61213EF263378F66AE701CB3A61A7E1B54ADA82662295BD88125014202843E6E42CE406DA0B72B5345731FF8293537BD9841AF410240472BDF63C3B3FC14D319440B2A05448B1C88624F45A6A7144B42AF0B1B6682F51917ADF934A8EFFDCD93E03B6D21F4EAB875A148CA9BA2D0DE9A6C25F3223A0902401E673CB216C11DCCD41D99F4892C2027A03ADE42E64C7B4410BF9C4D1B0A58C51CF33FA3BA1D8F4D693A2C5CCB0D42A4A787EE32729871FEF4FC143DFFA170A2
It's in a hex format, but OpenSSL's PEM_read_bio_RSAPrivateKey() function returns NULL. I searched for help about this and found out the private key must be in "PKCS8" format. So how can I convert the hex format above to PKCS8 format in my code?
The following uses OpenSSL 1.1.0 (I needed to do some other testing). It also uses d2i_RSAPrivateKey rather than PEM_read_bio_RSAPrivateKey. The key is in ASN.1/DER, so you can't use the PEM routines.
Compile and link with something similar to gcc -I/usr/local/ssl/1.1.0/include test.cc -o test.exe /usr/local/ssl/1.1.0/lib/libcrypto.a.
You should also add code to cleanup the library. Also see Library Initialization on the OpenSSL wiki.
#include <stdio.h>
#include <unistd.h>
#include <openssl/opensslconf.h>
#include <openssl/crypto.h>
#include <openssl/rsa.h>
const char encoded[] =
"3082025C02010002818100AB13EC000380B4E489F717440D42473BD1C0"
"F0B4F36D765171B3868ADF1CCAA782B48C71560C48342DFAEA01C6DBAF"
"7A36F885B58BB24D7934352AA304941B1EB8373B561C4FBF11181C2E75"
"ED3356CAE5B0DC9759A42CE997F5E5321AA5A67C5A2AE923F4705E61C2"
"C7C8C2441CDCE6DE8638AB9294DA7A9A5B59E2E31C8A9B020301000102"
"8180037ECDB4965DBBD46B8933DD7D13DC96B94B62DF9F959DF43E0977"
"F74065BB323EF667642D68E4D4C417BB4E3BFCE311F12B94B7C7D9E5C1"
"5332BEE343C5AEE4223BF3ADE524C2726A685E62938C6B62ADA73529C7"
"62A61ABF707E936CFAC2233AD2C7DB0D8764194A7648C16A85FA54E0EB"
"A32BFAB616CBE0009E5E3B8B5349024100D463E0012A09AA1399B5AD6B"
"DDB47A7418F35109786899DDE1913647D3864A7897747D001122E3430C"
"A58F4E94391208E9059606AEA8389E045B31273EBD2C75024100CE347F"
"1CF65ACEFB6B21B758D7AB6B850F4BA1ECFC7DA1B0FC52538AF5D52803"
"93ADB06D0A0762E66526C5755ABC6F81C22A6463E8B0E27D69BDEFFA8F"
"3C38CF024100A10054FE8CEF668E1527339F61213EF263378F66AE701C"
"B3A61A7E1B54ADA82662295BD88125014202843E6E42CE406DA0B72B53"
"45731FF8293537BD9841AF410240472BDF63C3B3FC14D319440B2A0544"
"8B1C88624F45A6A7144B42AF0B1B6682F51917ADF934A8EFFDCD93E03B"
"6D21F4EAB875A148CA9BA2D0DE9A6C25F3223A0902401E673CB216C11D"
"CCD41D99F4892C2027A03ADE42E64C7B4410BF9C4D1B0A58C51CF33FA3"
"BA1D8F4D693A2C5CCB0D42A4A787EE32729871FEF4FC143DFFA170A2";
int main(int argc, char* argv[])
{
long length = 0;
unsigned char* decoded = NULL;
RSA* rsa = NULL;
decoded = OPENSSL_hexstr2buf(encoded, &length);
if (decoded == NULL || length == 0)
return 1;
const unsigned char* temp = decoded;
rsa = d2i_RSAPrivateKey(NULL, &temp, length);
if (rsa == NULL)
return 1;
RSA_print_fp(stdout, rsa, 0);
if (rsa)
RSA_free(rsa);
if (decoded)
OPENSSL_free(decoded);
return 0;
}
const unsigned char* temp = decoded is used because temp is temporary, and d2i_RSAPrivateKey increments the pointer to the next private key (if another parse will be attempted). You can still get to the original data through decoded.
It results in:
$ ./test.exe
Private-Key: (1024 bit)
modulus:
00:ab:13:ec:00:03:80:b4:e4:89:f7:17:44:0d:42:
47:3b:d1:c0:f0:b4:f3:6d:76:51:71:b3:86:8a:df:
1c:ca:a7:82:b4:8c:71:56:0c:48:34:2d:fa:ea:01:
c6:db:af:7a:36:f8:85:b5:8b:b2:4d:79:34:35:2a:
a3:04:94:1b:1e:b8:37:3b:56:1c:4f:bf:11:18:1c:
2e:75:ed:33:56:ca:e5:b0:dc:97:59:a4:2c:e9:97:
f5:e5:32:1a:a5:a6:7c:5a:2a:e9:23:f4:70:5e:61:
c2:c7:c8:c2:44:1c:dc:e6:de:86:38:ab:92:94:da:
7a:9a:5b:59:e2:e3:1c:8a:9b
publicExponent: 65537 (0x10001)
privateExponent:
03:7e:cd:b4:96:5d:bb:d4:6b:89:33:dd:7d:13:dc:
96:b9:4b:62:df:9f:95:9d:f4:3e:09:77:f7:40:65:
bb:32:3e:f6:67:64:2d:68:e4:d4:c4:17:bb:4e:3b:
fc:e3:11:f1:2b:94:b7:c7:d9:e5:c1:53:32:be:e3:
43:c5:ae:e4:22:3b:f3:ad:e5:24:c2:72:6a:68:5e:
62:93:8c:6b:62:ad:a7:35:29:c7:62:a6:1a:bf:70:
7e:93:6c:fa:c2:23:3a:d2:c7:db:0d:87:64:19:4a:
76:48:c1:6a:85:fa:54:e0:eb:a3:2b:fa:b6:16:cb:
e0:00:9e:5e:3b:8b:53:49
prime1:
00:d4:63:e0:01:2a:09:aa:13:99:b5:ad:6b:dd:b4:
7a:74:18:f3:51:09:78:68:99:dd:e1:91:36:47:d3:
86:4a:78:97:74:7d:00:11:22:e3:43:0c:a5:8f:4e:
94:39:12:08:e9:05:96:06:ae:a8:38:9e:04:5b:31:
27:3e:bd:2c:75
prime2:
00:ce:34:7f:1c:f6:5a:ce:fb:6b:21:b7:58:d7:ab:
6b:85:0f:4b:a1:ec:fc:7d:a1:b0:fc:52:53:8a:f5:
d5:28:03:93:ad:b0:6d:0a:07:62:e6:65:26:c5:75:
5a:bc:6f:81:c2:2a:64:63:e8:b0:e2:7d:69:bd:ef:
fa:8f:3c:38:cf
exponent1:
00:a1:00:54:fe:8c:ef:66:8e:15:27:33:9f:61:21:
3e:f2:63:37:8f:66:ae:70:1c:b3:a6:1a:7e:1b:54:
ad:a8:26:62:29:5b:d8:81:25:01:42:02:84:3e:6e:
42:ce:40:6d:a0:b7:2b:53:45:73:1f:f8:29:35:37:
bd:98:41:af:41
exponent2:
47:2b:df:63:c3:b3:fc:14:d3:19:44:0b:2a:05:44:
8b:1c:88:62:4f:45:a6:a7:14:4b:42:af:0b:1b:66:
82:f5:19:17:ad:f9:34:a8:ef:fd:cd:93:e0:3b:6d:
21:f4:ea:b8:75:a1:48:ca:9b:a2:d0:de:9a:6c:25:
f3:22:3a:09
coefficient:
1e:67:3c:b2:16:c1:1d:cc:d4:1d:99:f4:89:2c:20:
27:a0:3a:de:42:e6:4c:7b:44:10:bf:9c:4d:1b:0a:
58:c5:1c:f3:3f:a3:ba:1d:8f:4d:69:3a:2c:5c:cb:
0d:42:a4:a7:87:ee:32:72:98:71:fe:f4:fc:14:3d:
ff:a1:70:a2
If you don't cleanup, then Valgrind will report something like:
$ valgrind --leak-check=full --show-leak-kinds=all --track-origins=yes ./test.exe
==32773== Memcheck, a memory error detector
==32773== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==32773== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==32773== Command: ./test.exe
==32773==
...
==32773==
==32773== HEAP SUMMARY:
==32773== in use at exit: 63,005 bytes in 365 blocks
==32773== total heap usage: 547 allocs, 182 frees, 69,806 bytes allocated
==32773==
==32773== 200 bytes in 1 blocks are still reachable in loss record 55 of 83
==32773== at 0x4D11: malloc (vg_replace_malloc.c:303)
==32773== by 0x10010AA3E: CRYPTO_zalloc (in ./test.exe)
==32773== by 0x1001522BB: CRYPTO_THREAD_lock_new (in ./test.exe)
==32773== by 0x100104EC8: do_ex_data_init (in ./test.exe)
==32773== by 0xD8FBF: pthread_once (in /usr/lib/system/libsystem_c.dylib)
==32773== by 0x1001523A8: CRYPTO_THREAD_run_once (in ./test.exe)
==32773== by 0x100104768: CRYPTO_new_ex_data (in ./test.exe)
==32773== by 0x10012A9C3: RSA_new_method (in ./test.exe)
==32773== by 0x100129ED1: rsa_cb (in ./test.exe)
==32773== by 0x100027057: asn1_item_embed_new (in ./test.exe)
==32773== by 0x1000244CA: asn1_item_embed_d2i (in ./test.exe)
==32773== by 0x100024125: ASN1_item_d2i (in ./test.exe)
==32773==
==32773== 4,096 bytes in 1 blocks are still reachable in loss record 81 of 83
==32773== at 0x4D11: malloc (vg_replace_malloc.c:303)
==32773== by 0x1431D8: __smakebuf (in /usr/lib/system/libsystem_c.dylib)
==32773== by 0x104F34: __swsetup (in /usr/lib/system/libsystem_c.dylib)
==32773== by 0x142392: __sfvwrite (in /usr/lib/system/libsystem_c.dylib)
==32773== by 0x142994: fwrite (in /usr/lib/system/libsystem_c.dylib)
==32773== by 0x100031997: file_write (in ./test.exe)
==32773== by 0x100030A29: BIO_write (in ./test.exe)
==32773== by 0x10002CFFB: BIO_vprintf (in ./test.exe)
==32773== by 0x10002CEFC: BIO_printf (in ./test.exe)
==32773== by 0x100129B7C: do_rsa_print (in ./test.exe)
==32773== by 0x10012EDBC: RSA_print_fp (in ./test.exe)
==32773== by 0x1000010C4: main (test.cc:48)
==32773==
==32773== LEAK SUMMARY:
==32773== definitely lost: 0 bytes in 0 blocks
==32773== indirectly lost: 0 bytes in 0 blocks
==32773== possibly lost: 0 bytes in 0 blocks
==32773== still reachable: 4,296 bytes in 2 blocks
==32773== suppressed: 58,709 bytes in 363 blocks
==32773==
Related
I'm trying to write application in C which manipulates GSettings. Unfortunately I have encountered some kind of memory leaks so I tried to track them down. I'm not sure whether this is library bug or am I missing something. This is the smallest example I come up with that allocates memory till it crashes.
#include <glib.h>
#include <gio/gio.h>
int main() {
while (1) {
GSettings *settings = g_settings_new("com.linuxmint.updates");
g_object_unref(settings);
//g_clear_object(&settings); // This leaks as well but seems to leak "slower"
}
return 0;
}
Can anyone explain me why memory leaks in this example and how to fix it?
PS I'm using libglib-2.0 (version 2.56.3 which come with Ubuntu 18.04 LTS / Mint).
EDIT 1
As per request in comments I'm posting valgrind output. I'm using command: valgrind --tool=memcheck --leak-check=full --leak-resolution=high --num-callers=50 --show-leak-kinds=definite ./main. I have changed program a little to be finite (it looping while 100.000 times). Here is output for that changed parameter.
==16375== Memcheck, a memory error detector
==16375== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==16375== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==16375== Command: ./main
==16375==
==16375==
==16375== HEAP SUMMARY:
==16375== in use at exit: 297,081,397 bytes in 5,056,358 blocks
==16375== total heap usage: 26,147,615 allocs, 21,091,257 frees, 1,064,178,170 bytes allocated
==16375==
==16375== LEAK SUMMARY:
==16375== definitely lost: 0 bytes in 0 blocks
==16375== indirectly lost: 0 bytes in 0 blocks
==16375== possibly lost: 2,840 bytes in 27 blocks
==16375== still reachable: 297,066,261 bytes in 5,056,238 blocks
==16375== of which reachable via heuristic:
==16375== length64 : 1,384 bytes in 28 blocks
==16375== newarray : 1,808 bytes in 33 blocks
==16375== suppressed: 0 bytes in 0 blocks
==16375== Reachable blocks (those to which a pointer was found) are not shown.
==16375== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==16375==
==16375== For counts of detected and suppressed errors, rerun with: -v
==16375== ERROR SUMMARY: 27 errors from 27 contexts (suppressed: 0 from 0)
I'm not an expert but parameter still reachable grows with number of loop. How those objects (or rather structures) can be reached if I'm consistently using one variable? Am I missing something? I'm trying do what is advised here: https://developer.gnome.org/gobject/stable/gobject-memory.html
EDIT 2
I was digging deeper into this problem. Because I was unsure that my code actually is correct I decided to change it to another GObject like this:
#include <glib.h>
#include <gio/gio.h>
int main() {
while (1) {
GFile *file = g_file_new_for_path ("/path/to/some/file");
g_object_unref(file);
//g_clear_object(&settings);
}
return 0;
}
I'm aware this do not open any file and only creates handle to resource but this code have constant memory usage over time. If I remove unref then it obviously leaks and crashes.
This is how valgrind output looks for this snippet for 100.000 and 1.000.000 iterations.
iterations = 100.000
==13257== Memcheck, a memory error detector
==13257== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==13257== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==13257== Command: ./main
==13257==
==13257==
==13257== HEAP SUMMARY:
==13257== in use at exit: 159,435 bytes in 1,975 blocks
==13257== total heap usage: 205,209 allocs, 203,234 frees, 6,758,893 bytes allocated
==13257==
==13257== LEAK SUMMARY:
==13257== definitely lost: 0 bytes in 0 blocks
==13257== indirectly lost: 0 bytes in 0 blocks
==13257== possibly lost: 2,528 bytes in 26 blocks
==13257== still reachable: 144,699 bytes in 1,852 blocks
==13257== of which reachable via heuristic:
==13257== length64 : 1,688 bytes in 32 blocks
==13257== newarray : 1,840 bytes in 35 blocks
==13257== suppressed: 0 bytes in 0 blocks
==13257== Rerun with --leak-check=full to see details of leaked memory
==13257==
==13257== For counts of detected and suppressed errors, rerun with: -v
==13257== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
iterations = 1.000.000
==12440== Memcheck, a memory error detector
==12440== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==12440== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==12440== Command: ./main
==12440==
==12440==
==12440== HEAP SUMMARY:
==12440== in use at exit: 157,241 bytes in 1,936 blocks
==12440== total heap usage: 2,005,339 allocs, 2,003,403 frees, 64,363,746 bytes allocated
==12440==
==12440== LEAK SUMMARY:
==12440== definitely lost: 0 bytes in 0 blocks
==12440== indirectly lost: 0 bytes in 0 blocks
==12440== possibly lost: 2,528 bytes in 26 blocks
==12440== still reachable: 142,505 bytes in 1,813 blocks
==12440== of which reachable via heuristic:
==12440== length64 : 1,688 bytes in 32 blocks
==12440== newarray : 1,840 bytes in 35 blocks
==12440== suppressed: 0 bytes in 0 blocks
==12440== Rerun with --leak-check=full to see details of leaked memory
==12440==
==12440== For counts of detected and suppressed errors, rerun with: -v
==12440== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
This gives me some idea that second code have almost same number allocations vs frees (diffrence is in both cases < 2000 which are probably some static allocations for lifetime of library).
This is not the case in first snippet where I use GSettings object. Number of allocations vs frees is nowhere constant and it grows over time.
I will try running this program against latest glib when I'll have access to some rolling release distro (arch probably) because I think compiling latest glib and plug it into Ubuntu is too complicated to me.
This ‘leak’ of reachable memory is an artifact of your test program. Each time you construct a GSettings object, it needs to add some match rules to the D-Bus session bus so that it can receive signals from the dconf daemon. Adding a match rule means sending a D-Bus method call to the message bus daemon and then waiting for a reply.
By creating 100000 GSettings objects in a row, you are queueing up 100000 AddMatch calls to the message bus daemon, including 100000 allocations containing information about the pending method call replies. However, your program exits before the message bus daemon replies to the majority of the AddMatch calls; so a lot of those allocations detailing the pending replies are still allocated on exit.
If your program were to sleep for, say, a minute until the message bus daemon had replied to all the AddMatch calls, I would expect that the ‘still reachable’ allocations would be consistent with the GFile example you ran.
(Note that it’s OK to do a usleep() call in your main() function to demonstrate this, as the D-Bus method calls and replies are handled in a separate worker thread.)
I would like to run OrientDB on an EC2 micro (free tier) instance. I am unable to find official documentation for OrientDB that gives memory requirements, however I found this question that says 512MB should be fine. I am running an EC2 micro instance which has 1GB RAM. However, when I try to run OrientDB I get the JRE error shown below. My initial thought was that I needed to increase the jre memory using -xmx, but I guess it would be the shell script that would do this.. Has anyone successfully run OrientDB in an EC2 micro instance or run into this problem?
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000007a04a0000, 1431699456, 0) failed; error='Cannot allocate memory' (errno=12)
There is insufficient memory for the Java Runtime Environment to continue.
Native memory allocation (malloc) failed to allocate 1431699456 bytes for committing reserved memory.
An error report file with more information is saved as:
/tmp/jvm-14728/hs_error.log
Here are the contents of the error log:
OS:Linux
uname:Linux 4.14.47-56.37.amzn1.x86_64 #1 SMP Wed Jun 6 18:49:01 UTC 2018 x86_64
libc:glibc 2.17 NPTL 2.17
rlimit: STACK 8192k, CORE 0k, NPROC 3867, NOFILE 4096, AS infinity
load average:0.00 0.00 0.00
/proc/meminfo:
MemTotal: 1011168 kB
MemFree: 322852 kB
MemAvailable: 822144 kB
Buffers: 83188 kB
Cached: 523056 kB
SwapCached: 0 kB
Active: 254680 kB
Inactive: 369952 kB
Active(anon): 18404 kB
Inactive(anon): 48 kB
Active(file): 236276 kB
Inactive(file): 369904 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 36 kB
Writeback: 0 kB
AnonPages: 18376 kB
Mapped: 31660 kB
Shmem: 56 kB
Slab: 51040 kB
SReclaimable: 41600 kB
SUnreclaim: 9440 kB
KernelStack: 1564 kB
PageTables: 2592 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 505584 kB
Committed_AS: 834340 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 0 kB
VmallocChunk: 0 kB
AnonHugePages: 0 kB
ShmemHugePages: 0 kB
ShmemPmdMapped: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 49152 kB
DirectMap2M: 999424 kB
CPU:total 1 (initial active 1) (1 cores per cpu, 1 threads per core) family 6 model 63 stepping 2, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, avx2, aes, erms, tsc
/proc/cpuinfo:
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 63
model name : Intel(R) Xeon(R) CPU E5-2676 v3 # 2.40GHz
stepping : 2
microcode : 0x3c
cpu MHz : 2400.043
cache size : 30720 KB
physical id : 0
siblings : 1
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm cpuid_fault invpcid_single pti fsgsbase bmi1 avx2 smep bmi2 erms invpcid xsaveopt
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips : 4800.05
clflush size : 64
cache_alignment : 64
address sizes : 46 bits physical, 48 bits virtual
power management:
Memory: 4k page, physical 1011168k(322728k free), swap 0k(0k free)
vm_info: OpenJDK 64-Bit Server VM (24.181-b00) for linux-amd64 JRE (1.7.0_181-b00), built on Jun 5 2018 20:36:03 by "mockbuild" with gcc 4.8.5 20150623 (Red Hat 4.8.5-28)
time: Mon Aug 20 20:51:08 2018
elapsed time: 0 seconds
Orient can easily run in 512MB though your performance and throughput will not be as high. In OrientDB 3.0.x you can use the environment variable ORIENTDB_OPTS_MEMORY to set it. On the command line I can, for example run:
cd $ORIENTDB_HOME/bin
export ORIENTDB_OPTS_MEMORY="-Xmx512m"
./server.sh
(where $ORIENTDB_HOME is where you have OrientDB installed) and I'm running with 512MB of memory.
As an aside, if you look in $ORIENTDB_HOME/bin/server.sh you'll see that there is even code to check if the server is running on a Raspberry Pi and those range from 256MB to 1GB so the t2.micro will run just fine.
I am trying to use recently released amazon-ecs-plugin:1.15 which support fargate but I am getting below error.
WARNING: Slave {0} - Cannot create ECS Task
May 24, 2018 1:10:39 PM hudson.slaves.NodeProvisioner$2 run
WARNING: Unexpected exception encountered while provisioning agent ECS Slave ecs-jenkins-slave
com.amazonaws.services.ecs.model.ClientException: No Fargate configuration exists for given values. (Service: AmazonECS; Status Code: 400; Error Code: ClientException; Request ID:****-****)
Here is my config
Check your CPU and memory amounts; you need to follow specific combinations:
CPU value Memory value (MiB)
256 (.25 vCPU) 512 (0.5GB), 1024 (1GB), 2048 (2GB)
512 (.5 vCPU) 1024 (1GB), 2048 (2GB), 3072 (3GB), 4096 (4GB)
1024 (1 vCPU) 2048 (2GB), 3072 (3GB), 4096 (4GB), 5120 (5GB), 6144 (6GB), 7168 (7GB), 8192 (8GB)
2048 (2 vCPU) Between 4096 (4GB) and 16384 (16GB) in increments of 1024 (1GB)
4096 (4 vCPU) Between 8192 (8GB) and 30720 (30GB) in increments of 1024 (1GB)
Reference: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html
To add to user6327093's answer
I had a similar issue when trying to create resources on AWS ECS using Terraform. The error was:
Error: ClientException: No Fargate configuration exists for given values.
Here's how I fixed it:
You must adhere to the supported task CPU and memory values for tasks that are hosted on Fargate are as follows.
CPU value
Memory value (MiB)
256 (.25 vCPU)
512 (0.5GB), 1024 (1GB), 2048 (2GB)
512 (.5 vCPU)
1024 (1GB), 2048 (2GB), 3072 (3GB), 4096 (4GB)
1024 (1 vCPU)
2048 (2GB), 3072 (3GB), 4096 (4GB), 5120 (5GB), 6144 (6GB), 7168 (7GB), 8192 (8GB)
2048 (2 vCPU)
Between 4096 (4GB) and 16384 (16GB) in increments of 1024 (1GB)
4096 (4 vCPU)
Between 8192 (8GB) and 30720 (30GB) in increments of 1024 (1GB)
In my case, the CPU was 4096, while the memory was 32768 (32GB). However, 32768 (32GB) memory is not supported by Fargate from the table above, so I had to change the memory to 30720 (30GB).
Resources: Invalid CPU or memory value specified
That's all
I've been trying for days to establish a secure (SSL/TLS) connection to an IBM MessageSight Virtual Appliance using the equivalent of the latest Eclipse Paho C Client library, which is IBM WebSphere MQ Client Pack MA9B for Mobile V1.0.0.4.
Client Side:
Objective-C Code:
...
client = [client initWithHosts:hosts ports:ports clientId:clientId];
ConnectOptions *opts = [[ConnectOptions alloc] init];
opts.timeout = 3600;
/*
opts.userName = #"******";
opts.password = #"******";
*/
opts.cleanSession = cleanSession;
opts.willMessage = nil;
NSBundle *mainBundle = [NSBundle mainBundle];
NSString *ksFile = [mainBundle pathForResource: #"ClientKeyStore" ofType: #"pem"];
NSString *pkFile = [mainBundle pathForResource: #"ClientKey" ofType: #"pem"];
NSString *tsFile = [mainBundle pathForResource: #"RootCAKey" ofType: #"pem"];
if (DEBUG) {
NSLog(#"Bundle ==> %#", mainBundle);
NSLog(#"ClientKeyStore ==> %#", ksFile);
NSLog(#"ClientKey ==> %#", pkFile);
NSLog(#"TrustStore ==> %#", tsFile);
}
SSLOptions *ssl = [[SSLOptions alloc] init];
ssl.enableServerCertAuth = NO;
// ssl.enabledCipherSuites = #"SHA2";
ssl.keyStore = ksFile;
ssl.privateKey = pkFile;
ssl.privateKeyPassword = #"******";
ssl.trustStore = tsFile;
opts.sslProperties = ssl;
[client connectWithOptions:opts invocationContext:self onCompletion:callback];
This is the Objective-C code I've used to interact with the C library through it's wrapper (MqttOCClient.h/m).
Now the specs:
iOS 7.1 onwards, Xcode 6.3.1 (Simulator).
iOS 8.3, Xcode 6.3.1 (iPhone 5S).
MQTT iOS Client: IBM WebSphere MQ Client Pack MA9B for Mobile V1.0.0.4 (http://www-01.ibm.com/support/knowledgecenter/?lang=en#!/SSFKSJ_7.5.0/com.ibm.mm.tc.doc/tc10120_.htm).
Trace:
I've enabled trace mode on the library through the 2 Environment Variables:
MQTT_C_CLIENT_TRACE_LEVEL = MAXIMUM
MQTT_C_CLIENT_TRACE = ON
and this is the output:
D] >>MessagingClient::connectWithOptions
I] first serverURI is ssl://example.com:1884
=========================================================
Trace Output
Product name: Paho Asynchronous MQTT C Client Library
Version: ##MQTTCLIENT_VERSION_TAG##
Build level: ##MQTTCLIENT_BUILD_TAG##
OpenSSL version: OpenSSL 1.0.1h 5 Jun 2014
OpenSSL flags: compiler: /Applications/Xcode.app/Contents/Developer/usr/bin/gcc -arch i386 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator7.1.sdk -miphoneos-version-min=7.1 -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -O3 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator7.1.sdk -fomit-frame-pointer -fno-common
OpenSSL build timestamp: built on: Thu Jun 5 14:59:07 BST 2014
OpenSSL platform: platform: iphoneos-cross
OpenSSL directory: OPENSSLDIR: "/tmp/openssl-1.0.1h-iOS-i386"
=========================================================
19691231 200000.000 Allocating 32 bytes in heap at file /Users/asm/workspace/Client/client_ios/iosMQTT/iosMQTT/mqttCClient/LinkedList.c line 56 ptr 0x7a068df0
19691231 200000.000 (42856916) (1)> Socket_outInitialize:124
19691231 200000.000 (42856916) (2)> SocketBuffer_initialize:85
19691231 200000.000 Allocating 32 bytes in heap at file /Users/asm/workspace/Client/client_ios/iosMQTT/iosMQTT/mqttCClient/SocketBuffer.c line 73 ptr 0x79f75480
19691231 200000.000 Allocating 1008 bytes in heap at file /Users/asm/workspace/Client/client_ios/iosMQTT/iosMQTT/mqttCClient/SocketBuffer.c line 75 ptr 0x7a917000
19691231 200000.000 Allocating 32 bytes in heap at file /Users/asm/workspace/Client/client_ios/iosMQTT/iosMQTT/mqttCClient/LinkedList.c line 56 ptr 0x79f791c0
19691231 200000.000 (42856916) (2)< SocketBuffer_initialize:89
19691231 200000.000 Allocating 32 bytes in heap at file /Users/asm/workspace/Client/client_ios/iosMQTT/iosMQTT/mqttCClient/LinkedList.c line 56 ptr 0x79f792a0
19691231 200000.000 Allocating 32 bytes in heap at file /Users/asm/workspace/Client/client_ios/iosMQTT/iosMQTT/mqttCClient/LinkedList.c line 56 ptr 0x79f786f0
19691231 200000.000 Allocating 32 bytes in heap at file /Users/asm/workspace/Client/client_ios/iosMQTT/iosMQTT/mqttCClient/LinkedList.c line 56 ptr 0x7a26ba90
19691231 200000.000 (42856916) (1)< Socket_outInitialize:137
19691231 200000.000 Allocating 32 bytes in heap at file /Users/asm/workspace/Client/client_ios/iosMQTT/iosMQTT/mqttCClient/LinkedList.c line 56 ptr 0x7a26bcb0
19691231 200000.000 Allocating 32 bytes in heap at file /Users/asm/workspace/Client/client_ios/iosMQTT/iosMQTT/mqttCClient/LinkedList.c line 56 ptr 0x7a26ba20
19691231 200000.000 (42856916) (1)> SSLSocket_initialize:398
19691231 200000.000 Allocating 1808 bytes in heap at file /Users/asm/workspace/Client/client_ios/iosMQTT/iosMQTT/mqttCClient/SSLSocket.c line 414 ptr 0x7b8fd600
...
20150514 130126.866 (42856916) (2)< SSL_create_mutex:313 (0)
20150514 130126.866 (42856916) (2)> SSL_create_mutex:307
...
20150514 130126.867 (42856916) (1)< SSLSocket_initialize:438 (1)
20150514 130126.867 Allocating 144 bytes in heap at file /Users/asm/workspace/Client/client_ios/iosMQTT/iosMQTT/mqttCClient/MQTTAsync.c line 374 ptr 0x79f75f10
20150514 130126.867 Allocating 32 bytes in heap at file /Users/asm/workspace/Client/client_ios/iosMQTT/iosMQTT/mqttCClient/MQTTAsync.c line 386 ptr 0x79f75fb0
20150514 130126.867 Allocating 32 bytes in heap at file /Users/asm/workspace/Client/client_ios/iosMQTT/iosMQTT/mqttCClient/LinkedList.c line 56 ptr 0x79ec7160
20150514 130126.867 Allocating 16 bytes in heap at file /Users/asm/workspace/Client/client_ios/iosMQTT/iosMQTT/mqttCClient/LinkedList.c line 93 ptr 0x79e78970
20150514 130126.867 Allocating 96 bytes in heap at file /Users/asm/workspace/Client/client_ios/iosMQTT/iosMQTT/mqttCClient/MQTTAsync.c line 391 ptr 0x79ec1840
20150514 130126.867 Allocating 32 bytes in heap at file /Users/asm/workspace/Client/client_ios/iosMQTT/iosMQTT/mqttCClient/LinkedList.c line 56 ptr 0x79ec1780
20150514 130126.867 Allocating 32 bytes in heap at file /Users/asm/workspace/Client/client_ios/iosMQTT/iosMQTT/mqttCClient/LinkedList.c line 56 ptr 0x79ed00c0
20150514 130126.867 Allocating 32 bytes in heap at file /Users/asm/workspace/Client/client_ios/iosMQTT/iosMQTT/mqttCClient/LinkedList.c line 56 ptr 0x79ec9090
20150514 130126.867 Allocating 32 bytes in heap at file /Users/asm/workspace/Client/client_ios/iosMQTT/iosMQTT/mqttCClient/MQTTAsync.c line 397 ptr 0x79ec2110
20150514 130126.867 (42856916) (1)> MQTTPersistence_create:47
20150514 130126.867 (42856916) (1)< MQTTPersistence_create:93 (0)
20150514 130126.867 (42856916) (1)> MQTTPersistence_initialize:108
20150514 130126.867 (42856916) (1)< MQTTPersistence_initialize:116 (0)
20150514 130126.867 (42856916) (1)> MQTTAsync_restoreCommands:666
20150514 130126.867 0 commands restored for client 32c94ab93d29fda895b02f6
20150514 130126.867 (42856916) (1)< MQTTAsync_restoreCommands:698 (0)
20150514 130126.867 (42856916) (1)> MQTTAsync_restoreMessageQueue:1872
20150514 130126.867 0 queued messages restored for client 32c94ab93d29fda895b02f6
20150514 130126.867 (42856916) (1)< MQTTAsync_restoreMessageQueue:1903 (0)
20150514 130126.867 Allocating 16 bytes in heap at file /Users/asm/workspace/Client/client_ios/iosMQTT/iosMQTT/mqttCClient/LinkedList.c line 93 ptr 0x79e7cef0
20150514 130126.867 (42856916) (0)< MQTTAsync_create:416 (0)
2015-05-14 13:01:26.867 SmartBanking[2616:607] D] C Client created
20150514 130126.867 (42856916) (0)> MQTTAsync_setCallbacks:1658
20150514 130126.867 (42856916) (0)< MQTTAsync_setCallbacks:1672 (0)
2015-05-14 13:01:26.868 SmartBanking[2616:607] D] Calling C client to make connection
20150514 130126.867 (42856916) (0)> MQTTAsync_connect:1990
20150514 130126.867 (42856916) (0)< MQTTAsync_connect:2177 (-8)
E] C client connect failed
Connection Error: ErrorCode=8 ErrorMessage=Unable to connect
D] <<MessagingClient::connectWithOptions
Server Side:
IBM MessageSight v1.1.0
Server Certificate: 2048bits Self-Signed.
Security is enabled and the Endpoint is up.
User and Pass enabled.
Side Notes:
If I use a browser I can view the certificate, which means it can connect to the server.
Works if I use Eclipse Paho for Java, validating the Server Certificate with the same TrustStore (which contains the Self-Signed CA).
I will appreciate if somebody who has make an SSL connection to MQTT from iOS using this library can point me to the right direction.
Regards,
I suspect your problem is with the cipherspec/ssl version you are using. The Java default cipherspec may be supported by MessageSight 1.1, but the default iOS cipherspec may not be.
You should try setting a specific cipherspec that is in the supported list.
I am trying to figure out what is wrong with my Debian server - I am getting warnings of having not enough free memory - top (as you can see below) is saying that 1.8G is consumed, but I am unable to find which application is responsible for it. There is only Tomcat running, which consumes, according to top, ~25 % and so 530m. But There is more than 1 GB left, which I am unable to find!
Tasks: 54 total, 1 running, 53 sleeping, 0 stopped, 0 zombie
Cpu(s):100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 2150400k total, 1877728k used, 272672k free, 0k buffers
Swap: 0k total, 0k used, 0k free, 0k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3271 root 18 0 1559m 530m 12m S 0 25.2 1:44.31 java
1568 mysql 15 0 270m 71m 7332 S 0 3.4 0:50.79 mysqld
(Full top output here)
Linux systems always try to use as much ram as available for various functions like caching of executables our even just page reads from disk. That's what you bought your fast RAM for after all.
You can find out more about your system by doing a
cat /proc/meminfo
More info in this helpful blog post
If you find out a lot is used in cache then you don't have to worry about the system.if individual processes warn you about memory issues then you'll have to check their settings for any memory limiting settings. Many server processes have those, like php or java based processes.
Questions of this nature are also probably more at home at https://serverfault.com/
As i see, your 'Free' command returned NO swap space
Swap: 0k total, 0k used, 0k free, 0k cached
either there is no swap partition available
this swap space is not mounted
one can manual make a swapfile
and mount this file as becoming the active swap
howto make swap
To test your real usage
reboot the machine and
test the used amount
retest after 1 hour
some processes are memory hoggs
like apache or ntop
refer:
check memory
display sorted
memory usage