I'm trying to parse CBOR stream using tinyCBOR. Goal is to write a generic parsing code for "map type"(because i don't know how many keys are there and which are they, in cbor stream)but not for a json, I just want to get values using "key",but for getting values from key i have to know the key.
Im simply able to parse the value by passing "key" in function
cbor_value_map_find_value(&main_value,"Age",&map_value);
but few things still not clear to me.
What sequence to follow, for getting key and values from CBOR stream?
For eg. following is my data in map format -
{"Roll_number": 7, "Age": 24, "Name": "USER"}
here is this binary format from cbor.me link -
A3 # map(3)
6B # text(11)
526F6C6C5F6E756D626572 # "Roll_number"
07 # unsigned(7)
63 # text(3)
416765 # "Age"
18 18 # unsigned(24)
64 # text(4)
4E616D65 # "Name"
64 # text(4)
55534552 # "USER"
1.How to get key from stream. like - Roll_number or AGE from stream?(sequentially getting key and values also fine).
2.After getting Roll_number value, how can i jump to next element ("Age") for getting "key" and "value".
3.How to identify that i'm reached at the "end of stream" and now there is no data ??
Any snippet code, that how to parse and which sequence of function need to follow.
Any help is appreciate.
Thanks!!!
The example code is pretty helpful for understanding the API. To iterate over the keys and values of a map, you call cbor_value_enter_container, then cbor_value_advance until cbor_value_at_end returns true (as long as there are no nested maps or arrays you want to look inside). For example:
cbor_parser_init(input, sizeof(input), 0, &parser, &it);
if (!cbor_value_is_map(&it)) {
return 1;
}
err = cbor_value_enter_container(&it, &map);
if (err) return 1;
while (!cbor_value_at_end(&map)) {
// get the key. Remember, keys don't have to be strings.
if (!cbor_value_is_text_string(&map)) {
return 1;
}
char *buf;
size_t n;
// Note: this also advances to the value
err = cbor_value_dup_text_string(&map, &buf, &n, &map);
if (err) return 1;
printf("Key: '%*s'\n", (int)n-1, buf);
if (strncmp(buf, "Age", n-1) == 0) {
if (cbor_value_is_integer(&map)) {
// Found the expected key and value type
err = cbor_value_get_int(&map, &val);
if (err) return 1;
printf("age: %d\n", val);
}
// note: can't break here, have to keep going until the end if you want
// `it` to still be valid.
}
free(buf);
err = cbor_value_advance(&map);
if (err) return 1;
}
err = cbor_value_leave_container(&it, &map);
if (err) return 1;
Related
I have a script which calls functions from this script:
https://www.baselinesinc.com/dxl-repository/?filerepoaction=showpost&filepost_id=13
which is distributed under the LGPL. I have found several references to this script online, indicating that it works. For my own purposes, I removed the part of the code which executes on load (some 20 lines at the bottom of the file) and converted the file to an .inc, to serve as a library. I haven't done any other changes.
My own script has worked for a few months at this point. But now, all of a sudden, when I try to run it, I get these errors:
-E- DXL: <DOORSImporter\Import_From_Excel.inc:291> incorrect use of identifier (rtfValue)
-E- DXL: <DOORSImporter\Import_From_Excel.inc:292> incorrect use of identifier (nonRTFValue)
-E- DXL: <DOORSImporter\Import_From_Excel.inc:346> incorrect use of identifier (temp)
-E- DXL: <DOORSImporter\Import_From_Excel.inc:464> incorrect use of identifier (columnHeading)
-E- DXL: <DOORSImporter\Import_From_Excel.inc:1127> incorrect use of identifier (cellValue)
-I- DXL: All done. Errors reported: 5. Warnings reported: 0.
Below is a snippet of code which shows this error. I marked lines 291 and 292
bool richTextTruncated(int rowNumber, int columnNumber) {
Buffer rtfValue = create
Buffer nonRTFValue = create
rtfValue = ""
nonRTFValue = ""
rtfValue = copyRichTextFromCell(""intToCol(columnNumber) "" rowNumber"") // get the rich text
rtfValue = trimWhitespace(stripRichText(tempStringOf(rtfValue))) // strip off RTF formatting. <----- 291
nonRTFValue = trimWhitespace(getCellValue(rowNumber, columnNumber)) //<----- 292
if(tempStringOf(rtfValue) != tempStringOf(nonRTFValue)) { // if the two strings aren't equal
delete(rtfValue)
delete(nonRTFValue)
return true // return that text was truncated
}
delete(rtfValue)
delete(nonRTFValue)
return false
}
The function trimWhitespace itself is defined as
string trimWhitespace(string s) so it's OK to assign its output to a buffer.
So this is all very confusing because, like I said, this has worked and the code has not been changed (last check in of the file DOORSImporter\Import_From_Excel.inc is from last November, whereas I have used the script even in January.
Our IT department might have updated DOORS recently, I don't know. I checked the latest version of the dxl reference manual for any recent changes which might have caused this, but I couldn't find any change to the way buffers are handled.
To be fair, I have found working with buffers quite confusing. In my own code, I ended up discarding them altogether. However, the script Import_From_Excel has worked unchanged for a long time now.
Any support with this will be greatly appreciated.
********** Edit 27.02.2020 **************
The function copyRichTextFromCell is defined as below. Its sourced is from the link below.
https://www.baselinesinc.com/dxl-repository/?filerepoaction=showpost&filepost_id=10
string copyRichTextFromCell(string loc) {
if(!getCell(loc)) {
return("error");
}
if(!checkResult(oleMethod(objCell, cMethodSelect))) { // selects the cell
return("error");
}
if(!checkResult(oleMethod(objCell, cMethodCopy))) { // copies the contents of the cell to the clipboard
return("error");
}
return stringOf(richClip)
}
********** Edit 28.02.2020 **************
The function trimWhitespace is as below.
string trimWhitespace(string s) {
int first = 0;
int last = length(s) - 1;
while(last > 0 && (isspace(s[last]) || s[last] == '\n' || s[last] == '\r' || s[last] == '\t')) {
last--;
}
while(isspace(s[first]) && first < last) {
first++;
}
if(s[first:last] == " ") {
return("");
}
return(s[first:last]);
}
While playing around, I discovered that if I modify as below, one of the errors disappears. It is beyond me as to why that might happen, because the code is equivalent, as far as I can tell...
bool richTextTruncated(int rowNumber, int columnNumber) {
Buffer rtfValue = create
Buffer nonRTFValue = create
string fooString
rtfValue = ""
nonRTFValue = ""
rtfValue = copyRichTextFromCell(""intToCol(columnNumber) "" rowNumber"") // get the rich text
fooString = trimWhitespace(stripRichText(tempStringOf(rtfValue)))
rtfValue = fooString // strip off RTF formatting.
fooString = trimWhitespace(getCellValue(rowNumber, columnNumber))
nonRTFValue = fooString
if(tempStringOf(rtfValue) != tempStringOf(nonRTFValue)) { // if the two strings aren't equal
delete(rtfValue)
delete(nonRTFValue)
return true // return that text was truncated
}
delete(rtfValue)
delete(nonRTFValue)
return false
}
So the lines with fooString do not throw an error anymore. In the initial version, these lines were just e.g. rtfValue = trimWhitespace(stripRichText(tempStringOf(rtfValue))) and this would throw an error.
I am facing the problem in generic extract object fixture .
My service layer provides me Json something like this. It is an array/list of values
[{"id":1,"description":"Anti-takeover Provision"},
{"id":2,"description":"Capital Structure"},
{"id":3,"description":"Director"},
{"id":4,"description":"Equity Plan"},
{"id":5,"description":"Executive Compensation"},
{"id":6,"description":"General Governance"},
{"id":7,"description":"Merger or Acquisition"},
{"id":12,"description":"Other"},
{"id":8,"description":"Proxy Contest"},
{"id":9,"description":"Reincorporation"},
{"id":10,"description":"Shareholder Proposal"},
{"id":11,"description":"Shareholder Rights"}]
But my database json
{"document":[{"VALUE":"Anti-takeover Provision","TOPIC_ID":1},
{"VALUE":"Capital Structure","TOPIC_ID":2},
{"VALUE":"Director","TOPIC_ID":3},
{"VALUE":"Equity Plan","TOPIC_ID":4},
{"VALUE":"Executive Compensation","TOPIC_ID":5},
{"VALUE":"General Governance","TOPIC_ID":6},
{"VALUE":"Merger or Acquisition","TOPIC_ID":7},
{"VALUE":"Other","TOPIC_ID":12},
{"VALUE":"Proxy Contest","TOPIC_ID":8},
{"VALUE":"Reincorporation","TOPIC_ID":9},
{"VALUE":"Shareholder Proposal","TOPIC_ID":10},
{"VALUE":"Shareholder Rights","TOPIC_ID":11}]}
How do i compare these two values easily?
Check that the lengths are the same
Sort the two lists by their corresponding ID fields
Check that the values match
// stop as soon as you find a difference
for(i=0; i < listLength; i++) {
if (listA[i].id != listB[i].TOPIC_ID) { return false; }
if (listA[i].description != listB[i].VALUE) { return false; }
}
// if you get here, they must be the same
return true;
I'm trying to modify an Order, but I always get Error #1.
From my research, I have discovered that error 1 means I have input parameter in a wrong way. How can I fix my OrderModify() function?
stoploss = NormalizeDouble(Ask - Point * TrailingStop,Digits);
int ticket;
takeprofit = NormalizeDouble(Ask + Point * TrailingStopTP,Digits);
double minstoplevel = MarketInfo( Symbol(), MODE_STOPLEVEL );
if(stoploss > NormalizeDouble(Ask - Point*minstoplevel,Digits)) {
stoploss = NormalizeDouble(Ask - Point*minstoplevel,Digits);
}
if(takeprofit < NormalizeDouble( Ask + Point*minstoplevel2, Digits )) {
takeprofit = NormalizeDouble( Ask + Point*minstoplevel2, Digits );
}
if(AccountFreeMarginCheck(Symbol(),OP_SELL,lotsize)>0) {
ticket=OrderSend(Symbol(),OP_BUY,lotsize,Ask, 0, 0.0, 0.0, "comment", MagicNumber, 0, Lime);
if(ticket<0) {
Print("Order send failed with error #",GetLastError());
} else {
Print("Order send sucesso!! Ticket#", ticket);
res=OrderModify(ticket,OrderOpenPrice(),stoploss,takeprofit,0,Blue);
if(res==false) {
Print("Error modifying order!, error#",GetLastError());
} else {
Print("Order modified successfully, res#", res);
}
}
} else {
Print("Sem dinheiro na conta D=");
}
}
Not exactly "wrong", OrderModify() legally sets _LastError == 1
There might be a bit surprise, but OrderModify() has an obligation to signal _LastError == 1 in case, the call was both syntactically and semantically correct, however, the values supplied for modification(s) were actually the very same, as the identified ticket# has already had in database.
This means, there was nothing to modify, as all the attributes already had the "quasi-new" target value(s).
One may pre-check all fields for a potential identity, which might allow our code to skip the OrderModify() call in this very case of an identity-of-{ current | target } values.
ERR_NO_RESULT == 1 // No error returned, but the result is unknown
GetLastError() - returns a last generated error-code. The same value is available via a system variable named _LastError. It's value can be reset before a critical activity to zero by calling ResetLastError().
Error codes are defined in stderror.mqh.
To print the error description you can use the ErrorDescription() function, defined in stdlib.mqh file
#include <stderror.mqh>
#include <stdlib.mqh>
The problem is that even though the entry price, stoploss, and take profit parameters to the OrderModify() call appear to be the same, they likely differ by a fraction of a unit ( smaller than "Digits" ).
To fix this,simply normalize the parameters to make sure they are a maximum of Digits decimal places long.
double entryPrice = NormalizeDouble( entryPrice, Digits );
double stoploss = NormalizeDouble( stoploss, Digits );
double target = NormalizeDouble( target, Digits );
Then pass them into the OrderModify() call.
This function returns a ctypes.unsigned_char.array() and I do read string on it. It is getting the titles of windows. The problem is sometimes it throws TypeError.
try {
console.error('straight readString on XWindowGetProperty data:', rez_GP.data.readString());
} catch (ex) {
console.error('ex on straight readString:', ex);
}
Please notice the rez_GP.data.readString()
For example this instance: TypeError: malformed UTF-8 character sequence at offset 48. In this situation the window title is Editing js-macosx/bootstrap.js at base-template · Noitidart/js-macosx - Mozilla Firefox The 48th offset is that dot chracter you see, it's chracter code is 183. How to do readString() on this buffer without getting this error?
Thanks
readString expects a utf-8 encoded string. This is true for strings returned by _NET_WM_NAME, but not for WM_NAME.
I found a way to read the string propertly even ifs not utf-8, but im not sure if its he best way or the recommended way. This works though, i have to cast it to unsigned_char (must be this, so not char or jschar) then do fromCharCode:
function readAsChar8ThenAsChar16(stringPtr, known_len, jschar) {
// when reading as jschar it assumes max length of 500
// stringPtr is either char or jschar, if you know its jschar for sure, pass 2nd arg as true
// if known_len is passed, then assumption is not made, at the known_len position in array we will see a null char
// i tried getting known_len from stringPtr but its not possible, it has be known, i tried this:
//"stringPtr.contents.toString()" "95"
//"stringPtr.toString()" "ctypes.unsigned_char.ptr(ctypes.UInt64("0x7f73d5c87650"))"
// so as we see neither of these is 77, this is for the example of "_scratchpad/EnTeHandle.js at master · Noitidart/_scratchpad - Mozilla Firefox"
// tries to do read string on stringPtr, if it fails then it falls to read as jschar
var readJSCharString = function() {
var assumption_max_len = known_len ? known_len : 500;
var ptrAsArr = ctypes.cast(stringPtr, ctypes.unsigned_char.array(assumption_max_len).ptr).contents; // MUST cast to unsigned char (not ctypes.jschar, or ctypes.char) as otherwise i dont get foreign characters, as they are got as negative values, and i should read till i find a 0 which is null terminator which will have unsigned_char code of 0 // can test this by reading a string like this: "_scratchpad/EnTeHandle.js at master · Noitidart/_scratchpad - Mozilla Firefox" at js array position 36 (so 37 if count from 1), we see 183, and at 77 we see char code of 0 IF casted to unsigned_char, if casted to char we see -73 at pos 36 but pos 77 still 0, if casted to jschar we see chineese characters in all spots expect spaces even null terminator is a chineese character
console.info('ptrAsArr.length:', ptrAsArr.length);
//console.log('debug-msg :: dataCasted:', dataCasted, uneval(dataCasted), dataCasted.toString());
var charCode = [];
var fromCharCode = []
for (var i=0; i<ptrAsArr.length; i++) { //if known_len is correct, then will not hit null terminator so like in example of "_scratchpad/EnTeHandle.js at master · Noitidart/_scratchpad - Mozilla Firefox" if you pass length of 77, then null term will not get hit by this loop as null term is at pos 77 and we go till `< known_len`
var thisUnsignedCharCode = ptrAsArr.addressOfElement(i).contents;
if (thisUnsignedCharCode == 0) {
// reached null terminator, break
console.log('reached null terminator, at pos: ', i);
break;
}
charCode.push(thisUnsignedCharCode);
fromCharCode.push(String.fromCharCode(thisUnsignedCharCode));
}
console.info('charCode:', charCode);
console.info('fromCharCode:', fromCharCode);
var char16_val = fromCharCode.join('');
console.info('char16_val:', char16_val);
return char16_val;
}
if (!jschar) {
try {
var char8_val = stringPtr.readString();
console.info('stringPtr.readString():', char8_val);
return char8_val;
} catch (ex if ex.message.indexOf('malformed UTF-8 character sequence at offset ') == 0) {
console.warn('ex of offset utf8 read error when trying to do readString so using alternative method, ex:', ex);
return readJSCharString();
}
} else {
return readJSCharString();
}
}
I have successfully encrypted data in BlackBerry in AES format. In order to verify my result, I am trying to implement decryption in BlackBerry using the following method:
private static byte[] decrypt( byte[] keyData, byte[] ciphertext )throws CryptoException, IOException
{
// First, create the AESKey again.
AESKey key = new AESKey( keyData );
// Now, create the decryptor engine.
AESDecryptorEngine engine = new AESDecryptorEngine( key );
// Since we cannot guarantee that the data will be of an equal block length
// we want to use a padding engine (PKCS5 in this case).
PKCS5UnformatterEngine uengine = new PKCS5UnformatterEngine( engine );
// Create the BlockDecryptor to hide the decryption details away.
ByteArrayInputStream input = new ByteArrayInputStream( ciphertext );
BlockDecryptor decryptor = new BlockDecryptor( uengine, input );
// Now, read in the data. Remember that the last 20 bytes represent
// the SHA1 hash of the decrypted data.
byte[] temp = new byte[ 100 ];
DataBuffer buffer = new DataBuffer();
for( ;; ) {
int bytesRead = decryptor.read( temp );
buffer.write( temp, 0, bytesRead );
if( bytesRead < 100 ) {
// We ran out of data.
break;
}
}
byte[] plaintextAndHash = buffer.getArray();
int plaintextLength = plaintextAndHash.length - SHA1Digest.DIGEST_LENGTH;
byte[] plaintext = new byte[ plaintextLength ];
byte[] hash = new byte[ SHA1Digest.DIGEST_LENGTH ];
System.arraycopy( plaintextAndHash, 0, plaintext, 0, plaintextLength );
System.arraycopy( plaintextAndHash, plaintextLength, hash, 0,
SHA1Digest.DIGEST_LENGTH );
// Now, hash the plaintext and compare against the hash
// that we found in the decrypted data.
SHA1Digest digest = new SHA1Digest();
digest.update( plaintext );
byte[] hash2 = digest.getDigest();
if( !Arrays.equals( hash, hash2 )) {
throw new RuntimeException();
}
return plaintext;
}
I get an exception thrown "BadPaddingException" at the following line
int bytesRead = decryptor.read( temp );
Can anybody please help.
I think the problem might be in this block:
for( ;; ) {
int bytesRead = decryptor.read( temp );
buffer.write( temp, 0, bytesRead );
if( bytesRead < 100 ) {
// We ran out of data.
break;
}
}
When read returns -1, you are also writing it to the buffer. And the exit condition is also wrong. Compare that to the block in CryptoDemo sample project:
for( ;; ) {
int bytesRead = decryptor.read( temp );
if( bytesRead <= 0 )
{
// We have run out of information to read, bail out of loop
break;
}
db.write(temp, 0, bytesRead);
}
Also there are a few points you should be careful about, even if they are not causing the error:
AESDecryptorEngine engine = new AESDecryptorEngine( key );
If you read the docs for this constructor, it says:
"Creates an instance of the AESEncryptorEngine class given the AES key
with a default block length of 16 bytes."
But in the previous line, when you create the key, you are doing this:
AESKey key = new AESKey( keyData );
Which according to the docs, it "Creates the longest key possible from existing data.", BUT only "the first 128 bits of the array are used". So it does not matter what length your keyData has, you will always be using a 128 bit key length, which is the shortest of the 3 available sizes (128, 192, 256).
Instead, you could explicitly select the algorithm block key length. For instance, to use AES-256:
AESKey key = new AESKey(keyData, 0, 256); //key length in BITS
AESDecryptorEngine engine = new AESDecryptorEngine(key, 32); //key lenth IN BYTES
Finally, even if you get this working, you should be aware that directly deriving the key from the password (which might be of an arbitrary size) is not secure. You could use PKCS5KDF2PseudoRandomSource to derive an stronger key from the key material (password), instead of just using PKCS5 for padding.
Your encrypted data should be correctly padded to the block size (16 bytes).
Try to decrypt the data without padding, and see if tail bytes correspond to PKCS#5 padding (for instance, if it was needed 5 bytes of padding, it should be appended with 0x05 0x05 0x05 0x05 0x05 bytes).
The problem is that any data with the correct block size will decrypt. The issue with that is that it will likely decrypt to random looking garbage. Random looking garbage is not often compatible with the PKCS#7 padding scheme, hence the exception.
I say problem because this exception may be thrown if the key data is invalid, if the wrong padding or block mode was used or simply if the input data was garbled during the process. The best way to debug this is to make 100% sure that the algorithms match, and that the binary input parameters (including default ones by the API) match precisely on both sides.