I have a D program with Tango and I'm trying to uncompress a gzipped string. Unfortunately I don't have A stream to it, but the compressed data is stored in a char[]. How can I uncompress it using tangos tango.io.compress.ZlibStream? I need another char[] with the uncompressed data.
I've been trying this for hours now. I'm not very familiar with tango.
Thank you
Edit: my code looks something like this:
char[] rawData; // decoded data goes here
Array array = new Array(e.value[4..(e.value.length-3)]); // e.value is a char[]
// array slice, castet to char[] is "H4sIAAAAAAAAA2NkYGBgHMWDBgMAjw2X0pABAAA="
// array.readable returns 40 (matches the above string)
// decoded string is expected to be 33 repeatitions of "AQAAAAEAAAABAAAA"
// followed by "AQAAAA=="
auto reader = new ZlibInput(array);
ubyte[1024] buffer;
reader.read(buffer); // throws Z_DATA_ERROR
well, nevermind. It appears, the guy who designed this file format compressed the data, before he encoded it with base64. I tried to decompress still base64 encoded data.
When I decoded the string with base64 and used gzip on the resulting ubyte array, it did the trick!
sorry about that.
Related
Im sending in data through post and get a png back in the response. Convert it to base64 so i dont have to save the file. The image is a qr code and it seems to suffer from some quality loss since iPhones cant seem to scan it, androids are fine.
Ive tried setting some encoding settings but that didnt do anything.
The data seems to be read correctly but im guessing that the center logo is to choppy to be read by the iphones.
Any ideas?
Public Sub updateSwish()
Dim getPrice As Integer = 100
Try
Dim data As String = "{'format':'png','size':300,'message':{'value':'test','editable':false},'amount':{'value':" + Total.ToString + ",'editable':false},'payee':{'value':'123456789','editable':false}}"
Dim json As JObject = JObject.Parse(data)
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12
Dim s As HttpWebRequest = HttpWebRequest.Create("https://mpc.getswish.net/qrg-swish/api/v1/prefilled")
Dim enc As UTF8Encoding
Dim postdata As String
Dim postdatabytes As Byte()
enc = New System.Text.UTF8Encoding()
postdata = json.ToString
postdatabytes = enc.GetBytes(postdata)
s.Method = "POST"
s.ContentType = "application/json"
s.ContentLength = postdatabytes.Length
Dim myresponse As HttpWebResponse
Dim returnval As System.Drawing.Image
Using stream = s.GetRequestStream()
stream.Write(postdatabytes, 0, postdatabytes.Length)
End Using
Using mStream As New MemoryStream()
Dim imgByte As Byte()
myresponse = CType(s.GetResponse(), HttpWebResponse)
returnval = Image.FromStream(myresponse.GetResponseStream(), True, True)
returnval.Save(mStream, returnval.RawFormat)
imgByte = mStream.ToArray()
Dim base64String As String = Convert.ToBase64String(imgByte, 0, imgByte.Length)
imgSwish.Src = "data:image/png;base64," & base64String
End Using
Catch ex As Exception
Me.Page.ClientScript.RegisterStartupScript(Me.GetType(), "ex", "alert('" + ex.Message + "');", True)
End Try
End Sub
EDIT:
Turns out that the provider had a v2 coming out due to problems from iphones. The code was fine all along, and the base64 conversion worked as it should. I tried doing the same project i PHP and got the same result.
The problem should not the base64 encoding in here. Encodings do not change the data itself, it just represents it in an other format. Like a substitution cipher called ROT13 you can change the representation of your plaintext of
EXAMPLE
to
RKNZCYR
You can play with this behaviour with a lot of online tool like this one. Therefore BASE64 must not be the problem here. (You can also test your generated image before the encode and decode, this will be problematic as well for some products to be read.)
The line where you specify the data
Dim data As String = "{'format':'png','size':300,'message':{'value':'test','editable':false},'amount':{'value':" + Total.ToString + ",'editable':false},'payee':{'value':'123456789','editable':false}}"
seems suspicious. This is quite a lot of characters to be encoded in a QR code if the size variable refers to the pixel size. The minimum size I can read with that size of data is at least 400px*400px sized. What I see from your code it should work if you can increase the size to generate somethig like this image:
TL.DR.: increase the resolution of your QR code for your large data, to allow every platform to read the QR code in a reliable manner or simply change the generator.
I have the following code:
buff=esp.flash_read(esp.flash_user_start(),50)
print(buff)
I get the following output from print:
bytearray(b'{"ssid": "mySSID", "password": "myPASSWD"}\xff\xff\xff\xff\xff\xff')
What I want to do is get the json in buff. What is the correct "Python-way" to do that?
buff is a Python bytes object, as shown by the print output beginning with b'. To convert this into a string you need to decode it.
In standard Python you could use
buff.decode(errors='ignore')
Note that without specifying errors=ignore you would get a UnicodeDecodeError because the \xff bytes aren't valid in the default encoding, which is UTF-8; presumably they're padding and you want to ignore them.
If that works on the ESP8266, great! However this from the MicroPython docs suggests the keyword syntax might not be implemented - I don't have an ESP8266 to test it. If not then you may need to remove the padding characters yourself:
textLength = find(buff, b'\xff')
text = buff[0:textLength].decode()
or simply:
text = buff[0:buff.find(b'\xff')].decode()
If decode isn't implemented either, which it isn't in the online MicroPython interpreter, you can use str:
text = str(buff[0:find(buff, b'\xff')], 'utf-8')
Here you have to specify explicitly that you're decoding from UTF-8 (or whatever encoding you specify).
However if what you're really after is the values encoded in the JSON, you should be able to use the json module to retrieve them into a dict:
import json
j = json.loads(buff[0:buff.find(b'\xff')])
ssid = j['ssid']
password = j['password']
I'm using MongoDB for my Go and Rails projects (using same database) and i have a bson.Binary data in my document (contain Base64 encoded publicKey)
type Device struct {
Id string `json:"id" form:"id" bson:"_id"`
PublicKey bson.Binary `json:"pub_key" form:"pub_key" bson:"public_key"`
Token string `json:"token" form:"token" bson:"token"`
CreatedAt time.Time `json:"created_at" bson:"created_at"`
UpdatedAt time.Time `json:"updated_at" bson:"updated_at"`
}
but when i retrieve it in my Go projects and the data is somehow corrupted (4 characters are missing)
device := models.Device{}
err = db.C(models.CollectionDevice).Find(bson.M{"_id": deviceId}).One(&device)
publicKey := device.PublicKey
publicKeyBase64 := base64.StdEncoding.EncodeToString(publicKey.Data)
fmt.Println("publicKey BinaryData: ", publicKey.Data)
t
"pub_key": {
"Kind": 0,
"Data": "Axj5A9BgkKJohGh/0BAQEFAAMBjYAAMYmCAYGACqW0smeNiaIgRgXJv7dLZ0n5gTeXxiI9q6h8EdbiPDLiFsv7Jgatwxm+OpucrUBp4sZp/othrSQWnJQR5vpnCZYZnNJUdjTAYw+PyjQ2qq9YEeLHMnMzhYgaq2ata+CSsCjalkOyzLt/rDEBn5WHaCfYm0Vm+QbbEWVLltUqcjvScOyAwEAAQn"
}
(220 characters length in Data)
the original Base64 encoded string is
Axj5A9BgkKJohGh/0BAQEFAAMBjYAAMYmCAYGACqW0smeNiaIgRgXJv7dLZ0\n5gTeXxiI9q6h8EdbiPDLiFsv7Jgatwxm+OpucrUBp4sZp/othrSQWnJQR5vp\nCZYZnNJUdjTAYw+PyjQ2qq9YEeLHMnMzhYgaq2ata+CSsCjalkOyzLt/rDEB\n5WHaCfYm0Vm+QbbEWVLltUqcjvScOyAwEAAQ\n
with 224 characters
i use this code in my Go projects:
fmt.Println("publicKey: ", publicKey.Data)
//publicKey is bson.Binary data type
i have tried to retrieve the data from my rails projects (same database) and it retrieve correctly
public_key.as_json["$binary"]
the result is exactly right (224 characters) :
"Axj5A9BgkKJohGh/0BAQEFAAMBjYAAMYmCAYGACqW0smeNiaIgRgXJv7dLZ0\nn5gTeXxiI9q6h8EdbiPDLiFsv7Jgatwxm+OpucrUBp4sZp/othrSQWnJQR5v\npnCZYZnNJUdjTAYw+PyjQ2qq9YEeLHMnMzhYgaq2ata+CSsCjalkOyzLt/rD\nEBn5WHaCfYm0Vm+QbbEWVLltUqcjvScOyAwEAAQn\n"
As you can see, there is still a \n at the last of string. Any one know why 4 characters are missing in Go ?
===== Additional information =====
When i store it in my mongoDB from my Rails API, i receive public_key params as Base64 format, but i decode it to binary and then i stored it with this code
def create
params = device_params
public_key = Base64.decode64 device_params[:public_key]
params[:public_key] = BSON::Binary.new(public_key, :generic)
device = Device.find_or_create_by(id: device_params[:id])
render_success device.update_attributes(params), device
end
Your original data contains exactly 4 newline characters \n:
Axj5A9BgkKJohGh/0BAQEFAAMBjYAAMYmCAYGACqW0smeNiaIgRgXJv7dLZ0\n
5gTeXxiI9q6h8EdbiPDLiFsv7Jgatwxm+OpucrUBp4sZp/othrSQWnJQR5vp\n
CZYZnNJUdjTAYw+PyjQ2qq9YEeLHMnMzhYgaq2ata+CSsCjalkOyzLt/rDEB\n
5WHaCfYm0Vm+QbbEWVLltUqcjvScOyAwEAAQ\n
These newline characters are not part of the Base64 encoded data, they are just to format the input.
Problem arises likely when you try to store this as a raw string literal, because this is an interpreted string literal, the \n sequences are to be discarded / removed.
Since the backslash is an invalid symbol for "normal" Base64, they are removed, though the subsequent n is not (because n is valid in Base64). As a result, you get corrupted Base64 data.
You should either completely remove the line endings and then you get a valid Base64 which you may store either as a string or binary; or store it as a general string which will preserve the line endings (which have to be dealt with during base64 decoding).
I'm working in a java application called Mirth where I need to read a saved word document that is saved in a database table in a Microsoft word binary data format. Currently I can retrieve data from column in my java application but I need to convert this to readable text or XML or HTML format.
Looking online there is a java library call Aspose.words but I can't find any methods that would read in this binary data and convert it to something readable. Has anyone use Aspose.words before to do a task like this or does anyone have an alternative solution
Load document from Database
You can load the Word document using ByteArrayInputStream, if it's in a database table. Please refer to http://www.aspose.com/docs/display/wordsjava/How+to++Load+and+Save+a+Document+to+Database for an article that explains saving and reading a Word document to/from database. I have copied the relevant code from there.
public static Document readFromDatabase(String fileName) throws Exception
{
// Create the SQL command.
String commandString = "SELECT * FROM Documents WHERE FileName='" + fileName + "'";
// Retrieve the results from the database.
ResultSet resultSet = executeQuery(commandString);
// Check there was a matching record found from the database and throw an exception if no record was found.
if(!resultSet.isBeforeFirst())
throw new IllegalArgumentException(MessageFormat.format("Could not find any record matching the document \"{0}\" in the database.", fileName));
// Move to the first record.
resultSet.next();
// The document is stored in byte form in the FileContent column.
// Retrieve these bytes of the first matching record to a new buffer.
byte[] buffer = resultSet.getBytes("FileContent");
// Wrap the bytes from the buffer into a new ByteArrayInputStream object.
ByteArrayInputStream newStream = new ByteArrayInputStream(buffer);
// Read the document from the input stream.
Document doc = new Document(newStream);
// Return the retrieved document.
return doc;
}
Read text
Once the file is loaded, you can read it's paragraphs, tables, images etc using DOM, see the related documentation at http://www.aspose.com/docs/display/wordsjava/Programming+with+Documents.
But, if you just want to get all the text from a document, you can do it easily by calling toString() method as below
System.out.println(doc.toString(SaveFormat.TEXT));
I work with Aspose as Developer Evangelist.
I'm have domain class with property that represents files uploaded on my GSP. I've defined that file as byte array (byte [] file). When some specific action happens I'm sending mail with attachments from. This is part of my SendMail service:
int i = 1;
[requestInstance.picture1, requestInstance.picture2, requestInstance.picture3].each(){
if(it.length != 0){
DataSource image = new ByteArrayDataSource(it, "image/jpeg");
helper.addAttachment("image" + i + ".jpg", image);
i++;
}
}
This works fine with image files. But now I want to be able to work with all file types and I'm wondering how to implement this. Also, I want to save real file name in database. All help is welcomed.
You can see where the file name and MIME type are specified in your code. It should be straightforward to save and restore that information from your database along with the attachment data.
If you're trying to figure out from the byte array of data what the MIME type is and what a good filename would be, that's a harder problem. Try this.