This code:
local client = require("socket.http")
local resp = client.request("some_server.com/some_image123.png")
How do I get the body -- as binary data / image -- of a response? Here "resp" isn't that.
You know, had you read the manual, you'd know this:
If successful, the simple form returns the response body as a string, followed by the response status code
I'll give you the benefit of the doubt and assume you really didn't know where to find that information, but even a quick google search for "luasocket reference" would have brought that page up.
EDIT: Or maybe you already knew that, and instead didn't realize that a string in Lua is also a representation of binary data. Same as in any other language, Lua strings are just arrays of bytes under the hood, but other than, for example C, they can contain any byte, including 0x00, so you can use them to store any sort of binary data you want.
Related
When I call json.decode on financial data returned from a server, I would like to either convert my numerics to decimal (pub.dev package) (or even leave them as strings so I can manually do that later). Everything else I would like to be converted as normal
There's a reviver callback which is passed to _parseJson(), but I can't find the implementation of this external function to study.
Update:
Looks like reviver() is too late: basic conversion has already happened by here and doubles get passed in. Is there any alternative callback that can be used?
I'd use the jsontool package (disclaimer: I wrote it, so obviously that's what I'd turn to first).
It's a low-level JSON processor which allows you (and requires you) to take control of the processing.
It has an example where it parses numbers into BigInts. You could probably adapt that to use Decimal instead.
I have stored an Erlang Term into a bucket/key in Riak.
This Term is stored as what I believe is binary by Erlang.
For example this is the original erlang Term: {someAtom,[1,2,3],{"text1","text2"}}
Using a java program I am trying to obtain the data using a riak client, but the data is encoded. Riak stores data as binary as well, but the java client has a getValueAsString() method, but that does not work (obviously) since the source data was binary.
The point is, if I read this data as byte[] in java, how do I decode it to a string that reppresents this text : {someAtom,[1,2,3],{"text1","text2"}} ? To be more specific, is erlang storing as bytes that reppresent individual character's ASCII code ?
I'm almost sure you use wrong interface to access data from Java. Take a look at Riak Java client quick start. They don't use getValueAsString() and there is a code where they use RiakResponse.getRiakObjects(). Try it.
is it possible to parse an incoming google protocol buffers datagram without any .proto file? I merely now its been serialized using protocol buffers but have no idea about the IDL file.
I'm looking for a way to just iterate through any value by some sort of reflection? Is this possible?
Thank you!
protoc --decode_raw < my_file
You need to take the following things into account when inspecting the output:
None of the field names are visible, just the tag numbers.
All varint-fields are shown as integers. This is ok for most types, but sint* will appear in the "zigzagged" format.
Doubles and floats will be shown as hex.
Bytes, string fields and submessages all appear the same, i.e. just a bunch of bytes.
If you want to decode the messages programmatically, you can write your own .proto file after you have figured out what the fields mean using the above method.
I've started learning Erlang and recently wrapped up the section on bit syntax. I feel I have a firm understanding of how they can be constructed and matched but failed to come up with an example of when I would want to change the default values of "unit" inside the TypeSpecifierList.
Can anyone share a situation when this would prove useful?
Thanks for your time.
Sometimes, just for convenience: you've got a parameter from somewhere (e.g., from a file header) specifying a count of units of a given size, such as N words of 24-bit audio data, and instead of doing some multiplication, you just say:
<<Audio:N/binary-unit:24, Rest/binary>> = Data
to extract that data (as a chunk) from the rest of the file contents. After parsing the rest of the file, you could pass that chunk to some other function that splits it up into samples.
I have a series of messages that are defined by independent structs. These structs share a common header are sent between applications. I am creating a decoder that will take the raw data captures in the messages that were built using these structs and decode/parse them to some plain text.
I have over 1000 different messages that need to be decoded so I am not sure if defining all the struct formats in XML and then using XSL or some translation is the way to go or if there is a better way to do this.
There are times when I will need to decode logs containing over a million messages so performance is a concern.
Any recommendations for techniques/tools/algorithms to go about creating the decoder/parser?
struct:
struct {
dword messageid;
dword datavalue1;
dword datavalue2;
} struct1;
Example raw data:
0101010A0A0A0A0F0F0F0F
Decoded message (desired output):
message id: 0x01010101, datavalue1: 0x0A0A0A0A, datavalue2: 0x0F0F0F0F
I am using C++ to do this development.
Regarding "performance" - if you are using disk IO and possible display IO I doubt your parser/decoder will have much effect unless you use a truly horrible algorithm.
I am also unsure about what the problem is - Given the question right now - you have 3 DWORDs in a struct and you claim that there are over 1000 unique messages based on these values.
Your decoded message does not imply to me that you need any kind of parsing - just straight output seems to work (convert from bytes to ascii representation of a hex value)
If you do have a mapping from a value to a string, then a big switch statement is simple - or alternatively if you want to be able to have these added dynamically or change the display, then I would provide the key/value pairs (mapping) in a config file (text, xml, etc) and then do a lookup as the log file/raw data is read.
map is what I would use in that case.
Perhaps if you provide another specific example of the values and decoded output I can come up with a more appropriate suggestion.
If you have the message definitions already given in the syntax that you've used in your example, you should definitely not try to convert it manually into some other syntax (XML or otherwise).
Instead, you should try to write a compiler that takes these method definitions, and compiles them into a decoder function.
These days, the recommendation is to use ANTLR as the parser generator, using any of the ANTLR languages for the actual compiler (Java, Python, Ruby, C#, C++). That compiler then should output C code, which does the entire decoding and pretty-printing.
You can use yacc or antlr, add appropriate parsing rules, populate some data structure out of it(a tree may be) while parsing, then traverse the data structure and do whatever you like.
I'm going to assume that all you need to do is format the records and output them.
Use a custom code generator. The generated code will look something like this:
typedef struct { word messageid; } Header;
//repeated for each record type
typedef struct {
word messageid;
// <members here>
} Record_##;
//END
void Process(Input inp, Output out) {
char buffer[BIG_ENOUGH];
char *offset;
offset = &buffer[BIG_ENOUGH];
while(notEnd) {
if(&offset[sizeof(LargestStruct)] >= &buffer[BIG_ENOUGH])
// move remaining buffer to start and fill tail from inp
Header *hpt = (Header*)offset;
switch(hpt->messageid)
{
//repeated for each record type
case <recond ID for given type>:
{
Record_##* rpt = (Record_##*)offset;
outp.format("name1: %t, ...\n", rpt->name1, ...);
offset += sizeof(Record_##);
break;
}
//END
}
}
}
Most of that's boiler plate so writing a program to generate it shouldn't be to hard.
If you need more processing, I think this idea could be tweaked some to make that work as well.
Edit: after re-reading the question, it looks like you might have the structs defined already. In that cases you can just #include them and use them directly. However then you end up with the issue of how to parse the structs to generate the input to the formating function. Awk or sed might be handy there.