I met a strange problem whne tying to convert a Lua string to C char arry.
local str = "1234567890abcdef"
local ffi = require "ffi"
ffi.cdef[[
int printf(const char *fmt, ...);
]]
print(#str)
print(str)
local cstr = ffi.new("unsigned char[?]", #str, str)
run this code get:
[root#origin ~]# luajit test.lua
16
1234567890abcdef
Segmentation fault
I know ffi.new("unsigned char[?]", #str+1, str) will solve this, but I dont know why.
I don't think it is the \0 problem because I found some strange point.
if str is not 16 bytes, this will not happen.
if I delete ffi.cdef which I didn't use , this will not happen.
if I put ffi.cdef behind the ffi.new ,this will not happen .
[root#origin ~]# luajit test.lua
17
1234567890abcdefg
// this is the result that I only append a 'g' to `str`.
I tried on Luajit 2.0.5 and Luajit 2.1.0-beta3 with default compiler arguments.
So, is there anyone knows how this happen, thanks.
It is exactly because of string size is 17 but array allocated only for 16 bytes. https://github.com/LuaJIT/LuaJIT/blob/0c0e7b168ea147866835954267c151ef789f64fb/src/lj_cconv.c#L582 is the code that copies string to resulting array. As you can see, if target is array and its size is smaller than string length, it shrinks string; however your type is VLA (variable-length array), and for VLA size is not specified (it is 2**32-1, actually, which is way bigger than 17).
"I don't get error if" is not an argument here - you stomp memory that is used for something else. Sometimes stomping extra byte with 0 is not fatal (e.g. due to alignment this byte wasn't used anyway, or just happened to be 0 already) or doesn't result in hard crash - it doesn't make it correct.
Related
I have a file I opened as binary like this: local dem = io.open("testdem.dem", "rb")
I can read out strings from it just fine: print(dem:read(8)) -> HL2DEMO, however, afterwards there is a 4-byte little endian integer and a 4-byte float (docs for the file format don't specify endianess but since it didn't specify little like the integer I'll have to assume big).
This cannot be read out with read.
I am new to the LuaJIT FFI and am not sure how to read this out. Frankly, I find the documentation on this specific aspect of the FFI to be underwhelming, although I'm just a lua programmer and don't have much experience with C. One thing I have tried is creating a cdata, but I don't think I understand that:
local dem = io.open("testdem.dem", "rb")
print(dem:read(8))
local cd = ffi.new("int", 4)
ffi.copy(cd, dem:read(4), 4)
print(tostring(cd))
--[[Output
HL2DEMO
luajit: bad argument #1 to 'copy' (cannot convert 'int' to 'void *')
]]--
Summary:
Goal: Read integers and floats from binary data.
Expected output: A lua integer or float I can then convert to string.
string.unpack does this for Lua 5.3, but there are some alternatives for LuaJIT as well. For example, see this answer (and other answers to the same question).
I am trying to open a binary file that I have some knowledge of its internal structure, and reinterpret it correctly in Julia. Let us say that I can load it already via:
arx=open("../axonbinaryfile.abf", "r")
databin=read(arx)
close(arx)
The data is loaded as an Array of UInt8, which I guess are bytes.
In the first 4 I can perform a simple Char conversion and it works:
head=databin[1:4]
map(Char, head)
4-element Array{Char,1}:
'A'
'B'
'F'
' '
Then it happens to be that in the positions 13-16 is an integer of 32 bytes waiting to be interpreted. How should I do that?
I have tried reinterpret() and Int32 as function, but to no avail.
You can use reinterpret(Int32, databin[13:16])[1]. The last [1] is needed, because reinterpret returns you a view.
Now note that read supports type passing. So if you first read 12 bytes of data from your file e.g. like this read(arx, 12) and then run read(arx, Int32) you will get the desired number without having to do any conversions or vector allocation.
Finally observe that what conversion to Char does in your code is converting a Unicode number to a character. I am not sure if this is exactly what you want (maybe it is). For example if the first byte read in has value 200 you will get:
julia> Char(200)
'È': Unicode U+00c8 (category Lu: Letter, uppercase)
EDIT one more comment is that when you do a conversion to Int32 of 4 bytes you should be sure to check if it should be encoded as big-endian or little-endian (see ENDIAN_BOM constant and ntoh, hton, ltoh, htol functions)
Here it is. Use view to avoid copying the data.
julia> dat = UInt8[65,66,67,68,0,0,2,40];
julia> Char.(view(dat,1:4))
4-element Array{Char,1}:
'A'
'B'
'C'
'D'
julia> reinterpret(Int32, view(dat,5:8))
1-element reinterpret(Int32, view(::Array{UInt8,1}, 5:8)):
671219712
I'm want to read and filter data from a list in redis. I want to inspect the first 4 bytes (an int32) of data in a blob to compare to an int32 I will pass in as an ARG.
I have a script started, but how can I check the first 4 bytes?
local updates = redis.call('LRANGE', KEYS[1], 0, -1)
local ret = {}
for i=1,#updates do
-- read int32 header
-- if header > ARGV[1]
ret[#ret+1] = updates[i]
end
return ret
Also, I see there is a limited set of libraries: http://redis.io/commands/EVAL#available-libraries
EDIT: Some more poking around and I'm running into issues due to how LUA stores numbers - ARGV[1] is a 8 byte string, and cannot be safely be converted into a 64 bit number. I think this is due to LUA storing everything as doubles, which only have 52 bits of precision.
EDIT: I'm accepting the answer below, but changing the question to int32. The int64 part of the problem I put into another question: Comparing signed 64 bit number using 32 bit bitwise operations in Lua
The Redis Lua interpreter loads struct library, so try
if struct.unpack("I8",updates) > ARGV[1] then
Below is the C code
#include <stdio.h>
void read_input()
{
char input[512];
int c = 0;
while (read(0, input + c++,1) == 1);
}
int main ()
{
read_input();
printf("Done !\n");
return 0;
}
In the above code, there should be a buffer overflow of the array 'input'. The file we give it will have over 600 characters in it, all 2's ( ex. 2222222...) (btw, ascii of 2 is 32). However, when executing the code with the file, no segmentation fault is thrown, meaning program counter register was unchanged. Below is the screenshot of the memory of input array in gdb, highlighted is the address of the ebp (program counter) register, and its clear that it was skipped when writing:
LINK
The writing of the characters continues after the program counter, which is maybe why segmentation fault is not shown. Please explain why this is happening, and how to cause the program counter to overflow.
This is tricky! Both input[] and c are in stack, with c following the 512 bytes of input[]. Before you read the 513th byte, c=0x00000201 (513). But since input[] is over you are reading 0x32 (50) onto c that after reading is c=0x00000232 (562): in fact this is little endian and the least significative byte comes first in memory (if this was a big endian architecture it was c=0x32000201 - and it was going to segfault mostly for sure).
So you are actually jumping 562 - 513 = 49 bytes ahead. Than there is the ++ and they are 50. In fact you have exactly 50 bytes not overwritten with 0x32 (again... 0x3232ab64 is little endian. If you display memory as bytes instead of dwords you will see 0x64 0xab 0x32 0x32).
So you are writing in not assigned stack area. It doesn't segfault because it's in the process legal space (up to the imposed limit), and is not overwriting any vital information.
Nice example of how things can go horribly wrong without exploding! Is this a real life example or an assignment?
Ah yes... for the second question, try declaring c before input[], or c as static... in order not to overwrite it.
I am using memset function in C and having a problem. Here is my problem:
char* tail;
tail = //some memory address
int pbytes = 5;
When I call memset like:
**memset(tail+pbytes, 0 , 8); // It gives no error**
When I call memset like:
**memset(tail+pbytes, 0 , 9); // It goes into infinite loop**
When I call memset like:
**memset(tail+pbytes, 0 , 10); // last parameter (10 or above). It gives Segmentation fault**
What can be the reason of this? The program runs and gives output as desired but it gives segmentation fault in the end. I am using Linux 64 virtual machine.
Any help would be appreciated.
OK. Let me clarify more with what i am doing. I am making 128 bytes (0-127 in array) data. I write 0(NULL) from byte 112 to 119 (it goes well) but when I try to write 0 on 120th byte and run the program, it goes into infinite loop. If I write 1,2,4,6 at 120th byte, program runs well. If I write other numbers at 120th byte, program gives segmentation fault. Basically there is something wrong with bytes from 120 to 127.
This is nothing wrong with memset. It's something wrong with how you defined your pointer variable tail.
If you simply wrote
char tail[128];
memset(tail+5, 0, 9);
of course it would work fine. Your problem is that you're not doing anything that simple and correct; you're doing something obscure and incorrect, such as
char tail[1];
memset(tail+5, 0, 9);
or
void foo(int x) {
char *tail = &x;
memset(tail+5, 0, 9);
}
To paraphrase Charles Babbage: When you put wrong code into the machine, wrong answers come out.
The segfault is probably because you're trying to write to a virtual address that has not yet been allocated. The infinite loop might be because you're overwriting some part of the memset's return address, so that it returns to the wrong place (such as into the middle of an infinite loop) instead of returning to the place it was called from.