I intend to use Qemu to generate a memory trace for the execution of a x86 guest operating system.
According to tcg wiki page, Qemu uses a handful of helpers to generate load/stores to the target(guest) memory.
This list of instructions is tcg_gen_qemu_ld8s/u, tcg_gen_qemu_ld16s/u, tcg_gen_qemu_ld32s/u, tcg_gen_qemu_ld64. (We have a similar set for store instructions).
I am trapping all calls to the above functions in the target-i386/translate.c file
However, I am still missing load/stores of certain instructions like
cmp ecx, [r12+0x4]
mov r10b, [r13+0x0]
mov byte [rax+0xf0000], 0x0
mov byte [rax+rdx], 0x0
Questions :
Can someone please point to other load/store points (direct or indirect) that I am missing ??
Does qemu provide a single entry point function for accesses to guest memory (like guest_read()) which can be instrumented for tracing all loads from the guest memory ???
Can somebody please point to a good documentation where I can understand how qemu maintains the state of the guest memory ??
Sorry friends for the misleading instructions in the previous mail.
cmp ecx, [r12+0x4]
mov r10b, [r13+0x0]
mov byte [rax+0xf0000], 0x0
mov byte [rax+rdx], 0x0
It seems all the above instructions are getting covered with the tcg_gen_ld/st helpers.
But now I have stumbled upon another problem :
I initially thought that all the interactions with the guest memory happen through the helper instructions in the translate.c file.
However, I found that the helper functions for some instructions like cmpxcgh8b and cmpxchg16b are actually accessing guest memory.
So, does it mean there are more than one entry points for reading guest memory.
Can some one please explain how are the ldq and stq instructions translated to access the guest memory ??
The other functions that load data are called cpu_ld*_data and cpu_st*_data, or cpu_ld*_data_ra and cpu_st*_data_ra. The _ra version have an additional argument, which is the address of the caller in the generated code. It is used to compute the address of the faulting instruction in case the load or store generates a page fault.
For example, grepping for cmpxchg8b gives
target/i386/mem_helper.c:void helper_cmpxchg8b(CPUX86State *env, target_ulong a0)
and inside that function:
uintptr_t ra = GETPC();
...
oldv = cpu_ldq_data_ra(env, a0, ra);
newv = (cmpv == oldv ? newv : oldv);
/* always do the store */
cpu_stq_data_ra(env, a0, newv, ra);
Related
I am using intel pin as my primary DBI tool.
I am interested to know how can I trace all variables allocated in a program .
suppose, we have the following snippet in C:
int *ptr_one, *ptr_two, g;
ptr_one = (int *)malloc(sizeof(int));
ptr_two = (int *)malloc(sizeof(int));
*ptr_one = 25;
*ptr_two = 24;
g = 130;
free(ptr_two);
g = 210;
*ptr_two = 50;
I want to know how can I trace specific variables / memory references in my program . for example on the above code, I like to trace the variable "g" in my program with Intel Pin, how it can be done?
for dynamically allocated variables, I'm monitoring malloc/free calls & follow their addresses, but for static ones I do not have any idea .
Another matter is, for dynamically allocated variables, I like to trace them across the whole program, suppose in the above code, I want to monitor (ptr_two) variable changes and modification during my program from start to finish .
If anyone have some idea about that, it can be nice to share it here, sample codes appreciated in Intel Pin .
thank you all .
Simply stated, you can't associate a name from your source code (be it variable or function name) with a memory location on the compiled binary: this information is (probably) lost on the final binary.
This is not true in two cases:
1) If your binary is exporting functions: in this case other binaries must have a means to call the function by name (minus some subtleties), in which case the information must be available somewhere; for example on Windows, binaries that export functions, variables or classes have an export table.
2) You have symbolic information: in your example, either for the global variable or other local variable, you have to use the symbolic information provided by the compiler.
On Linux you will need an external tool / library / program (e.g. libelf.so or libdwarf.so) to parse the symbolic information from the symbol tables (usually dynsym / symtab) if the binary is not stripped.
On windows you have to rely on the program database (*.pdb files); the format is mostly undocumented (although MS is trying to document it) and you have to use either the DbgHelp API or the DIA SDK.
As stated by the PIN user guide (emphasis is mine):
Pin provides access to function names using the symbol object (SYM).
Symbol objects only provide information about the function symbols in
the application. Information about other types of symbols (e.g. data
symbols), must be obtained independently by the tool.
If you have symbolic information you can then associate a variable name - obtained from an external tool - with an address (relative to the module base for global vars or a stack location for local ones). At runtime it is then just a matter of converting the relative address to a virtual one.
I need a function to get the Physical Sector Size for all kind of system drives, in Win7 or higher.
This is the code that I've used until today, when I found out that it's not working with my external USB HDD (exFAT file system) and with my USB MP3 Player (FAT16). In these cases the function DeviceIoControl fails and I get the exception: "System Error. Code 50. The request is not suported". But it works very well with NTFS volumes.
function GetSectorSize(Drive:Char):DWORD;
var h:THandle;
junk:DWORD;
Query:STORAGE_PROPERTY_QUERY;
Alignment:STORAGE_ACCESS_ALIGNMENT_DESCRIPTOR;
begin
result:=0;
h:=CreateFileW(PWideChar('\\.\'+UpperCase(Drive)+':'),0,FILE_SHARE_READ or FILE_SHARE_WRITE,nil,OPEN_EXISTING,0,0);
if h=INVALID_HANDLE_VALUE then RaiseLastOSError;
try
FillChar(Query,SizeOf(Query),0);
Query.PropertyId:=StorageAccessAlignmentProperty;
Query.QueryType:=PropertyStandardQuery;
if not DeviceIoControl(h,IOCTL_STORAGE_QUERY_PROPERTY,#Query,SizeOf(Query),#Alignment,SizeOf(Alignment),junk,nil) then RaiseLastOSError;
result:=Alignment.BytesPerPhysicalSector;
finally
CloseHandle(h);
end;
end;
According to MSDN:
File Buffering
Most current Windows APIs, such as IOCTL_DISK_GET_DRIVE_GEOMETRY and GetDiskFreeSpace, will return the logical sector size, but the physical sector size can be retrieved through the IOCTL_STORAGE_QUERY_PROPERTY control code, with the relevant information contained in the BytesPerPhysicalSector member in the STORAGE_ACCESS_ALIGNMENT_DESCRIPTOR structure. For an example, see the sample code at STORAGE_ACCESS_ALIGNMENT_DESCRIPTOR. Microsoft strongly recommends that developers align unbuffered I/O to the physical sector size as reported by the IOCTL_STORAGE_QUERY_PROPERTY control code to help ensure their applications are prepared for this sector size transition.
This same quote also appears in the following MSDN document:
Advanced format (4K) disk compatibility update
Which includes the following additional information:
The below list summarizes the new features delivered as part of Windows 8 and Windows Server 2012 to help improve customer and developer experience with large sector disks. More detailed description for each item follow.
...
•Provides a new API to query for physical sector size (FileFsSectorSizeInformation)
...
Here’s how you can query for the physical sector size:
Preferred method for Windows 8
With Windows 8, Microsoft has introduced a new API that enables developers to easily integrate 4K support within their apps. This new API supports even greater numbers of scenarios than the legacy method for Windows Vista and Windows 7 discussed below. This API enables these calling scenarios:
•Calling from an unprivileged app
•Calling to any valid file handle
•Calling to a file handle on a remote volume over SMB2
•Simplified programming model
The API is in the form of a new info class, FileFsSectorSizeInformation, with associated structure FILE_FS_SECTOR_SIZE_INFORMATION
FILE_FS_SECTOR_SIZE_INFORMATION structure
This information can be queried in either of the following ways:
•Call FltQueryVolumeInformation or ZwQueryVolumeInformationFile, passing FileFsSectorSizeInformation as the value of FileInformationClass and passing a caller-allocated, FILE_FS_SECTOR_SIZE_INFORMATION-structured buffer as the value of FileInformation.
•Create an IRP with major function code IRP_MJ_QUERY_VOLUME_INFORMATION.
•Call FsRtlGetSectorSizeInformation with a pointer to a FILE_FS_SECTOR_SIZE_INFORMATION-structured buffer. The FileSystemEffectivePhysicalBytesPerSectorForAtomicity member will not have a value initialized by the file system when this structure is returned from FsRtlGetSectorSizeInformation. A file system driver will typically call this function and then set its own value for FileSystemEffectivePhysicalBytesPerSectorForAtomicity.
Your principal error is that you try to get physical sector size from a volume handle rather than from that of an underlying physical device (\\.\PhysicalDriveX). Device's physical sector size doesn't depend on FS and shouldn't be confused with a logical sector size defined by FS properties.
I'm working on bare metal Programming on the Beaglebone Black with a Segger J-link under Ubuntu linux with the arm-none-eabi toolchain.
So now i get every time the error No source aviailable for "0x...."
When I pause the bone I get following in the Disassembly:
b6e93ce1: inc %ebx
b6e93ce2: fdivl -0x4922eb40(%esi)
b6e93ce8: call 0xdfa019f7
b6e93ced: add %ch,%bl
b6e93cef: mov $0x50,%dh
b6e93cf1: add $0xdd,%al
b6e93cf3: mov $0xa4,%dh
b6e93cf5: or %ebp,%ebx
b6e93cf7: mov $0xcc,%dh
b6e93cf9: inc %eax
b6e93cfa: fnsave -0x4922bf2c(%esi)
b6e93d00: add %al,(%eax)
b6e93d02: add %al,(%eax)
b6e93d04: add %al,(%eax)
b6e93d06: add %al,(%eax)
b6e93d08: js 0xb6e93d4c
b6e93d0a: fnsave -0x4922bb54(%esi)
b6e93d10: xor $0x42,%al
b6e93d12: fnsave -0x4922cfc0(%esi)
b6e93d18: mov $0x42,%ah
b6e93d1a: fnsave -0x4922cf98(%esi)
b6e93d20: cmp $0x31,%al
b6e93d22: fnsave -0x4922b92c(%esi)
b6e93d28: xorb $0xdd,(%edx)
b6e93d2b: mov $0xe0,%dh
b6e93d2d: xor %ebx,%ebp
b6e93d2f: mov $0xf0,%dh
When I start debugging the programm the J-Link restarts the bone correct but then the linux starts and not my Programm.
You may have several problems here, but the most severe is that you are using a version of gdb targeted for Intel processors: ebx, esi and friends are Intel 32 bits registers.
You have to use arm-none-eabi-gdb, not the gdb that comes with Ubuntu.
An easy way to avoid confusion is to explicitly use arm-none-eabi-gcc, arm-none-eabi-as, arm-none-eabi-ld and friends in your scripts and make files.
J-Links comes with a gdb-server software that acts as an intermediary between GDB and the JTAG hardware. Once the gdb-server is running, you can start a debugging session, assuming you are using a version of GDB debugger that matches your target processor.
This is obviously not your case, since your disassembled code looks pretty much like x86 assembly language code. This makes me think you are using a GDB executable compiled for an x86 target. You therefore really do need to get a version of GDB compiled for use with ARM targets.
You will find a version suitable to your needs in the bin directory of a Linaro toolchain, either for Windows or Linux. Its name will be arm-none-eabi-gdb.exe (Windows) or arm-none-eabi-gdb (Linux).
This should be a very simple,very quick qustion. These are the first 3 lines of a program in C I wrote:
Dump of assembler code for function main:
0x0804844d <+0>: push ebp
0x0804844e <+1>: mov ebp,esp
0x08048450 <+3>: and esp,0xfffffff0
... ... ... ... ... ... ...
What is 0x0804844d and 0x0804844e and 0x08048450? It is not affected by ASLR. Is it still a memory address, or a relative point to the file?
If you look at the Intel Developer Manual instruction-set reference you can see that 0x0804846d <+32>: eb 15 jmp 0x8048484 encodes a relative address. i.e. it's the jmp rel8 short encoding. This works even in position-independent code, i.e. code which can run when mapped / loaded at any address.
ASLR means that the address of the stack (and optionally code+data) in the executable can change every time you load the file into memory. Obviously, once the program is loaded, the addresses won't change anymore, until it is loaded again. So if you know the address at runtime, you can target it, but you can't write an exploit assuming a fixed address.
GDB is showing you addresses of code in the virtual-memory space of your process, after any ASLR. (BTW, GDB disables ASLR by default: set disable-randomization on|off to toggle.)
For executables, it's common that only the stack pointer is ASLRed, while the code is position-dependent and loaded at a fixed address, so code and static data addresses are link-time constants, so code like push OFFSET .LC0 / call puts can work, hard-coding the address of the string constant into a push imm32.
Libraries usually need to be position-independent anyway, so ASLR can load them at a randomized address.
But ASLR for executables is possible and becoming more common, either by making position-independent executables (Linux), or by having the OS fix-up every hard-coded address when it loads the executable at a different address than it was compiled for (Windows).
Addresses only have a 1:1 relation to the position within the file only in a relative sense within the same segment. i.e. the next byte of code is the next byte of the file. The headers of the executable describe which regions of the file are what (and where they should be mapped by the OS's program loader).
The meaning of the addresses shown differs in three cases:
For executable files
For DLLs (Windows) or shared objects (.so, Linux and Un*x-like)
For object files
For executables:
Executable files typically cannot be loaded to any address in memory. In Windows there is the possibility to add a "relocation table" to an executable file (required for very old Windows versions); if this is not present (typically the case when using GCC) then it is not possible to load the file to another memory location. In Linux it is never possible to load the executable to another location.
You may try something like this:
static int a;
printf("%X\n", &a);
When you execute the program 100 times you see that the address of a is always the same so no ASLR is done for the executable file itself.
The addresses dumped by objdump are absolute addresses.
For DLLs / .so files:
The addresses are relative to the base address of the DLL (under Linux) or they are absolute addresses (under Windows) that will change when the DLL is loaded into another memory area.
For object files:
When dumping an object file the addresses are relative to the currently displayed section. If there are multiple ".text" sections in a file the addresses will start at 0 for each section.
Example
function Test: Boolean;
var
a, b, c: Integer;
begin
...
end;
When a program containing such code is executed, are a, b, and c allocated each time Test is called, or are they allocated only once somewhere in the initialization phase of the execution? I ask this because such an information is not available in the debugger.
Local variables are created in the stack, after the call to the function. They are removed by the called function by default when the function returns.
Here is a more exact version.
Local variables are allocated:
Usually on the stack;
In registers if the optimizer can use it: for instance, a simple method with just a loop and a var i: integer declared as local variable will likely allocate i as a CPU register, for better speed.
How is stack allocated?
On both x86 and x64 scheme, the compiler has the same process:
It first computes all the space needed, at compile time;
It generates code to reserve this space on the stack (e.g. a MOV EBP,ESP; SUB ESP,16);
It generates code to initialize reference-counted variables allocated on the stack (e.g. string) - other kind of variables (like integer) have no default value, and can be any random content which is on the stack;
It generates an hidden try..finally block if there are some reference-counted variables;
It generates the code for the internal of the function/method;
Now here the finally part of the function/method: it generates code to free all reference-counted variables;
It generates code to release the space on the stack (e.g. an MOV ESP,EBP);
It generates code to return to the caller function.
Most of the time, a "stack frame" (pointed by register EBP) is created: it is used to access directly all the variables allocated on the stack.
There is a specific handling of the result variable of a function: sometimes, it is a CPU/FPU register, sometimes, it is a variable initialized by the caller, and passed as an additional parameter.
On x64, it is a bit more complicated, since exceptions are not handled the same, and all registers need to have some space allocated on the stack, if there is an inner call.
On Mac OS, there are some alignment issues.
All this stack allocation / initialization process is the reason why for some small functions/methods, declaring them inline will make the code faster to execute: all this stack handling is sometimes slow, if the process within the function is very simple.
For more details, see the official Delphi documentation.