Firstly, what I want to do is to intercept an arbitrary standard C function (like fopen, read, write, malloc, ...) of an iOS application.
I have a libtest.dylib with this code:
typedef struct interpose_s {
void *new_func;
void *orig_func;
} interpose_t;
FILE *vg_fopen(const char * __restrict, const char * __restrict);
static const interpose_t interposing_functions[] \
__attribute__ ((section("__DATA, __interpose"))) = {
{ (void *)vg_fopen, (void *)fopen },
};
FILE *vg_fopen(const char * __restrict path, const char * __restrict mode) {
printf("vg_fopen");
return fopen(path, mode);
}
After compiled the dylib, I go to the binary of the host iOS app and add an LC_LOAD_DYLIB to the end of the LC_LOAD_COMMANDS list and point it to #executable_path/libtest.dylib
What I expect is that it will override the implementation of fopen, and print "vg_fopen" whenever fopen is called. However, I do not get it, so the interposition might have been failed.
I'd like to know what might be the reason. This is for in-house development for learning purpose only, so please don't mention about the impact or warn me about inappropriate use.
Thanks in advance.
From the dyld source:
// link any inserted libraries
// do this after linking main executable so that any dylibs pulled in by inserted
// dylibs (e.g. libSystem) will not be in front of dylibs the program uses
if ( sInsertedDylibCount > 0 ) {
for(unsigned int i=0; i < sInsertedDylibCount; ++i) {
ImageLoader* image = sAllImages[i+1];
link(image, sEnv.DYLD_BIND_AT_LAUNCH, ImageLoader::RPathChain(NULL, NULL));
// only INSERTED libraries can interpose
image->registerInterposing();
}
}
So no, only libraries inserted via DYLD_INSERT_LIBRARIES have their interposing applied.
Related
I have a GUI written in R that utilizes Tcl/TK package as well a C .dll that also uses Tcl library. I have done some research on this issue, and it seems to be memory related. I am an inexperienced programmer, so I am not sure where I should be looking for this memory issue. Each call of malloc() has a matching free(), and same with the analogous Tcl_Alloc() and Tcl_Free(). This error is very hard to reproduce as well, thus I am afraid I cannot provide a reproducible example as it is seemingly random in nature. One pattern is however that it seems to only happen upon closure of the program, though this is very inconsistent.
By making this post, I am hoping to gain a logical process that one should take in an attempt to debug this problem in a general context under Tcl/Tk - C - R applications. I am not looking for a solution specific to my code, but rather what an individual should think about when encountering this problem.
The message comes from the function Ptr2Block() in tclThreadAlloc.c (or there's something else about which produces the same error message; possible but unlikely) which is Tcl's thread-specific memory allocator (which is used widely inside Tcl to reduce the number of times global locks are hit). Specifically, it's this bit:
if (blockPtr->magicNum1 != MAGIC || blockPtr->magicNum2 != MAGIC) {
Tcl_Panic("alloc: invalid block: %p: %x %x",
blockPtr, blockPtr->magicNum1, blockPtr->magicNum2);
}
The problem? Those zeroes should be MAGIC (which is equal to 0xEF). This indicates that something has overwritten the memory block's metadata — which also should include the size of the block, but that is now likely hot garbage — and program memory integrity can no longer be trusted. Alas, at this point we're now dealing with a program in a broken state where the breakage happened some time previously; the place where the panic happened is merely where detection of the bug happened, not the actual location of the bug.
Debugging further is usually done by building a version of everything with fancy memory allocators turned off (in Tcl's code, this is done by defining the PURIFY symbol when building) and then running the resulting code — which hopefully still has the bug — with a tool like electricfence or purify (hence the special symbol name) to see what sort of out-of-bounds errors are found; they're very good at hunting down this sort of issue.
I would advise you to start by having a closer look to the sizeof() values provided to your Tcl_Alloc() calls in this C .dll.
I'm writing myself a Tcl binding for a C library and I faced recently exactly the same problem and therefore I'm assuming you may have the same error than me in your code.
Here below a minimal example that reproduces the problem:
#include <tcl.h>
#include <stdlib.h> // malloc
static unsigned int dataCtr;
struct tDataWrapper {
const char *str; // Tcl_GetCommandName(interp, cmd)
unsigned int n; // dataCtr value
void *data; // pointer to wrapped object
};
static void wrapDelCmd(ClientData clientData)
{
struct tDataWrapper *wrap = (struct tDataWrapper *) clientData;
if (wrap != NULL) {
/* with false sizeof value provided while creating the wrapper
* (see above), this data pointer would overwrite the
* overhead section of the allocated tcl memory block
* from what I understood and this is what can be causing
* the panic with message like following one when the
* memory is freed with ckfree (here after calling unload)
* alloc: invalid block: 0000018F2624E760: 0 0 */
printf("DEBUG: #%s(%s) &wrap->data #%p\n",
__func__, wrap->str, &wrap->data);
if (wrap->data != NULL) {
// call your wrapped API to deinstantiate the object
}
ckfree(wrap);
}
}
static int wrapCmd(ClientData clientData, Tcl_Interp *interp,
int objc, Tcl_Obj *const objv[])
{
struct tDataWrapper *wrap = (struct tDataWrapper *) clientData;
if (wrap == NULL)
return TCL_ERROR;
else if (wrap->data != NULL) {
// call your wrapped API to do something with instantiated object
return TCL_OK;
} else {
Tcl_Obj *obj = Tcl_ObjPrintf("wrap: {str=\"%s\", n=%u, data=%llx}",
wrap->str, wrap->n, (unsigned long long) wrap->data);
if (obj != NULL) {
Tcl_SetObjResult(interp, obj);
return TCL_OK;
} else
return TCL_ERROR;
}
}
static int newCmd(ClientData clientData, Tcl_Interp *interp,
int objc, Tcl_Obj *const objv[])
{
struct tDataWrapper *wrap;
Tcl_Obj *obj;
Tcl_Command cmd;
// 3) this is correct
// if ((wrap = attemptckalloc(sizeof(struct tDataWrapper))) == NULL)
// 2) still incorrect but GCC gives more warning regarding the inconsistent pointer handling
// if ((wrap = malloc(sizeof(struct tDataWrapper *))) == NULL)
// 1) this is incorrect
if ((wrap = attemptckalloc(sizeof(struct tDataWrapper *))) == NULL)
Tcl_Panic("%s:%u: attemptckalloc failed\n", __func__, __LINE__);
else if ((obj = Tcl_ObjPrintf("data%u", dataCtr+1)) == NULL)
Tcl_Panic("%s:%u: Tcl_ObjPrintf failed\n", __func__, __LINE__);
else if ((cmd = Tcl_CreateObjCommand(interp, Tcl_GetString(obj),
wrapCmd, (ClientData) wrap, wrapDelCmd)) == NULL)
Tcl_Panic("%s:%u: Tcl_CreateObjCommand failed\n", __func__, __LINE__);
else {
wrap->str = Tcl_GetCommandName(interp, cmd);
wrap->n = dataCtr;
wrap->data = NULL; // call your wrapped API to instantiate an object
dataCtr++;
Tcl_SetObjResult(interp, obj);
}
return TCL_OK;
}
int Allocinvalidblock_Init(Tcl_Interp *interp)
{
dataCtr = 0;
return (Tcl_CreateObjCommand(interp, "new",
newCmd, (ClientData) NULL, NULL)
== NULL) ? TCL_ERROR : TCL_OK;
}
int Allocinvalidblock_Unload(Tcl_Interp *interp, int flags)
{
Tcl_Namespace *ns = Tcl_GetGlobalNamespace(interp);
Tcl_Obj *obj;
Tcl_Command cmd;
unsigned int i;
for(i=0; i<dataCtr; i++) {
if ((obj = Tcl_ObjPrintf("data%u", i+1)) != NULL) {
if ((cmd = Tcl_FindCommand(interp,
Tcl_GetString(obj), ns, TCL_GLOBAL_ONLY)) != NULL)
Tcl_DeleteCommandFromToken(interp, cmd);
Tcl_DecrRefCount(obj);
}
}
return TCL_OK;
}
Once built (for example with Code::Blocks as shared library project linking against C:/msys64/mingw64/lib/libtcl.dll.a), the error can be triggered when more than a data object is created and the library immediately unloaded:
load bin/Release/libAllocInvalidBlock.dll
new
new
unload bin/Release/libAllocInvalidBlock.dll
If used otherwise the crash may even be not triggered... Anyway, such an error in the C code is not particularly obvious to identify (although easy to fix) because the compilation is running without any warning (although -Wall compiler flag is set).
I read all I could find about memory management in the Tcl API, but haven't been able to solve my problem so far. I wrote a Tcl extension to access an existing application. It works, except for a serious issue: memory leak.
I tried to reproduce the problem with minimal code, which you can find at the end of the post. The extension defines a new command, recordings, in namespace vtcl. The recordings command creates a list of 10000 elements, each element being a new command. Each command has data attached to it, which is the name of a recording. The name subcommand of each command returns the name of the recording.
I run the following Tcl code with tclsh to reproduce the problem:
load libvtcl.so
for {set ii 0} {$ii < 1000} {incr ii} {
set recs [vtcl::recordings]
foreach r $recs {rename $r ""}
}
The line foreach r $recs {rename $r ""} deletes all the commands at each iteration, which frees the memory of the piece of data attached to each command (I can see that in gdb). I can also see in gdb that the reference count of variable recs goes to 0 at each iteration so that the contents of the list is freed. Nonetheless, I see the memory of the process running tclsh going up at each iteration.
I have no more idea what else I could try. Help will be greatly appreciated.
#include <stdio.h>
#include <string.h>
#include <tcl.h>
static void DecrementRefCount(ClientData cd);
static int ListRecordingsCmd(ClientData cd, Tcl_Interp *interp, int objc,
Tcl_Obj *CONST objv[]);
static int RecordingCmd(ClientData cd, Tcl_Interp *interp, int objc,
Tcl_Obj *CONST objv[]);
static void
DecrementRefCount(ClientData cd)
{
Tcl_Obj *obj = (Tcl_Obj *) cd;
Tcl_DecrRefCount(obj);
return;
}
static int
ListRecordingsCmd(ClientData cd, Tcl_Interp *interp, int objc,
Tcl_Obj *CONST objv[])
{
char name_buf[20];
Tcl_Obj *rec_list = Tcl_NewListObj(0, NULL);
for (int ii = 0; ii < 10000; ii++)
{
static int obj_id = 0;
Tcl_Obj *cmd;
Tcl_Obj *rec_name;
cmd = Tcl_NewStringObj ("rec", -1);
Tcl_AppendObjToObj (cmd, Tcl_NewIntObj (obj_id++));
rec_name = Tcl_NewStringObj ("DM", -1);
snprintf(name_buf, sizeof(name_buf), "%04d", ii);
Tcl_AppendStringsToObj(rec_name, name_buf, (char *) NULL);
Tcl_IncrRefCount(rec_name);
Tcl_CreateObjCommand (interp, Tcl_GetString (cmd), RecordingCmd,
(ClientData) rec_name, DecrementRefCount);
Tcl_ListObjAppendElement (interp, rec_list, cmd);
}
Tcl_SetObjResult (interp, rec_list);
return TCL_OK;
}
static int
RecordingCmd(ClientData cd, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[])
{
Tcl_Obj *rec_name = (Tcl_Obj *)cd;
char *subcmd;
subcmd = Tcl_GetString (objv[1]);
if (strcmp (subcmd, "name") == 0)
{
Tcl_SetObjResult (interp, rec_name);
}
else
{
Tcl_Obj *result = Tcl_NewStringObj ("", 0);
Tcl_AppendStringsToObj (result,
"bad command \"",
Tcl_GetString (objv[1]),
"\"",
(char *) NULL);
Tcl_SetObjResult (interp, result);
return TCL_ERROR;
}
return TCL_OK;
}
int
Vtcl_Init(Tcl_Interp *interp)
{
#ifdef USE_TCL_STUBS
if (Tcl_InitStubs(interp, "8.5", 0) == NULL) {
return TCL_ERROR;
}
#endif
if (Tcl_PkgProvide(interp, "vtcl", "0.0.1") != TCL_OK)
return TCL_ERROR;
Tcl_CreateNamespace(interp, "vtcl", (ClientData) NULL,
(Tcl_NamespaceDeleteProc *) NULL);
Tcl_CreateObjCommand(interp, "::vtcl::recordings", ListRecordingsCmd,
(ClientData) NULL, (Tcl_CmdDeleteProc *) NULL);
return TCL_OK;
}
The management of the Tcl_Obj * reference counts looks absolutely correct, but I do wonder whether you're freeing all the other resources associated with a particular instance in your real code. It might also be something else entirely; your code is not the only thing in Tcl that allocates memory! Furthermore, the default memory allocator in Tcl does not actually return memory to the OS, but instead holds onto it until the process ends. Figuring out what is wrong can be tricky.
You can try doing a build of Tcl with the --enable-symbols=mem passed to configure. That makes Tcl build in an extra command, memory, which allows more extensive checking of memory management behaviour (it also does things like ensure that memory is never written to after it is freed). It's not enabled by default because it has a substantial performance hit, but it could well help you track down what's going on. (The memory info subcommand is where to get started.)
You could also try adding -DPURIFY to the CFLAGS when building; it completely disables the Tcl memory allocator (so memory checking tools like — commercial — Purify and — OSS — Electric Fence can get accurate information, instead of getting very confused by Tcl's high-performance thread-aware allocator) and may allow you to figure out what is going on.
I found where the leak is. In function ListRecordingsCmd, I replaced line
Tcl_AppendObjToObj (cmd, Tcl_NewIntObj (obj_id++));
with
Tcl_Obj *obj = Tcl_NewIntObj (obj_id++);
Tcl_AppendObjToObj (cmd, obj);
Tcl_DecrRefCount(obj);
The memory allocated to store the object id was not released. The memory used by the tclsh process is now stable.
My cocoa application uses one library written in 'C' which is tryings write file at '/tmp' path. This creates sandbox violations. In Cocoa we can use 'NSTemporaryDirectory' API. To fix sandbox violation Is it safe to use 'tmpfile' API in sandboxed environment? Are there in any other solutions?
EDITED After actually testing it
No, tmpnam() won't work and I think the only way to get a temporary filename is to provide a .m file with your library specifically for use with iOS and OSX, which can be used return the temporary directory as a C-String:
apple.h:
#pragma once
extern size_t getTemporaryDirectory(char *buffer, size_t len);
apple.m:
size_t getTemporaryDirectory(char *buffer, size_t len)
{
#autoreleasepool
{
NSString *tempDir = NSTemporaryDirectory();
if (tempDir != nil)
{
const char *utf = [tempDir UTF8String];
strncpy(buffer, utf, len);
return strlen(utf);
}
}
return 0;
}
I searched and tried for days. The problem is this:
I wrote a script which load a shared library locker.so, it runs well with lua interpretor, but I can not write out the correct host program.
My lua script load_so.lua is very simple:
locker = require("locker")
print(type(locker))
for k, v in pairs(locker) do
print(k, v)
end
My host program is:
int main(int argc, const char *argv[])
{
lua_State * L = luaL_newstate();
luaL_openlibs(L);
if (luaL_dofile(L, "load_so.lua") != 0) {
fprintf(stderr, "luaL_dofile error: %s\n", lua_tostring(L, -1));
lua_pop(L, 1);
}
lua_close(L);
return 0;
}
When I run my host program, error print out:
luaL_dofile error: error loading module 'locker' from file './locker.so':
./locker.so: undefined symbol: lua_pushstring
And the locker.c:
static int elock_get(lua_State * L) {...}
static int elock_set(lua_State * L) {...}
static const struct luaL_Reg lockerlib[] = {
{"get", elock_get},
{"set", elock_set},
{NULL, NULL}
};
int luaopen_locker(lua_State *L)
{
//luaL_newlib(L, lockerlib);
//lua_pushvalue(L, -1);
//lua_setglobal(L, LOCKER_LIBNAME);
//set_info(L);
luaL_newlibtable(L, lockerlib);
luaL_setfuncs(L, lockerlib, 0);
return 1;
}
Most articles, books, questions shows how to do it in Lua 5.1, and yes, the program runs correctly in Lua 5.1. But how can I make it support Lua 5.2, and why?
P.S: I don't want to load the library in my C host program like luaL_requiref(L, "locker", luaopen_locker, 1), because I don't know which .so library will load in Lua script.
Thanks.
In Linux, if you're linking liblua.a statically into your main program, you need to use -Wl,-E when linking to export the Lua API symbols; this is how the standard command line interpreter is built.
Is there any support of OpenCV graphics library is available for Windows Phone 8 and Windows 8. I made a search on Google but didn't find any resource related with OpenCV to connect with Windows Phone 8 / Windows 8. If any of you know more about this please help me, and provide some link to reach the library.
This is the latest information what I get from OpenCV team.
OpenCV development team is working on port for Windows RT. Here is current development branch for WinRT(https://github.com/asmorkalov/opencv/tree/winrt). You can build it for ARM using Visual Studio Express for Windows 8 and Platform SDK.
Open Visual Studio development console.
Setup environment for cross compilation by command "C:\Program Files(x86)\Microsoft
Visual Studio 11.0\VC\bin\x86_arm\vcvarsx86_arm.bat"
cd <opencv_source_dir>/platforms/winrt/
run scripts/cmake_winrt.cmd
run ninja
Alternatively you can use nmake instead ninja. You need to edit cmake_winrt.cmd and change project generator fro -GNinja to -G "NMake Makefiles". Only algorithmic part of the library is supported now, no tbb, no UI, no video IO.
Please check the below given URL from more details.
http://answers.opencv.org/question/9847/opencv-for-windows-8-tablet/?answer=9851#post-id-9851
By windows-8, I guess you mean winRT ? AFAIK, there is no official port to winRT. You need to compile it by yourself as a Win8 Store DLL for instance, so that you can reference it from a Win8 Store Application.
Just start by opencv-core, then add the lib you need, one by one, because all the components will not be able to compile (for instance, opencv-highgui is highly dependant on Windows API which is not fully compatible with Win8 Store Apps).
You'll also need to code by yourself some Win32 methods used by OpenCV and not accessible from Win8 App like GetSystemInfo(), GetTempPathA(), GetTempFileNameA() and all methods related to thread local storage (TLS).
I've been able to use a small subset of OpenCV in WinRT by compiling opencv_core, opencv_imgproc and zlib, as 3 seperate static libs. I've added one another, called opencv_winrt, that contains only the two following files:
opencv_winrt.h
#pragma once
#include "combaseapi.h"
void WINAPI GetSystemInfo(
_Out_ LPSYSTEM_INFO lpSystemInfo
);
DWORD WINAPI GetTempPathA(
_In_ DWORD nBufferLength,
_Out_ char* lpBuffer
);
UINT WINAPI GetTempFileNameA(
_In_ const char* lpPathName,
_In_ const char* lpPrefixString,
_In_ UINT uUnique,
_Out_ char* lpTempFileName
);
DWORD WINAPI TlsAlloc();
BOOL WINAPI TlsFree(
_In_ DWORD dwTlsIndex
);
LPVOID WINAPI TlsGetValue(
_In_ DWORD dwTlsIndex
);
BOOL WINAPI TlsSetValue(
_In_ DWORD dwTlsIndex,
_In_opt_ LPVOID lpTlsValue
);
void WINAPI TlsShutdown();
# define TLS_OUT_OF_INDEXES ((DWORD)0xFFFFFFFF)
opencv_winrt.cpp
#include "opencv_winrt.h"
#include <vector>
#include <set>
#include <mutex>
#include "assert.h"
void WINAPI GetSystemInfo(LPSYSTEM_INFO lpSystemInfo)
{
GetNativeSystemInfo(lpSystemInfo);
}
DWORD WINAPI GetTempPathA(DWORD nBufferLength, char* lpBuffer)
{
return 0;
}
UINT WINAPI GetTempFileNameA(const char* lpPathName, const char* lpPrefixString, UINT uUnique, char* lpTempFileName)
{
return 0;
}
// Thread local storage.
typedef std::vector<void*> ThreadLocalData;
static __declspec(thread) ThreadLocalData* currentThreadData = nullptr;
static std::set<ThreadLocalData*> allThreadData;
static DWORD nextTlsIndex = 0;
static std::vector<DWORD> freeTlsIndices;
static std::mutex tlsAllocationLock;
DWORD WINAPI TlsAlloc()
{
std::lock_guard<std::mutex> lock(tlsAllocationLock);
// Can we reuse a previously freed TLS slot?
if (!freeTlsIndices.empty())
{
DWORD result = freeTlsIndices.back();
freeTlsIndices.pop_back();
return result;
}
// Allocate a new TLS slot.
return nextTlsIndex++;
}
_Use_decl_annotations_ BOOL WINAPI TlsFree(DWORD dwTlsIndex)
{
std::lock_guard<std::mutex> lock(tlsAllocationLock);
assert(dwTlsIndex < nextTlsIndex);
assert(find(freeTlsIndices.begin(), freeTlsIndices.end(), dwTlsIndex) == freeTlsIndices.end());
// Store this slot for reuse by TlsAlloc.
try
{
freeTlsIndices.push_back(dwTlsIndex);
}
catch (...)
{
return false;
}
// Zero the value for all threads that might be using this now freed slot.
for each (auto threadData in allThreadData)
{
if (threadData->size() > dwTlsIndex)
{
threadData->at(dwTlsIndex) = nullptr;
}
}
return true;
}
_Use_decl_annotations_ LPVOID WINAPI TlsGetValue(DWORD dwTlsIndex)
{
ThreadLocalData* threadData = currentThreadData;
if (threadData && threadData->size() > dwTlsIndex)
{
// Return the value of an allocated TLS slot.
return threadData->at(dwTlsIndex);
}
else
{
// Default value for unallocated slots.
return nullptr;
}
}
_Use_decl_annotations_ BOOL WINAPI TlsSetValue(DWORD dwTlsIndex, LPVOID lpTlsValue)
{
ThreadLocalData* threadData = currentThreadData;
if (!threadData)
{
// First time allocation of TLS data for this thread.
try
{
threadData = new ThreadLocalData(dwTlsIndex + 1, nullptr);
std::lock_guard<std::mutex> lock(tlsAllocationLock);
allThreadData.insert(threadData);
currentThreadData = threadData;
}
catch (...)
{
if (threadData)
delete threadData;
return false;
}
}
else if (threadData->size() <= dwTlsIndex)
{
// This thread already has a TLS data block, but it must be expanded to fit the specified slot.
try
{
std::lock_guard<std::mutex> lock(tlsAllocationLock);
threadData->resize(dwTlsIndex + 1, nullptr);
}
catch (...)
{
return false;
}
}
// Store the new value for this slot.
threadData->at(dwTlsIndex) = lpTlsValue;
return true;
}
// Called at thread exit to clean up TLS allocations.
void WINAPI TlsShutdown()
{
ThreadLocalData* threadData = currentThreadData;
if (threadData)
{
{
std::lock_guard<std::mutex> lock(tlsAllocationLock);
allThreadData.erase(threadData);
}
currentThreadData = nullptr;
delete threadData;
}
}
And I modify the file cvconfig.h: I've commented out every #define, except PACKAGE* and VERSION, and I added #include "opencv_winrt.h" at the end.
Just a hint - there is a C# wrapper for OpenCV called EmguCV (http://www.emgu.com/wiki/index.php/Main_Page), by looking at the forum posts I see that there is some activity towards using it on Windows 8 but it's hard to tell if it's now working since the posts claiming issues are quite old. I'd suggest you just give it a try and see if this C# wrapper is able to run on Windows Phone 8, I think it should definitely run on Windows 8.