Is it possible to detect if another process reading the memory of my application? If so, can you give me any examples on how to accomplish this? (examples in C++ would be great)
Thank You
To detect a process that opens a handle to your process and calls ReadProcessMemory you must either, hook OpenProcess & ReadProcessMemory in every process or have a kernel mode driver that intercepts these calls.
Detecting ReadProcessMemory solely from inside the target process is usermode is not possible to my knowledge. I have seen this question come up many times and have never seen an acceptable answer.
Related
I am profiling a multi threaded application that uses pthread library, to get information about its most executed Basic Blocks (BBLs).
The problem I have encountered is that there are a lot of blocks belonging to __pthread_mutex_lock function, which are not only in the TOP 100 most executed BBLs, but in the TOP 10. This is really annoying because I am not interested in BBLs belonging to pthread library functions at all.
My mentor and I were speculating if busy wait had something to do with this, because in MPI there are ways to change locks to behave from busy waiting mode to blocking mode, so when profiling apps these kind of pthread functions don't appear that much, not even in the top 10. I have never worked with MPI so excuse me if what I say is not very accurate.
What I wanted to ask here is if anyone knows a way to do something like that to pthreads. Changing the code of the application is not an option, but if there is a compiling knob to compile the application, appart from -lpthread, that can change its behaviour not to get so many BBLs from its libraries, would be great.
Thank you for your time.
We have an application made up by a host (exe) and a lot of modules (dlls) containing gui etc.
Sometimes, the application freezes on shutdown. Mostly it happens during testing through TestComplete. We are not able to reproduce the behavior during debugging.
How can we find out why the application freezes?
I would guess that it is related to threads, but I do not know for sure.
Are there any tools or techniques we should try out?
I think that good old logging would help you. Add some logging to every unit finalization, add such logging to destructors of global data (database connection, global configuration etc). Of course do not destroy logger object.
If your application is multithreaded then add some logging to working threads such as writing '[date] thread [name of class] working' and write it every few seconds (you can use some debug mode). Also add logging when thread discovers that it should terminate.
Also use some system utilites such as ProcessMonitor, Handles, Process Explorer (all by Sysinternals/Microsoft). Monitor disk reads/writes, handle count, memory usage, network connections. Maybe your application dumps some big structures on disk at exit? Maybe it allocated a lot of memory and must release it?
Rig it with EurekaLog or MadExcept, and that may show you where the exception is, or where the memory leak is that is causing the exception. Both of those are excellent tools with fully-featured trial versions. Try 'em, buy 'em. Good stuff.
If the debugger's presence keeps the problem from occurring, then wait for the problem to occur, and then attach the debugger to it. Pause execution and you can inspect each thread's call stack.
If you use lots of GUI components in DLL's and/or do plenty of multi-threading then you'd first have to discover which DLL or thread is causing the problem. Or maybe it's a combination of both. Basically, you should create log events for every DLL and thread that gets loaded/started. Try to get a situation where you have a minimum of DLL's and threads loaded to generate the freeze. Then you've localized the problem to one of those.Also, create simple test-applications or use a unit-testing framework to test specific modules. For example, there is a Delphi version of NUnit available, which might help. (It's called DUnit...) Such a test framework is helpful to isolate the threads and DLL's to check each of them.
If your app is happenning under windows 7 and there are threads running in DLL, you must shutdown/terminate all running DLL threads before closing the main form of executable.
Good luck
There are software applications, such as ArtMoney, that edit the memory of other applications.
Is there a way to detect when some other application is editing the memory of my application?
The basic idea to protect from basic memory modification is to encrypt the parts of memory you care about, and have redundant checks to ensure against modification.
None of which will stop a determined hacker, but it's sufficient to keep the script kiddies out of your address space.
One method, used by many virus checkers, is to perform a checksum of your executable or memory and save it. When running, occasionally calculate a new checksum and compare with the original. Most programs don't intentionally modify their executables.
The short answer is no, it's not possible in the general case. Even if you implement some of the suggestions that have been given, there's nothing stopping someone from patching the code that performs the checks.
I don't know the specifics of how ArtMonkey works, but if it functions as a debugger you could try checking regularly to see if DebugHook <> 0, and reacting appropriately if it is. (Just make sure to put that code in a {$IFNDEF DEBUG} block so it doesn't cause trouble for you!)
You might want to ask yourself why you want to prevent people from patchimg your memory, though. Unless there's a genuine security issue, you probably shouldn't even try. Remember that the user's computer, that your program will be running on, is their property, not yours, and if you interfere too much with the user's choices as to what to do with their property, your program is morally indistinguishable from malware.
I do not know how it works, I think it can be done in 3 ways:
ReadProcessMemory and WriteProcessMemory Windows API
using a debugger (check for debughook, but that's almost too easy so it won't use that)
injects a dll so it can acces all memory (because it is in the same process)
The last one is easier (check for injected dll or something like that). The first one is trickier, but I found some articles about it:
Memory breakpoints: http://www.codeproject.com/KB/security/AntiReverseEngineering.aspx?fid=1529949&fr=51&df=90&mpp=25&noise=3&sort=Position&view=Quick#BpMem
Hook "WriteProcessMemory" api: http://www.codeproject.com/KB/system/hooksys.aspx
I asked a similar question, and the conclusion was basically that you cannot stop this.
How can I increase memory security in Delphi
I'm making a wrapper for the pthread library that allows each thread to have its own set of non-shared memory. Right now the way c is set up if any thread tries to rwe another threads data, the program segfaults. This is fine, I can catch it with a sighandler and call pthread_exit() and continue on with the program.
But not every segfault is going to be the result of a bad rwe. I need to find a way to use the siginfo type to determine if the segfault was bad programming or this error. Any ideas?
Since I am using mmap to manage the memory pages I think using si_addr in siginfo will help me out.
It sounds like what you're really after is thread local storage which is already solved much more portably than this. GCC provides __thread, MSVC provides __declspec(thread). boost::thread provides portable thread local storage using a variety of mechanisms depending on platform/toolchain etc.
If you really do want to go down this road it can be made to work however the path is fraught with dangers. Recovering from SIGSEGV is undefined behaviour technically, although it can be made to work on quite a few platforms it is neither robust nor portable. You need to be very careful what you do in the signal handler though too -- the list of async-safe functions, i.e. those which may legally be safely called from a signal handler is very small.
I've used this trick successfully a few times in the past, normally for marking "pages" as "dirty" in userspace. The way I did this was by setting up a hashtable which contained the base address of all the "pages" of memory that I was interested in. When you catch a SIGSEGV in a handler you can then map an address back to a page with simple arithmetic operations. Provided the hashtable can be read without locks you can then lookup if this is a page that you care about or a segfault from somewhere else and decide how to act.
What is the normal way people writing network code in Delphi use Windows-style overlapped asynchronous socket I/O?
Here's my prior research into this question:
The Indy components seem entirely synchronous. On the other hand, while ScktComp unit does use WSAAsyncSelect, it basically only asynchronizes a BSD-style multiplexed socket app. You get dumped in a single event callback, as if you had just returned from select() in a loop, and have to do all the state machine navigation yourself.
The .NET situation is considerably nicer, with Socket.BeginRead / Socket.EndRead, where the continuation is passed directly to Socket.BeginRead, and that's where you pick back up. A continuation coded as a closure obviously has all the context you need, and more.
I have found that Indy, while a simpler concept in the beginning, is awkward to manage due to the need to kill sockets to free threads at application termination. In addition, I had the Indy library stop working after an OS patch upgrade. ScktComp works well for my application.
#Roddy - Synchronous sockets are not what I'm after. Burning a whole thread for the sake of a possibly long-lived connection means you limit the amount of concurrent connections to the number of threads that your process can contain. Since threads use a lot of resources - reserved stack address space, committed stack memory, and kernel transitions for context switches - they do not scale when you need to support hundreds of connections, much less thousands or more.
What is the normal way people writing
network code in Delphi use
Windows-style overlapped asynchronous
socket I/O?
Well, Indy has been the 'standard' library for socket I/O for a long while now - and it's based on blocking sockets. This means if you want asynchronous behaviour, you use additional thread(s) to connect/read/write data. To my mind this is actually a major advantage, as there's no need to manage any kind of state machine navigation, or worry about callback procs or similar stuff. I find the logic of my 'reading' thread is less cluttered and much more portable than non-blocking sockets would allow.
Indy 9 has been mostly bombproof, fast and reliable for us. However the move to Indy 10 for Tiburon is causing me a little concern.
#Mike: "...the need to kill sockets to free threads...".
This made go "huh?" until I remembered our threading library uses an exception-based technique to kill 'waiting' threads safely. We call QueueUserAPC to queue a function which raises a C++ exception (NOT derived from class Exception) which should only be caught by our thread wrapper procedure. All destructors get called so the threads all terminate cleanly and tidy up on the way out.
"Synchronous sockets are not what I'm after."
Understood - but I think in that case the answer to your original question is that there just isn't a Delphi idiom for async socket IO because it's actually a highly specialized and uncommon requirement.
As a side issue, you might find these links interesting. They're both a little old, and more *nxy than Windows. The second one implies that - in the right environment - threads might not be as bad as you think.
The C10K problem
Why Events Are A Bad Idea (for High-concurrency Servers)
#Chris Miller - What you've stated in your answer is factually inaccurate.
Windows message-style async, as available through WSAAsyncSelect, is indeed largely a workaround for lack of a proper threading model in Win 3.x days.
.NET Begin/End, however, is not using extra threads. Instead, it is using overlapped I/O, using the extra argument on WSASend / WSARecv, specifically the overlapped completion routine, to specify the continuation.
This means that the .NET style harnesses the Windows OS's async I/O support to avoid burning a thread by blocking on a socket.
Since threads are generally speaking expensive (unless you specify a very small stack size to CreateThread), having threads blocking on sockets will stop you from scaling to 10,000s of concurrent connections.
This is why it's important that async I/O be used if you want to scale, and also why .NET is not, I repeat, is not, simply "using threads, [...] just managed by the Framework".
#Roddy - I've already read the links you point to, they are both referenced from Paul Tyma's presentation "Thousands of Threads and Blocking I/O - The old way to write Java Servers is New again".
Some of the things that don't necessarily jump out from Paul's presentation, however, are that he specified -Xss:48k to the JVM on startup, and that he's assuming that the JVM's NIO implementation is efficient in order for it to be a valid comparison.
Indy does not specify a similarly shrunken and tightly constrained stack size. There are no calls to BeginThread (the Delphi RTL thread creation routine, which you should use for such situations) or CreateThread (the raw WinAPI call) in the Indy codebase.
The default stack size is stored in the PE, and for the Delphi compiler it defaults to 1MB of reserved address space (space is committed page by page by the OS in 4K chunks; in fact, the compiler needs to generate code to touch pages if there are >4K of locals in a function, because the extension is controlled by page faults, but only for the lowest (guard) page in the stack). That means you're going to run out of address space after max 2,000 concurrent threads handling connections.
Now, you can change the default stack size in the PE using the {$M minStackSize [,maxStackSize]} directive, but that will affect all threads, including the main thread. I hope you don't do much recursion, because 48K or (similar) isn't a lot of space.
Now, whether Paul is right about non-performance of async I/O for Windows in particular, I'm not 100% sure - I'd have to measure it to be certain. What I do know, however, is that arguments about threaded programming being easier than async event-based programming, are presenting a false dichotomy.
Async code doesn't need to be event-based; it can be continuation-based, like it is in .NET, and if you specify a closure as your continuation, you get state maintained for you for free. Moreover, conversion from linear thread-style code to continuation-passing-style async code can be made mechanical by a compiler (CPS transform is mechanical), so there need be no cost in code clarity either.
There is a free IOCP (completion ports) socket components : http://www.torry.net/authorsmore.php?id=7131 (source code included)
"By Naberegnyh Sergey N.. High
performance socket server based on
Windows Completion Port and with using
Windows Socket Extensions. IPv6
supported. "
i've found it while looking better components/library to rearchitecture my little instant messaging server. I haven't tried it yet but it looks good coded as a first impression.
For async stuff try ICS
http://www.overbyte.be/frame_index.html?redirTo=/products/ics.html
Indy uses synchronous sockets because it's a simpler way of programming. The asynchronous socket blocking was something added to the winsock stack back in the Windows 3.x days. Windows 3.x did not support threads and there you couldn't do socket I/O without threads. For some additional information about why Indy uses the blocking model, please see this article.
The .NET Socket.BeginRead/EndRead calls are using threads, it's just managed by the Framework instead of by you.
#Roddy, Indy 10 has been bundled with Delphi since at Delphi 2006. I found that migrating from Indy 9 to Indy 10 to be a straight forward task.
With the ScktComp classes, you need to use a ThreadBlocking server rather than an a NonBlocking server type. Use the OnGetThread event to hand off the ClientSocket param to a new thread of your devising. Once you've instantiated an inherited instance of TServerClientThread you'll create a instance of TWinSocketStream (inside the thread) which you can use to read and write to the socket. This method gets you away from trying to process data in the event handler. These threads could exist for just the short period need to read or write, or hang on for the duration for the purpose of being reused.
The subject of writing a socket server is fairly vast. There are many techniques and practices you could choose to implement. The method of reading and writing to the same socket with in the TServerClientThread is straight forward and fine for simple applications. If you need a model for high availability and high concurrency then you need to look into patterns like the Proactor pattern.
Good luck!