Say I have a couple of java runtime environments running on my system which are used by several applications. I would like to programmatically interact with these applications by reading their memory.
A typical approach would be to directly look into this application's memory, however for java applications this seems to be practically impossible because of the java runtime environment. Instead, one has to look into the memory of the java runtime environment, or debug it.
[ the above is what I think I have learned from several searches on the web, if anything is false, please correct me ]
Note: keep in mind that the application I want to monitor is not owned by me and thus I do not have the source code nor the ability to launch the application in "debug mode" or something.
Now, as this is a non-production project, I would prefer an easy way out: using an existing windows GUI application which can already monitor variables of a java runtime environment and it's applications to programmatically crawl these from this GUI application for usage in my own project. If any such program exists, I would really appreciate the help.
If the above is not possible, how would I (programmatically) retreive these variables otherwise?
It's difficult to answer this precisely without knowing much more about the application involved, its structure etc. Note that objects move around in the JVM's memory, and so you can't monitor the actual application memory directly.
So the first question is, how do you know what you want to monitor without the source code ? e.g. which variables/objects etc.?
Given that you've worked this out, it strikes me that you have two options.
decompile and instrument the application (perhaps statically, perhaps using AOP), and recompile it. This assumes that the application is not obfuscated, and you're not in breach of licensing etc.
wrap the application in a thin layer that uses reflection to identify the variables you're interested in, and tracks the values of those variables as the process executes. I suspect you'll still have to decompile to identify these variables.
You can monitor these values remotely by creating an MBean and exposing via JMX, and monitoring via JConsole. That's pretty trivial compared to the initial step of finding those variables you're interested in.
Related
You know, when an application opens a file and write to it, the system chooses in which cluster will be stored. I want to choose myself ! Let me tell you what I really want to do... In fact, I don't necessarily want to write anything. I have a HDD with a BAD range of clusters in the middle and I want to mark that space as it is occupied by a file, and eventually set it as a hidden-unmoveable-system one (like page file in windows) so that it won't be accessed anymore. Any ideas on how to do that ?
Later Edit:
I think THIS is my last hope. I just found it, but I need to investigate... Maybe a file could be created anywhere and then relocated to the desired cluster. But that requires writing, and the function may fail if that cluster is bad.
I believe the answer to your specific question: "Can I write a file to a specific cluster location" is, in general, "No".
The reason for that is that the architecture of modern operating systems is layered so that the underlying disk store is accessed at a lower level than you can access, and of course disks can be formatted in different ways so there will be different kernel mode drivers that support different formats. Even so, an intelligent disk controller can remap the addresses used by the kernel mode driver anyway. In short there are too many levels of possible redirection for you to be sure that your intervention is happening at the correct level.
If you are talking about Windows - which you haven't stated but which appears to assumed - then you need to be looking at storage drivers in the kernel (see https://learn.microsoft.com/en-us/windows-hardware/drivers/storage/). I think the closest you could reasonably come would be to write your own Installable File System driver (see https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/_ifsk/). This is really a 'filter' as it sits in the IO request chain and can intercept and change IO Request Packets (IRPs). Of course this would run in the kernel, not in userspace, and normally this would be written in C and I note your question is tagged for Delphi.
Your IFS Driver can sit at differnt levels in the request chain. I have used this technique to intercept calls to specific file system locations (paths / file names) and alter the IRP so as to virtualise the request - even calling back to user space from the kernel to resolve how the request should be handled. Using the provided examples implementing basic functionality with an IFS driver is not too involved because it's a filter and not a complete storgae system.
However the very nature of this approach means that another filter can also alter what you are doing in your driver.
You could look at replacing the file system driver that interfaces to the hardware, but I think that's likely to be an excessive task under the circumstances ... and as pointed out already by #fpiette the disk controller hardware can remap your request anyway.
In the days of MSDOS the access to the hardware was simpler and provided by the BIOS which could be hooked to allow the requests to be intercepted. Modern environments aren't that simple anymore. The IFS approach does allow IO to be hooked, but it does not provide the level of control you need.
EDIT regarding suggestion by the OP of using FSCTL_MOVE_FILE
For simple environment this may well do what you want, it is designed to support a defragmentation process.
However I still think there's no guarantee that this actually will do what you want.
You will note from the page you have linked to it states that it is moving one or more virtual clusters of a file from one logical cluster to another within the same volume
This is a code that's passed to the underlying storage drivers which I have referred to above. What the storage layer does is up to the storage layer and will depend on the underlying technology. With more advanced storage there's no guarantee this actually addresses the physical locations which I believe your question is asking about.
However that's entirely dependent on the underlying storage system. For some types of storage relocation by the OS may not be honoured in the same way. As an example consider an enterprise storage array that has a built in data-tiering function. Without the awareness of the OS data will be relocated within the storage based on the tiering algorithms. Also consider that there are technologies which allow data to be directly accessed (like NVMe) and that you are working with 'virtual' and 'logical' clusters, not physical locations.
However, you may well find that in a simple case, with support in the underlying drivers and no remapping done outside the OS and kernel, this does what you need.
Since you problem is to mark bad cluster, you don't need to write any program. Use the command line utility CHKDSK that Windows provides.
I an elevated command prompt (Run as administrator), run the command:
chkdsk /r c:
The check will be done on the next reboot.
Don't forget to read the documentation.
I have a related post - Assertion failure in DBAccess.pas but thought this was worth asking separately.
We are licensed for the full source-code release of DevArt ODAC but have been experiencing tremendous difficulties performing an upgrade. In the course of investigating this I have noticed that there is no .pas file for OraNet.dcu.
This is making it difficult to establish what the cause of our difficulties is as we cannot fully debug the code.
Also - what is this unit? From its name and the directives in the code it would be reasonable to suppose it is a .NET required unit - not something we are interested in.
There is the Direct DB connection mode implemented in the OraNet.dcu module, and we don't distribute the source code of this module, this limitation is specified at our website (the reference at the bottom of the page). If you don't use the Direct mode, and work via the Oracle client (the OCI mode), you can specify DEFINE NONET in your project settings, in this case, the Direct mode will be unavailable, and this module won't be used.
The client usage (even Oracle Instant Client) really allows using more features, than our Direct mode, but the Direct mode even surpasses OCI by performance in some cases. Besides, the Direct mode significantly simplifies application deployment and decreases the application size on disk due to the fact that there is no need to supply and deploy additional libraries, and set additional registry parameters and environment variables. The Direct mode also allows application deployment to systems, for which there are no native Oracle clients, for example, iOS. Choosing of the way of working with the DB (Direct or OCI) depends on developer and tasks resolved by each particular application. As it was mentioned above, if the Direct mode is not used, additional module plugging can be disabled by setting DEFINE NONET
As far as I know Erlang provides advanced features for error handling and isolation of processes.
I'm building a system that allow user to submit their code to be executed on the shared server environment and need to make it safe.
Requirements are:
limit CPU and Memory usage individually for each user-process.
forbid user-process to communicate with other processes (except some processes specially designed for such purpose).
forbid access to all sytem resources (shell, file system, ...).
terminate user-process in case of errors or high resource consumption.
Is it possible to to all this with Erlang and keep it performance efficient?
In general, Erlang doesn't provide means to sandbox code which a user can inject. You can try writing your own piece of protection code, but it is rather hard.
A better choice would probably be a language like "safe haskell":
http://www.haskell.org/ghc/docs/7.4.2/html/users_guide/safe-haskell.html
which is specifically built to do this kind of thing.
The isolation provided by Erlang is not intended to protect against malicious modules being injected. In fact, there is no such protection in the distributed case either. As soon as two machines are connected, there is no limit to what you can do to the other machine.
There has been work done on Safe Erlang in the past and you can find several papers about it.
The ErlHive project addresses the problem in an interesting way.
I have a media application (written in Delphi 2010 but I am not sure that's entirely relevant) and it only allows one instance (via mutex).
One of my customers would like to run 2 instances of the app by duplicating its install and all of its application data as this will allow him to run the output to two different sound cards, giving him two audio zones.
Now I can allow the second instance via command line switch, thus creating a differently named mutex and even allowing him to send controls to either instance of the appliction via command line switches or windows message passing.
My application also exposes a COM interface for automation purposes, obviously this provides a much richer interface than command line and makes it much easier to get information out of the application.
So my problem is that, as far as I am aware, I can only expose the COM interface to one executable. Now I know that makes sense, but I am wondering if anyone can think of a workaround to this.
I had a quick try at duplicating the registry keys for my HKLM\Software\Classes\AppID thus making AppIDv2 and got as far as it lanching the other copy of my app, but I guess it all came unstuck when it hit the more specific GUIDS for the TypeLib etc. Mind you, I know I overstepped the bounds of my knowledge!
My thought is that if I can create a different AppID string and ultimately target the exe sitting in different locations then we'd at least be able to do some automation via scripting COM Automation but I suspect that the requirement for GUIDs is ultimately going to let me down.
Another option may be to move my COM to inprocess and then have multiple compiled versions of my application that expose an instance of the main interface via new AppIDs, but that gets messy when you want the DLL to know all about the running instance of your application.
Any ideas welcome. Thanks in advance.
It sounds like you want to register yourself in the Running Objects Table (ROT).
i'm likening your problem to that of multiple copies of Excel running. COM has a mechanism to allow someone to find my the running instances of Excel and connect to one of them.
Out of process COM objects are expected to register themselves with the ROT. Callers can then use GetActiveObject to find your instance:
To automate an Office application that is already running, you can use the GetActiveObject() API function to obtain the IDispatch pointer for the running instance. Once you have this IDispatch pointer for the running instance, you can use the methods and the properties of the running instance.
You might not like it, but i believe the solution is that there is one application interface, and that "first" application acts as the gateway to other "instances" of your application (i.e. your automation server).
i'm not an expert in out-of-process COM automation, but i think i've read enough to believe that's the (unfortunate) answer.
See also
Registering the Active Object with API Functions
How To Attach to a Running Instance of an Office Application
You do indeed need the Running Object Table (IRunningObjectTable). Ian's answer is largely correct.
http://msdn.microsoft.com/en-us/library/ms695276(v=VS.85).aspx
However, it is possible to have two distinguishable instances in the ROT, allowing both copies of your app to be accessed because their monikers are distinguished.
Martyn
What, at a minimum, should an application health-monitoring system do for you (the developer) and/or your boss (the IT Manager) and/or the operations (on-call) staff?
What else should it do above the minimum requirements?
Is monitoring the 'infrastructure' applications (ms-exchange, apache, etc.) sufficient or do individual user applications, web sites, and databases also need to be monitored?
if the latter, what do you need to know about them?
ADDENDUM: thanks for the input, i was really looking for application-level monitoring not infrastructure monitoring, but it is good to know about both
Whether the application is running.
Unusual cpu/memory/network usage.
Report any unhandled exceptions.
Status of various modules (if applicable).
Status of external components (databases, webservices, fileservers, etc.)
Number of pending background tasks (if applicable).
Maybe track usage of the application and report statistics on most/less used functionalities so you know where optimizations are most beneficial.
The answer is 'it depends'. Why do you need to monitor? How large is your operations staff? Do you need reporting? What is the application environment? Who cares if the application fails? Who cares if an exception happens? Are any of the errors recoverable? I could ask questions like these for a long time.
Great question.
We've been looking for some application-level monitoring solution for our needs some time ago without any luck. Popular monitoring solution are mostly addressed to monitor infrastrcture and - in my opinion - they are too complicated for a requirements of most of small and mid-sized companies.
We required (mainly) following features:
alerts - we wanted to know about
incident as fast as possible
painless management - hosted service wouldbe
the best
visualizations - it's good to know what is going on and take some knowledge from the data
Because we didn't find suitable solution we started to write our own. Finally we've ended with up-and-running service called AlertGrid. (You can check it for free of course.)
The idea behind it is to provide an easy way to handle custom monitoring scenarios. Integration API is very simple (one function with two required parameters). At the momment we and others are using it for:
monitor scheduled tasks (cron jobs)
monitor entire application logic execution
alert on errors in applications
we are also working on examples of basic infrastructure monitoring using AlertGrid
This is such an open ended question, but I would start with physical measurements.
1. Are all the machines I think are hosting this site pingable?
2. Are all the machines which should be serving content actually serving some content? (Ideally this would be hit from an external network.)
3. Is each expected service on each machine running?
3a. Have those services run recently?
4. Does each machine have hard drive space left? (Don't forget the db)
5. Have these machines been backed up? When was the last time?
Once one lays out the physical monitoring of the systems, one can address those specific to a system?
1. Can an automated script log in? How long did it take?
2. How many users are live? Have there been a million fake accounts added?
...
These sorts of questions get more nebulous, and can be very system specific. They also usually can be derived reactively when responding to phsyical measurements. Hard drive fill up, maybe the web server logs got filled up because a bunch of agents created too many fake users. That kind of thing.
While plan A shouldn't necessarily be reactive, it is the way many a site setup a monitoring system.
Minimum: make sure it is running :)
However, some other stuff would be very useful. For example, the CPU load, RAM usage and (in multiuser systems) which user is running what. Also, for applications that access network, a list of network connections for each app. And (if you have access to client computer(s)) it would be cool to be able to see the 'window title' of the app - maybe check each 2-3 minutes if it changed and save it. Also, a list of files open by the application could be very useful, but it is not a must.
I think this is fairly simple - monitor so that you can be warned early enough before something goes wrong. That means monitor dependencies and the application itself.
It's really hard to provide specifics if you're not going to give details on the application you're monitoring, so I'd say use that as a general rule.
At a minimum you want to know that the system is healthy. This is subjective in what defines your system is healthy. Is it computers are up, the needed resources exist, the data is flowing through the system, the data is properly producing results, etc, etc.
In my project we do monitoring of most of this and then some. It really comes down to what is the highest level that you can use to analyze that everything is working. In our case we need to know down to the data output. If you just need to know down to the are these machines up it saves you on trying to show an inexperienced end user what is wrong.
There are also "off the shelf" tools that will do a lot of the hard work for you if you are just looking too hard into data results. I particularly liked Nagios when I was looking around but we needed more than it could easily show so I wrote our own monitoring system. Basically we also watch for "peculiarities" in the system, memory / cpu spikes, etc...
thanks everyone for the input, i was really looking for application-level monitoring not infrastructure monitoring, but it is good to know about both
the difference is:
infrastructure monitoring would be servers plus MS Exchange Server, Apache, IIS, and so forth
application monitoring would be user machines and the specific programs that they use to do their jobs, and/or servers plus the data-moving/backend applications that they run to keep the data flowing
sometimes it's hard to draw the line - an oversimplified definition might be "if your team wrote it, it's an application; if you bought it, it's infrastructure"
i think in practice it is best to monitor both
What you need to do is to break down the business process of the application and then have the software emit events at major business components. In addition, you'll need to create end to end synthetic transactions (eg. emulating end users clicking on a website). All that data would be fed into an monitoring tool. In the past, I've done JMX for applications of which flowed into Tivoli Monitoring's JMX Adapter and then I've done scripts that implement a "fake user" and then pipe in the results into Tivoli Monitoring's Script Adapter. Tivoli Monitoring takes the data and then creates application health and performance charts from that raw data.