So I have log4j2 setup to log hourly. I know this happens in a background thread. Unfortunately, our products logs a lot, and zips over 500MB hourly. We notice a small hickup at each hour there is being logged, so it looks like the log4j2 background thread takes too much CPU. It is a small hickup, but unfortunately, this already causes internal errors to our product.
Is there any way to improve the performance of the log4j zipping? Could we for example, say to the background thread, that it is only allowed to use a certain percent of the CPU?
edit:
I've seen CPU usage going through the roof at the moment it happens, so it is definetely a CPU usage thing.
As far as I know there is no built-in way to avoid the high CPU usage on rollover so you have to implement that yourself.
log4j2 is designed to be easily extended. I cannot give you a full solution. But you should have a look at the org.apache.logging.log4j.core.appender.rolling.action.GzCompressAction class which performs the zipping (in case you are using gz). I think it should be possible to implement your own action class and put a delay to the zipping as you like.
The zip format allows a “compressionLevel” configuration attribute. This is something you can experiment with to see if it makes a difference. See the DefaultRolloverStrategy parameters in https://logging.apache.org/log4j/2.x/manual/appenders.html#RollingFileAppender
Another thing to try is simply zip smaller files. You can tell Log4j2 to rollover based on size. You’ll have more smaller log files, but you can post process them offline where it doesn’t disrupt the application.
Related
I'm considering overwriting the same small file 1,000's - 100,000's of times in an iOS app. Are there any downsides to this, given that flash memory is rated for 1000's of writes (but not, say, 100,000's)?
Will the system file cache save me if I stick to standard FileHandle operations? (without me having to implement my own such cache)
This has been addressed before: Reading/Writing to/from iPhone's Documents folder performance
Any new insights?
Update in response to some comments below: in general I agree with you that sometimes examining the choice of solution is more critical than helping with the proposed solution itself.
However, for this case, I feel the question is legit. Basically, it applies to any program where there is a small amount of very volatile data that needs to be persisted often: say, a position in a game, or a stock tick, or some counter, or the last key pressed, or something like that. It needs to be reliably read after process restart, so the app can pick up where it left off, hence the question:
Can I use the iOS file system for that? I know I can't write 10,000's of times to actual flash memory - that would burn it out. But will file system operations solve this for me, through some form of caching? Or do I need to do that myself, 'by hand'?
I sort of assume 'yes' (file system will solve) - otherwise other apps that do this (there must be some) would be burning out phones all the time! But: hard to know for sure...
Update again: asked this question on apple forums:
https://forums.developer.apple.com/thread/116740
Still no clear answer. Some answers are: just cache it yourself to avoid any such potential problems (and there can be: a file write can fail, and increasing the frequency increases the probability of failure in weird ways). Another is: iOS logs so much stuff, there's no way I can write more frequently than that, and that's fine, so no worries... I guess I'll leave this question open for now.
I am using gocraft/web in a project and am trying to debug some high memory usage. gocraft/web uses reflection to call handlers. I've set up the net/http/pprof profiler which works very well, but the largest block of memory, and the one that I am iterested in, only shows reflect.Value.call as the function. That's not very helpful.
How can I get around the fact that gocraft/web is using reflection and dig deeper into the memory profile?
Here's an example of the profile output I am seeing:
Thanks to #thwd for filing http://golang.org/issue/11786 about this. This is a display issue in pprof. All the data is there, just being hidden. You can get the data you need by invoking pprof with the -runtime flag. It will also show data you don't need, but it should serve as a decent workaround until Go 1.6 is out.
The short answer is that you can't directly. reflect.Value.call calls reflect.call which forwards to runtime.reflectcall which is an assembly routine implemented in the runtime, for example for amd64, here. This circumvents what the profiler can see.
Your best bet is to invoke your handlers without reflection and test them like that individually.
Also, enabling the profiler to follow reflective calls would arguably be an acceptable change to propose for the next Go iteration. You should follow the change proposal process for this.
Edit: issue created.
I have a part of my application that creates an export file. The export file process is fairly quick for the vast majority of users however, there are users that generate 10,000 or more records. This complicates things. First, the tool that imports the files, blows up on files larger than about 4,000 records. Secondly, the process for 10,000 records takes about 20 minutes. There has a tendency for the users to start doing other things and then for what ever reason, the process seems to time out and they never get their file. However, if you click the process button, and just leave your machine alone, 20 minutes later you will get the file.
I need to make this more user-friendly and robust. Here's my ideas:
1) automatically create separate files of 4,000 a pop
2) provide a status bar for the file generation
3) background the process so a user can click the button and come back say an hour later and download their files
So I have been doing research on the background plugins and gems. Most seem to be fairly out of date, which make me nervous and may seem to be major overkill for what I need. So Spawn seemed to be simple and straight forward but I'm unclear on how to do a status bar for that type of product.
Then we have something like Delayed_job. This seems like it would work but also seems a little heavy but it does provide the hooks to generate some kind of status update. Anyone have an example of this? The README is a little light.
Another issue is the file generation, how do I get this multiple files to download? Anyway, I can store the generated file for the live of the user session?
Finally, most of the solutions are looking like a major change, this issue is painful but technically works. So the time that I am being allotted to solve it is minimal so I am trying to KISS. Thanks for any help and or direction you can provide.
If your looking for background processing job I guess you must look for resque it supereasy run on redis as against delayed_job which poll your databases changes
as per gathering progress info I guess there bunch of resque plugin here one that can help you in the quest
Lastly
Another issue is the file generation, how do I get this multiple files to download? Anyway, I can store the generated file for the live of the user session?
Not sure what you actually meant but if you wanted multiple file to download can zipping into one can help
This has happened to me on more than one occasion and has led to many lost hours chasing a ghost. As typical, when I am debugging some really difficult timing-related code I start adding tons of OutputDebugString() calls, so I can get a good picture of the sequence of related operations. The problem is, the Delphi 6 IDE seems to be able to only handle that situation for so long. I'll use a concrete example I just went through to avoid generalities (as much as possible).
I spent several days debugging my inter-thread semaphore locking code along with my DirectShow timestamp calculation code that was causing some deeply frustrating problems. After having eliminated every bug I could think of, I still was having a problem with Skype, which my application sends audio to.
After about 10 seconds the delay between my talking and hearing my voice come out of Skype on the second PC that I was using for testing, the far end of the call, started to grow. At around 20 - 30 seconds the delay started to grow exponentially and at that point triggered code I have that checks to see if a critical section was being held too long.
Fortunately it wasn't too late at night and having been through this before, I decided to stop relentlessly tracing and turned off the majority of the OutputDebugString(). Thankfully I had most of them wrapped in a conditional compiler define so it was easy to do. The instant I did this the problems went away, and it turned out my code was working fine.
So it looks like the Delphi 6 IDE starts to really bog down when the amount of OutputDebugstring() traffic is above some threshold. Perhaps it's just the task of adding strings to the Event Log debugger pane, which holds all the OutputDebugString() reports. I don't know, but I have seen similar problems in my applications when a TMemo or similar control starts to contain too many strings.
What have those of you out there done to prevent this? Is there a way of clearing the Event Log via some method call or at least a way of limiting its size? Also, what techniques do you use via conditional defines, IDE plug-ins, or whatever, to cope with this situation?
A similar problem happened to me before with Delphi 2007. Disable event viewing in the IDE and instead use DebugView from Sysinternals.
I hardly ever use OutputDebugString. I find it hard to analyze the output in the IDE and it takes extra effort to keep several sets of multiple runs.
I really prefer a good logging component suite (CodeSite, SmartInspect) and usually log to various files. Standard files for example are "General", "Debug" (standard debug info that I want to collect from a client installation as well), "Configuration", "Services", "Clients". These are all set up to "overflow" to a set of numbered files, which allows you to keep the logs of several runs by simply allowing more numbered files. Comparing log info from different runs becomes a whole lot easier that way.
In the situation you describe I would add debug statements that log to a separate logfile. For example "Trace". The code to make "Trace" available is between conditional defines. That makes turning it on pretty simple.
To avoid leaving in these extra debug statements, I tend to make the changes to turn on the "Trace" log without checking it out from source control. That way, the compiler of the build server will throw out "identifier not defined" errors on any statements unintentionally left in. If I want to keep these extra statements I either change them to go to the "Debug" log, or put them between conditional defines.
The first thing I would do is make certain that the problem is what you think it is. It has been a long time since I've used Delphi, so I'm not sure about the IDE limitations, but I'm a bit skeptical that the event log will start bogging down exponentially over time with the same number of debug strings being written in a period of 20-30 seconds. It seems more likely that the number of debug strings being written is increasing over time for some reason, which could indicate a bug in your application control flow that is just not as obvious with the logging disabled.
To be sure I would try writing a simple application that just runs in a loop writing out debug strings in chunks of 100 or so, and start recording the time it takes for each chunk, and see if the time starts to increase as significantly over a 20-30 second timespan.
If you do verify that this is the problem - or even if it's not - then I would recommend using some type of logging library instead. OutputDebugString really loses it's effectiveness when you use it for massive log dumps like that. Even if you do find a way to reset or limit the output window, you'd be losing all of that logging data.
IDE Fix Pack has an optimisation to improve performance of OutputDebugString
The IDE’s Debug Log View also got an optimization. The debugger now
updates the Log View only when the IDE is idle. This allows the IDE to
stay responsive when hundreds of OutputDebugString messages or other
debug messages are written to the Debug Log View.
Note that this only runs on Delphi 2007 and above.
Looking for a little help getting started on a little project i've had in the back of my mind for a while.
I have log file(s) varying in size depending on how often they are cleaned from 50-500MB. I'd like to write a program that will monitor the log file while its actively being written to. when in use it's being changed pretty quickly easily several hundred lines a second or so. Most if not all of the examples i've seen for reading log/text files are simply open and read file contents into a variable which isn't really feasible to do every time the file changes in this situation. I've not settled on a language to write this in but its on a windows box and I can work in .net flavors / java / or php ( heh dont think php will fly to well for this), and can likely muddle through another language if someone has a suggestion for something well built for handling this.
Essentially I believe what I'm looking for would probably be better described to as a high speed way of monitoring a text file for changes and seeing what those changes are. Each line written is relatively small. (less than 300 characters, so its not big data on each line).
EDIT: to change the wording to hopefully better describe what i'm trying to do. Which is write a program to keep an eye on a log file for a trigger then match a following action to that trigger. So my question here is pertaining to file handling inside a programming language.
I greatly appreciate any thoughts/comments.
If it's incremental then you can just read the whole file the first time you start analyzing logs, then you keep the current size as n. Next time you check (maybe a timed action to check last modified date) just skip first n bytes, read all new bytes and update size.
Otherwise you could use tail -f by getting its stdout and using it for your purposes..
The 'keep an eye on a log file' part of what you are describing is what tail does.
If you plan to implement it in Java, you can check this question: Java IO implementation of unix/linux "tail -f" and add your trigger logic to lines read.
I suggest not reinventing the wheel.
Try using the elastic.co
All of these applications are open source and free and are capable of monitoring (together) and trigger actions based on input.
filebeats - will read the log file line by line (supports multiline log messages as well) and will send it across to logstash. There are loads of other shippers you can use.
logstash - will take the log messages, filter them, add tags and send the messages to elasticsearch
elasticsearch - will take the log messages and index them, the store them. It is also capable of running actions based on input
kibana - is a user friendly web interface to query and analyze the data. Or just simply put it up on a dashboard.
Hope this helps.