guys
I met a problem.I use logg4j and apache-flume to collect logs.the architecture is use logg4j remote print,the config like this:
log4j.appender.flume=org.apache.flume.clients.log4jappender.Log4jAppender
log4j.appender.flume.Hostname=192.168.152.49
log4j.appender.flume.Port=44446
log4j.appender.flume.layout=org.apache.log4j.PatternLayout
while the configure of flume like this:
a1.sources.r1.type=avro
a1.sources.r1.bind=192.168.152.49
a1.sources.r1.port=44446
it works!but the question is when the flume closed.the application which use logg4j can't print log!so is anybody can tell me.
how to fix this problem
It depends on how you want to handle Flume being down. With the regular Log4jAppender, you can enable unsafe mode which will log the error in the log4j LogLog, but otherwise fail silently. To do that you can set log4j.appender.flume.UnsafeMode = true. You can see an example here:
https://github.com/kite-sdk/kite-examples/blob/master/logging/src/main/resources/log4j.properties#L20
With unsafe enabled, any events you log while Flume is down will be lost.
If you want to be able to point to multiple Flume agents and have it balance the load between them as well as fail over if one of them goes down, you can use the LoadBalancingLog4jAppender instead. The docs here should help:
http://flume.apache.org/FlumeUserGuide.html#load-balancing-log4j-appender
Related
When using Apache Beam (GCP Dataflow) I see the following warning in worker logs:
No session file found: /var/opt/google/dataflow/pickled_main_session.
Functions defined in __main__ (interactive session) may fail.
My Dataflow job seems to be fine regardless, but I'm wondering what this warning is all about.
I have seen the following in some sample code (which I am NOT currently doing):
pipeline_options.view_as(SetupOptions).save_main_session = True
where pipeline_options is the main way of specifying options for the Beam/Dataflow pipeline, as in the following later in the code:
with beam.Pipeline(options=pipeline_options) as p:
# actual pipeline code here
I am curious if the two are related. Does the presence of the warning mean I should always be saving the main session? Are these two things related? Unrelated?
You should be able to safely ignore this warning. No need to set save_main_session if it's not required for your pipeline.
I have a situation where I want to import my graph data to database.I am having janusgraph(latest version) running with cassandra(version 3) and elasticsearch(version 6.6.0) using Docker.I have been suggested to use gryo format.So I have tried this command
graph.io(IoCore.gryo()).reader().create().readGraph(ToInputStream.from("my_graph.kryo"), graph);
but ended up with an error
No such property: ToInputStream for class: Script4
The documentation I am following is here.Please take a look and put me in a right procedure. Thanks in advance!
ToInputStream is not a function of Gremlin or JanusGraph. I believe that it is only a function of IBM Compose so unless you are running JanusGraph on that specific platform, this command will not work.
Versions of JanusGraph that utilize TinkerPop 3.4.x will support the io() step and this is the preferred manner in which to load gryo (as well as graphson and graphml) files.
Graph graph = ... // setup JanusGraph instance
GraphTraversalSource g = traversal().withGraph(graph); // might use withRemote() here instead depending on how you are connecting I suppose
g.io("graph.kryo").read().iterate()
Note that if you are connecting remotely - it seems you are sending scripts to the Docker instance given your error - then be sure that that "graph.kryo" file path is accessible to Docker. That's what's nice about ToInputStream from Compose as it allows you to access remote sources.
Keep forgetting that Logging chapter of the the Kernel User's Guide already has an answer.
Paraphrasing the 2.9 Example: Add a handler to log info events to file section in the Logging chapter of the Kernel User's Guide:
1. Set log level (default: notice)
Globally: logger:set_primary_config/2
For certain modules only: logger:set_module_level/2
Accepted log levels (from least severe to most):
debug, info, notice, warning, error, critical, alert, emergency
Note:
The default log level in the Erlang shell is notice, so if you leave it as is, but set a lower level (such as debug or info) when adding a log handler in the next step, those level of logs will never get through.
Example:
logger:set_primary_config(level, debug).
2. Configure and add log handler
Specify the handler configuration map, for example:
Config = #{config => #{file => "./sample.log"}, level => debug}.
And add the handler:
logger:add_handler(to_file_handler, logger_std_h, Config).
logger_std_h is the standard handler for Logger.
3. Filter out logs below a certain level on Erlang shell
Following the examples above, all levels of logs will be printed. To restore the notice default, but still save every level of logs in the file), use logger:set_handler_config/3.
logger:set_handler_config(default, level, notice).
Work in progress: Log each process' events into their own logfile
This module documents my (partially successful) attempts; will revisit and expand on this section when time permits. My use case was that the FreeSWITCH phone server would spawn an Erlang process to handle the call, and so logging each of them into their own files made sense at the time.
I am writing a small program in Progress that needs to write an error message to the system's standard error. What ways, simple if at all possible, can I use to print to standard error?
I am using OpenEdge 11.3.
When on Windows (10.2B+) you can use .NET:
System.Console:Error:WriteLine ("This is an error message") .
together with
prowin32 2> stderr.out
Progress doesn't provide a way to write to stderr - the easiest way I can think of is to output-through an external program that takes stdin and echoes it to stderr.
You could look into LOG-MANAGER:WRITE-MESSAGE. It won't log to standard output or standard error, but to a client-specific log. This log should be monitored in any case (specifically if the client is an application server).
From the documentation:
For an interactive or batch client, the WRITE-MESSAGE( ) method writes the log entries to the log file specified by the LOGFILE-NAME attribute or the Client Logging (-clientlog) startup parameter. For WebSpeed agents and AppServer servers, the WRITE-MESSAGE() method writes the log entries to the server log file. For DataServers, the WRITE-MESSAGE() method writes the log entries to the log file specified by the DataServer Logging (-dslog) startup parameter.
LOG-MANAGER:WRITE-MESSAGE("Got here, x=" + STRING(x), "DEBUG1").
Will write this in the log:
[04/12/05#13:19:19.742-0500] P-003616 T-001984 1 4GL DEBUG1 Got here, x=5
There are quite a lot of options regarding the LOG-MANAGER system, what messages to display, where the file is placed, etc.
There is no easy way, but in Unixen you can always do something like this using OUTPUT THROUGH (untested):
output through "cat >&2" no-echo unbuffered.
Alternatively -- and this is tested -- if you just want error messages from a batch-mode program to go to standard out then
output through "tee" ...
...definitely works.
Say I want to use FTP in Python using the ftplib. I begin with this:
from ftplib import ftp
ftp = FTP('10.10.10.151')
If the FTP server is not online, however, it will hang right there indefinitely. The only thing that can kick it out is a keyboard interrupt as far as I know. I've tried this:
ftp.connect('10.10.10.151','21', 5)
With the five being a five second timeout. But the problem here is that I do not know of any way to use that line without first assigning ftp something. But if the server is offline, then the "ftp =" line will hang. So what use is ftp.connect()'s timeout function?!?
Does anybody know a workaround or anything? Is there a way to time out the "ftp = FTP(xxx)" command that I haven't found? Thanks.
I'm using Python 2.7 on Linux Mint.
Your call to connect() is redundant since FTP() method documentation states:
When host is given, the method call connect(host) is made.
Also, since Python 2.6, FTP() does have a timeout parameter:
class ftplib.FTP([host[, user[, passwd[, acct[, timeout]]]]])
The optional timeout parameter specifies a timeout in seconds for blocking operations like the connection attempt (if is not specified, the global default timeout setting will be used).