Is there any way that I can send a bunch of existing log files (json content) to graylog? I found few posts googling, but all deal with sending live data to graylog, and unable to find any instructions on how to send existing log file
There are various options for sending existing log messages (text files) to Graylog.
The most basic option to send line-delimited log messages (i. e. not multiline) would be to create a Raw/Plaintext TCP input and send the complete file using something like netcat (nc, netcat, ncat, socat, etc.).
# Raw/Plaintext TCP on port 5555/tcp
$ nc graylog.example.org 5555 < /path/to/file
If you like it a bit more fancy (with some preprocessing of the log file and buffering of log messages), you could create a Beats input and use Filebeat to send the file.
Related
I am trying to use TwinCat 3 with TcAnalytics to create a Data Logger which logs values to MQTT. I have created this Data Logger, and set the settings to 127.0.0.1 and also created the same connection in my Target Browser under TcAnalytics.
The problem is that the PlcStream1 created for the Data Logger is showing 'Disconnected' in the Online tab at the same time as the Target Browser connection shows everything is fine and connected properly.
I have seen a video of someone setting this up and the data stream from the Data Logger appears nested under the connection in the Target Browser once it's setup. For me this does not show up either.
I have tested my local MQTT server with a third party tool that can publish and subscribe to it just fine.
Why won't the Data Logger connect to MQTT and publish data to it?
My setting in the Data Logger 1 Parameter (Init) tab look accurate to me. They are shown below.
Does anyone know how to get TcAnalytics to log data to MQTT? It seems like a really straight forward setup but it's just not connecting.
The problem here was that my MQTT server was not serving on all interfaces. Contrary to some data I found, Mosquitto doesn't serve to all interfaces by default. I needed to add this to my mosquitto.conf file.
listener 1883
allow_anonymous true
I am writing a ROS Node, and I am currently using ROS_INFO and ROS_ERROR for messages that are shown directly on the console. Now I want to switch to syslog and I want to use the syslog function for C.
That already works fine when I duplicate the log message and send it with syslog and ROS_INF/ERROR at the same time. But now I always have two line codes for the same error message in the code. Is there an easy way to show the syslog messages also on the console?!
Br Harald
Seems like rosconsole has multiple backends and you might be able to change that at compile time. Take a look at this thread
I have two graylog servers, one on an older version of graylog (server 1) that is receiving logs (version 1).
I have another graylog server (version 2), let's call it server 2 that I would like to send all the logs received at server 1 to. I would like to have a way to send all the logs received by server 1 to server 2...
You can create a "Catchall" stream which will include all ingested messages (e. g. check for the presence of the timestamp message field) and then assign a GELF output to that stream.
This would, additionally to indexing everything in ES, send all messages to your second Graylog cluster.
I am using Cocoa async socket library in my iOS application to make connection with an IRC server via Tcp sockets. All the IRC commands are working perfectly except /TOPIC command.
When I send a topic command it always trim the first two characters in the response. Please check the example wireshark report below.
It works on other IRC clients (mIRC). How can I identify the root cause of the issue?
If you send your own IRC commands (most IRC clients let you easily inject arbitrary commands with /command here), you have to format it correctly, with a : before the final parameter if it contains spaces:
TOPIC #abcdereh :My name is Clement
Most IRC clients have an alias for /TOPIC that does that stuff for you.
If you are writing your own client, make sure that you handle the following cases:
/TOPIC: send TOPIC #currentchannel
/TOPIC #somechannel send TOPIC #somechannel
/TOPIC Not a channel send TOPIC #currentchannel :Not a channel
/TOPIC #somechannel New Topic send TOPIC #somechannel :New Topic
I am trying to hash out how I would create an e-mail parser. I understand technically how to do it, but I cannot figure out implementation details.
So, user sends an e-mail to an address, mail server receives and my app parses it based upon subject, content and drops it in a bucket (e-mail account or database) and then I can act upon it.
So do I use an existing mail server software (like Zimbra, which we already have running) or do I create an app that listens on port 25 and does specifically what I need? (meaning no mail server sofware running on this box, etc)
My goal here is to create myself a series of organization tools for personal use in an automated way based upon what I e-mail myself.
Writing something to listen on port 25 and act as an SMTP server will be involved and probably overkill for what you want.
I think there are two main options. The first is to leave your existing mail server in place and then poll an account on that mail server over IMAP (or POP3) to retrieve the emails and then process them using a script. It really doesn't matter what language you're comfortable with as there are libraries for handling IMAP connections and then parsing the email in most languages.
Alternatively you could look at a service like http://CloudMailin.com that does this for you. It will receive the email and send it to a web app that you could create via an http post in something like JSON format.
I would go for a python script which polls the mailbox (basing on a cron job). Python allows you to access IMAP very easily and has powerful regular expression functions to parse the email content.
Try something like:
import imaplib, email
import re
M= imaplib.IMAP4_SSL('imap.gmail.com')
M.login('user', 'pass')
M.select('Imap_folder')
typ, data = M.search(None, 'FROM', '"*"')
for num in data[0].split():
typ, data = M.fetch(num, '(RFC822)')
email_body = data[0][1] # getting the mail content
mail = email.message_from_string(email_body) # parsing the mail content to get a mail object
foo = re.compile("your regular expr here", re.MULTILINE)
res = foo.search(email_body)