What the difference in ArcSight's CEF Syslog configuration between RAW TCP and UDP ?
is it possible that Udp syslog is sending without endline eg \n
How it possibole to send the syslog from ArcSight that Flumes syslogtcp will read it as syslog
The difference
The RAW syslog - Send the information over TCP protocol, the "RAW" is simply sends CEF payload via TCP as original data (without normalization) with \n ending
CEF:0.......
CEF:0.......
The UDP syslog - Send the information over UDP protocol, the "RAW" is simply sends CEF payload via TCP without \n ending - looks like stream
CEF:0.......CEF:0.......
In my tests the ArcSight version 6.9.1 not sending Syslog format as should
Should be:
Sep 10 15:19:01 host CEF:0|.............
Actualy:
CEF:0|..........
According https://activate.lab1.semplicityinc.com/foswiki/pub/ArcSightActivate/PLinuxOSConnectorInstallation/SyslogNGDaemonConfig.pdf and https://www.protect724.hpe.com
When selecting Syslog Daemon, with Raw TCP, connections remain idle in a CLOSE_WAIT state until
closed explicitly by the application.
More reference:
https://www.hpe.com/h20195/v2/getpdf.aspx/4AA6-8664ENW.pdf?ver=1.0
Apache Flume
The best practices for ArcSight version 6.9.1 and Flume is:
in ArcSight use CEF Syslog and RAW Tcp
Flume should use NetCat source and parse it
Enjoy !
Related
I use TIdUDPServer in this scenario:
Host 192.168.1.10: IdUDPServer1.Active := True using dynamic port (e.g.: 60112)
Host 192.168.1.20: IdUDPServer2.Active := True using dynamic port (e.g.: 50332)
When I invoke IdUDPServer1.Send('192.168.1.20', 50332, 'Hello'), the host
192.168.1.20 didn't receive any UDP message.
I then invoke IdUDPServer2.Send('192.168.1.10', 60112, 'Hello'), the host 192.168.1.10 receive the UDP message. Both hosts are able to receive UDP messages from each other then.
Similarly, if I send message from IdUDPServer2 first, the IdUDPServer1 will not receive any UDP message.
This strange problem always happens to UDP server that first send a message.
This problem doesn't happen if both IdUDPServer1 and IdUDPServer2 started in same host (e.g.: localhost or same pc).
I didn't assign any port number to both TIdUDPServer instance and let the library to assign a dynamic port for me. Hence, Windows 10 didn't prompt any Firewall warning.
I am new to using Wireshark. Can I install Wireshark on server which is hosting Web Service to capture incoming requests and out going responses?
Example end point URL of my Web Service: http://MyIP:9086/WebService
For example my web service is using 9086 port. If I start capturing traffic on 9086, will it give me all request and response (SOAP messages)?
I have installed Wireshark on local laptop and can packets when SOAP UI send request to Web Service. But I want to install it on server and want to capture from that end. Is that feasible?
If the server is a linux box, you can use tcpdump, and tell it dump the traffic into a pcap file. This pcap file you can transfer to a local machine and load into wireshark.
From https://www.wireshark.org/docs/wsug_html_chunked/AppToolstcpdump.html
D.3. tcpdump: Capturing with tcpdump for viewing with Wireshark
It’s often more useful to capture packets using tcpdump rather than wireshark. For example, you might want to do a remote capture and either don’t have GUI access or don’t have Wireshark installed on the remote machine.
Older versions of tcpdump truncate packets to 68 or 96 bytes. If this is the case, use -s to capture full-sized packets:
$ tcpdump -i <interface> -s 65535 -w <some-file>
You will have to specify the correct interface and the name of a file to save into. In addition, you will have to terminate the capture with ^C when you believe you have captured enough packets.
I have following script to get given file from given remote directory by accepting following parameters
Host Name that you are connecting to get File
User Name of the Host
Local Directory where you wanted to transfer file
Remote Directory from where you wanted to get file
File name that you wanted to get from Remote server
FSERVER=$1
FUSER=$2
SRC_DIR=$3
REMOTE_SRC_DIR=$4
FILE_NAME=$5
cd $SRC_DIR
sftp $FUSER#$FSERVER <<GOTO
cd $REMOTE_SRC_DIR
ascii
get $FILE_NAME
bye
To access the files from $REMOTE_SRC_DIR to SRC_DIR do I need port open from both side? I.e. bi-directional or just one port from Remote Server to Source and it should need "INITIATE" session from the source. And what is the reason?
As per my understanding we are connecting to remote server path and then writing the query Get File name. So we need to bi-directional access.
SFTP uses a single TCP connection. In general, TCP connection is stateful. As such, once opened both sides can send data to each other. Only the passive side of the connection needs to initially have a well known port number opened (22 for SSH/SFTP in this case). The active side opens a random port number that the passive side learns from the TCP connection initiation packed. This passive-side port closes with the TCP connection. While the active-side port is kept open for future TCP connections.
The SFTP protocol uses strictly request-response model. I.e. although the TCP allows both sides to send data anytime, with the SFTP, the server never sends data on its own, but always in a response to client request. Note that this does not mean, that no unsolicited data flows from the server to the client on network level, as in both underlying protocols of the SFTP (the TCP and the SSH) both sides of connection can send (and send) packets anytime.
Simplified flow is:
SFTP client initiates TCP connection to remote port 22 (this causes implicit open of random local port on client side, this is done by operating system).
SSH protocol initialization and authentication occurs.
SFTP client requests SSH server to start SFTP server. Note that SFTP server is not a continuously running process. It is a sub-process/sub-service of SSH server, which is continuously running (=listening on port 22)
SFTP protocol initialization occurs.
SFTP (contrary to FTP protocol) is stateless, as such it does not have a concept of a working directory. As such changing remote working directory with the cd command is simulated on client side. The SFTP server is not aware at all of client remote working directory. SFTP client typically only verifies existence of the new working directory with the SFTP server.
The ascii command: The OpenSSH sftp client does not have ascii command. You should get "Invalid command." Unless you use other client than OpenSSH.
The get command: For file transfers the SFTP protocol offers a similar block-level API as most operating systems (contrary to a stream API of FTP protocol). So SFTP client sends "open file" request, over the existing connection, followed by repetitive "read block" requests and "close file" request. As with any SFTP requests, responses go back over the same TCP connection.
At the end, the TCP connection is terminated and connection-specific random local port is closed.
I made a C# program with a client and a server that comunicates fine between them. Information is received by the server and stored in a MySQL Database with no problems. However I wanted to use wireshark to monitor how safe/encrypted was the information and couldn't find the communication using wireshark. My program uses sockets to connect the server and client, on my test the server and client are running at adress 127.0.0.1 or local IP machine (something like 192.168.13.191), server listens/connects using port 3608 and client a much higher random port, and all the filters I used on wireshark return nothing.
So far I tried:
ip.addr == 127.0.0.1 (show only packages whose origin or destination are 127.0.0.1)
ip.addr == 192.168.13.191 (same as above)
tcp.port == 3608 (show only packages whose origin or destination is this port)
tcp.port == client_port (same as above)
tcp contains 01:00:00:3B (only packages with a few of the bytes I'm sending)
My idea was to find at least one message from the client to server and use the "Follow TCP stream" option from wireshark, but all these filters return a black screen when used individually. I have no idea why, because when I run netstat -a on a windows command I can see the program have a established connection and is also listening for new connections on port 3608. Anyone have any idea what is going on or what filter should I use? Operational system somehow is "redirecting" traffic from port 3608 to another?
Thanks!
Wireshark doesn't pick up loopback packets. You'll have to install the client or server on another machine and then try again.
EDIT: After doing some reading I've discovered that this really doesn't have anything to do with Wireshark but with the way WinPcap works. In any case, running the client or server on another machine will solve the problem.
I want pulseaudio (remote instance) to push audio sent to the primary alsa device straight to RTP over the internet so that I could listen to it on VLC on my home computer.
in my /etc/pulse/default.pa
load-module module-rtp-send source=alsa_output.0.analog-stereo.monitor destination=x.x.x.x port=8080 loop=1
Where x.x.x.x is the internet IP of my server
After starting pulseaudio,netstat shows (I cannot connect from remote VLC to this)
udp 0 0 10.170.94.16:58606 x.x.x.x:9875 ESTABLISHED 2109/pulseaudio
udp 0 0 10.170.94.16:35597 x.x.x.x:8080 ESTABLISHED 2109/pulseaudio
It works with cvlc to produce the intended results (I can connect from remove VLC to this)
cvlc -vvv pulse://alsa_output.0.analog-stereo.monitor --sout '#transcode{acodec=mpga,ab=32,channels:1}:rtp{sdp=rtsp://0.0.0.0:8080/test.sdp}'
which produces
tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 1944/pulseaudio
And I can connect to it.
So what I think I understand is that pulseaudio is working correctly, and is sending the audio to port 8080. However, it's not listening for incoming connections to route the data to. How would I tell pulseaudio, or set up the routing so that port 8080 transmits the audio provided by rtp when an inbound connection occurs. I also see that cvlc is using rtsp as the source.
Still a bit confused..
RTP uses the stateless UDP protocol as a transport: it does not require the destination end to acknowledge any incoming packets. It cannot tell you if the connection has actually been established, other than if the remote end explicitly rejected it. This feature allows for doing broadcast on a LAN because multiple clients can listen to the packets as they wish, but it makes debugging unicast connections slightly more complicated.
When doing unicast UDP (like you are trying to achieve), once it's out on the Internet, it should get to your destination. Do you have a firewall? Unless your machine is connected directly to your modem, it is very likely that you have at least network address translation which would prevent the packets from reaching your computer. Look for port forwarding in your gateway/router configuration.
You can test if you are actually receiving the packets by using tcpdump. When running it, you will see some noise from other connections, along with packets that look like this:
08:19:38.483895 IP y.y.y.y.zzzz > x.x.x.x:8080: UDP, length 1292
Also, PulseAudio does not use RTSP, so VLC needs to open an RTP stream only. Simply typing rtp://0.0.0.0:8080 should do it.