I previously posted a question entitled "Writing a Wireshark Dissector to Count Number of TCP Flows." I got some feedback to use Lua/Tap instead so I set out to write one but I need assistance with the code. I currently have the following functions that a tap must have:
Listener.new,
listener.packet,
listener.draw,
listener.reset.
To get a better understanding of what I want to do, please review my previous question here:
Writing a Wireshark dissector to count number of TCP flows
My new question is, would I need to write a code to do the equivalent of the tshark's command:
tshark -r 1min.pcap -q -n -z conv,tcp
in Lua/Tap to extract the statistics information first before I proceed to write code to count the TCP flows? Or all I need to do is write a code in Lua/Tap to to extract the TCP flow count. In either case, can someone help me with the code? I've search the web but can't find an example close to what I'm looking for so I can customize to suit what I'm trying to achieve. Thanks.
I don't have time to write code for you, but here's some information copied from an edit I made to the answer to your other question:
The C structure passed to TCP taps for each packet is:
/* the tcp header structure, passed to tap listeners */
typedef struct tcpheader {
guint32 th_seq;
guint32 th_ack;
gboolean th_have_seglen; /* TRUE if th_seglen is valid */
guint32 th_seglen;
guint32 th_win; /* make it 32 bits so we can handle some scaling */
guint16 th_sport;
guint16 th_dport;
guint8 th_hlen;
guint16 th_flags;
guint32 th_stream; /* this stream index field is included to help differentiate when address/port pairs are reused */
address ip_src;
address ip_dst;
/* This is the absolute maximum we could find in TCP options (RFC2018, section 3) */
#define MAX_TCP_SACK_RANGES 4
guint8 num_sack_ranges;
guint32 sack_left_edge[MAX_TCP_SACK_RANGES];
guint32 sack_right_edge[MAX_TCP_SACK_RANGES];
} tcp_info_t;
So, for C-language taps, the "data" argument to the tap listener's "packet" routine points to a structure of that sort.
For Lua taps, the "tapinfo" table passed as the third argument to the tap listener's "packet" routine is described as "a table of info based on the Listener's type, or nil.". For a TCP tap, the entries in the table include all the fields in that structure except for sack_left_edge and sack_right_edge; the keys in the table are the structure member names.
The th_stream field identifies the connection; each time the TCP dissector finds a new connection, it assigns a new value. As the comment indicates, "this stream index field is included to help differentiate when address/port pairs are reused", so that if a given connection is closed, and a later connection uses the same endpoints, the two connections have different th_stream values even though they have the same endpoints.
So you'd have a table using the th_stream value as a key. The table would store the endpoints (addresses and ports) and counts of packets and bytes in each direction. For each packet passed to the listener's "packet" routine, you'd look up the th_stream value in the table and, if you don't find it, you'd create a new entry, starting the counts off at zero, and use that new entry; otherwise, you'd use the entry you found. You'd then figure out whether the packet was going from A to B or B to A, and increase the appropriate packet count and byte count.
You'd also keep track of the time stamp. For the first packet, you'd store the time stamp for that packet. For each packet, you'd look at the time stamp and, if it's one minute or more later than the stored time stamp, you'd:
dump out the statistics from the table of connections;
empty out the table of connections;
store the new packet's time stamp, replacing the previous stored time stamp.
Related
After that I send different runicast messages with the function runicast_send, how can I understand which message was acknowledged when the callback sent_runicast is triggered?
The runicast.h file states:
The runicast primitive adds two packet attributes: the single-hop
packet type and the single-hop packet ID. The runicast primitive
uses the packet ID attribute as a sequence number for matching
acknowledgement packets to the corresponding data packets.
but I didn't understand how to do it in practice. Can somebody provide an example?
One way would be to look at the field sndnxt of struct runicast_conn *c before you send the packet, and then compare that value of the packetbuf_attr(PACKETBUF_ATTR_PACKET_ID) in the "sent" callback of your code.
However note that by default the runicast packet ID is just 2 bits long. Enough to demultiplex the ACK in most cases, but may be insufficient for your purposes. (The packet ID size in bits can be changed by redefining RUNICAST_PACKET_ID_BITS.)
Also Rime is obsolete. Don't use it in your code, especially production code unless you know what you're doing. runicast was never one of the highlights of Rime, I doubt there are no better alternatives (e.g. the uIPv6 stack) for what you want to do.
I have a very large tcpdump file that I split into 1 minute intervals. I am able to use tshark to extract TCP statistics for each of the 1 minute files using a loop code and save the results as a CSV file so I can perform further analysis in Excel. Now I want to be able to count the number of TCP flows in each 1 minute file for all the 1 minute files and save the data in a CSV file. A TCP flow here represents group of packets going from a specific source to a specific destination. Each flow has statistics such as source IP, dest IP, #pcakets from A->B, #bytes from A->B, #packets from B->A, #bytes from B->A, total packets, total bytes, etc. And I just want to count the number of TCP flows in each of the 1 minute files. From what I’ve read so far, it seems I need to create a dissector to do that. Can anyone give me pointers or code on how to get started? Thanks.
Tshark has a command to dump all of the necessary information: tshark -qz conv,tcp -r FILE. This writes one line per flow (plus a header and footer) so to count the flows just count the lines and subtract the header/footer.
Not a dissector, but a tap. See the Wireshark README.tapping document, and see the TShark iousers tap for a, sadly, not at all simple example in C.
It's also possible to write taps in Lua; see, for example, the Lua/Taps page in the Wireshark Wiki and the Lua Support in Wireshark section of the Wireshark User's Manual.
The C structure passed to TCP taps for each packet is:
/* the tcp header structure, passed to tap listeners */
typedef struct tcpheader {
guint32 th_seq;
guint32 th_ack;
gboolean th_have_seglen; /* TRUE if th_seglen is valid */
guint32 th_seglen;
guint32 th_win; /* make it 32 bits so we can handle some scaling */
guint16 th_sport;
guint16 th_dport;
guint8 th_hlen;
guint16 th_flags;
guint32 th_stream; /* this stream index field is included to help differentiate when address/port pairs are reused */
address ip_src;
address ip_dst;
/* This is the absolute maximum we could find in TCP options (RFC2018, section 3) */
#define MAX_TCP_SACK_RANGES 4
guint8 num_sack_ranges;
guint32 sack_left_edge[MAX_TCP_SACK_RANGES];
guint32 sack_right_edge[MAX_TCP_SACK_RANGES];
} tcp_info_t;
So, for C-language taps, the "data" argument to the tap listener's "packet" routine points to a structure of that sort.
For Lua taps, the "tapinfo" table passed as the third argument to the tap listener's "packet" routine is described as "a table of info based on the Listener's type, or nil.". For a TCP tap, the entries in the table include all the fields in that structure except for sack_left_edge and sack_right_edge; the keys in the table are the structure member names.
The th_stream field identifies the connection; each time the TCP dissector finds a new connection, it assigns a new value. As the comment indicates, "this stream index field is included to help differentiate when address/port pairs are reused", so that if a given connection is closed, and a later connection uses the same endpoints, the two connections have different th_stream values even though they have the same endpoints.
So you'd have a table using the th_stream value as a key. The table would store the endpoints (addresses and ports) and counts of packets and bytes in each direction. For each packet passed to the listener's "packet" routine, you'd look up the th_stream value in the table and, if you don't find it, you'd create a new entry, starting the counts off at zero, and use that new entry; otherwise, you'd use the entry you found. You'd then figure out whether the packet was going from A to B or B to A, and increase the appropriate packet count and byte count.
You'd also keep track of the time stamp. For the first packet, you'd store the time stamp for that packet. For each packet, you'd look at the time stamp and, if it's one minute or more later than the stored time stamp, you'd:
dump out the statistics from the table of connections;
empty out the table of connections;
store the new packet's time stamp, replacing the previous stored time stamp.
Going to do some statistics operation on a trace with Lua. Each IP packet can have multiple TCAP and each TCAP may have multiple CAP operation, like
IP {[SCTP-M3UA-SCCP-TCAP-CAP,CAP] [SCTP-M3UA-SCCP-TCAP-CAP,CAP,CAP]}
Now I want to access the whole tree or iterate somehow in TCAP layer in Lua listener tap. The purpose of this kind of iteration is that something like follow TCP stream because the transaction ID is kept in TCAP layer while operation and parameters in sequence of Camel (CAP) layer should be considered.
How can I access the dissector tree in listener tap or dissect upper layer if get the lower layer data part ?
For example the node ID come in first operation of 1 new session in highest layer(CAP) along with another sessions in same packet. Then another parameter that needed to be counted comes in another operation/packet, while same TID in TCAP to be checked to be sure on it belongs to same node.
It is not a reply to your question, only tip.
You should use array for fields extractor instead of plain value.
For example:
tap_diameter = nil
diaSessionIdExtr = Field.new("diameter.Session-Id")
tap_diameter = Listener.new("frame", "diameter && !tcp.analysis.retransmission && !tcp.analysis.lost_segment")
function tap_diameter.packet(pinfo,tvb,userdata)
local answers = {diaSessionIdExtr()} -- this is how to do it
for i in pairs(answers) do
debug(answers[i])
end
end
I want to send files(text or binary) through winsock,I have a buffer with 32768 byte size, In the other side the buffer size is same,But when the packet size <32768 then i don't know how determine the end of packet in buffer,Also with binary file it seems mark the end of packet with a unique character is not possible,Any solution there?
thx
With fixed-size "packets," we would usually that every packet except the last will be completely full of valid data. Only the last one will be "partial," and if the recipient knows how many bytes to expect (because, using Davita's suggestion, the sender told it the file size in advance), then that's no problem. The recipient can simply ignore the remainder of the last packet.
But your further description makes it sound like there may be multiple partially full packets associated with a single file transmission. There is a similarly easy solution to that: Prefix each packet with the number of valid bytes.
You later mention TCustomWinSocket.ReceiveText, and you wonder how it knows how much text to read, and then you quote the answer, which is that it calls ReceiveBuf(Pointer(nul)^, -1)) to set the length of the result buffer before filling it. Perhaps you just didn't understand what that code is doing. It's easier to understand if you look at that same code in another context, the ReceiveLength method. It makes that same call to ReceiveBuf, indicating that when you pass -1 to ReceiveBuf, it returns the number of bytes it received.
In order for that to work for your purposes, you cannot send fixed-size packets. If you always send 32KB packets, and just pad the end with zeroes, then ReceiveLength will always return 32768, and you'll have to combine Davita's and my solutions of sending file and packet lengths along with the payload. But if you ensure that every byte in your packet is always valid, then the recipient can know how much to save based on the size of the packet.
One way or another, you need to make sure the sender provides the recipient with the information it needs to do its job. If the sender sends garbage without giving the recipient a way to distinguish garbage from valid data, then you're stuck.
Well, you can always send file size before you start file transfer, so you'll know when to stop writing to file.
When writing a dissector in Wireshark, is the dissect function in the dissector's source called on each packet in order, only once?
What could be possible reasons for tree values changing as I click on packets multiple times?
It is called once when the packet is first to display the high level information.
if (check_col(pinfo->cinfo, COL_PROTOCOL))
or
if (check_col(pinfo->cinfo,COL_INFO))
And called again when showing the body, ie when you click on that one packet.
if (tree)
I'd assume that the second call results are discarded, as if you have a large number of packets to decode keeping the details for each would be too large an overhead.
But as always some quick testing would be able to show if this is the case. (via a static counter)