Payload dumper
Contents
Introduction
This tutorial discusses the plugin payloadDumper, a tcpflow replacement.
A plugin which does not match with the T2 traffic mining philosophy. You select first the relevant flows and
then look into more detail. But a good lad we know wanted it badly, so we wrote it. In future it
will do much more than tcpflow, e.g., layer 2, etc
Preparation
First, restore T2 into a pristine state by removing all unnecessary or older plugins from the plugin folder ~/.tranalyzer/plugins:
t2build -e -y
Are you sure you want to empty the plugin folder '/home/wurst/.tranalyzer/plugins' (y/N)? yes
Plugin folder emptied
Then compile the following plugins:
t2build tranalyzer2 basicFlow payloadDumper txtSink
...
BUILD SUCCESSFUL
If you did not create a separate data and results directory yet, please do it now in another bash window, that facilitates your workflow:
mkdir ~/data ~/results
The PCAP used in this tutorial can be downloaded here: faf-exercise.pcap.
Please save it in your ~/data folder.
Now you are all set for dumping payload!
payloadDumper
Let’s look at the plugin default configuration first:
payloadDumper
vi src/payloadDumper.h
...
/* ========================================================================== */
/* ------------------------ USER CONFIGURATION FLAGS ------------------------ */
/* ========================================================================== */
#define PLDUMP_L2 0 // Extract payload for layer 2 flows (require ETH_ACTIVATE > 0)
#define PLDUMP_TCP 1 // Extract payload for TCP flows
#define PLDUMP_UDP 1 // Extract payload for UDP flows
#define PLDUMP_SCTP 0 // Extract payload for SCTP stream flows (require SCTP_ACTIVATE > 0)
#define PLDUMP_ETHERTYPES {} // Only extract L2 payloads for those ethertypes, e.g., {0x2000,0x2003}
#define PLDUMP_TCP_PORTS {} // Only extract TCP payloads on those ports, e.g., {80,8080}
#define PLDUMP_UDP_PORTS {} // Only extract UDP payloads on those ports, e.g., {80,8080}
#define PLDUMP_SCTP_PORTS {} // Only extract SCTP payloads on those ports, e.g., {80,8080}
#define PLDUMP_MAX_BYTES 0 // Max number of bytes per flow to dump (use 0 for no limits)
#define PLDUMP_START_OFF 0 // Start dumping bytes at a specific offset (L2 and UDP only)
#define PLDUMP_NAMES 0 // Format for filenames:
// 0: flowInd '_' [AB]
// 1: srcIP.srcPort-dstIP.dstPort-l4Proto[_sctpStream],
// srcMac-dstMac-etherType
// 2: Same as 1, but prefixed with timestampT
/* +++++++++++++++++++++ ENV / RUNTIME - conf Variables +++++++++++++++++++++ */
#define PLDUMP_RMDIR 1 // Empty PLDUMP_FOLDER before starting
#define PLDUMP_FOLDER "/tmp/payloadDumper" // output folder
#define PLDUMP_PREFIX "" // prefix for output files
#define PLDUMP_SUFFIX "" // suffix for output files
/* ========================================================================== */
/* ------------------------- DO NOT EDIT BELOW HERE ------------------------- */
/* ========================================================================== */
...You can choose whether layer 2, TCP, SCTP and/or UDP is your target. You can also choose to only extract data for specific ports or ethertypes, set a maximum for the payload length to be extracted and choose the name of the files and their location. By default the plugin extracts all payload from TCP and UDP.
Note that the following core switches found in networkHeaders.h must be activated for layer 2 and SCTP respectively:
- For layer 2:
ETH_ACTIVATEmust be1or2. - For SCTP,
SCTP_ACTIVATEmust be1.
The PLDUMP_START_OFF defines the offset to start the extraction, this is experimental currently.
Let’s say we are only interested in FTP and SMTP packets to be extracted. So the appropriate t2conf command is the following:
t2conf payloadDumper -D PLDUMP_TCP_PORTS="{25,21}"
t2build payloadDumper
Note that the port row value has to be enclosed by ".
Now run t2 on the supplied pcap.
t2 -r ~/data/faf-exercise.pcap -w ~/results/ -s
tawk -V pldStat=0x03
The pldStat column with value 0x03 is to be interpreted as follows:
bit | pldStat | Description
=============================================================================
0 | 0x01 | Match for this flow
1 | 0x02 | dump payload for this flow
Let’s show only the flows which extract content:
tawk 'bitsanyset($pldStat, 0x02)' ~/results/faf-exercise_flows.txt | tcol
%dir flowInd flowStat timeFirst timeLast duration numHdrDesc numHdrs hdrDesc srcMac dstMac ethType ethVlanID srcIP srcIPCC srcIPOrg srcPort dstIP dstIPCC dstIPOrg dstPort l4Proto pldStat
A 12 0x0400000000004000 1258563573.941668 1258563576.594009 2.652341 1 3 eth:ipv4:tcp 00:0b:db:63:5b:d4 00:19:e3:e7:5d:23 0x0800 192.168.1.103 07 "Private network" 1397 192.168.1.1 07 "Private network" 25 6 0x03
B 12 0x0400000000004001 1258563573.941709 1258563576.594045 2.652336 1 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:0b:db:63:5b:d4 0x0800 192.168.1.1 07 "Private network" 25 192.168.1.103 07 "Private network" 1397 6 0x03
A 13 0x0400000000004000 1258565030.304653 1258565030.420837 0.116184 1 3 eth:ipv4:tcp 00:0b:db:63:5b:d4 00:19:e3:e7:5d:23 0x0800 192.168.1.103 07 "Private network" 1749 192.168.1.1 07 "Private network" 25 6 0x03
B 13 0x0400000000004001 1258565030.304696 1258565030.420877 0.116181 1 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:0b:db:63:5b:d4 0x0800 192.168.1.1 07 "Private network" 25 192.168.1.103 07 "Private network" 1749 6 0x03
A 14 0x0400000000004000 1258565174.919134 1258565175.037809 0.118675 1 3 eth:ipv4:tcp 00:0b:db:63:5b:d4 00:19:e3:e7:5d:23 0x0800 192.168.1.103 07 "Private network" 1755 192.168.1.1 07 "Private network" 25 6 0x03
B 14 0x0400000000004001 1258565174.919179 1258565175.037828 0.118649 1 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:0b:db:63:5b:d4 0x0800 192.168.1.1 07 "Private network" 25 192.168.1.103 07 "Private network" 1755 6 0x03
...
B 32 0x0400000000004001 1258583614.298161 1258583615.323218 1.025057 1 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:0b:db:63:58:a6 0x0800 192.168.1.1 07 "Private network" 25 192.168.1.102 07 "Private network" 1911 6 0x03
A 35 0x0400000000004000 1258594162.928342 1258594185.618346 22.690004 1 3 eth:ipv4:tcp 00:08:74:38:01:b4 00:19:e3:e7:5d:23 0x0800 192.168.1.105 07 "Private network" 49329 143.166.11.10 us "Dell" 21 6 0x03
B 35 0x0400000000004001 1258594163.008594 1258594185.427515 22.418921 1 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:08:74:38:01:b4 0x0800 143.166.11.10 us "Dell" 21 192.168.1.105 07 "Private network" 49329 6 0x03In the packet file we select now the packets which are dumped or only matched the rule.
tawk 'bitsanyset($pldStat, 0x03)' ~/results/faf-exercise_packets.txt | tcol
%pktNo flowInd flowStat time pktIAT pktTrip flowDuration numHdrs hdrDesc ethVlanID srcMac dstMac ethType srcIP srcIPCC srcIPOrg srcPort dstIP dstIPCC dstIPOrg dstPort l4Proto pldStat l7Content
247 12 0x0400000000004000 1258563573.941668 0.000000 0.000000 0.000000 3 eth:ipv4:tcp 00:0b:db:63:5b:d4 00:19:e3:e7:5d:23 0x0800 192.168.1.103 07 Private network 1397 192.168.1.1 07 Private network 25 6 0x01
248 12 0x0400000000004001 1258563573.941709 0.000000 0.000041 0.000000 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:0b:db:63:5b:d4 0x0800 192.168.1.1 07 Private network 25 192.168.1.103 07 Private network 1397 6 0x01
249 12 0x0400000000004000 1258563573.942161 0.000493 0.000452 0.000493 3 eth:ipv4:tcp 00:0b:db:63:5b:d4 00:19:e3:e7:5d:23 0x0800 192.168.1.103 07 Private network 1397 192.168.1.1 07 Private network 25 6 0x01
250 12 0x0400000000004001 1258563574.317353 0.375644 0.375192 0.375644 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:0b:db:63:5b:d4 0x0800 192.168.1.1 07 Private network 25 192.168.1.103 07 Private network 1397 6 0x03 220 domex ESMTP Postfix\r\n
251 12 0x0400000000004000 1258563574.319182 0.377021 0.001829 0.377514 3 eth:ipv4:tcp 00:0b:db:63:5b:d4 00:19:e3:e7:5d:23 0x0800 192.168.1.103 07 Private network 1397 192.168.1.1 07 Private network 25 6 0x03 EHLO m57pat\r\n
252 12 0x0400000000004001 1258563574.319211 0.001858 0.000029 0.377502 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:0b:db:63:5b:d4 0x0800 192.168.1.1 07 Private network 25 192.168.1.103 07 Private network 1397 6 0x01
253 12 0x0400000000004001 1258563574.319380 0.000169 0.000198 0.377671 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:0b:db:63:5b:d4 0x0800 192.168.1.1 07 Private network 25 192.168.1.103 07 Private network 1397 6 0x03 250-domex\r\n250-PIPELINING\r\n250-SIZE 10485760\r\n250-VRFY\r\n250-ETRN\r\n250-AUTH LOGIN PLAIN CRAM-MD5 GSSAPI\r\n250-STARTTLS\r\n250-ENHANCEDSTATUSCODES\r\n250-8BITMIME\r\n250 DSN\r\n
254 12 0x0400000000004000 1258563574.319928 0.000746 0.000548 0.378260 3 eth:ipv4:tcp 00:0b:db:63:5b:d4 00:19:e3:e7:5d:23 0x0800 192.168.1.103 07 Private network 1397 192.168.1.1 07 Private network 25 6 0x03 STARTTLS\r\n
255 12 0x0400000000004001 1258563574.319950 0.000570 0.000022 0.378241 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:0b:db:63:5b:d4 0x0800 192.168.1.1 07 Private network 25 192.168.1.103 07 Private network 1397 6 0x01
...
1279 35 0x0400000000004001 1258594163.407582 0.080461 0.079737 0.398988 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:08:74:38:01:b4 0x0800 143.166.11.10 us Dell 21 192.168.1.105 07 Private network 49329 6 0x03 227 Entering Passive Mode (143,166,11,10,251,78)\r\n
1283 35 0x0400000000004000 1258594163.487490 0.159645 0.079908 0.559148 3 eth:ipv4:tcp 00:08:74:38:01:b4 00:19:e3:e7:5d:23 0x0800 192.168.1.105 07 Private network 49329 143.166.11.10 us Dell 21 6 0x03 SIZE /video/R79733.EXE\r\n
1284 35 0x0400000000004001 1258594163.565990 0.158408 0.078500 0.557396 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:08:74:38:01:b4 0x0800 143.166.11.10 us Dell 21 192.168.1.105 07 Private network 49329 6 0x03 213 4255056\r\n
1285 35 0x0400000000004000 1258594163.566694 0.079204 0.000704 0.638352 3 eth:ipv4:tcp 00:08:74:38:01:b4 00:19:e3:e7:5d:23 0x0800 192.168.1.105 07 Private network 49329 143.166.11.10 us Dell 21 6 0x03 RETR /video/R79733.EXE\r\n
1286 35 0x0400000000004001 1258594163.644188 0.078198 0.077494 0.635594 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:08:74:38:01:b4 0x0800 143.166.11.10 us Dell 21 192.168.1.105 07 Private network 49329 6 0x03 125 Data connection already open; Transfer starting.\r\n
1303 35 0x0400000000004000 1258594163.838277 0.271583 0.194089 0.909935 3 eth:ipv4:tcp 00:08:74:38:01:b4 00:19:e3:e7:5d:23 0x0800 192.168.1.105 07 Private network 49329 143.166.11.10 us Dell 21 6 0x01
5898 35 0x0400000000004001 1258594185.427515 21.783327 21.589237 22.418921 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:08:74:38:01:b4 0x0800 143.166.11.10 us Dell 21 192.168.1.105 07 Private network 49329 6 0x03 226 Transfer complete.\r\n
5900 35 0x0400000000004000 1258594185.618346 21.780069 0.190831 22.690004 3 eth:ipv4:tcp 00:08:74:38:01:b4 00:19:e3:e7:5d:23 0x0800 192.168.1.105 07 Private network 49329 143.166.11.10 us Dell 21 6 0x01
5902 37 0x0400000000004001 1258594491.683288 0.000000 0.000000 0.000000 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:08:74:38:01:b4 0x0800 143.166.11.10 us Dell 21 192.168.1.105 07 Private network 49329 6 0x01Note that packet 247-249,252,255… 1303,5900,5902 match our rule, but were not extracted due to no payload. You may load basicStat to verify that fact. Now look under /tmp/ for the extracted payload:
ls /tmp/payloadDumper
12_A 13_A 14_A 15_A 16_A 17_A 18_A 19_A 20_A 21_A 22_A 23_A 24_A 25_A 26_A 27_A 28_A 29_A 30_A 31_A 32_A 35_A
12_B 13_B 14_B 15_B 16_B 17_B 18_B 19_B 20_B 21_B 22_B 23_B 24_B 25_B 26_B 27_B 28_B 29_B 30_B 31_B 32_B 35_B
Let’s have a look at one of the files:
cat /tmp/payloadDumper/35_A
USER anonymous
PASS IEUser@
TYPE I
PASV
SIZE /video/R79733.EXE
RETR /video/R79733.EXEOups, what is that? Murky….
As homework try look into file 28_A and then change PLDUMP_NAME to 1, recompile and rerun T2, now the file names look like tcpflow.
Play around with PLDUMP_TCP_PORTS extract some ports, e.g., 80 traffic.
Conclusion
Another method to extract the content and display it like Wireshark’s Follow TCP/UDP stream is to use tawk and the follow_stream() function on the packet file.
Refer to the packet mode tutorial for a practical example.
Don’t forget to reset the plugin configuration for the next tutorial.
t2conf payloadDumper --reset && t2build payloadDumper
Have fun analyzing.