IP anonymization

ANONYM_IP anonymization IP IPv4 IPv6 layer 3

Introduction

Sometimes it is necessary to hide your personal or public addresses from anybody you share flow or packet files with. T2 implements a very simplistic mechanism for this purpose, it just blocks the output in the specific files.

Preparation

First, restore T2 into a pristine state by removing all unnecessary or older plugins from the plugin folder ~/.tranalyzer/plugins:

t2build -e -y

Are you sure you want to empty the plugin folder '/home/wurst/.tranalyzer/plugins' (y/N)? yes
Plugin folder emptied

Then compile the core (tranalyzer2) and the following plugins:

t2build tranalyzer2 basicFlow basicStats connStat txtSink

...
BUILD SUCCESSFUL

If you did not create a separate data and results directory yet, please do it now in another bash window, that facilitates your workflow:

mkdir ~/data ~/results

The sample PCAP used in this tutorial can be downloaded here: faf-exercise.pcap.

Please save it in your ~/data folder.

Now you are all set!

ANONYM_IP

The control for IP anonymization is located in the core, tranalyzer.h. You can see the current value using t2conf -G option:

t2conf tranalyzer2 -G ANONYM_IP

ANONYM_IP = 0

Set it to 1, recompile the core and all plugins and run t2 on the pcap:

t2conf tranalyzer2 -D ANONYM_IP=1 && t2build -R

t2 -r ~/data/faf-exercise.pcap -w ~/results/ -s

So you do not see IP addresses in the report summary anymore. IP addresses, including Teredo, L2TP and GRE entries, are also removed in the flow and packet files. Yes it is coarse, but if will be more elaborate in the future.

tcol ~/results/faf-exercise_flows.txt

%dir  flowInd  flowStat            timeFirst          timeLast           duration  srcMac             dstMac             ethType  ethVlanID  srcPort  dstPort  l4Proto  numPktsSnt  numPktsRcvd  numBytesSnt  numBytesRcvd  minPktSz  maxPktSz  avePktSize  stdPktSize  minIAT  maxIAT    aveIAT       stdIAT       pktps     bytps     pktAsm       bytAsm      connSip  connDip  connSipDip  connSipDprt  connF
A     1        0x0400000000004000  1258544215.037210  1258544215.372742  0.335532  00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800              1258     80       6        6           7            367          3547          0         367       61.16667    113.5846    0       0.166651  0.055922     0.06351066   17.88205  1093.785  -0.07692308  -0.8124681  2        1        10          18           9
B     1        0x0400000000004001  1258544215.202900  1258544215.537951  0.335051  00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800              80       1258     6        7           6            3547         367           0         1380      506.7143    474.1748    0       0.167371  0.04786443   0.06323512   20.89234  10586.45  0.07692308   0.8124681   1        2        9           17           17
A     2        0x0400000000004000  1258544216.385370  1258544216.723144  0.337774  00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800              1259     80       6        5           4            322          464           0         322       64.4        106.4982    0       0.16665   0.06755479   0.06471679   14.8028   953.3001  0.1111111    -0.1806616  2        1        8           16           8
B     2        0x0400000000004001  1258544216.551313  1258544216.888595  0.337282  00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800              80       1259     6        4           5            464          322           0         464       116         150.9982    0       0.169645  0.0843205    0.0658242    11.85951  1375.703  -0.1111111   0.1806616   1        2        7           15           15
A     3        0x0400000000004000  1258544216.908284  1258544217.008468  0.100184  00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800              1260     80       6        18          73           319          95603         0         319       17.72222    61.73868    0       0.013738  0.005565778  0.00363383   179.6694  3184.141  -0.6043956   -0.9933488  2        1        8           14           7
B     3        0x0400000000004001  1258544216.915576  1258544217.008019  0.092443  00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800              80       1260     6        73          18           95603        319           0         1380      1309.63     274.7284    0       0.021251  0.001266342  0.003059902  789.6758  1034183   0.6043956    0.9933488   1        2        7           13           13
A     4        0x0400000000004000  1258544217.003718  1258544217.348506  0.344788  00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800              1261     80       6        5           4            323          466           0         323       64.6        106.829     0       0.166899  0.0689576    0.06384765   14.50166  936.8075  0.1111111    -0.1812421  2        1        6           12           6
B     4        0x0400000000004001  1258544217.169421  1258544217.513942  0.344521  00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800              80       1261     6        4           5            466          323           0         466       116.5       151.649     0       0.177128  0.08613025   0.06695305   11.61032  1352.603  -0.1111111   0.1812421   1        2        5           11           11
A     5        0x0400000000004000  1258544217.349751  1258544217.413719  0.063968  00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800              1262     80       6        9           26           320          30820         0         320       35.55556    84.1992     0       0.009743  0.007107555  0.002012674  140.6953  5002.501  -0.4857143   -0.9794477  2        1        6           10           5
...

Same applies to the packet file.

tcol ~/results/faf-exercise_packets.txt

%pktNo  flowInd  flowStat            time               pktIAT    pktTrip   flowDuration  ethVlanID  srcMac             dstMac             ethType  srcPort  dstPort  l4Proto  pktLen  l7Len  l7Content
1       1        0x0400000000004000  1258544215.037210  0.000000  0.000000  0.000000                 00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800   1258     80       6        66      0
2       1        0x0400000000004001  1258544215.202900  0.000000  0.165690  0.000000                 00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   80       1258     6        62      0
3       1        0x0400000000004000  1258544215.203358  0.166148  0.000458  0.166148                 00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800   1258     80       6        64      0
4       1        0x0400000000004000  1258544215.203850  0.000492  0.000950  0.166640                 00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800   1258     80       6        425     367    GET /softw/90/update/avg9infoavi.ctf HTTP/1.1\r\nUser-Agent: AVGINET9-WXPPX86 90 AVI=270.14.71/2510 BUILD=707 LOC=1033 LIC=9I-ASXNN-X4WGW-M0XFR-T84VX-3VX02 DIAG=51E OPF=0 PCA=\r\nHost: backup.avg.cz\r\nAccept: */*\r\nAccept-Encoding: identity,deflate,gzip\r\nIf-Modified-Since: Tue, 17 Nov 2009 20:41:10 GMT\r\nPragma: no-cache\r\nCache-Control: no-cache\r\nx-avg-id:78-175947826\r\n\r\n
5       1        0x0400000000004001  1258544215.370055  0.167155  0.166205  0.167155                 00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   80       1258     6        1434    1380   HTTP/1.1 200 OK\r\nDate: Wed, 18 Nov 2009 11:39:48 GMT\r\nServer: Apache\r\nLast-Modified: Wed, 18 Nov 2009 09:04:15 GMT\r\nETag: "15c007-cea-478a186e401c0"\r\nAccept-Ranges: bytes\r\nContent-Length: 3306\r\nConnection: close\r\nContent-Type: text/plain\r\n\r\nAVG CTF Index File;ver(10)\r\nbin(u7avi1777ff.bin)grp(avi:1777)tm(0911180855)pri(2)len(6637487)\r\nbin(u7avi1777u1323ff.bin)grp(avi:1777)dep(avi:1323)tm(0911180855)pri(2)len(590030)\r\nbin(u7avi1777u1705ff.bin)grp(avi:1777)dep(avi:1705)tm(0911180855)pri(2)len(95323)\r\nbin(u7iavi2511ff.bin)grp(iavi:2511)tm(0911180855)pri(2)len(45641416)\r\nbin(u7iavi2511u2251fo.bin)grp(iavi:2511)dep(iavi:2251)tm(0911180855)pri(2)len(6175342)\r\nbin(u7iavi2511u2353fn.bin)grp(iavi:2511)dep(iavi:2353)tm(0911180855)pri(2)len(4506824)\r\nbin(u7iavi2511u2404fm.bin)grp(iavi:2511)dep(iavi:2404)tm(0911180855)pri(2)len(3342546)\r\nbin(u7iavi2511u2431fl.bin)grp(iavi:2511)dep(iavi:2431)tm(0911180855)pri(2)len(2609643)\r\nbin(u7iavi2511u2458fk.bin)grp(iavi:2511)dep(iavi:2458)tm(0911180855)pri(2)len(1380674)\r\nbin(u7iavi2511u2480fj.bin)grp(iavi:2511)dep(iavi:2480)tm(0911180855)pri(2)len(681743)\r\nbin(u7iavi2511u2490fi.bin)grp(iavi:2511)dep(iavi:2490)tm(0911180855)pri(2)len(555496)\r\nbin(u7iavi2511u2500fh.bin)grp(iavi:2511)dep(iavi:2500)tm(0911180855)pri(2)len(332906)\r\nbin(u7iavi2511u2505fh.bin)grp(iavi:2511)dep(iavi:2505)tm(0911180855)pri(2)len(204942)\r\nbin(u7iavi2511u2508
6       1        0x0400000000004001  1258544215.370067  0.000012  0.166217  0.167167                 00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   80       1258     6        375     321    fg.bin)grp(iavi:2511)dep(iavi:2508)tm(0911180855)pri(2)len(72274)\r\nbin(u7iavi2511u2510ff.bin)grp(iavi:2511)dep(iavi:2510)tm(0911180855)pri(2)len(30540)\r\nbin(u9ichjw4qt.bin)grp(ichjw:4)pri(2)tm(0907131859)len(113488)\r\nbin(u9ichjw4u2ia.bin)grp(ichjw:4)dep(ichjw:2)pri(2)tm(0907131859)len(94470)\r\nbin(u9ichjw4u3gq.bin)grp(ic
7       1        0x0400000000004000  1258544215.370501  0.166651  0.000434  0.333291                 00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800   1258     80       6        64      0
8       1        0x0400000000004001  1258544215.370560  0.000493  0.000059  0.167660                 00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   80       1258     6        1434    1380   hjw:4)dep(ichjw:3)pri(2)tm(0907131859)len(13313)\r\nbin(u9idat249b243tc.bin)grp(idat:249)dif(idat:243)pri(2)tm(0911051401)len(621430)\r\nbin(u9idat249b245bp.bin)grp(idat:249)dif(idat:245)pri(2)tm(0911051401)len(197648)\r\nbin(u9idat249b246mj.bin)grp(idat:249)dif(idat:246)pri(2)tm(0911051400)len(159289)\r\nbin(u9idat249b247uh.bin)grp(idat:249)dif(idat:247)pri(2)tm(0911051400)len(95680)\r\nbin(u9idat249le.bin)grp(idat:249)pri(2)tm(0911051400)len(1927074)\r\nbin(u9ifw42jx.bin)grp(ifw:42)pri(2)tm(0911131400)len(541853)\r\nbin(u9ifw42u38we.bin)grp(ifw:42)dep(ifw:38)pri(2)tm(0911131400)len(123575)\r\nbin(u9ifw42u39ke.bin)grp(ifw:42)dep(ifw:39)pri(2)tm(0911131400)len(92745)\r\nbin(u9ifw42u40dc.bin)grp(ifw:42)dep(ifw:40)pri(2)tm(0911131400)len(67292)\r\nbin(u9ifw42u41tz.bin)grp(ifw:42)dep(ifw:41)pri(2)tm(0911131400)len(18393)\r\nbin(x8all234yc.bin)grp(xplph:0012;xplsb:0099;xplsb2:0118;xplsc:0149)tm(0911180700)pri(2)len(1146352)\r\nbin(x8xplph_12gj.bin)grp(xplph:12)tm(0910280700)pri(2)len(4551)\r\nbin(x8xplsb2_118c8.bin)grp(xplsb2:118)tm(0911180700)pri(2)len(4989)\r\nbin(x8xplsb_9946.bin)grp(xplsb:99)tm(0911160700)pri(2)len(985460)\r\nbin(x8xplsb_99d9546.bin)grp(xplsb:99)dif(xplsb:95)tm(0911160700)pri(2)len(32267)\r\nbin(x8xplsb_99d9646.bin)grp(xplsb:99)dif(xplsb:96)tm(0911160700)pri(2)len(3505)\r\nbin(x8xplsb_99d9746.bin)grp(xplsb:99)dif(xplsb:97)tm(0911160700)pri(2)len(644)\r\nbin(x8xplsb_99d9846.bin
9       1        0x0400000000004001  1258544215.370571  0.000011  0.000070  0.167671                 00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   80       1258     6        520     466    )grp(xplsb:99)dif(xplsb:98)tm(0911160700)pri(2)len(581)\r\nbin(x8xplsc_149c8.bin)grp(xplsc:149)tm(0911180700)pri(2)len(151922)\r\nbin(x8xplsc_149d145c8.bin)grp(xplsc:149)dif(xplsc:145)tm(0911180700)pri(2)len(2561)\r\nbin(x8xplsc_149d146c8.bin)grp(xplsc:149)dif(xplsc:146)tm(0911180700)pri(2)len(1921)\r\nbin(x8xplsc_149d147c8.bin)grp(xplsc:149)dif(xplsc:147)tm(0911180700)pri(2)len(1767)\r\nbin(x8xplsc_149d148c8.bin)grp(xplsc:149)dif(xplsc:148)tm(0911180700)pri(2)len(1411)\r\n
...

Conclusion

It is a very rudimentary anonymization and it does not currently include the other plugins providing IP addresses, e.g. dnsDecode. The Anteater knows that removing IP’s is not the solution, but it worked for us in an urgent job. And then there is content anonymization, a large challenge. Be patient he’s working on it.

Do not forget to reset the configuration for the next tutorials:

t2conf tranalyzer2 -D ANONYM_IP=1 && t2build -R

Or use the new command:

t2conf --reset && t2build -R

Have fun!