by Don
Parker
06/17/2004
Anyone who has worked with an intrusion detection system knows that it can produce an enormous amount of data. For many network security analysts this vast ocean of packets flagged for further inspection quickly becomes an unruly beast to tame. How then to tame the beast?
The simplest and most efficient way to extract needed data from the ever-growing database logging these packets is to use a combination of Berkeley packet filters (bpf) and bitmask filters. Once you're familiar with their syntax and usage, filtering out specific data is easy. Instead of manually checking 200MB of packet data one packet at a time, you can tailor that down to the interesting 500KB. This represents enormous savings in time and trouble.
Many people do not even know of the existence of tools such as bpf and bitmask filters, and fewer still know how to use them. Their usage can be confusing, which is why many people I know shun them. To add to the dislike, they're normally used from the command-line interface, whether a DOS shell or in a Unix or Linux terminal window. Many people are far more comfortable in a GUI environment.
When it comes to using bpf and bitmask filters, you can have the best of both worlds by using Ethereal. This free tool allows you to interpret packets and accepts bpf syntax. No matter your preference, the command-line interface or the GUI, you can employ these filtering tools to greatly enhance your analysis efforts.
Effectively using bpf filters and bitmasks requires that you have some knowledge of TCP/IP itself. These filters use various bytes in the TCP packets to narrow down your searches. Let's discuss the TCP/IP three-way handshake and the TCP packet flags.
When setting up communications with another computer, the source sends out a TCP SYN packet. This packet's purpose is to synchronize sequence numbers. The destination computer then responds with a SYN/ACK packet to acknowledge receipt of the SYN packet. Finally, the originating computer responds with an ACK packet. At this point both computers are ready for the exchange of data with PSH (push) packets.
To sum up, you will see the following packet order used to set up communications: SYN, SYN/ACK, ACK. Then you'll see PSH/ACK packets for the exchange of actual data.
The 13th byte offset in the TCP header contains all of the above mentioned flags are contained. Each flag has a binary value within that byte as seen below:
|
128 |
64 |
32 |
16 |
8 |
4 |
2 |
1 |
|
CWR |
ECE |
URG |
ACK |
PSH |
RST |
SYN |
FYN |
Bit positions 128 and 64 apply for congestion notification. They're not really germane for this article, though you can find packets with the CWR and ECN flags set. They're showing up more frequently in network reconnaissance attempts as of late so it may be a good idea to keep an eye on these types of packets.
Now that we understand the TCP flags, we can look at some filtering examples. Furthermore, I will also explain a scenario in which you may want to invoke the shown bpf and bitmask filter. We will also progress in a logical sequence of complexity when building these filters.
When I was a new IDS Analyst, I manually reviewed each and every packet that a scan generated, even though the scan generated a huge file. An analyst does not have the time to sort through each packet. Instead, he has to winnow down the deluge of packets to those of interest. To illustrate this, please consider the following scenario.
You just arrived at work after the weekend. Skimming the IDS log, you see is a lot of scanning from a particular IP address. The target computer offers several services for your corporate network. You need to know which services responded back to the machine doing the scanning and, more importantly, what they sent in response. It is entirely possible that a specially crafted packet sent to a service on your machine — which an attacker may have already compromised — opened up another port. The scanning computer may be in contact with the target computer on that port.
Bearing this possibility in mind, you've performed a cursory log check to ascertain the offender's IP address. With that in hand, you build the following filter:
-nXvs 0 ip and host 192.168.1.100
We have not yet covered these switches, so I'll elaborate on them here.
-: You need this
hyphen before putting in all of the following bpf switches.
n: Keep IP addresses
in numerical format; do not resolve them.
X: Print packet
information in both hex and ASCII.
v: Enable verbose
logging; print all header metrics.
s: The snaplength of
the packet; how many bytes to capture. (Ethernet packets are 1540
bytes long.)
0: The amount in
decimal of the snaplength, measured in bytes.
ip: Show all packets
with valid IP headers.
and host: Follow a
single host, as opposed to an entire subnet.
192.168.1.100: The IP address for which you want
packets.
With the above filter in place, you'll see a lot of information
back from the packet database. This example shows a very basic
filter. Realistically it is not very useful, as it will show all
traffic associated with that IP address. By that I mean all flags
such as syn and psh/ack and such will be
included.
If that host communicated with a DNS server, for example, you'll also see UDP traffic, as DNS uses both protocols to communicate. The ability to filter the query further is ever more important.
Many organizations have a central database that stores all of the flagged packet output. Depending on the size of your network and average traffic flow, this database may be sizable. Ideally, you can retain records for quite a while. Consider the possible legal requirements intend to pursue someone whose malicious activity you have logged. Central logging is also a good idea, because keeping track of varied query logging paths could quickly become a logistics problem.
The hypothetical scan has probably generated a large amount of traffic. How can you find the appropriate data in this sea of packets? Since TCP communications require the three-way handshake, we could build a bpf filter to find only TCP traffic. If the computer that is sending out SYN packets to various ports on the network computer you are monitoring answers back it will answer back with a SYN/ACK packet. The below filter will accommodate our needs:
-nXvs 0 tcp and host 192.168.1.100
This filter will show all TCP traffic that responds to the computer at the IP address 192.168.1.100. We have now managed to pare down a large packet file to a manageable size. For analysis sake, can we pare it down further? Yes!
At this stage of the game, knowledge of TCP/IP really becomes a factor in the creation of filters. All packets transmitted contain certain headers, whether ubiquitous TCP/IP packets or less popular protocols such as ICMP or ARP. Packets of each protocol have specific fields that contain specific information, though the structure of packets differs between protocols. Building a filter for a TCP-based packet will not work for say a UDP-based one or another protocol. To create custom bpf filters for your needs, you'll need to understand the structure and fields of the packets of each protocol. This knowledge will pay huge dividends when creating these filters and bitmasks to cull the data that you require from the database.
Many of you already know that attacks against your network assets can come in various shapes and sizes. TCP packets are not the only threat vectors to consider. Looking at various sites online that host exploit code will reveal that many exploits use other protocols to deliver or activate system vulnerabilities.
The next part of this article will demonstrate how to draw upon various fields within each of the major protocols used on network. Furthermore, we will also learn how to differentiate between source and destination computers as well as ports. With this kind of knowledge in hand you will be able to create esoteric filters to suit your specific needs. I'll provide realistic examples that you can incorporate into your work environment.
Don Parker is an intrusion detection specialist who works for Rigel Kent Security and Advisory Services.
Return to the Security DevCenter.
Copyright © 2004 O'Reilly Media, Inc.