.~e~----------------------------------------------------------~e~. ; *15* chapter sixteen -- ktwo ; `----------------------------------------------------------------' y0y0y0y0y0y0, hey ladies and gents, I just thought maybe I would take a risk and include chapter 16 of Hack Proofing Your Network before the book is released. This is the unedited version, fresh off the printing press. Have fun, and remember, I will be autogr- aphing copies of Know Your Enemy for free at this upcoming defcon. Chapter 16: Ryan, suggest places for inclusion of code and screenshots, as requested. Id like to see some screenshots, packet prints, command-line options or something for the fragrouter section. Grammar and style was a bit awkward and punctuation was sparse. I did one pass-through (didn't track the easy edits, so it wouldn't be too hard on the eyes), please highlight anything you feel is still awkward and needs to be clarified by the author. I like the material a lot. Just need to fix wording in a few spots, as mentioned. Chapter 16 IDS Evasion Solutions in this chapter: Understanding How Signature-Based IDSs Work Using Packet Level Evasion Using Protocol and Application Protocol Level Evasion Using Code Morphing Evasion Chapter suggestions for: Examples and Exercises: Check for the specific code called for in each section Screen Shots: Screenshots for each program called for Introduction One of the laws of security is that all signature-based detection mechanisms can be bypassed. This is as true for Intrusion Detection System (IDS) signatures as it is for virus signatures. IDS systems, which have all the problems of a virus scanner, plus the job of modeling network state, must operate at several layers simultaneously, and they can be fooled at each of those layers. IDS’ have all the problems of a virus scanner, plus the job of modeling network state. This chapter covers techniques for evading IDSs. These techniques include playing games at the packet level, application level, and morphing the machine code. Each of these types can be used individually, or together, to evade detection by an IDS. In this chapter, we present several examples of how an attack might evade detection. Understanding How Signature-Based IDSs Work An IDS is quite simply the high-tech equivalent of a burglar alarm—a burglar alarm configured to monitor access points, hostile activities and known intruders. These systems typically trigger on events by referencing network activity against an attack signature database. If a match is made, an alert will take place and will be logged for future reference. It is the makeup of this signature database that is the Achilles heel of these systems. Attack signatures consist of several components used to uniquely describe an attack. An ideal signature would be one that is specific to the attack while being as simple as possible to match with the input data stream (large complex signatures may pose a serious processing burden). Just as there are varying types of attacks, there must be varying types of signatures. Some signatures will define the characteristics of a single IP option, perhaps that of a nmap portscan, while others will be derived from the actual payload of an attack. Most signatures are constructed by running a known exploit several times, monitoring the data as it appears on the network and looking for a unique pattern that is repeated on every execution. This method works fairly well at ensuring that the signature will consistently match an exploit attemptattempt by that particular exploit. Although I have seen my share of shoddy signatures, some so simplistic in nature that the amazingly hostile activity of browsing a few Websites may set them off, remember the idea is for the unique identification of an attack, not merely the detection of attacks. Tools & Traps… Signature Components The following are Eexample snort Snort signatures: Breaks and indents for wrapping lines OK? Looks good to me. alert tcp $EXTERNAL_NET any -> $HOME_NET 8080 (msg:"SCAN Proxy attempt";flags:S; classtype:attempted-recon; sid:620; rev:1;) alert ip $EXTERNAL_NET any -> $HOME_NET :1023 (msg:"SHELLCODE linux shellcode"; content:"|90 90 90 e8 c0 ff ff ff|/bin /sh"; classtype:attempted-admin; sid:652; rev:2;) alert tcp $EXTERNAL_NET any -> $HOME_NET 21 (msg:"FTP CWD ..."; flags:A+; content:"CWD ..."; classtype:bad-unknown; sid:1229 ; rev:1;) alert icmp $EXTERNAL_NET any -> $HOME_NET any (msg:"ICMP traceroute ipopts"; ipopts: rr; itype: 0; classtype: attempted-recon; sid:475; rev:1;) alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS 80 (msg:"WEB- ATTACKS chgrp command attempt"; flags:A+; content:"/usr/bin/ chgrp";nocase; sid:1337; rev:1; classtype:web-application -attack;) Here are some basics of snort signatures. Snort implements a description language used to construct any rule. I’m not going toTo avoid getting into the rather complex details of writing your own signatures, but just point out enough to get us on our way. Llet’s simply go left to right through the examples above and try to discern what exactly theyse mean. We can see that these all define a type of alert. These alerts are then classified into a type of protocol, then the specific details are given, : IP address ($EXTERNAL_NET and $HOME_NET are variables usually defined as 10.10.10.0/24 CIDR style) and port numbers to restrict the scope. The msg keyword defines the message that will be sent out if the rule is matched;, flags will define which of the TCP flags are set in the stream;, just as ipopts dictates the options of an IP packet; and content is used to specify a unique series of data that appears in the actual contents of the packet. In a content field, anything between vertical bars is in hex format, while the rest is ASCII. The first rule watches for any attempt from the outside to connect to an inside host at TCP port 8080, which is a port often used for web proxies. The second rule looks for a commonly-used shellcode sequence inside any IP packet going to a port less than 1024. (The :1023 is shorthand for a range of ports between 0 and 1023, inclusive.) The third rule is checking for a “CWD …” command to TCP port 21, the FTP port. The fourth rule is watching for IP packets with the rr (Record Route) option on. The final rule is checking for the string “/usr/bin/chgrp” going to port 80, the HTTP port. Computing systems, in their most basic abstraction, can be defined as a finite state machine, which literally means that there are only a specific predefined number of states that a system may attain. This crux limitation hinders the IDS in that it can to be only well armed at only a single point in time (i.e. as well armed as the size of its database). First, Hhow can one have foreknowledge of the internal characteristics that make up an intrusion attempt that has not yet occurred? You can’t alert on attacks you’ve never seen before. There Second, there can be only educated guesses that what has happened in the past may again transpire in the future. You can create a signature for a past attack after the fact, but that’s no guarantee you’ll ever see that attack again.,. Third, most an IDS are may be incapable of discerning a new attack from the background white noise of any network. The network utilization may be too high or many false positives cause rules to be diabled.; aAnd finally, it may be incapacitated by even the slightest modification to a known attack. It is ether a weakness in the signature matching process or more fundamentally a weakness in the packet analysis engine (packet sniffing/reconstruction) that will thwart any detection capability. You’re getting too abstract for me to follow here. I don’t follow where you’re going with the state-machine discussion. Are you trying to point out that the external IDS has to model the state of the victim? I think what you’re saying is that 1) You can’t alert on attacks you’ve never seen before, 2) You can create a signature for a past attack after the fact, but that’s no guarantee you’ll ever see that attack again… is 3) relating to anomaly detection? Point 4 is understandable as-is. How about we embed a few of these statements for clairity. The state discussion was just to get the reader accustomed to the idea of state and modeling? The goals of an attacker as it relates to IDS evasion are twofold: To evade detection completely, or to use techniques and methods that will increase the processing load of the IDS sensor significantly. The more methods employed by attackers at large, on a wide scale, the more vendors will be forced to implement more complex signature matching and packet analysis engines. These complex systems will undoubtedly have lower operating throughputs and more opportunities for evasion. The paradox is that the more complex a system becomes, the more opportunities there are for vulnerabilities! Some say the ratio for bugs to code may be as high as 1:1000, and even conservatives say a ratio of 1:10000 may exist. With these sorts of figures in mind, a system of increasing complexity will undoubtedly lead to new levels of increased insecurity. Judging False Positives and Negatives To be an effective tool, an IDS must be able to digest and report information efficiently. A false positive is an event that was triggered that did not actually occur, which may be as innocuous as the download of a signature database (downloading of an IDS signature database may trigger every alarm in the book) or some unusual traffic generated by a networked game. This, although annoying, is usually not of much consequence but can easily happen and is usually tuned down by an initial configuration and burn-in of a Network IDS (NIDS) configuration. However, more dangerous is the possibility for false negatives, which is the failure to alert to an actual event. This would occur in a failure of one of the key functional units of a NIDS. False negatives are the product of a situation in which an attacker modifies their attack payload in order to subvert the detection engine. False positives have a significant impact on the effectiveness of an IDS sensor. If you are charged with the responsibility of monitoring a device, you will find you become accustomed to its typical behavior. If there is a reasonable number of false positives being detected, the perceived urgency of an alert may be diminished by the fact that there are numerous events being triggered on a daily basis that turn into wild goose chases. In the end, all the power of IDS is ultimately controlled by the single judgment call on whether or not to take action. Alert Flooding This problem of making sense of what an IDS reports is apparent again in a flood scenario. Flooding, as you may have guessed, is the process of overloading the IDS by triggering a deluge of alerts. This attack has a number of beneficial actions for an attacker. If the attacker can muster enough firepower in terms of network bandwidth, a Denial of Service (DoS) attack is possible. Many IDS sensors exasperate this condition by the first match (or multiple match) paradox, in which the sensor has to essentially decide whether or not to alert based on the first match in its database or to attempt further matches. The issue here is that an attacker may identify a low-priority or benign signature common to many IDS signature databases and attempt to reproduce this in a more damaging exploit attempt. If the sensor were to use a first match method, it would produce an alert for the less severe vulnerability and not signal to the true nature of the attack. However, in using the multiple match approach, the IDS allows itself to be more vulnerable to alert flooding attacks. The attacker may simply package an entire signature database into some network traffic and watch the IDS crumble to the ground. Aside from the desirable condition of failing an IDS sensor, there is the added bonus of having generated an excessive amount of alerts (in excess of 10,000 is no problem at all) that the admin must then somehow make sense of. The intended target host may be totally lost within a dizzying display of messages, beeps and red flags. Trying to identify a real intrusion event may be arduous at best. Let us not forget the psychological impact of seeing what may be construed as an all-out Internet wide assault on your networking equipment. If this style of attack were to somehow become routine, how effective would your IDS solution be then? Using Packet Level Evasion Are you going to cover Hailstorm here as indicated in the original outline. Not in this portion of text – but somewhere within this Level One Head Section? I spoke with Ryan about using fragrouter and such in place of Hailstorm. Clicktosecure.com is down and I am unable to get much information about it at this time. Yes, that is correct. Network IDSs have the dubious task of making sense of literally millions of pieces of information per second, analyzing information while providing acceptable response times (typically as close to real-time as possible is desired). To break down the effort of data analysis, a NIDS will function on several discrete layers of the network protocol stack. The first layers under inspection will be the network and transport layers, where the attacker has a great opportunity to confuse, circumvent or eliminate a NIDS sensor. If an attacker were to devise a technique that would enable them to evade detection this would be an ideal location to begin, as all other detection capabilities of the IDS rely on the ability to correctly interpret network traffic just as the target host would. Unfortunately for the defender the characteristics of IP and TCP do not lend themselves to well-defined inspection. These protocols were developed to operate in a dynamic environment, defined by permissive standards that are laden with soft “SHOULD" and "MAY” statements, “MUST” being reserved for all butonly the most basic requirements. This lax definition of protocol standards leads to many complications when an attempt is made to interpret network communications. This will leave the door open for an attacker to desynchronize the state of the IDS, such that it does not correctly assemble traffic in the same manner that the target host will. For example, if an IDS signature was crafted to search for the string “CODE-RED” in any HTTP request, it may be possible for the attacker to fragment his traffic in such a way that it will assemble differently for the IDS as it will for the target host. Therefore, the attacker may exploit the target host without the IDS being able to interpret the event accordingly. Notes from the Underground… TCP/IP Specification Interpretation The difficulties inherent in interpreting the TCP/IP specification is is what also leads to many TCP/IP stack fingerprinting opportunities, . anything Anything from the initial TCP sequence number to packet fragment and options handling characteristics may be used to identify a remote OS. This uniqueness of implementation (nmap has over 300 entries in its nmap-os-fingerprints database) has produced some of the most devastating and complex problems for IDS developers to overcome. How to understandThe challenge of decoding what a particular stream of communications may look to the end host without intimate knowledge of the inner workings of its protocol stack is exceedingly complex. Author: Rephrase for clear grammar in last sidebar sentence. Several years ago a paper was written to discuss the many issues facing NIDS development. Essentially the attacks discussed in 1998 Thomas Ptacek and Timothy Newsham’s published1998 "Insertion, Evasion, and Denial of Service: Eluding Network Intrusion Detection ()", vary in style from insertion to evasion attacks. Insertion and evasion are the basis for evading a signature match. Insertion is the technique which relies upon a situation in whichwhere an IDS will accept some information with the assumption that the target host will also. However, if the IDS does not interpret the network stream in the same manner that the target does, the IDS will have a different understanding of what the communication looks like and will be ineffective in properly alerting to the presence of an attack. The IDS signature will simply not match the data acquired from the network. Our “CODE-RED” example may be seen to the IDS as “CODE-NOT-RED”, this I suppose iswhich may be enough for the IDS to feel safe, whereas the target host will actually receive “CODE-RED”, having dropped the “NOT” in the middle due to the packet containing it not matching the target’s understanding of the standards.. Evasion is the converse of insertion; where it relies upon a situation in which a target system will accept data that the IDS will ignore. An attack may then look something like “CODE” to the IDS where the target will receive “CODE-RED”. These sorts of attacks can be enabled in a number of ways. At any time a TCP/IP communication may be terminated by either party. If the IDS were to incorrectly interpret a RST or FIN from an attacker that was not accepted by the target host (e.g. if the IDS did not correctly monitor sequence numbers), the attacker would be free to communicate with impudence. Denial of Service in IDS implementations is commonplace. The opportunities to subvert the operation of a sensor are quite apparent. System resources are finite; there are only so many pages of memory that can be allocated, ; CPUs are bound and even network IO cards may not be able to maintain consistent throughput despite their speed rating. A Because a computer is a system of queues, some will inevitably fill and spill faster then the data contained may be examined. These issues vary from the micro scale when we are concerned with exhausting the relatively few network IO buffers, to macro issues similar to running low on disk resources. Management of system resources is a complex task that is made exceedingly difficult by requirements to monitor an unknown amount of communication streams and a limited view of the actual internal TCP/IP stack state for each host. IP Options Upon examination of an IP header, there are a number of fields in which, with methodical alteration, some insertion or evasion vulnerabilities will become apparent. Mangling the IP header must be done with care; our traffic must still be valid such that it can be routed across the Internet. Modifying the size of a packet may make it difficult for the IDS to understand where the upper layers of the packet begin (evasion). The IP checksum is another good start; if we can interleave invalid IP packets in our stream, the IDS may accept them as valid (if it does not manually calculate the checksum for every packet) where the end system does not (insertion). Time To Live Attacks In a typical network configuration, a NIDS would most often be placed on the perimeter of a network. This would enable the NIDS to monitor all communication across the Internet. Unfortunately if an attacker is able to traceroute or methodically reduce the Time to Live (TTL) of the traffic to the target and identify the exact amount of hops required to reach the host, they would then be able to send some packets with an insufficient TTL value. This would have the effect of ensuring the packets with a lower TTL would never reach the target system, but would instead be possessed by the IDS as part of the stream, as seen in Figure 16.1. Luckily administrators may be able to combat this attack by configuring their IDS on the same network segment as the hosts they wish to monitor. Figure 16.1 TTL Insertion Attack IP Fragmentation IP fragmentation reassembly is the basis for a number of attacks. If a NIDS sensor does not reassemble IP fragments in a similar fashion as the target host, it will not be able to match the packet to its signature database. In normal network operations, IP fragments will typically arrive in the order in which they are sent. However, this is not always the case; IP supports difficult-to-analyze out-of-order transmission and overlapping fragment reassembly behaviors. Assembling IP fragments can also become complicated by the requirement to keep fragments in memory until the final fragment is received, in order to complete the assembly of the entire packet. This raises yet another DoS issue; many fragments can be transmitted to consume any internal buffers or structures so that the IDS may begin to drop packets or even crash. We can further elaborate on this issue when we add the complexity of internal garbage collection. An IDS listening to the wire may have to account for the sessions of several thousand hosts, whereas each host need only be concerned with its own traffic. A host system may allow an excessive amount of time for fragments to arrive in the stream whereas the IDS may have more aggressive timeouts in order to support the management of an exponentially larger system. If the attacker were to send an attack consisting of three fragments and withhold the final fragment until a significant amount of time has expired, and if the NIDS does not have identical internal fragment management processes (something tells me this is next to impossible to attain), it will not have a consistent view of the IP packet and will therefore be incapacitated from any signature matching processes. Fragmentation Tests A number of tests conducted by Ptacek and Newsham revealed that at the time of testing none of the IDS platforms that were analyzed could properly interpret a number of IP fragmentation issues. The first two tests covered involved an in-order fragmented payload that was sent in two different sizes (8 and 24 bytes). Further testing was done where 8-byte fragments were sent—with one fragment sent out of order (evasion), with a fragment twice (insertion), with all fragments out of order and one duplicate (combination), by sending the fragment marked as the last fragment first (evasion), and by sending a series of fragments that would overlap the previous (evasion). Startling as it may seem, none of the four products (RealSecure, NetRanger, SessionWall and NFR) were able to handle any of the fragmentation attacks. Currently most NIDS have updated their fragmentation assembly engines such that they are capable of reconstructing streams with some degrees of success. TCP Header The TCP header contains a number of fields that are open to exploitation, and so opportunities for evasion and insertion exist if an IDS were not to fully inspect the TCP header. The CODE field defines the type of message being sent for the connection; if someone were to send an invalid combination or a packet missing the ACK flag it would be possible that the target host would reject the packet where the IDS would not (insertion possible). Segments marked as a SYN may also include data; due to the relative infrequent use of this option for data, an IDS may ignore the contents of these types as well (evasion). We can examine many of the fields in the TCP header and look for any opportunity where a target host will either accept traffic that the IDS does not or vice-versa. Another great example is the “Checksum” filed, where if the IDS were not manually calculating the checksum for every TCP segment, we may intermix segments with an invalid checksum into our legitimate session with the hope that the IDS will not validate all segments (the vendor may have assumed the processing overhead too great). TCP recently added a number of new TCP options with RFC 1323, `TCP Extensions for High Performance,'' by V. Jacobson, R. Braden and D. Borman introduce (amongst other things) , Protection Against Wrapped Sequence numbers (PAWS) and the option for non-SYN packets to contain new option flags. This means that if an IDS does not know how a target system may deal with non-SYN packets containing options, there are multiple opportunities for insertion and evasion. The target system may reject this newer form of TCP where the IDS will not, and again the converse is also true. PAWS is a mechanism where a system will have a timestamp associated with each TCP segment. If the target host were to receive a segment with timestamp less then its internal threshold value, it will be dropped. Again and again we see the difficulty with examining TCP data on the wire. There is simply not enough state information transmitted to give an accurate picture of what the behavior will be of a potential target host. I’d love to see a reference to the RFC that covers PAWS here. TCP Synchronization Just as there are a number of attack vectors available against strictly IP communications, when we begin to analyze layers above IP, the added complexity and requirements for functionality produce new synchronization challenges. Today most IDS platforms have implemented “stateful’ inspection for TCP. Stateful inspection requires a number of design decisions about how to identify a communication stream when you examine TCP data. An IDS must be capable of reconstruction a stream in an identical manner as the destination host—if it can not, there will be opportunities for an attacker to subvert the analysis engine. The state information for a TCP session is held in a structure known as a TCP Control Block (TCB). A TCB (containing information like source and destination, sequence numbers and current state) will be required for each session that a NIDS will monitor. The three attack vectors that Ptacek and Newsham identified are as follows: TCB creation Stream reassembly TCB teardown. An IDS would have to participate in these processes to identify new sessions, monitor open connections, and to identify when it is appropriate to stop monitoring. TCB Creation Understanding how to begin monitoring a connection poses some interesting challenges. Should the NIDS simply monitor the TCP handshake processes and build a TCB at this time? Can the NIDS effectively establish a TCB for a connection for which it did not see a SYN (connections that were active before the monitor)? There are unique challenges with any technique used to establish a TCB. It would be desirable for the IDS to be able to monitor connections for which it did not see an initial Three Way Handshake (3WH). If not, an attacker could establish a connection and wait a significant amount of time; the IDS may reboot and then be unable to track the already established connection. It is possible to only use ACK packets for TCB creation. This is known as “synching on data''. With the added benefit of being able to identify sessions for which a 3WH has not been inspected. There are a number of drawbacks, one being that the IDS will likely inspect excessive amounts of data as it will not be able to differentiate packets not part of a stream from established connections. Another issue is that syncing on data causes a dependence on accurate sequence number checking. The attacker may be able to desynchronize the IDS by spoofing erroneous data before attempting the attack. An alternate technique to TCB creation is to require a SYN+ACK combination to be seen. This will have the added benefit that it is nearly impossible for the attacker to effect the ACK from the target network. This will enable the IDS to identify which host is the server and client. However, the IDS may be able to be tricked into opening tracking many connections for non-existent hosts (DoS). A SYN+ACK can be easily spoofed without requiring the final ACK from the originating host and care should be taken when relying on this mechanism for TCB creation. A combination of methods is usually the best strategy, building on the strengths while attempting to eliminate the weaknesses of each technique. Stream Reassembly A number of similar issues exist for TCP stream reassembly as for IP fragmentation assembly. The TCP segments may arrive out of order, overlap and possibly be redundant. The IDS must take special care to monitor the sequence numbers of each connection to ensure they do not get desynchronized (difficult to do in a heavily loaded environment). Again, the difficulty with interpreting the possible behavior of the destination host, while not knowing the particulars about its TCP/IP stack implementation, is quite challenging. In the case of a redundant TCP segment, some hosts may retain the older frame, while others may discard in favor of the most recently received. If an IDS hopes to maintain a consistent view of the traffic being evaluated, it must also be weary of the advertised windows size for each connection; this value is often tuned during a session to ensure maximum throughput. If an IDS were to lose sight of the size of the TCP window, it may be vulnerable to an easy insertion attack where the attacker simply sends in excess of the window size, in which case the destination host will simply drop packets that were received outside of its advertised size. TCB Teardown To ensure that a DoS condition does not occur, proper garbage collection must take place. There are some challenges here. Connections may terminate at any time, with or without notice. Some systems may not require RST segments to be properly sequenced. The Internet Control Message Protocol (ICMP) may even terminate a connection; most hosts will respect an ICMP destination unreachable message as an appropriate signal for termination. If the IDS is not aware of these semantics it may become desynchronized and unable to track new connections with similar parameters. There will almost undoubtedly be some timeout for any established connection to prevent some logic error from eventually leaking memory. This will also lead to an attack that we had eludedalluded to earlier. Most hosts do not employ keep-alive messages for all connections. This leaves an IDS in an undesirable position where an attacker may simply wait for an excessive amount of time and possibly simultaneously provoke the IDS to become more aggressive with its garbage collection (by establishing many new connections). If successful, the attacker will be able to send whatever attacks they wish, undetected. Using Fragrouter and Congestant Theory is not enough for some to make a judgment on the performance of security products. We have seen time and time again that many vendors do not heed the warning of the research community. To adequately illustrate the vulnerabilities that NIDS face, Dug Song released fragrouter in September 1999 (). Fragrouter's benefit is that it will enable an attacker to use the same tools and exploits they have always used without modification. Fragrouter functions, as its name suggests, as a sort of fragmenting router. It implements most of the attacks described in the Ptacek and Newsham paper. Congestant is another great tool that implements a number of anti-IDS packet mangling techniques. This is a product ofwas authored by "horizon" and was first released in December 1998 in his paper, “Defeating Sniffers and Intrusion Detection Systems” (www.phrack.org/show.php?p=54&a=10) for phrack 54. The difference here is that congestant Congestant is implemented as a shared library or a kernel patch to OpenBSD. You may find that it is possible to use these tools concurrently for some added confusion for the IDS sensor. Increasing the processing overhead and complexity of IDS sensors is of benefit to an attacker; these systems become more prone to DoS and less likely to perform in an environment of extreme stress (large amount number of packets per second). It is a certainty that there will always be more features and options added to IDSs as they mature, as an attacker will always attempt to identify the critical execution path (the most CPU intensive operation an IDS may make) in attempts to stress an IDS sensor. I’d love to see some detail here. These programs are designed to be pretty transparent. And chance you could print a before-and-after packet going through fragrouter? Here is the output when running fragrouter from a shell, it’s pretty plug-and-play, you just need to ensure that your system will route through the “fragrouter” host to reach the target. storm:~/dl/fragrouter-1.6# ./fragrouter -F5 fragrouter: frag-5: out of order 8-byte fragments, one duplicate truncated-tcp 8 (frag 21150:8@0+) 10.10.42.9 > 10.10.42.3: (frag 21150:8@16+) 10.10.42.9 > 10.10.42.3: (frag 21150:8@8+) 10.10.42.9 > 10.10.42.3: (frag 21150:8@16+) 10.10.42.9 > 10.10.42.3: (frag 21150:4@24) truncated-tcp 8 (frag 57499:8@0+) 10.10.42.9 > 10.10.42.3: (frag 57499:8@8+) 10.10.42.9 > 10.10.42.3: (frag 57499:8@8+) 10.10.42.9 > 10.10.42.3: (frag 57499:4@16) truncated-tcp 8 (frag 57500:8@0+) 10.10.42.9 > 10.10.42.3: (frag 57500:8@8+) 10.10.42.9 > 10.10.42.3: (frag 57500:8@8+) 10.10.42.9 > 10.10.42.3: (frag 57500:4@16) truncated-tcp 8 (frag 58289:8@0+) 10.10.42.9 > 10.10.42.3: (frag 58289:8@8+) 10.10.42.9 > 10.10.42.3: (frag 58289:8@8+) 10.10.42.9 > 10.10.42.3: (frag 58289:4@16) Here is a comparison of what the tcpdump output from the F5 “fragrouter: frag-5: out of order 8-byte fragments, one duplicate” technique would appear against normal traffic. Note the DF (Don’t Fragment) flags on every packet of a normal connection and that the fragrouter stream has several fragmented packets. Before (no fragrouter): 19:36:52.469751 10.10.42.9.32920 > 10.10.42.3.7: S 1180574360:1180574360(0) win 24820 (DF) 19:36:52.469815 10.10.42.9.32920 > 10.10.42.3.7: S 1180574360:1180574360(0) win 24820 (DF) 19:36:52.470822 10.10.42.9.32920 > 10.10.42.3.7: . ack 4206722337 win 24820 (DF) 19:36:52.470841 10.10.42.9.32920 > 10.10.42.3.7: . ack 1 win 24820 (DF) 19:36:53.165813 10.10.42.9.32920 > 10.10.42.3.7: F 0:0(0) ack 1 win 24820 (DF) 19:36:53.165884 10.10.42.9.32920 > 10.10.42.3.7: F 0:0(0) ack 1 win 24820 (DF) 19:36:53.171968 10.10.42.9.32920 > 10.10.42.3.7: . ack 2 win 24820 (DF) 19:36:53.171984 10.10.42.9.32920 > 10.10.42.3.7: . ack 2 win 24820 (DF) After (with fragrouter): 19:37:29.528452 10.10.42.9.32921 > 10.10.42.3.7: S 1189855959:1189855959(0) win 24820 (DF) 19:37:29.528527 10.10.42.9.32921 > 10.10.42.3.7: S 1189855959:1189855959(0) win 24820 (DF) 19:37:29.529167 10.10.42.9.32921 > 10.10.42.3.7: [|tcp] (frag 21150:8@0+) 19:37:29.529532 10.10.42.9.32921 > 10.10.42.3.7: . ack 4211652507 win 24820 (DF) 19:37:29.529564 10.10.42.9.32921 > 10.10.42.3.7: . ack 1 win 24820 (DF) 19:37:29.530293 10.10.42.9.32921 > 10.10.42.3.7: [|tcp] (frag 57499:8@0+) 19:37:30.309450 10.10.42.9.32921 > 10.10.42.3.7: F 0:0(0) ack 1 win 24820 (DF) 19:37:30.309530 10.10.42.9.32921 > 10.10.42.3.7: F 0:0(0) ack 1 win 24820 (DF) 19:37:30.310082 10.10.42.9.32921 > 10.10.42.3.7: [|tcp] (frag 57500:8@0+) 19:37:30.316337 10.10.42.9.32921 > 10.10.42.3.7: . ack 2 win 24820 (DF) 19:37:30.316357 10.10.42.9.32921 > 10.10.42.3.7: . ack 2 win 24820 (DF) 19:37:30.316695 10.10.42.9.32921 > 10.10.42.3.7: [|tcp] (frag 58289:8@0+) Countermeasures For those wishing to implement NIDS throughout their network infrastructure, fortunately there are some emerging technologies that help eliminate a great many of these lower-layer protocol vulnerabilities. Protocol normalization, as discussed by Mark Handley and Vern Paxson in May 2001 in “Network Intrusion Detection: Evasion, Traffic Normalization, and End-to-End Protocol Semantics” (), is an attempt to scrub or rewrite network traffic as it enters a destination network. This scrubbing process should eliminate many of the difficulties in reconstructing a consistent view of network traffic. If an IDS and target host were both behind a network protocol scrubber, they would both receive an identical picture of the network traffic. Tools & Traps… Honeynets Recently there has been an upsurge in the use of honeynets as a defensive tool. A honeynet is a system that is deployed with the intended purpose of being compromised. These are hyper defensive tools that can be implemented at any location inside a network. The current best known configuration type for these tools is where two systems are deployed, one for the bait, the other configured to log all traffic. The logging host should be configured as a bridge (invisible to any remote attacker) with sufficient disk space to record all network traffic for later analysis. The system behind the logging host can be configured in any fashion. Most systems are quite simply bait, meaning that they are designed to be the most attractive target on a network segment. It is the hope of the defender that all attackers would see this easy point of presence and target their attacks in that direction. Although it has been seen that there is cause to have bait systems configured identically to other production systems on the target network (hopefully hardened), so that if an attackers presence is detected on the honeynet (nobody can transmit any data to this system without detection), the defender can be sure that there are vulnerabilities in their production configuration. And with the added benefit of detailed logging, some low level forensics will typically reveal the vulnerability information along with any backdoors the intruder used to maintain their foothold. Luckily However, no system is foolproof. Attackers should be able to discern that they are behind a bridge by the lack of layer2 traffic and the discrepancy in Media Access Control (MAC) addresses in the bait systems arp cache. See http://project.honeynet.org for more details. Using Application Protocol Level Evasion IDS sensors have the ability to inspect the protocol internals of a communications stream to aid in the detection process. There are two basic strategies that vendors employ: application protocol decoding, where the IDS will attempt to parse the network input to determine the legitimacy of the service request, and simple signature matching. Both of these approaches have their own unique challenges and benefits; we will see that most IDSs probably implement a hybrid of these solutions. Opportunities to evade detection are available at every layer of the protocol stack. Security as an Afterthought Application developers are typically motivated by features and dollars. We all know that the end user is the ultimate decision maker on the success or failure of software. In an effort to please end users, provide maximum compatibility, and eliminate erroneous conditions, developers make many concessions towardsomit strict compliance of protocol specifications in favor of error correction. It is uncommon for an application to immediately terminate requests upon the first deviation from specified protocols—quite to the contrary, every effort is made to recover from any error in an attempt to service every request possible (thereby maximizing compatibility and possibly increasing interoperability). As security researcher Rain Forest Puppy (known as RFP) stated at the CanSecWest Security Conference 2001, “You would be surprised with what passes for legitimate http traffic…” – RFP (CanSecWest Security Conference 2001). These practices are the downfall of application security they only serve to aid an attacker in allowing additional latitude in which to operate. That section could be read as either favoring strict compliance, or the opposite. Please re-word to make less ambiguous. Perhaps “developers forego strict compliance…” Also, RFP has a couple of chapters before this one, so you can assume the reader has heard of him by the time they get here. Evading a Match Upgrades, patches and variation of implementation may change the appearance (on the wire) of an application. Signatures, —too specific, too general and just plain too stale, —are thesea basic issues that continues to thwart IDS attack identification efforts. If we look back towards our snort signatures, we can see that quite clearly one of them specifies the complete path name for the chgrp command. This signature is supposed to alert to the execution of some command through a Web server. Any attacker who is aware of the semantics for these rules could easily modify their attack to play any number of tricks in hopes of evading this match. This rule itself is quite specific about the path and name for the chgrp command. We can plainly see that if the command resided in a different directory then /usr/bin, this signature would fail. Also, if the attacker were to simply ensure that their path environment variable were correctly set, they may just issue chgrp, without the complete path to evade a signature match. Should the IDS be configured to alert when any of these variations are present? How many signatures would our IDS have if we were to account for these many variations? Alternate Data Encodings Largely implemented to support multiple languages, the standard text sent between a web client and server may be encoded so that it should be interpreted as Unicode. Unicode gives the capability to represent any known symbol (the Unicode value for Yung is U+6C38). It also presents all new challenges to IDS vendors, as these values must be inspected and converted into ASCII ANSI (American National Standards Institute) for standard processing. This challenge is not that difficult to overcome; most systems implement a practice known as protocol normalization. Protocol normalization will take an input string and digest all known encodings, white space, and any protocol-specific delimiters in an attempt to produce the most basic form of the input. Did you mean ASCII? Yes Unfortunately all of the normalizations imaginable cannot overcome the challenge of monitoring closed source software packages. Without detailed information of the inner workings of a system there can be no accounting for undocumented nonstandard features. IIS had one such “special-feature:” %u#### encoding was allowed as an alternate to the normal Unicode encodings (%####). The famed “Code Red” worm had used this previously unknown technique to bypass many IDS signatures tuned to match for the specific .ida buffer overflow vulnerability. Lack of information is the worst enemy of a network defender. Consider the following imaginary attack: Attack String: GET /vulnerable.cgi?ATTACK=exploit-code Signature: alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS 80 (msg:"WEB-ATTACKS vulnerable.cgi attempt"; flags:A+; content:"get /vulnerable.cgi?ATTACK=exploit-code";nocase; sid:1337; rev:1; classtype:web-application-attack;) Modified Attack String: GET /vulnerable.cgi?ATTACK=exploit-code The attack here seems to exploit some Common Gateway Interface (CGI) application, and a simple signature is developed to alert to the known vulnerability. This signature would provide a very high level assurance that there would be relatively few false positives, as the exploit-code is embedded right into the signature. However, we can see that if the attacker were able to send a modified attack string, through the use of some additional white space, they should be able to bypass a signature match. This exercise again illustrates the difficulty of signature development. If the signature left out the portion of the exploit code, there may be a great number of false positives, whereas if they embed some of the exploit code, the chance for evasion is greatly increased. This is an incredibly simplistic example and is not that difficult to overcome. Adequate normalizations should be able to eliminate white space and allow for a signature match. Web Attack Techniques A number of Web attack issues have been analyzed by RFP; see for instance “A look at whisker's anti-IDS tactics” from December 1999 () He has implemented a number of them into his whisker vulnerability scanner. We'll take a look at some of them in the following sections. Since RFP is working on this book – he should probably take a look at this section. From past experience I like him to look at what people are writing about him. THX. Ryan has sent the chapter to RFP. Method Matching The method of a HTTP request informs the server what type of connection to anticipate (GET, HEAD, POST, etc). RFP found that many IDS signatures had completely failed to recognize any other methods. This is a somewhat depressing fact as many IDS vendors claim to be not totally dependent on signature matching to generate an alert. Directory and File Referencing A slash, the character that specifies a separation between directory and file names (/), can be represented in a couple of different ways. The simplest form is double or multiple slashes (/some//file.html = /some////file.html). These tricks will fool the simplest signature matches, providing there are no normalizations to counteract. Another form of the same trick (this works only on IIS Web servers), is to use the DOS slash character (\). If an IDS were not aware of this convention, it would not be able to generate a match. These tricks work because they can reference a file by a different pathname. Amazingly enough, resolving a pathname is substantially harder then you would think (this is what has lead to a number of remote compromises in IIS, remember Unicode). Dot, the path to the current directory, and double dot, the path to the previous directory, can be used to obfuscate a file reference. An attacker may only need to use his or her imagination in constructing unique paths; all of these are equivalent requests: GET /some/file.cgi HTTP/1.0 GET /.././some////file.cgi HTTP/1.0 GET /./some//..\..///some/./file.cgi HTTP/1.0 A form of the aforementioned evasions is what RFP calls parameter hiding. This evasion is based on the assumption that some IDSs may only evaluate a request until it encounters a question mark (?) , a hex-encoded value of %3f). This character is typically what will denote that any further parameters are arguments to a Web application. If the IDS simply wanted to alert to the request of a file, it may not fully evaluate the expression. The following two requests are equivalent: GET /real.file HTTP/1.0 GET /%3f/file/does/not/exist/../../../../../real.file HTTP/1.0 Countermeasures As discussed previously, a signature based IDS may be able to normalize the communications stream. That is, as it inputs data destined for a HTTP server, it should apply some logic to reduce the input into its lowest common denominator (a single /, or resolving directory references). Partial signature matches may also help, if a sensor does not enforce a strong 100% match, they should be able to account for some variation of many exploit types. Using Code Morphing Evasion Polymorphism is the ability to exist in multiple forms, and morphing is the processes that is used to achieve polymorphism. The objective of polymorphic code is to retain the same functional properties while existing in a structurally unique form. A NIDS has only the opportunity to inspect information as it exists on the wire; this would then only allow the structure of the exploit to be inspected. This feature had allowed viruses to remain undetected for quite some time. The only difference is that a virus scanner has the opportunity to inspect disk files instead of network data. The way that most virus scanning engines have tackled this problem is through the use of heuristic scanning techniques; this is similar to what a host based IDS would do (identifying suspicious events, inappropriate file access). Polymorphism is achieved through taking the original attack payload and encoding it with some form of a reversible algorithm. All of the nop-sled instructions are substituted with suitable replacements. This encoded payload is then sent over the network with a small decoding function prefixed (this decoder is also dynamically generated to avoid a signature match). When the exploit runs on the target, the decoder will unwrap the original payload and execute it. This way, the original functionality is maintained. nop-sled? Yup, check out the buffer-overflow chapter. Polymorphic shellcode is discussed thoroughly in this author's paper that was released in early 2001 (). An engine is included for use in any current or future vulnerabilities. The basis for polymorphic code generation is that there is always more then one way to calculate a value. If, to exploit a vulnerability, we had to calculate the value of 4, we could do any of 2+2, 3+1, 6-2 and so on. There are literally endless methods to calculate a given value—this is the job of an exploit, the possessing of some machine instructions. To a NIDS examining network traffic there is no way to identify 2+2 being equivalent to 3+1. The NIDS is only given the low-level machine instructions to evaluate against a known pattern; it does not interpret the instructions as the target host will. This technique has the ability to mask any exploit from detection, from any specific rule to the general. The only opportunity for a signature based NIDS to formulate a match is if a signature for the small decoder is able to be determined. To date I have not seen any signatures or techniques developed for this class of polymorphic shellcode. Table 16.1 shows a side by side view of two executions of a polymorphic shellcode engine. What should alignment of multiple items within table columns be? Table 16.1 Insert Title HereShellcode Variations Addresses Normal Shellcode Possible Polymorphic shellcode #1 Possible Polymorphic shellcode #2  0x8049b00 0x8049b01 0x8049b02 0x8049b03 0x8049b04 0x8049b05 0x8049b06 0x8049b07 0x8049b08 0x8049b09 0x8049b0a 0x8049b0b 0x8049b0c 0x8049b0d 0x8049b0e 0x8049b0f 0x8049b10 0x8049b11 0x8049b12 0x8049b13 0x8049b14 0x8049b16 0x8049b17 0x8049b19 0x8049b1b 0x8049b1e 0x8049b20 0x8049b21 0x8049b23 0x8049b25 0x8049b26 0x8049b28 0x8049b2a 0x8049b2b 0x8049b2d 0x8049b2f 0x8049b31 0x8049b33 0x8049b35 0x8049b36 nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop jmp 0x8049b38 pop %esi mov %esi,%ebx mov %esi,%edi add $0x7,%edi xor %eax,%eax stos %al,%es:(%edi) mov %edi,%ecx mov %esi,%eax stos %eax,%es:(%edi) mov %edi,%edx xor %eax,%eax stos %eax,%es:(%edi) mov $0x8,%al add $0x3,%al int $0x80 xor %ebx,%ebx mov %ebx,%eax inc %eax int $0x80 push %ebx cmc pop %edx xchg %eax,%edx lahf aas push %esi push %esp clc push %edx push %esi xchg %eax,%ebx dec %ebp pop %ecx inc %edi dec %edi inc %ecx sahf pop %edi sti push %esp repz dec %eax push %ebp dec %esp pop %eax loope 0x804da1b js 0x804d994 daa sbb $0x15,%al pop %eax out %eax,(%dx) push %ebp dec %edi jp 0x804d966 movl %es:(%ecx),%ss mov $0x15d5b76c,%ebp adc %edi,(%edi) loopne 0x804d9a0 push %ebp xchg %eax,%ecx das pushf inc %ecx xchg %eax,%ebp pop %edi push %edi dec %ebp dec %ebx lahf xchg %eax,%edx push %ebx pushf inc %esp fwait lahf pop %edi dec %ecx dec %eax cwtl dec %esp xchg %eax,%ebx sarb $0x45,(%ecx) mov 0xffffff90(%ebx),%ebp dec %edi mov $0xd20c56e5,%edi imul $0x36,0xee498845(%esi),%ebx dec %ecx and %ah,%cl jl 0x804da3d out %al,$0x64 add %edi,%eax sarl %cl,0x4caaa2a0(%ebp,%eax,2) nop cmp 0x5cd8733(%eax),%ebx movsl %ds:(%esi),%es:(%edi) push %ss int $0x14 push $0xbffff586 xchg %dh,%ch (bad)   As you can plainly see, there is very little correlation between the three executions. There are very many possibilities There are a huge number of permutations that can be used. Unfinished thought? What’s the (bad) there? That is part of the encoded shellcode, it’s value did not decode by gdb as a valid instruction so it just pop’s up as “(bad)”. Countermeasure It is apparent that most IDSs are not always quite ready to run out of the box. They require frequent updating and maintenance to yield long-term success. Some The IDSs that do have hope of detecting unknown forms of attack are anomaly detection based. These systems do not use signatures at all. They instead monitor all network communications as they occur and attempt to build a high level image of typical traffic. A statistical anomaly would then trigger an alert. As the system matures and gains more entropy into its database, it would then theoretically become more accurate. There is some question whether or not a purely anomaly-based detection engine would be very effective, as exploit attempts seem to be quite normal in day-to-day network operation and may fall into the baseline of these systems. As in all things, a little of each is not a bad idea. A strong signature based system supplemented by an anomaly based detection engine should yield a high level of assurance that most intrusion events are monitored. In the endless security game of cat and mouse, one can forecast the generation of polymorphic statistically normalized attack engines that should provide one more hurdle for NIDS developers to overcome. Summary Signature based IDS sensors have many variables to account for when attempting to analyze and interpret network data. Many challenges continue to elude these systems. The lack of information that is available for inspection is difficult to overcome. However, the rate at which many IDS sensors have been maturing is quite promising; Gigabit speeds and flexible architectures supported by an ever growing security community push forward to achieve configure systems that are capable of detecting all but the most obtuse and infrequent attack scenarios. At every layer of the network stack there are difficulties with maintaining a consistent view of network traffic and the effect of every packet being transmitted. It is quite clear that an attacker has certain advantages, being able to hide in a sea of information while being the only one aware of their true intension. Packet layer evasions have been well documented throughout the past several years. IDS vendors are quite aware of the many issues surrounding packet acquisition and analysis. Most networks are beginning to filter “suspicious” packets in any case, that is any types with options and excessive fragmentations. Perhaps in the coming years network layer normalizations will become commonplace and many of these evasion possibilities will evaporate. The difficulty with analyzing the application layer protocols continues to cause ongoing headaches. Some proxy solutions have begun to take hold but the bottleneck that these systems cause is often too great. They also suffer from similar issues as IDSs, unable to identify classes of attacks that they were not originally intended for. It is simple quite acceptable to quash malformed TCP/IP packets in the case of an error; the a legitimate end system will would eventuallysimply retransmit. The same is not true for higher layers; a NIDS may have an extremely limited understanding of application protocols and the information they transmit. Polymorphic attacks present a significant challenge that cannot be easily solved with a purely signature based system. These attacks may exist in virtually limitless combinations. Fix grammar IDS evasion will continue to be a way of life on the Internet. There will beis an ever-flowing renewing tide of tools and techniques that are developed and refined designed for large-scale implementation (eventually raising the everyday script kitty kiddie into a previously reserved more advanced skill set) to make the job of detection more difficult.. One should continually monitor and investigate network activity to gain an understanding of what to expect on day-to-day operations. hold dear the principles of least privilege, segmentation and auditing to ensure their overall network posturing remains as secure as possible. Clarify? Solutions Fast Track Author: Please fortify these bullet points so they are all full sentences. They should also be a bit more informative and useful to the reader who wants to use them to brush up on the chapter material—you could add a sentence to each that makes it clearer why that particular point is relevant to the chapter. Understanding How Signature-Based IDSs Work Capabilities defined in signature database The capabilities of a NIDS are defined by a signature database. This enforces the requirement for frequent updates to combat the frequency of new vulnerabilities. Difficult to extrapolate from defined database Most NIDS do not alert even to slight variations of the defined signatures. This affords an attacker the ability too vary there attack to evade a signature match. Signatures are very specific to a vulnerability slight variations will be missed Increase the processing overhead required for detection Attackers will continue to vary there evasion techniques such that the processing required to monitor and detect is greatly increased. This would contribute to DoS and evasion possibilities. Using Packet Level Evasion Many vendors implement TCP/IP with slight variations. A NIDS has a difficult time in constructing a view of network communications as they appear to other systems. This inconsistent view is what allows an attacker to evade detection. Hosts may not adhere to RFC specifications and allow some packets where the NIDS would may not. drop NIDS do not have enough information from the wire to reconstruct TCP/IP communications. With the options and states available in a TCP/IP stack, some ambiguities form as to how a host would interpret information, there is an insufficiently of information transmitted between systems when communicating. Fragrouter and congestant are effective evasion tools. They implement a number of documented NIDS evasion techniques. Using Protocol and Application Protocol Level Evasion Application protocols are difficult to interpret Application protocols are verbose and rich in function. There are many subtle, antiquated and obscure application nuances that make effective application protocol decoding difficult. An attacker may compromise even the slightest oversight. Applications tend to allow for slight variation, developers intentionally build in error correcting cases that attempt to make sense of any request, no matter how malformed. With a lax of strict compliance to defined specifications, it is difficult for the NIDS to determine the behavior of a network application. Multiple encoding options exist for data representation, Unicode, uuencoded or hex encoded options exist in many application protocols. These alternate representations complicate the development of detection engines. Using Code Morphing Evasion There is always more then one way to do it. When detection hinges on the identification of application code, there are many alternatives to code generation. Code may be randomly generated The code of an attack may be pseudo randomly generated. Any number of instructions can accomplish similar tasks, the code must simply function there is no requirement of performance or other optimization benefits. Most exploits will vary from host to host. Variations can be incorporated even when restrictions are placed on the length or type of codes possible. Frequently Asked Questions Q: How many IDSs do I need to make them more effective? A: All networks are different and require varying levels of monitoring. Your particular risk tolerance should help you find this out though. A network witch desired a high level of assurance that they are detecting many intrusion events, should have at least one sensor per network segment (layer 2). It is also desirable to have multiple vendor types implemented when an even hirer higher level of security is needed (one vendor’s strengths would hopefully fill in gaps from another) Q: Aren’t these techniques too advanced for most attackers? A: Just like most other technologies, attack methodologies and techniques are eventually turned into boilerplate applications that anybody can wield. The layout of the virtual battlefield may change in an instant. The next big worm might wield these techniques, and force a sea-change in the IDS market. Q: How do I choose the best IDS? A: Continue to do as much research as possible. The biggest tool that a network architect has is that only they are aware of all the tricks that are deployed on the enterprise. Be creative, use multiple vendors’ technologies, and implement honeypots for advanced warning of unknown techniques and vulnerabilities. Homegrown technologies (if properly tested and implemented) often do a great service when a high degree of security is required. Q: Where can I get information about new evasion attacks? A: The “underground” scene is typically the catalyst for advancements in security technologies. Frequent online publications, get a feel for where useful information may come from. There is no single source for where all new papers will be distributed. Check out: antisec (http://anti.security.is) Phrack (http://) Packetstorm () Technotronic (http://www.technotronic.com/) Drop a couple of names, if you would if you like Phrack, some mailing list, etc please mention here. Q: What do I do if I am inundated with alerts? A: Secure systems rely on compartmentalization to hopefully contain intruders. If you see that you are being attacked at an abnormal pace, isolate and separate the troubled systems and attempt to identify if there are some hosts with some well-known vulnerabilities or exposures. Correlate your logs and IDS events to give you a better picture of what may be going on. Do not rely on authorities and the network administrators of the attacking networks; they are usually far too overworked or uninterested to give a respectable amount of support. Q: How do I know that my IDSs are working? A: Ongoing auditing and testing should be done to ensure that networking systems are properly implemented. Independent reviewers should always be apart of secure systems to ensure a fresh set of eyes evaluate a network architecture and IDS implementation.