J. A. Drew Hamilton, Jr., Ph.D.web.cse.msstate.edu/~hamilton/P3I/CEH/lessons/9_Malware_Trojan… ·...

Mississippi State University Center for Cyber Innovation 1

J. A. “Drew” Hamilton, Jr., Ph.D. Director, Center for Cyber Innovation

Professor, Computer Science & Engineering

CCI Post Office Box 9627 Mississippi State, MS 39762

Voice: (662) 325-2294 Fax: (662) 325-7692 [email protected]


Section Objectives

1.  Describe malware types and their purpose 2.  Identify malware deployment methods 3.  Describe the malware analysis process 4.  Identify malware countermeasures 5.  Describe DoS attacks and techniques 6.  Identify DoS detection and countermeasure

action 7.  Describe session hijacking and sequence

prediction


Malware

Dr. Drew Hamilton Reference: Dr. John Copeland

Reference: Matt Walker All-in-One CEH Certified Ethical Hacker


Virus - code that copies itself into other programs

Payload - harmful things it does, after it has had time to spread.

Worm - a program that replicates itself across the network (usually riding on email messages or attached documents (e.g., macro viruses).

Trojan Horse - instructions in an otherwise good program that cause bad things to happen (sending your data or password to an attacker over the net).

Logic Bomb - malicious code that activates on an event (e.g., date).

Trap Door (or Back Door) - undocumented entry point written into code for debugging that can allow unwanted users.

Viruses, Worms, and Trojan Horses


1.  Have a well-known virus protection program, configured to scan disks and downloads automatically for known viruses.

2.  Do not execute programs (or "macro's") from unknown sources (e.g., PS files, HyperCard files, MS Office documents, Java, ...), if you can help it.

3.  Avoid the most common operating systems and email programs, if possible.

Virus Protection

Mississippi State University Center for Cyber Innovation 6 6

Professor John Copeland

Need Host Program

Independent

Trapdoors

Logic Bombs

Trojan Horses

Viruses

Bacteria

Worms

Taxonomy of Malicious Programs


Virus Phases •  Dormant - waits for a trigger to start replicating •  Propagation - copies itself into other programs of the

same type on a computer. –  Spreads when the user shares a file with another computer. –  Usually searches a file for it’s own signature before

infecting. •  Worms spread over a network connection as

executable attachments to email. •  Triggering - starts delivering payload.

–  Sometimes triggered on a certain date, or after a certain time after infection.

•  Execution - payload function is done. –  Perhaps it put a funny message on the screen, or wiped the

hard disk clean. –  It may become start the first phase over again.


Virus Types •  Parasitic Virus - attaches itself to executable files

as part of their code. Runs whenever the host program runs.

•  Memory-resident Virus - Lodges in main memory as part of the residual operating system.

•  Boot Sector Virus - infects the boot sector of a disk, and spreads when the operating system boots up (original DOS viruses).

•  Stealth Virus – explicitly designed to hide from Virus Scanning programs.

•  Polymorphic Virus - mutates with every new host to prevent signature detection.


CEH Virus Types •  Ransomeware

–  locks (encrypts) you out of your own system resources and demands an online payment of some sort in order to release them back to you

–  ransomeware “family” includes Cryptorbit, CryptoLocker, CryptoDefense, & police-themed names.

•  Boot sector virus –  Also known as a system virus, this virus type actually

moves the boot sector to another location on the hard drive, forcing the virus code to be executed first.

•  Shell virus –  Working just like the boot sector virus, this virus type

wraps itself around an application’s code, inserting its own code before the application code.

–  Every time the application is run, the virus code is run first.


CEH Virus Types

•  Cluster virus –  modifies directory table entries so that user or system

processes are pointed to the virus code itself instead of the application or action intended.

•  Multipartite virus –  Attempts to infect both files and the boot sector at the

same time. –  This generally refers to a virus with multiple infection

vectors. •  Macro virus

–  Usually written with Visual Basic for Applications (VBA), this virus type infects template files created by Microsoft Office, normally Word and Excel.


CEH Virus Types •  Polymorphic code virus

–  This virus mutates its code using a built-in polymorphic engine.

–  These viruses are difficult to find and remove because their signatures constantly change.

•  Encryption virus –  use encryption to hide the code from antivirus scanners.

•  Metamorphic virus –  This virus type rewrites itself every time it infects a new file.

•  Stealth virus –  Also known as a “tunneling virus,” this one attempts to

evade antivirus (AV) applications by intercepting the AV’s requests to the operating system (OS) and returning them to itself instead of OS.


CEH Virus Types

•  Cavity virus –  Cavity viruses overwrite portions of host files so as not

to increase the actual size of the file. –  This is done using the null content sections of the file

and leaves the file’s actual functionality intact. •  Sparse infector virus

–  These only infect occasionally. •  File extension virus

–  These viruses change the file extensions of files to take advantage of most people having file extension view turned off.


CEH Worm Types •  Code Red

–  Exploited indexing software on IIS servers in 2001. •  Darlloz

–  The worm for the “Internet of Things,” darlloz is a Linux-based worm that targets running ARM, MIPS, and PowerPC architectures.

•  Slammer –  Also known as SQL Slammer, this was a denial-of-service worm

attacking buffer overflow weaknesses in Microsoft SQL Services. •  Nimda A

–  successful file infection virus that modified and touched nearly all web content on a machine.

•  Bug Bear –  Propagating over open network shares and e-mail, Bug Bear

terminated AV applications and set up a backdoor for later use.

•  Pretty Park –  Pretty Park spread via e-mail (attempting a send every 30 minutes)

and took advantage of IRC to propagate stolen passwords.


Viruses, Trojan Horses, Worms, etc.

•  Resources –  http://www.wildlist.org/ –  http://www.iwar.org.uk/comsec/ –  ‘Viruses Revealed” by Harley,

Slade and Gatticker, McGraw-Hill

–  “Modern Operating Systems,” by Tanenbaum

Virus researchers owe this man a debt of gratitude


The three pillars of Information Security

•  Confidentiality: protecting information from unauthorized disclosure;

•  Integrity: protecting information from unauthorized modifications, and ensure that information is accurate and complete;

•  Availability: ensuring information is available when needed;


Direct Damage from Virus & Trojan Payloads

•  Availability –  Deletion of files and directories –  Renaming of files –  Encryption of files, disks, system calls –  Unauthorized calls to system software such as FORMAT,

FDISK, etc. •  Integrity

–  Corruption of system files and system areas (MBRs, FAT, etc.) –  Garbling data such as spreadsheet formulas... –  Corruption of both application and data files by unauthorized

file writes •  Confidentiality

–  Capture and forwarding of passwords –  Forwarding of personal and confidential files to newsgroups

and elsewhere


Defining a Computer Virus

•  A virus is an entity that uses the resources of the host to spread and reproduce itself, usually without informed operator action.

•  A virus cannot execute on its own. •  Strong viruses use normal computer operations

to achieve the virus design goals. •  There is no single characteristic that can be used

to identify a previously unknown virus program. •  Consequently, there is some academic

disagreement as to just how many viruses have been released, what variants define different strains.


Virus Structure

•  Infection: The infection mechanism may be defined as the way or ways in which the virus spreads.

•  Payload: The payload mechanism is defined as what (if anything) the virus does in addition to replicating.

•  Trigger: The trigger mechanism is defined as the routine that decides what time to deliver the payload if there is a payload.


Virus Damage

•  Deliberate damage inflicted by the virus payload mechanism, if it exists, such as the trashing or intentional corruption of files.

•  Accidental damage caused when the virus attempts to install itself on the victim system (the newly infected host), such as corruption of system areas preventing the victim system from booting.

•  Incidental damage that may not be obvious but is nevertheless inherent in the fact of infection. Nearly all viruses entail damage in this category, since their presence involves loss of performance due to theft of memory, disk space, clock cycles, system modifications or combination of these,


Some Social Impacts

•  Scapegoating of virus victims •  Secondary damage to systems caused by

inappropriate responses to a perceived virus threat (ex. low-level formatting of a hard disk to eradicate a macro virus.)

•  Legal or quasi-legal issues such as failure to comply with data-protection legislation and policies.

•  Inappropriate security responses –  reformatting –  passwords –  change in business models


A Few Examples of Virus Damage

•  The disappearance of Word menu options relating to the presence of macros.

•  Encryption or displacement of system areas, such as the Master Boot Record.

•  Manipulation of the Windows Registry •  Trashing or corruption of legitimate macros as

part of the installation of a macro virus.


Latency

•  Unexecuted viruses are latent or dormant –  ex. mailbox full of unread, infected mail –  ex. PC-specific virus residing on a Mac or a UNIX

server. •  “Heterogeneous virus transmission.”


Some Useful Terms

•  Intendeds: reproductive mechanism never triggers, or if triggered, code never attaches to host. –  ex. virus intended to execute on Sundays and uses

DOS system call Get Date. Virus waits for Get Date to return “7” but Get Date only returns values between “0..6.”

•  Corruptions: may be caused by system transfers, incomplete “cleansing” and poorly maintained virus collections. –  Antivirus programs often detect corrupted non-viral

programs simply to avoid being penalized by incompetent testers and reviewers.


Virus Design Considerations

•  Polymorphic Viruses – change structure in attempts to avoid detection

•  Non-Resident (direct action) versus Memory-Resident viruses. –  Hybrids –  Macros

•  Payload versus reproduction •  Damage

–  In general, a virus can do anything any other software can do

•  Boot Sector


Attaching viral code to an existing program

•  Overwrite existing program code (overwriting viruses)

•  Add code to the beginning of the program (prependers)

•  Add code to the end of the program (appenders) •  Insert viral code into the command chain so that

it is run when the legitimate code is executed (parasitic viruses or file infectors) –  Macro viruses are a special case of a file infector

•  These methods are becoming less common as VBScript, AOL programs and MS Office macros continue to ease the task of virus writers.


Polymorphic Virus Techniques

•  Objective: fool scanners, make signatures harder to identify

•  Methods –  Encryption: Start with a “random” number such as the

value of seconds in system time then use that as a key to encrypt part of the payload.

–  Arbitrary code relocation: rearrange code after each infection.

•  Detection –  change detection –  activity monitoring –  detecting the mutating engine in kit-produced viruses –  bankruptcy of scanners that cannot detect polymorphic

viruses


Stealth Technology

•  Even if the virus is new or polymorphic, it still adds code to the infected program increasing size. –  If overwrites are carefully managed to avoid increasing

size, new code will still fail a CRC. •  Traps – a stealthy virus will try and intercept

system calls to avoid detection –  Determine another program is trying to access the

memory the virus is occupying so hang the system –  Trap the attempt to read the disk information and return

an image of the disk information before infection


Virus Symptoms

•  System software, applications, or utilities display unpredictable behavior.

•  GPFs and similar conflicts and errors are encountered

•  Parity and checksum errors are encountered •  Loss of performance

–  ex. loss of 32-bit access •  Loss of access to system areas may be observed,

possibly entailing lost access to normally mounted volumes and subsequent unavailability of data and/or applications.


Review of Boot Sequence

1.  The user powers up the computers 2.  Computer runs a power supply self-test 3.  ROM BIOS code is executed 4.  ROM BIOS performs a test of the central

hardware 5.  Computer runs a video test 6.  Computer runs a memory test 7.  On a cold boot, the full POST is run, skipped on a

warm boot. 8.  Computer tests for the partition boot record at

the first sector of the default boot drive (specified in the BIOS).


Review of Boot Sequence (2)

9.  The partition boot record is executed. 10. The computer initializes specified system files, or displays

a message if these are not available (in DOS IO.SYS and MSDOS.SYS, in NT, NTLDR, NTDETECT.COM checks hardware and NTOSKRNL.EXE initializes the OS)

11. The base device drivers are initialized and device status is checked.

12. The computer reads configuration files (config.sys, system.dat, user.dat, as per OS).

13. The comand shell (command.com for example) is loaded. 14. The shell’s start-up command files (autoexec for example)

are executed.


Virus Writing

•  Assembler versus High-Level Language •  For DOS-based attacks, a wide variety of file

extensions are available. •  Consider:

Program Screen_Virus; const attack = ‘Get a life’ begin

writeln(attack); end.

•  Turbo Pascal compiles this program in 1,920 bytes, MS Assembler takes 30 bytes.

•  Much easier to access the boot sector or other low-level mechanisms via assembler.


Tripartite Structure of a Virus (1)

•  Infection –  Begin

If (infectable_object_found) AND (object_not_already_infected) THEN (infect_object)

–  may entail •  writing of a new section of code to the boot sector •  addition of code to a program file •  addition of macro code to MS Word NORMAL.DOT file •  addition of code o a standard system program to intercept

network services so as to send an infected file attachment to harvested email addresses


Tripartite Structure of a Virus (2)

•  Trigger Begin

IF (date_is_Friday_13th) THEN (set_trigger_status_to_yes)

End

•  Payload Begin

IF (trigger_status_is_yes) THEN (execute_payload)

End


Operating System Security Trojan Horses

•  Does NOT self-replicate •  Free program made available to unsuspecting user

–  Actually contains code to do harm

•  Place altered version of utility program on victim's computer –  trick user into running that program –  la –  /usr/mal/ls

•  Rootkits •  Remote Access Tools

–  PCAnywhere –  Laplink –  Back Orifice


ECC Categorization of DoS/DDoS •  Fragmentation attacks

–  These attacks take advantage of the system’s ability (or lack thereof) to reconstruct fragmented packets.

•  Volumetric attacks –  Also known as bandwidth attacks, these consume all

available bandwidth for the system or service. •  Application attacks

–  These attacks consume the resources necessary for the application to run, effectively making it unavailable to others.

•  TCP state-exhaustion attacks –  These attacks go after load balancers, firewalls, and

application servers by attempting to consume their connection state tables.


Types of DoS/DDoS Attacks •  SYN attack

–  The hacker will send thousands upon thousands of SYN packets to the machine with a false source IP address.

–  The machine will attempt to respond with a SYN/ACK but will be unsuccessful (because the address is false).

–  Eventually, all the machine’s resources are engaged, and it becomes a giant paperweight.

•  SYN flood –  In this attack, the hacker sends thousands of SYN packets

to the target but never responds to any of the return SYN/ACK packets.

–  Because there is a certain amount of time the target must wait to receive an answer to the SYN/ACK, it will eventually bog down and run out of available connections.


Types of DoS/DDoS Attacks

•  ICMP flood –  Here, the attacker sends ICMP Echo packets to the

target with a spoofed (fake) source address. –  The target continues to respond to an address that

doesn’t exist and eventually reaches a limit of packets per second sent.

•  Application level –  A simple attack whereby the hacker sends more

“legitimate” traffic to a web application than it can handle, causing the system to crash.

–  Usually these attacks are designed to exploit weak programming code.


Types of DoS/DDoS Attacks •  Smurf

–  The attacker sends a large number of pings to the broadcast address of the subnet, with the source IP spoofed to that of the target.

–  The entire subnet will then begin sending ping responses to the target, exhausting the resources there.

–  A fraggle attack is similar but uses UDP for the same purpose. •  Ping of death

–  In the ping of death, an attacker fragments an ICMP message to send to a target.

–  When the fragments are reassembled, the resultant ICMP packet is larger than the maximum size and crashes the system.

•  (Note that this isn’t a valid attack with modern systems, but is still a definition you may need.)


Types of DoS/DDoS Attacks •  Teardrop

–  In a teardrop attack, a large number of garbled IP fragments with overlapping, oversized payloads are sent to the target machine.

•  On older operating systems (such as Windows 3.1x, Windows 95, and Windows NT operating systems), this takes advantage of weaknesses in the fragment reassembly functionality of their TCP/IP stack, causing the system to crash or reboot.

•  Peer to peer –  In this attack, clients of a peer-to-peer file sharing hub are

disconnected and directed to connect with the target system.

•  Permanent Phlashing –  refers to a DoS attack that causes permanent damage to a

system. Usually this includes damage to the hardware and can also be known as bricking a system.


Distributed Reflection Denial of Service

References: “The Distributed Reflection DoS Attack,” Steve Gibson, Gibson Research “Understanding BGP Misconfiguration,” Mahajan, Wetherall and Anderson


Attack Overview

•  Perhaps the most startling aspect of this attack was that the apparent source was hundreds of the Internet's "core routers", web servers belonging to yahoo.com, and even a machine with an IP resolving to "gary7.nsa.gov". –  Cyberarmy.com

•  Victims appeared to be under attack by hundreds of very powerful and well-connected machines.

•  In some cases a distributed reflection DoS is synonymous with “botnet”.


IP Security Overview •  IP Packets have no inherent security

–  Relatively easy to •  forge contents of IP packets •  modify contents of IP packets •  inspect the contents of IP packets in transit

•  Therefore, there is no guarantee that IP datagrams received: –  are from the claimed sender (source address in the IP

header) –  contain the original data that the sender placed in them –  were not inspected by a third party while the packet was

being sent from source to destination


TCP Review

•  The establishment of a TCP connection typically requires the exchange of three Internet packets between two machines in an interchange known as the TCP three-way handshake. Here's how it works: –  SYN: A TCP client (such as a web browser, ftp client,

etc.) initiates a connection with a TCP server by sending a SYN packet to the server.

–  SYN/ACK: When a connection-requesting SYN packet is received at an ‘open’ TCP service port, the server's operating system replies with a connection-accepting SYN/ACK packet.

–  ACK: When the client receives the server's acknowledging SYN/ACK packet for the pending connection, it replies with an ACK packet.


Bandwidth Consumption DoS

•  Traditional SYN flooding DoS attacks are either one-on-one –  (one machine sending out enough SYN packets to the

target machine to effectively choke off access to the other machine)

•  or many-on-one –  (SYN flooding ‘zombie’ programs loaded by the

attacker into compromised machines and commanded by the attacker to send huge volumes of SYN commands to the target machine).


Review of SYN Packets

SYN: A TCP client (such as a web browser, ftp client, etc.) initiates connection with a TCP server by sending a "SYN" packet to the server.


Review of SYN Packets SYN/ACK: When a connection-requesting SYN packet is

received at an "open“ TCP service port, the server's operating system replies with a connection accepting the "SYN/ACK" packet.

ACK: When the client receives the server's acknowledging SYN/ACK packet for the pending connection, it replies with an ACK packet.


SYN Packet with Deliberately Spoofed Return Address

Through the use of "Raw Sockets", the packet's "return address" (source IP) can be overridden and falsified. When a SYN packet with a spoofed source IP arrives at the server, it appears as any other valid connection request.


Raw Socket Review

•  Data is exchanged across the Internet by either establishing a bi-directional "TCP Connection" between two machines, or by sending a uni-directional "UDP Datagram" message from one machine to another. Both of these data transferring operations employ standard sockets.


Raw Sockets Review •  Smooth and orderly traffic flow across the Internet requires

machines to inform each other of various non-data events such as closed ports, network congestion, unreachable IP addresses, etc. The ICMP (Internet Control Message Protocol) was created to fill this need.

•  The operating system's built-in TCP/IP stack automatically and transparently generates and receives most of these "Internet plumbing" ICMP messages on behalf of the machine. To facilitate the creation of Internet plumbing applications, such as "ping" and "traceroute", which also employ ICMP messages, the Berkeley designers allowed programmers to manually generate and receive their own ICMP, and other, message traffic. As shown in the diagram, the Berkeley Sockets system provides this power through the use of a so-called "Raw Socket".

•  A Raw Socket short-circuits the TCP/IP stack to open a "backdoor" directly into the underlying network data transport.

–  This provides full and direct "packet level" Internet access to any Unix sockets programmer.


SYN Packet: Destination Unknown •  The server will allocate the required memory buffers,

record the information about the new connection, and send an answering SYN/ACK packet back to the client.

•  But since the source IP contained in the SYN packet was deliberately falsified (it is often a random number), the SYN/ACK will be sent to a random IP address on the Internet.

•  If the packet were addressed to a valid IP, the machine at that address might reply with a "RST" (reset) packet to let the server know that it did not request a connection.

•  But with over 4 billion Internet addresses, the chances are that there will be no machine at the address and the packet will be discarded.


Reflection SYN Flooding

•  With a reflection SYN flooding attack the attacking machines send out huge volumes of SYN packets but with the IP source address pointing to the target machine.

•  The TCP three-way handshake requires that any TCP based service that receives a SYN packet must respond with a SYN/ACK packet.

•  The servers and routers that receive these fraudulent SYN packets dutifully send out the SYN/ACK packet to the machine pointed to by the SYN packets IP source address.


SYN Reflector Capability •  Consider this, any general-purpose TCP connection-

accepting Internet server could be used to reflect SYN packets.

•  Here is a short list of the more popular TCP ports: –  22 (Secure Shell) –  23 (Telnet) –  53 (DNS) –  80 (HTTP/web) –  And, virtually all of the Internet’s routers will accept TCP

connections on port 179. •  To fully comprehend the potential of this new form of DoS

attack consider this: –  it uses a fundamental Internet communications protocol; –  machines that use this protocol exist in the millions; –  it is extremely easy to generate a list of ‘SYN packet

reflectors’.


Generating and Using the ‘SYN Packet Reflector’ List

•  A simple script can be constructed to collect a large number of ‘SYN packet reflection’ capable routers and servers. –  Well-known web server farms, such as eBay and Yahoo, are

easily available. –  Simple port scans through high bandwidth IP regions will

reveal thousands, if not millions, of available TCP servers. –  Readily available tools such as Trace Route provide the IP

address of every Internet router between the tracer and any other IP address.

•  Given a large list of SYN packet reflectors, each SYN spoofing attack host can distribute its fraudulent SYN packets evenly across every reflector on its list.


Load Balancing the Attack

•  The big win for the attacker is that since the SYN flooding machine is distributing its packets across a huge number of SYN packet reflectors, none of the innocent reflectors will experience significant levels of incomplete TCP connections.

•  And, since routers generally do not retain any record of previously routed packets, it makes tracking an attack from the victim to the attacker extremely difficult.


Force Multipliers •  As if ease of attack and ubiquity of reflectors were not bad

enough, it turns out that the reflectors will generate three or four times more SYN/ACK packets than the number of SYN packets they receive.

•  Since the TCP connection that receives the SYN command is expecting to receive an ACK back from the machine it sent the SYN/ACK response to, it will send out three or four more SYN/ACK responses over the next few minutes.

•  This TCP protocol feature essentially multiplies the number of malicious SYN/ACK packets being sent to the target machine by a factor of three or four.

•  It also means that the flood of SYN/ACK packets will continue to disable the target site for a minute or two even after the attacker has called off the attack.


Collateral Damage •  The basic connection unit in the Internet is the router.

–  Some routers serve only a small number of machines while other ‘aggregation routers’ collect and disperse large amounts of packet traffic from smaller networks.

•  During normal operations, the traffic flowing through the aggregation routers can be sorted and forwarded to the router's various lower bandwidth client networks.

•  Now imagine a SYN/ACK flood that is so large that it starts to degrade the performance of the aggregation router. –  Having to process and disperse so many packets to the client

networks, the router will drop and discard a portion of the packets. –  Legitimate Internet clients, trying to access resources that have

nothing to do with the target under attack, will also experience degraded, or complete denial of, service.


Solutions to SYN Spoofing •  Operating system vendors responded to spoofed SYN

packet DoS attacks by strengthening their TCP "protocol stacks" in various ways.

•  Most of these were quantitative improvements to make their systems less vulnerable, but they did not eliminate the problem.

•  Two complete, robust, and practical solutions were developed: –  The Unix community invented a clever "stateless" TCP

connection system known as "SYN-cookies". –  Steve Gibson implemented a different solution which was

dubbed "GENESIS". •  Both of these DoS solutions arrange to stay compatible

with all important aspects of the standard TCP protocol. •  They operate by eliminating all allocation of server

resources after receiving a SYN packet and generating a SYN/ACK reply.


Bandwidth Consumption

•  Unlike a DoS-style attack, in which a low rate of fraudulent SYN packets consumes a vulnerable server's TCP connection resources, a bandwidth attack creates a brute force flood of malicious "nonsense" Internet traffic to swamp and consume the target server's or its network connection bandwidth.

•  This malicious packet flood competes with, and overwhelms, the network's valid traffic so that "good packets" have a low likelihood of surviving the flood.

•  The network's servers become cut off from the rest of the Internet, and their service is denied.


Internet Aggregation Router


•  The computers and/or networks shown to the right are serviced by the central "aggregation router." –  This router is placed at the "customer edge" of the Internet service

provider's network to collect and disperse traffic from many smaller customer networks.

–  Thus, many lower-bandwidth Internet connections are "aggregated" into a single high-bandwidth Internet connection for routing to the public Internet.

•  During normal operation, the traffic coming from the Internet down the "Big Pipe" will be sorted and forwarded to the router's various lower bandwidth client networks.

•  When the Big Pipe is filled by a high volume of packets bound for just one of the router's client networks. –  Faced with the task of squeezing too many packets from the big pipe

into the much smaller pipe, the router has no choice but to deliberately drop and discard a large percentage of the packets struggling to get through the smaller pipe.

–  Valid Internet clients, trying to access the resources on the far side of the smaller pipe, will resend their dropped packets. But these clients will generally give up after a few attempts. The victim's network is effectively blasted off the Internet by the flood of malicious traffic.


DoS versus DDoS


Distributed zombie traffic aggregation •  As the

individual streams of traffic move across the Internet from their many separate sources, they are combined by the Internet's routers to form a single massive flood . . .


SYN FLOODING INTERNET ROUTERS (Bandwidth Attack)

•  TCP servers were sending SYN/ACK packets to grc.com in the well-meaning belief that WE wanted to open a TCP connection with their built-in BGP servers.


Blocking the reflection attack

•  Gibson Research reaction to DRDoS: •  First, block any inbound traffic originating from

the BGP service port 179. –  Since the malicious hacker's SYN packets were aimed at

the intermediate routers' port 179, any reflected packets would be originating from that port.

–  Verio's engineer added a "filter" to the aggregation router servicing our Internet connection to block (drop) any packets inbound to us from port 179.

–  The flood of packets coming in from port 179 immediately stopped.

•  But we did NOT return to the Internet.


Secondary Flooding

•  A fresh packet capture revealed that Gibson Research was now being actively flooded by an entirely new set of Internet servers.

•  Since this second set of traffic appeared only after the port 179 router traffic had been blocked, it appeared that this second wave of reflection traffic had been unable to compete with the routers' flood. –  (You know you're in trouble when packet floods are competing

to flood you.) •  With the routers traffic blocked, we were now being flooded

by a SYN/ACK packets pouring in from ports 22 (Secure Shell), 23 (Telnet), 53 (DNS), and 80 (HTTP/Web).

•  There were also some packets coming from port 4001 (a proxy server port) and 6668 (IRC chat).


Packet Path Diffusion

•  The big win for the attacker is the extreme degree of "packet path diffusion" made possible when attack traffic can be bounced off a large number of intermediate TCP servers. This diagram is a representation of the path of traffic between a single attacker and victim.


Packet Path Diffusion with Reflectors •  The addition of innocent reflection servers

substantially transforms the attack.


Packet Path Diffusion with Reflectors

•  Upon leaving an attacking machine, the malicious SYN packets immediately fan out.

•  No longer aimed at the victim, these attack packets are instead being sent to widely spread TCP servers.

•  As we know, these servers are potentially located throughout the entire Internet.

•  Just a few "router hops" away from the attacker, the heavy packet flow will no longer be discernible because it will have diffused into neighboring routers rather than following a single path.


Defending against DRDoS •  Routers can be configured to filter (drop) packets destined

for a particular address or group of addresses. –  Router port 179 can be blocked as a reflector.

•  Since reflected SYN/ACK packets must bounce off a TCP server, and since almost all common service ports fall within the range from 1 to 1023, blocking all inbound packets originating from the service port range will block most of the traffic being innocently generated by reflection servers. Holes in the reflection filter may have to be created to allow legitimate traffic to pass through.

•  Block all inbound packets to high-numbered service ports. This has the undesirable effect that legitimate clients of the protected server could be generating connections from those blocked ports.


Defending against DRDoS •  End-user client machines cannot be protected. Most client

machines spend all of their time connecting to remote servers all over the Internet and require access to data coming back from many of the most common low-numbered service ports.

•  Servers could be programmed to recognize a SYN source IP address that never completes its connections and has an anomalous number of failed connections occurring within a period of time. The target of the reflection attack could be easily determined and the SYN/ACK response could be temporarily turned off.

•  ISPs could prevent the transmission of fraudulently addressed packets (packets with an IP source address not within their source address space) from within their controlled networks. This control mechanism alone would have a major dampening effect on this type of attack.


Session Hijacking


Session Hijacking •  Accomplished by

–  Brute force –  Calculation –  Eavesdropping –  Transmitting a preconfigured session ID to the target

•  When target opens •  Wait for authentication then insert your session

•  Sequence attacks take advantage of the fact that TCP is a session-oriented protocol –  Synchronized packets set up the sequence numbers

(SN) –  For example an ack packet of 100 with a window of 300

means you expect packet sequence numbers between 100 and 400.


Session Hijacking (EC-Council)

1.  Sniff the traffic between the client and the server.

2.  Monitor the traffic and predict the sequence numbering.

3.  Desynchronize the session with the client.

4.  Predict the session token and take over the session.

5.  Inject packets to the target server.


IP Security Overview •  IP Packets have no inherent security

–  Relatively easy to •  forge contents of IP packets •  modify contents of IP packets •  inspect the contents of IP packets in transit

•  Therefore, there is no guarantee that IP datagrams received: –  are from the claimed sender (source address in the IP

header) –  contain the original data that the sender placed in them –  were not inspected by a third party while the packet was

being sent from source to destination

IPSec is a means to limit the spoofing of routers


Virtual Private Networks •  A VPN is a way to simulate a private network over

a public network, such as the Internet –  “Virtual” because it depends on the use of virtual

connections –  temporary connections that have no real physical

presence, but consist of packets routed over various machines on the Internet on an ad hoc basis

–  secure virtual connections are created between machines and networks as follows:

•  two machines •  a machine and a network •  two networks


Origins of VPNs

•  WANs –  T1/T3 –  ATM –  Frame Relay –  ISDN –  X.25

•  Forerunner of VPNs was the idea of a virtual circuit –  A virtual circuit creates a logical path from the source to

the destination


Virtual Circuits

•  In packet switched networks, the network makes dynamic decisions concerning the pathway each packet will take

•  To improve reliability, a decision could be made prior to any data being sent –  In this manner, a single static path could be set up

between two communicating parties and used exclusively between them

–  This pathway is known as a virtual circuit •  When creating a virtual circuit, sender and

receiver agree on which path will be used and on packet size. –  During communications, acknowledgements are sent,

including flow control info and error control info


Tunneling

•  Tunneling enables one network to send its data over another network’s connections

•  Tunneling creates circuit-like connections across the packet-oriented Internet

Internet

VPNs designed to create the logical equivalent below


VPNs versus long haul connections

•  Long Haul connections –  leased line –  frame relay network –  ISDN –  ........

•  For two remote offices, much cheaper to each get an ISP POP (point of presence) –  Then deploy an VPN between the two routers at the two

offices over the Internet


How VPNs Solve Internet Security Issues

•  Firewalls –  discussed next lecture

•  authentication –  multiple means including IPSec –  Challenge Handshaking Authentication Protocol (CHAP) –  RSA

•  encryption –  multiple means including IPSec –  private key encryption –  public key encryption


IP Spoofing

•  An attacker compromises the routing packets to redirect a file or transmission to a different destination –  most routing information is not encrypted

•  easy to modify source data or change destination –  also used to mask attacker’s identity

•  Best solutions –  screen packets at router and firewall, reject any that

appear to come from an internal address –  encryption to safeguard the payloads of the packets –  authentication to verify sender


IPSec

•  IPSec is a method of protecting IP datagrams. •  This protection takes the form of

–  data origin authentication –  connectionless data integrity authentication –  data content confidentiality –  anti-replay protection –  limited traffic flow confidentiality

•  Protection via Encapsulating Security Payload (ESP) or Authentication Header (AH) –  Ultimate security dependent upon the cryptographic

algorithm applied –  Symmetric key cryptography used – why?


What is Tunneling? •  Tunneling encloses one type of data packet into the

packet of another protocol –  Protocol of the encapsulating packet is understood by the

network and by the network entry and exit points •  Before encapsulation takes place, packets are

encrypted so that they payloads are unreadable during transit

•  Tunneling involves three different protocols –  Carrier protocol – used by the network that the information is

traveling over – usually TCP/IP –  Encapsulation protocol – protocol that the original data is

packaged in such as GRE, IPSec, L2F, PPTP or L2TP –  Passenger protocol – original or native data that is being

carried from the network where the originating host resides such as IPX, AppleTalk, IP


Tunneling Protocols •  Layer 2 tunneling protocols

–  Layer 2 protocols correspond to the Data Link layer and use frames as their unit of exchange. PPTP, L2TP and L2F are Layer 2 tunneling protocols. These protocols encapsulate the data in a Point-to-point Protocol (PPP) frame to send across an internetwork*

*an internet with a lower case i, is any collection of networks that are networked or connected together over a common infrastructure.

•  Layer 3 tunneling protocols –  Layer 3 protocols correspond to the network layer and

use packets. IP over IP and IPSec Tunnel Mode are examples of Layer 3 tunneling protocols. These protocols encapsulate IP packets in an additional IP header before sending them across an IP internetwork.


IPSec Overview

IPSec Roadmap, Doraswamy and Harkins

Architecture

ESP AH

Encryption Algorithm Authentication Algorithm

Domain of Interpretation

Policy Key Management


IPSec Architecture Revisited •  Defined by RFC 2401 •  Mandatory in IPv6 •  Internet Key Exchange (IKE)

–  Symmetric key cryptography is used for efficiency –  To exchange keys securely, a negotiation protocol is used that

allows users to agree on authentication methods, encryption methods and the keys to use.

–  It also specifies how long keys can be used before changing and how to accomplish key exchange

•  The IPSec protocols, AH and ESP can be used to protect an entire IP payload or the upper layer protocols of an IP payload. –  AH used for authentication –  ESP used for encryption

•  Two different modes of IPSec –  Transport mode to protect upper-layer protocols –  Tunnel mode to protect entire IP datagrams


Internet Key Exchange (IKE)

•  Compliant IKEs require adherence to three documents –  ISAKMP specification (RFC 2408) (Internet Security

Association and Key Management Protocol) –  Domain of Interpretation for (DOI) for for IPSec (RFC 2407) –  IKE specification (RFC 2409)

•  Security Associations (SAs) are used with IPSec to define the processing done on a specific IP packet.

•  IKEs establish shared security parameters and authenticated keys – SAs- between IPSec peers

•  IKE is a generic protocol with application beyond IPSec –  ex. RIPv2 or OSPF


Transforms

•  Transformation applied to the data to secure it. –  includes algorithm, key sizes, derivations –  specific information required in order for different

implementations to interoperate •  IKE – Internet Key Exchange

–  establishes shared security parameters and authenticated keys

•  i.e. security associations (SAs) between IPSec peers –  Actual negotiated parameters come up in the Domain of

Interpretation (DOI) •  Policy

–  Necessary but not sufficient for interoperability –  Determines transforms, representations and implementation


Overview of ISAKMP •  AH Transform Identifiers

–  AH_MD5 –  AH_SHA –  AH_DES –  AH_SHA2-256 (256 bit

message digest) –  AH_SHA2-384 –  AH_SHA2-512 –  AH_RIPEMD

•  Certificate Types –  PGP certificates –  DNS signed key –  x.509 cert – signature –  x.509 cert – key exchange –  Kerberos tokens –  CRL (Cert Revocation List) –  ARL (Auth Revocation List) –  SPKI cewrt –  x.509 cert - Attribute

•  ESP Transform Identifiers –  ESP_DES_IV64 (DES in CBC

mode with a 64 bit IV) –  ESP_DES (DES in CBC

mode) –  ESP_3DES –  ESP_RC5 –  ESP_IDEA –  ESP_CAST –  ESP_Blowfish –  ESP_3IDEA –  ESP_DES_IV32 (DES in CBC

mode with a 32-bit IV) –  ESP_RC4 –  ESP_NULL (NONE) –  ESP_AES


Security Associations

•  SAs form the basis for IPSec –  contract between two communicating entities –  determine the protocols used for securing packets

•  SAs are one-way, i.e. simplex –  If two hosts are communicating, host A will have an SAout and an

SAin •  SAs are protocol specific

–  Each host builds a separate SA for AH and ESP •  Security policy database

–  Works in conjunction with the security association database •  Security Parameter Index

–  32-bit entity that is used to uniquely identify an SA at the receiver –  SPI passed to AH and ESP headers using a tuple <spi,dst,protocol>


IPSec in Tunnel Mode

•  An IPSec tunnel mode packet has two headers – inner and outer –  Inner header constructed by the host –  Outer header is added by the device providing security

services

SA RA,RB Host A Host B Router A Router B

(Outer)IP Header ESP IP

Header Network Payload

IPSec tunneled mode packet format


Nested Tunnels

•  IPSec defines tunnel mode for both ESP and AH •  In the nested tunnel example above, host A is sending a

packet to host B. –  Policy requires authentication to router B –  VPN between the two networks bounded by router A and router B

SAAH Host A 1.1.1.1

Host B 3.3.3.2

Router A 1.1.1.2 2.2.2.1

Router B 2.3.2.2 3.3.3.1

SAESP

ESP IP Header IP Header Data IP Header AH

SRC = 2.2.2.1 Dest = 2.3.2.2

SRC = 1.1.1.1 Dest = 2.3.2.2

SRC = 1.1.1.1 Dest = 3.3.3.2

Nested Packet Format


Valid and Invalid Nested Tunnels

•  The requirement for the tunnel is that the inner header must be completely encompassed by the outer header.

Host A Host C Router A Router C Router B Tunnel 1

Tunnel 2

Host A Host C Router A Router C Router B Tunnel 1 - invalid

Tunnel 2 - invalid


Authentication Header

•  96 bits is selected to maintain compatibility with original IPSec spec •  Replay protection is provided by using the Sequence Number field

within the AH header whose value is covered by the authentication procedure

IP Header AH Header Payload

MD5 or SHA-1

MD5 or SHA-1

Authentication Key

1st 96 bits of second hash becomes Integrity Check Value (ICV)


Mutable IPv4 fields that cannot be protected by AH

•  Mutable IPv4 fields that cannot be protected by AH –  Type of Service (TOS) –  Flags –  Fragment Offset –  Time to Live (TTL) –  Header Checksum

•  When protection of these fields is required, tunneling should be used

•  Payloads of an IP packet are considered immutable and therefore always protected by AH

•  An IP packet with AH applied can be fragmented BUT AH cannot be applied to a fragmented packet


AH Transport and Tunnel Modes

•  In transport mode, the original datagram’s IP header is the outermost IP header

•  In tunnel mode, a new IP header is generated for use as the outer IP header of the resulting datagram –  Source and destination address of the new header will generally differ – i.e.

the destination address of the new IP header may be a corporate firewall.

IP Header Payload Original IP Datagram

IP Header AH Header Payload AH Transport Mode

IP Header AH Header New IP Header Payload AH Tunnel

Mode


Encapsulating Security Payload (ESP)

•  ESP adds approximately 24 bytes per packet •  For interoperability purposes, mandatory to

implement algorithms has been defined for ESP –  The must-implement cipher is DES-CBC with an explicit

IV (RFC 2405) –  The must-implement authenticators are HMAC-MD5-96

and HMAC-SHA-96 (RFCs 2403 AND 2404) •  Published prior to development of “deep crack” •  RFCs updated to indicate deprecated nature of

DES and suggesting stronger cipher algorithms


Outbound ESP Processing

•  Insert header (similar for both IPv4 and IPv6) •  Encrypt packet from beginning of the payload to the

next header field in the trailer using appropriate cipher specified in the SA (policy check)

•  Authenticate packet from ESP header through the ciphertext to the ESP trailer. –  Insert result in the authentication data field of the ESP

trailer •  Recompute checksum of the IP header the precedes

the ESP header


Inbound ESP Processing

•  SA determines what the incoming packet should be. –  No way to tell until packet is decrypted –  Makes unauthorized traffic analysis harder –  If no valid SA exits – drop the packet

•  Next, authenticate by checking the message digest –  pass appropriate key to authentication algorithm from the

SA •  Decrypt the packet -- from the beginning of the

payload data to the next header field –  decrypted using the key and cipher algorithm from the SA –  check decryption by checking the padding

•  padding is completely deterministic •  verifies whether packet was successfully decrypted.


ESP Transport and Tunnel Modes

•  ESP in transport mode provides neither authentication nor encryption for the IP header.

•  In tunnel mode, the new IP header is not encrypted – everything else is

IP Header Payload Original IP Datagram

New IP Header ESP Header Payload ESP

Transport Mode

IP Header ESP Header New IP Header Payload

ESP Tunnel Mode

ESP Auth.

ESP Trailer

ESP Auth.

ESP Trailer

Encrypted Authenticated (ESP)

Encrypted Authenticated (ESP)


Transport Mode •  AH and ESP intercept the packets moving from

the transport layer into the network layer. –  When security is NOT enabled, TCP and UDP flow into

IP which adds an IP header –  When security is enabled, TCP / UDP flow into the IPSec

component –  When both AH and ESP are used, ESP is applied first –

why?

IP Header

AH Header

ESP Header

TCP Payload

Packet format with AH and ESP


Tunnel Mode

•  IPSec in Tunnel mode is normally used when the ultimate destination of the packet is different from the security termination point. –  ex. security termination point may be a router rather

than a host. –  also used when a router provides security services for

packets it is forwarding –  In the case of tunnel mode, IPSec encapsulates an IP

packet with IPSec headers and adds an outer IP header

(Outer)IP Header ESP IP

Header Network Payload

IPSec tunneled mode packet format


Conclusion: IPSec Implementation

•  Can be implemented in end hosts, gateways / routers or both

•  Advantages of OS-level integration –  Efficiency: IPSec can use network services in the OS

such as user context (sockets) –  Ease of Implementation: Network connections, HTTP

connections – all can be configured from the host –  All IPSec modes are supported

•  BUMP-in-the-Stack (BITS) network level integration –  Supports multiple OSs –  Duplicated functionality causing unnecessary

complications –  Allows firewall vendors to integrate with their products


Summary – Section Objectives

1.  Describe malware types and their purpose 2.  Identify malware deployment methods 3.  Describe the malware analysis process 4.  Identify malware countermeasures 5.  Describe DoS attacks and techniques 6.  Identify DoS detection and countermeasure

action 7.  Describe session hijacking and sequence

prediction

J. A. Drew Hamilton, Jr., Ph.D.web.cse.msstate.edu/~hamilton/P3I/CEH/lessons/9_Malware_Trojan… ·...

Documents

Transcript of J. A. Drew Hamilton, Jr., Ph.D.web.cse.msstate.edu/~hamilton/P3I/CEH/lessons/9_Malware_Trojan… ·...