CS266 Software Reverse Engineering (SRE)Identifying, Monitoring, and Reporting Malware

Teodoro (Ted) Cipresso, teodoro.cipresso@sjsu.eduDepartment of Computer ScienceSan José State UniversitySpring 2015

The information in this presentation is taken from the thesis “Software reverse engineering education” available at http://scholarworks.sjsu.edu/etd_theses/3734/ where all citations can be found.

2

Identifying, Monitoring, and Reporting MalwareWhat Qualifies as Malware?

Malware describes a category of software that doesn’t always operate in a way that benefits the user.

Of course, those of us who have ever used software might contend that this definition of malware will cause programs that we use every day to be categorized as malware.

So let's qualify it a bit: the malicious or annoying behaviors of malware are intentional, not the result of one or more bugs.

3

Identifying, Monitoring, and Reporting MalwareTypes of Malware

There are several types of malware that affect computer systems [6] [7]:

Viruses: require some deliberate action to help them spread.

Worms: similar to a virus but can spread by itself over computer networks.

Trojan Horses: performs hidden malicious or annoying operations.

Backdoor: a vulnerability purposely embedded in software.

Rabbit: a program that exhausts system resources.

Ransomware: lock computer files, victim has to pay to unlock.

Criminalware: Steal sensitive information.

4

Identifying, Monitoring, and Reporting MalwarePrevalence of Malware Types

Malware usually isn't of just one type; for example, 4 of the top 10 malicious codes families reported in 2011 were Viruses with a Worm component.

Using the machine code and bytecode reversing experiences gained from the previous modules, one could attempt to reverse malware.

Using virtualization tools such as VMWare or Virtual Box to create secondary operating system images (Guests) on which to analyze malware can still result in infection of the primary operating system (Host).

Great care should be taken to isolate guest OSes from their host OS.

Networking, removeable storage, clipboard usage, etc…

5

6

Identifying, Monitoring, and Reporting MalwareSafe & Practical Malware Reversing

We want to become familiar with using tools to identify, monitor, and report software that might be malicious.

Reversing malware directly is especially challenging because several anti-reversing techniques will have been applied to the code.

Given that unexpected catastrophes can arise when installing a virus, worm, backdoor, etc… for academic purposes; we could still learn something from working with contrived or benign “malware”.

In 1996, Mark Russinovich founded a company called “Winternals Software” where he was the chief software architect on a comprehensive suite of tools for diagnosing, debugging, and repairing Windows® systems and applications [9].

7

Identifying, Monitoring, and Reporting MalwareWindows Sysinternals

Mark's company was purchased by Microsoft and the suite of tools have been rebranded as Windows Sysinternals which are are offered for free.

Mark's story is an interesting one because he is recognized as an expert on the internals of Windows even though he did not participate in its development—a true testament to what can be learned about software through reversing.

The Sysinternals suite contains 69 utilities, but we’ll focus on just one.

8

Identifying, Monitoring, and Reporting MalwareSysinternals Process Monitor

The Process Monitor can capture detailed information about a process in a Windows system including file system, registry, and network activity.

Process Monitor session for the Password Vault CPP application.

File systemactivity

9

Identifying, Monitoring, and Reporting MalwareSysinternals Process Monitor (cont’d)

The Process Monitor can capture detailed information about a process in a Windows system including file system, registry, and network activity.

Process Monitor session for the Password Vault CPP application.

Networkactivity

10

Identifying, Monitoring, and Reporting MalwareSysinternals Process Monitor (cont’d)

The Process Monitor can capture detailed information about a process in a Windows system including file system, registry, and network activity.

Process Monitor session for the Password Vault CPP application.

RegistryActivity

11

Identifying, Monitoring, and Reporting Malware Sysinternals Process Monitor (cont’d)

Process Monitor itself does not detect or identify malware. It simply monitors and records what processes are doing.

With a little bit of ingenuity, one can identify a software Trojan by looking for activities that don't seem to fit with the advertised functionality of a program.

It's common practice to download free software from the Internet:

Some believe that open-source software, should have the fewest number of vulnerabilities. The more eyes the better, right?

Becoming familiar with the Sysinternals suite can help you evaluate whether the software on your Windows machine is acting in your best interest.

12

Identifying, Monitoring, and Reporting MalwareBenign Malware Exercise

The Alarm Clock application is a benign software Trojan that, in addition to being a rudimentary alarm clock, performs unadvertised functions on background threads:

Logs information from the Windows® registry

Logs locations of “office” documents in the file system.

Scans for computers that respond to an ICMP ping.

Paced background threads are used.

13

Identifying, Monitoring, and Reporting MalwareBenign Malware Exercise (cont’d)

Background threads log information about the user’s system.

14

Identifying, Monitoring, and Reporting MalwareIs Open Source More Trustworthy?

The data on the number of vulnerabilities found in the 5 most popular Internet browsers does not support the proposition that open source is more secure.

Big 5: Google Chrome, Mozilla Firefox, Internet Explorer, Opera and Safari.

Mozilla Firefox was affected by 270 new vulnerabilities in 2013, more than any other browser; 245 new vulnerabilities were found in Google Chrome, 126 in Internet Explorer, 75 in Apple Safari, 11 in Opera [Secunia].

The two browsers containing the most open source (Chrome based Chromium, Firefox based on Mozilla), have the most vulnerabilities…

Of course we need temper this judgement with the observation that popular software is targeted more often.

15

Identifying, Monitoring, and Reporting MalwareReporting Suspected Malware

If you suspect a particular program to be malware, it can be submitted to online threat analysis services such as ThreatExpert or Virus Total.

ThreatExpert and Virus Total are Web-based tools that support submission of suspicious executables or URLs to detect possible malware.

Both services match against databases of existing malware, however ThreatExpert (itself) attempts to execute binaries in an isolated environment to perform heuristic detection of malware.

16

http://www.threatexpert.com/submit.aspx

http://www.threatexpert.com/report.aspx?md5=acdd4c2a377933d89139b5ee6eefc464

Heuristic anlaysis components

17

44 out of 56 antiviruses detect this as malware

18

Identifying, Monitoring, and Reporting MalwareSetting up a Lab for Analyzing Malware

Each of you have been assigned your own VMWare image (info on Canvas).

The images are only accessible through VMWare’s built-in VNC server.

The images are on a virtual network and have no connectivity to the Internet or the Host’s network. This is to prevent:

Infection of the Host (primary OS), worms from spreading*

downloading of additional threats,

transmission of sensitive data to hacker sites.

Virtualized Network Isolation for a Malware Analysis Lab

https://zeltser.com/vmware-network-isolation-for-malware-analysis/

19

20

21

End