PhishGuru: A System for Educating Users aboutSemantic Attacks

Ponnurangam Kumaraguru

Thesis ProposalDoctor of Philosophy

From

Computation, Organizations and Society ProgramInstitute for Software Research

School of Computer ScienceCarnegie Mellon University

Pittsburgh, PA

April 13, 2007

Thesis CommitteeLorrie Cranor(Chair)

Jason HongVincent AlevenRahul Tongia

Alessandro Acquisti

Contents

1 Introduction 5

2 Background and related work 7

2.1 Phishing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.1.1 Introduction to phishing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.1.2 Life cycle of phishing attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.1.3 Countermeasures for phishing . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.2 Role of Trust . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.2.1 Trust and user decision making . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.2.2 Trust models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.3 Learning Science . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.3.1 Instructional design principles . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.3.2 Measurement of learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

3 Past and present work contributing to this thesis 18

3.1 A Model of Trust in Phishing Scenario (MoTPS) . . . . . . . . . . . . . . . . . . . . 18

3.2 Protecting People from Phishing: The Design and Evaluation of an Embedded Train-ing Email System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.3 Teaching Johnny Not to Fall for Phish . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.4 Anti-Phishing Phil . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.5 Laboratory study I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

4 Proposed Research 24

4.1 Embedded training concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

4.2 HCI design methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

4.3 Instructional materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

4.4 Learning algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

4.5 The PhishGuru system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

4.5.1 The PhishGuru architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

4.5.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

4.6 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

4.6.1 Pilot studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4.6.2 Field-study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4.7 Application to other semantic attacks . . . . . . . . . . . . . . . . . . . . . . . . . . 34

2

4.7.1 Other semantic attack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

4.7.2 Laboratory study II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

5 Timeline 35

6 Outline of the thesis 37

7 Conclusion 39

A Appendix - Phishing training curriculum 40

List of Figures

1 Life cycle of phishing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2 Example of a phishing email . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3 Example of a phishing website . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

4 A model of trust in phishing scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

5 Earlier comic strip design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

6 Screen shot from the anti-phishing Phil game . . . . . . . . . . . . . . . . . . . . . . 22

7 Latest comic strip design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

8 A few production rules for phishing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

9 The PhishGuru architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

10 Proposed schedule for research and writing my dissertation . . . . . . . . . . . . . . 37

3

Abstract

Online security attacks are a growing concern among Internet users. Currently, the Inter-net community is facing three types of security attacks: physical, syntactic, and semantic. Asemantic attack is a type of security attack that takes advantage of the way humans interactwith computers or interpret messages. There are three major approaches to counter semanticattacks: silently eliminating the attacks, warning users about the attacks, and training usersnot to fall for attacks. The existing methods for silently eliminating the attack and warningusers about the attack are unlikely to perform flawlessly and as users are the weakest link inthese attacks, it is essential that user training complements other methods. Most of the existingonline training methodologies are less successful because, first, the organizations creating andhosting training materials expect users to proactively seek them, second, these organizationsexpect users to have some knowledge about semantic attacks, and last, the training materialslack learning science principles.

The goal of my thesis is to show that computer users trained using an embedded trainingsystem grounded in learning science are able to make more accurate online trust decisions thanthose who read traditional security training materials distributed via email or posted on websites. To achieve this goal, I will focus on “phishing” a type of semantic attack and develop asystem called “The PhishGuru” based on embedded training methodology and learning scienceprinciples. Embedded training is a methodology in which the training materials are integratedinto the primary tasks that users perform in their day–to–day lives. In contrast to existingtraining methodologies, the PhishGuru will show the training materials to users through emailswhen the users actually fall for phishing attacks.

I will evaluate the embedded training methodology through laboratory and field studies. Iwill also extend this methodology to other semantic attacks. The design principles establishedin this thesis will help researchers to develop systems that can train users in other online riskysituations.

Arise, awake and sleep not till the goal is reached.

∼ Swami Vivekananda, Philosopher, India

4

1 Introduction

As people conduct an increasing number of transactions using the Internet, Internet security con-cerns have also increased. Currently, the Internet community is facing three types of securityattacks: physical, where computers and electronics are targeted (e.g. power and data outages);syntactic, where operating logic and networks are targeted (e.g. security vulnerabilities in Mi-crosoft Windows operating system); and semantic, where the way users assign meaning to thecontent is targeted. A semantic attack is an attack that is aimed directly at people. Rather thantaking advantage of system vulnerabilities, semantic attacks take advantage of the way humansinteract with computers or interpret messages [132].

Recently, we have seen a dramatic increase in a semantic attack known as phishing, in which victimsget conned by spoofed emails and fraudulent websites. Phishing attacks exploit users’ inability todistinguish legitimate company websites from fake ones. Phishers exploit the difference betweenwhat the system thinks the user is doing (system model) and what the users think the system isdoing (user mental model) [101]. Currently, emails are an important threat vector for exploiting thisdifference [71, pp. 687]. Phishers send out spoofed emails that look as if they were sent by trustedcompanies. These emails lead to spoofed websites that are similar or virtually identical to legitimatewebsites, and lure people into disclosing sensitive information. Phishers use this information forcriminal purposes, such as identity theft, financial fraud, and corporate espionage [72,87].

The number of phishing emails that are sent to users has increased notably in the last few years. Thenumber of phishing incidents reported to the Anti-Phishing Working Group (APWG) increased from21 in Nov 2003 to 23,789 in Dec 2006. Seventy three million U.S. adults said that they “definitely”or “think they” received an average of more than 50 phishing e-mails in the year 2005 [53]. Thenumber of unique websites which were phished has also been increasing constantly [11].

The actual cost of phishing is difficult to calculate; broadly, the costs that are involved in phishingcan be classified as: direct cost, the cost that is directly incurred due to the phishing attack; indirectcost, the cost of handling customer support calls due to the phishing attack for an organization andthe emotional stress for the consumers; and opportunity cost, cost incurred due to users refrainingfrom using the Internet to do business and other financial transactions because of the distrusttowards the Internet. In a survey among 5000 US consumers, Gartner found that nearly 30% ofthe consumers changed their online banking behavior because of the influence of online attackslike phishing [53]. Gartner predicted that consumers will loose $2.8Billion in 2006 due to phishingattacks [97].

A variety of strategies to protect people from phishing have been proposed in the literature andimplemented. These strategies fall into three major categories: silently eliminating the threat byfiltering emails, warning users about the threat with toolbars and browser extensions, and trainingusers not to fall for attacks [84]. There is no single silver bullet solution for solving the problem ofphishing. Different approaches have to be developed to counter phishing. Most of the research hasfocused on solving the problem through the first (eliminating) or the second (warning) approach.Little work has been done on educating people about phishing and other semantic attacks.

In this thesis, I will study in depth the strategy of training users not to fall for semantic attacks. Toachieve this goal, I will develop a training system for phishing and extend the system to understandmethodologies to train users about other semantic attacks. In training, I aim at informing users

5

about what actions to take on such phishing emails. User education should be considered comple-mentary to other solutions and should be pursued in parallel with those solutions. The goal of usereducation is not to prevent the phishing emails reaching users because that is done by filtering andother techniques.

The thesis statement of my work is

Computer users trained using an embedded training system grounded inlearning science are able to make more accurate online trust decisions thanthose who read traditional security training materials distributed via emailor posted on web sites.

To evaluate the above thesis statement, I will develop a knowledge tracing algorithm that willbe used to measure user performance and provide insights as to which training materials canbe presented to them. Knowledge tracing will be helpful in implementing the mastery learning.Mastery learning is a technique in which users are trained until they have mastered the contentin the training materials [19]. I will use a scaffolding technique to train users to use differentcues and strategies to identify phishing emails and websites. The knowledge tracing algorithmwill individualize the training and evaluate whether the users have learned the skills that are beingpresented in the training materials. I plan to develop a curriculum of instructions for training users.I will use different learning science principles (learning-by-doing, providing immediate feedback,conceptual-procedural, contiguity, personalization, and story-based) for developing the instructionsand implementing the training methodology.

In order to keep users engaged in the training, I plan to develop a user rating algorithm, whichwill rate the users according to the skills they have acquired through the training system. Thesealgorithms will make the training system adaptive and intelligent, and provide users with a betterlearning experience. I plan to use the data driven and user driven design techniques for developingthe training system. In data driven technique, the design is evolved using the empirical data that iscollected and in user driven technique, the user is always kept in the loop of design and evaluationof the system.

This thesis is both timely and needed to reduce the negative consequences of semantic attackson society. This research is worth doing because it can potentially help reduce the increasingnumber of people falling for phishing and other semantic attacks. This research fits in the areaof Computation, Organizations and Society, because phishing is a societal problem and can besolved by the collective efforts of computer science researchers, education researchers, lawyers, andorganizations. I will build on existing knowledge in economics, learning science, computer science,human computer interaction, and security to build a system to help users make better online trustdecisions. The design principles established in this thesis will help researchers develop systems thatcan train users in other risky online situations.

The remainder of this proposal is organized as follows. In the following section, I will present abackground and related work on phishing, the role of trust in online situations, and learning science.In Section 3, I will discuss my past and present research which leads to this thesis. In Section 4, Iwill describe the proposed research that I will conduct to complete the thesis. Finally, in Section 5,I will present a schedule for completing my dissertation.

6

2 Background and related work

In this section, I will discuss the literature on phishing, the role of trust in online situations, andthe literature on learning science.

2.1 Phishing

In this section, I will discuss the definition of phishing, different types of phishing, and the life cycleof phishing attacks. I will also discuss different countermeasures for phishing attacks.

2.1.1 Introduction to phishing

Phishing is “a broadly launched social engineering attack in which an electronic identity is misrepre-sented in an attempt to trick individuals into revealing personal credentials (financial information,social security numbers, system access information and other personal confidential information)that can be used fraudulently against them [48].” Victims get conned by spoofed emails and fraud-ulent websites which masquerade as a legitimate organization [71,72,87]. Broadly, phishing attackscan be classified into three different types: deceptive attacks, malware based attacks, and DNS-based attacks. Deceptive attacks trick victims into giving their personal confidential informationto spoofed websites. Currently, this is the most prevalent attack. Malware based attacks send andexecute malicious software on to user’s machine through phishing emails. Keyloggers, session hi-jackers and web Trojans fall into this category of attacks. DNS-based attacks tamper the integrityof the lookup process for a domain name. Content-injection, man-in-the-middle, and search enginephishing belong to this category of attacks [42].

Phishing has evolved over time and phishers have spread across all sectors of business; however,the financial sector has been affected most by phishing [118].

2.1.2 Life cycle of phishing attacks

Phishing attacks involve six phases, from planning the attack until removing all evidence of theattack [48]. Figure 1 presents these phases; I describe each of these phases briefly below.

1. Planning : In this phase, the phisher identifies the organization to spoof, identifies the typeof personal information to collect, and develops a story line or plot to collect the personalinformation. In this phase, phishers also decide on the technical infrastructure to deploy theattack [48].

2. Setup: The phisher then designs the attack materials such as phishing emails (The lure) andthe phishing websites (The hook) [71]. Figure 2 shows an example of a phishing email spoofingthe organization eBay. The ‘sender’ address is masqueraded to look as though it has comefrom ‘ebay.com,’ but in reality it does not. The email also shows a ‘sense of urgency’ and‘an action’ to be taken by the user; the link in the email is disguised to take the user to aphishing website and not to the legitimate eBay website. Phishers generally use open relaysor zombie machines to send out the phishing emails [48].

7

3. Attack : Phishers use many vectors to perform the attacks. Some of the vectors are: websites,emails, instant messaging, auto phone dialers (vishing) [115], news, chat room, blogs, bulletinboards, wireless networks, malware [48, pp.8] and search engines [42]. The most commonlyused threat vectors are emails and websites [72].

In this phase, phishers send out phishing emails to victims. Email addresses are harvested andcollected from various sources; phishers send out emails to these harvested email addresses.These email addresses get traded and reused among different groups of phishers. Phishersrely on users to click on the link in the email and go to the spoofed website to give theirpersonal information. Figure 3 shows an example of a phishing website; phishers use sucha website to collect personal information from users. The example shows the ‘phishy URL’and the ‘spoofed status bar.’ Phishers have also become more sophisticated and use victim’spersonal information in the email (e.g. first four digits of the credit card number); this typeof attack is called spear phishing [99].

4. Collection: In this phase, phishers collect the personal information that victims provide tothe phishing website [48,71]. The personal information that phishers tend to collect are creditcard numbers, social security numbers, computer and account login information, address forcommunication, and other personal sensitive information [149]. The personal informationthat users enter on the phishing website is saved in files for the phisher to collect or is sentas an email to the phishers [87].

5. Fraud & abuse: Phishers sell, trade, or directly use the personal information collected fromvictims [48]. Phishers employ cashers or mules to convert this information to cash or toperform identity theft and other frauds. Most of the times, mules are innocent people whodo this conversion without knowing that they are being part of an illegal activity [71].

6. Post attack : During the post attack phase, phishers tend to remove all the trails from theattack and remove the phishing websites that they registered for the attack. It is believedthat phishers also monitor and track the success of their attacks and use the knowledge andexperience gained for future attacks [48].

2.1.3 Countermeasures for phishing

A variety of strategies to protect people from phishing have been proposed in the literature andimplemented. These strategies fall into three major categories: silently eliminating the threat,warning users about the threat, and training users not to fall for attacks.

1. Silently eliminating the threat : This strategy provides protection without requiring any aware-ness or action on the part of users. This includes finding phishing websites and shutting themdown (regulatory and policy solution), as well as detecting and deleting phishing emails auto-matically [47,139]. Other methods that fall into this strategy are: DomainKeys from Yahoo!,which verifies the DNS domain of an email sender and the message integrity [152], SenderPolicy Framework, which uses Simple Mail Transfer Protocol (SMTP) to reject the forged ad-dress in the SMTP MAIL FROM address [135]; and Remote-Harm Detection (RHD), whichcollects Internet browsing history from the client to identify phishing websites at the server

8

Figure 1: The life cycle of a phishing attack, presenting the different stages of phishing attack starting fromplanning until post attack. Figure reproduced from [48].

Figure 2: Example of a phishing email spoofing the organization eBay. Highlights the important character-istics of a phishing email.

9

Figure 3: Example of a phishing website spoofing the organization eBay. Highlights the phishy URL andthe spoofed status bar. Reproduced from APWG [11].

10

end [70]. If phishing threats could be completely eliminated using these methods, there wouldbe no need for other protection strategies. However, existing tools are unable to detect phish-ing emails with one hundred percent accuracy, and phishing websites stay online long enoughto snare unsuspecting victims. According to the Anti-Phishing Working Group (APWG),phishing websites stay online for an average of 4.8 days [11].

2. Warning users about the threat : A number of tools have been developed to warn users thatthe website they are visiting is likely to be fraudulent, either by providing explicit warningsor by providing interfaces that help people notice that they may be on a phishing website.Ye and Sean [154] and Dhamija and Tygar [38] have developed prototype “trusted paths”for the Mozilla web browser that are designed to assist users in verifying that their browserhas made a secure connection to a trusted website. More common are web browser toolbarsthat provide extra cues — such as a red or green light indicating overall safety to informusers that they may be at risk [1, 110, 140, 141]. However, there are three weaknesses withthis approach. First, it requires people to install special software (although newer versions ofweb browsers have such software included). Second, user studies have shown that users oftendo not understand or act on the cues provided by toolbars [101, 150]. Third, a recent studyshows that some anti-phishing toolbars are not very accurate, and even the best toolbars maymiss over 20% of phishing websites [155].

3. Training users not to fall for attacks: There are two schools of thought in the context ofuser education for phishing and semantic attacks. First, education will work because “humanfirewall [the brain]” is the greatest defense [64] and second, education will not work becauseit is “dumb users” who is the cause for most of the security problem [31]. Here, I arguethat technology alone cannot solve the problem of phishing and semantic attacks; instead, itshould be complemented with user education.

Security experts have concluded that user education is not a solution for phishing and securityattacks because “[education] doesn’t work,” “[education] puts the burden on the wrong shoul-der” [109] and “security user education is a myth” [59]. Since security is only a secondarygoal of users, researchers believe that user education cannot be a solution [43].

Researchers have also formally tested whether user education helps users make better deci-sions [6, 67]. However, these studies do not confirm that user education will not work. Onestudy conducted pre and post phishing IQ tests to evaluate the effectiveness of the FTCphishing education material. Results show an increase in the false positive (identifying le-gitimate emails as phishing emails) among the participants who read the training materials.Researchers attribute this behavior to increased concern rather than increased knowledge ofhow to identify phishing emails [6]. However, the training material used for this study doesnot provide any specific principles for identifying phishing emails or websites, and it is de-signed for identity theft in general and not specifically for phishing. Therefore, the trainingmaterials only raise the general awareness or concern for the phishing problem, rather thanproviding knowledge to identify phishing emails.

Another study evaluated the new extended validation feature in Internet Explorer (IE) 7 [67].This study measured the effects of extended validation certificates that appear only in le-gitimate websites and the effect of reading a help file about security features in IE7. Theresults show that participants who did not know about extended validation did not notice

11

the indicator. This study also shows that participants who read the training materials ongeneral security classified both legitimate and fake websites to be legitimate when the warn-ing did not appear. Based on this study, the researchers claim that education did not helpusers make better decisions. However, the training materials used in this study were designedto teach how extended validation works and how the indicator mechanism is used in IE7.The training materials did not provide any specific tips to identify phishing websites. Boththe above studies [6, 67] used training materials which had a broader purpose like teachingpeople about identity theft or general security. Therefore, these results do not necessarilyshow that user education does not work in the context of phishing. In general, most of theonline training materials increase the suspicion level rather than providing specific trainingtowards identifying phishing emails or websites. Hence, there is a need for developing trainingmaterials specifically about phishing and semantic attacks to aid users make better onlinetrust decisions.

Some researchers believe and argue that education and training can help users not to fallfor phishing and security attacks [14, 17, 42, 58, 72, 75], [37, pp. 281], [71, pp. 73]. Someresearchers also believe that educating the end user is one of the most prominent lines ofdefense that an organization can choose to combat security attacks [64]. Organizations spenda considerable amount of money in educating their employees [58]. Studies have shown that“the majority of users were security conscious, as long as they perceive the need for these[secure] behaviors [2, pp. 45].” Thus, user education is important in combating phishing andother semantic attacks.

There are many approaches to training and educating users about phishing. The most basicapproach is to post articles about phishing on websites, as has been done by governmentorganizations [44,45], non-profits [11] and businesses [39,100]. These materials fail to enhanceusers learning because, it is difficult to get large number of users to read these trainingmaterials. A more interactive approach is to provide web-based tests that allow users toassess their own knowledge of phishing. For example, Mail Frontier has set up a websitecontaining screenshots of potential phishing emails [51]. Users are scored based on how wellthey can identify which emails are legitimate and which are not. This approach has beenapplied more to test users than to train them. Phishing education can also be conducted ina classroom setting, as has been done by Robila and Ragucci [126]. However, it is difficult totrain large number of users through classroom sessions.

Another method for educating users is to send fake phishing emails to test users’ vulnerabilityfor these emails. Typically, at the end of such studies, all users are given additional materialsto teach them about phishing attacks. This approach has been used with Indiana Universitystudents [68] and West Point cadets [46], as well as with employees at a New York stateoffice [111]. The West Point and the New York state researchers conducted the study in twophases. In the first phase, participants did not have any prior preparation or training aboutphishing before being tested for their ability to detect phishing attacks. In the second phase,participants were given training materials and lectures about phishing before being testedagain. Both studies showed that education led to an improvement in the participants’ abilityto identify phishing emails. Researchers have also started looking at non-traditional ways,such as comic strips, to educate users about security attacks [69].

The work discussed in this thesis differs from all the above research, as I will focus on the

12

design and evaluation of email interventions to understand what kinds of designs are mosteffective in teaching people about phishing and actually protecting them in practice. Mywork aims to teach people what cues to look for to make better decisions in more generalcases. For example, rather than just teaching people not to fall for PayPal phishing attacks,we want people to learn how to identify phishing attacks in general.

Little research has been done on designing instructional materials to educate people aboutphishing and semantic attacks. This work focuses on how to present the training materialsand what to present in the training materials to educate users not to fall for phishing attacks.In this thesis, I argue that automated detection systems should be used as the first line ofdefense against phishing attacks, but since these systems are unlikely to perform flawlessly,they should be complemented with education to help people better recognize fraudulent emailsand websites. A better informed user will be the best defense for reducing the impact ofphishing.

2.2 Role of Trust

Phishers make use of trust, rather mistrust, that users apply to the phished emails and websites.To educate people about phishing, it is necessary to understand how people make decisions and tounderstand the trust models that have been developed. In this section, I discuss trust, user decisionmaking, and trust models.

2.2.1 Trust and user decision making

Trust is a precious asset in online transactions. Internet users often do not know whether to trust acertain online merchant with their personal information, or whether the email ostensibly sent froma legitimate company has actually been sent by an impostor. Making good online trust decisions isbecoming increasingly difficult even for experts. It requires specialized knowledge (such as computerexperience) and continually updated awareness of threats and attacks. It also requires the abilityto deal with uncertainty and to properly assess risks. The growing amount and sophistication ofspam, phishing, and other semantic attacks are making online trust decisions increasingly difficult.

Users fall for phishing because of the poor online trust decisions that they make. Psychologists haveshown that people do not consider options when they make decisions under stress (here it could beaccessing emails while busy at work). Studies have shown that people under stress tend to makedecisions that are not rational and without thinking of all possible solutions [74]. Psychologistshave termed this as singular evaluation approach. In this approach, people tend to evaluate thesolution options individually rather than comparing with other options and take the first solutionthat works [77, pp. 20]. Also, James Reason in his book Human Error has established that peopleuse patterns and context to make decisions rather than looking at the analytical solution to theproblem [119]. Generally, it is believed that people do not ask the right questions while makinga decision. People are also primed by visible similarities and their experiences while making anydecision [144,145]. In particular, research has shown that non-experts make decisions without muchthinking but by choosing the most obvious solution and least strenuous path. However, expertsmake better decisions by thinking about many strategies [78]. If we extend the above discussion tothe scenario of phishing or semantic attacks in general, one can argue that these (singular evaluation

13

approach, not asking the right questions, and looking for patterns) can be the likely reasons whypeople fall for semantic attacks.

2.2.2 Trust models

Trust is a concept with many dimensions [12, 36, 93, 114] that has been studied in many diversedisciplines: for instance, economics, where the focus is on agents’ reputations and their effect ontransactions [23,25,56,61,106,116,120]; marketing, where the focus is on strategies for consumers’persuasion and trust building [26,28,49,142]; human computer interaction, where the focus is on therelation between design and usability of a system and users’ reactions [36,122,123]; and psychology,where trust has been studied as an interpersonal and group phenomenon [129,134].

Researchers created trust models to find and show the different independent characteristics thatinfluence trust. Formal models of trust started appearing in the literature early in the second halfof the twentieth century. A common approach sees the trustor (the trusting party — for instance,an individual accessing her email) as considering engaging in an interaction with the trustee (theparty to be trusted — for instance, ‘sender address’ in the email) [93]. The trustor engages in thetransaction if and only if the level of trust is higher than her personal threshold trust value [143].Researchers have identified several ‘antecedents’ of trust in their models. Antecedents are factorsthat affect trustors’ trust level when the trustor is interacting with the trustee. Mayer et al. haveidentified trustee’s perceived integrity, benevolence, and ability as trust antecedents; they haveshown that these antecedents may vary independent of each other [93]. Similarly, comprehensiveinformation, shared values, prior trust in technology and the Internet, familiarity, experience, con-textual properties (motivation based on temporal, social, and institutional embeddedness), andintrinsic properties (ability, motivation, and benevolence) have been shown to be the antecedentsfor trustworthy behavior [16, 36, 41, 54, 86, 121–123]. Researchers have also worked on formaliz-ing trust with computational models [90, 105, 117]. These trust models have been used to buildtrustworthy agents [113].

2.3 Learning Science

Learning science is the body of research that examines the foundations of how people gain knowledgeand learn new skills. As pointed out earlier in this document, very little formal work has been donein connecting learning science literature and user education in the context of security. In thissection, I will discuss some relevant learning science literature which I will later connect to user-end security education.

According to Clark, there are five content types that can be learned: facts, concepts, procedures,process, and principles [29]. Through these content types, cognitive skills are developed for process-ing information and applying existing knowledge. Cognitive skills are developed through learning;these skills are developed by active processing of the content types in the memory system. Hu-man memory uses visual and auditory channels for processing information and developing knowl-edge [30]. According to ACT-R (Adaptive Control of Thought–Rational) theory of cognition andlearning, knowledge is distinguished as declarative knowledge (knowing-that) and procedural knowl-edge (knowing-how) [7]. Declarative knowledge is “factual knowledge that people can report ordescribe,” for example, the Internet is a publicly accessible network of interconnected computer

14

networks that transmits data using Internet protocol. Procedural knowledge is “the knowledgeof how to perform a task,” for example, the steps involved in going through the email inbox andchecking for new emails [7]. The ACT-R theory models procedural knowledge as production rules.Production rules are if-then or condition-action pairs [7]. In this thesis, I am interested in trainingusers to create the right production rules for making online trust decisions. In general, existingeducational solutions assume that users have declarative knowledge before they use the system [9].In the context of semantic attacks, since users do not have prior knowledge of how to protect them-selves from semantic attacks, I will aid users to gain declarative and procedural knowledge to avoidbeing victims.

The field of learning science has developed principles based on how humans process information toacquire new skills and to gain knowledge. In particular, the learning science area has developedinstructional design principles which are techniques that support the learning of the content pro-vided in the training materials. Research has shown that previously developed systems that applythese design principles enhance learning [4, 9].

2.3.1 Instructional design principles

Education researchers have developed instructional design principles which provide effective edu-cation to users. In this sub-section, I will discuss some of the instructional design principles thatare relevant to the work discussed in this thesis.

• Learning-by-doing principle: One of the fundamental hypothesis of ACT-R theory of cognitionand learning is that knowledge and skills are acquired and strengthened through practice (bydoing) [7]. Experiments in the cognitive tutor domain have shown that students who get topractice perform better than students who do not. Also students get deeper understanding ofinstruction materials when they are made to explain the steps along with their practice [5].Research has also shown that difficult and random practice sessions are necessary for effectivetransfer of learning [40,131]. Using the algebra cognitive tutors, Koedinger et al. have shownthat making students learn and practice interactively is better than active or passive mode oflearning by doing [80]. Clark Aldrich in his book Learning by Doing mentions that simulationand games are ways to make people learn and practice for better learning experience [3].Learning by doing is also being tried in traditional educational systems, where the coursesare taught using a Story-Centered Curriculum (SCC) [130].

• Immediate feedback principle: Researchers have shown that providing immediate feedbackduring the knowledge acquisition phase results in efficient learning, guidance towards correctbehavior, and reduces unproductive floundering [91,131]. One of the principles developed byAnderson et al. in the context of tutors is “provide immediate feedback on errors [9].” UsingLISP tutors, Corbett et al. showed that students who got immediate feedback performedsignificantly better than students who got delayed feedback [34]. Anderson et al. have alsodiscussed that the feedback should be immediate otherwise students think about somethingelse [8]. Research has also shown that providing simple feedback like “yes” or “no”, or detailedfeedback like “the shot was off target to the right by 22mm” provided effective learning [40].

• Conceptual-procedural principle: A concept is a mental representation or prototype of objectsor ideas for which multiple specific examples exist (e.g. phishing) [29, Chapter 4]. A procedure

15

is a series of clearly defined steps which results in the achievement of a given task (e.g. loggingonto a computer) [29, Chapter 3]. The conceptual-procedural principle states that “conceptualand procedural knowledge influence one another in mutually supportive ways and build in aniterative process [73].” Conceptual materials lead to large usage of working memory of thelearner thereby causing difficulty in learning the materials. Presenting procedural materials inbetween conceptual materials helps reiterate the concepts learned. In this way, the conceptsare re-inforced through procedures for better learning and vice versa. Understanding theconceptual and procedural knowledge is necessary in order to develop a competence in agiven area [32]. This principle can be used to improve learning by providing conceptual andprocedural knowledge iteratively. In an experiment to teach decimal places, students whowere given concepts and procedures iteratively, performed better than the group where thetraining information was presented consecutively [73,79].

• Contiguity principle: Mayer et al. proposed the contiguity principle, which states that “the ef-fectiveness of the computer aided instruction increases when words and pictures are presentedcontiguously (rather than isolated from one another) in time and space [96].” Psychologistsbelieve that humans make sense out of the presented content by creating meaningful relationsbetween words and pictures [30, Chapter 4]. In an experiment, students who learned aboutthe process of lightning performed better in understanding the materials when words andpictures were close to each other (spatial-contiguity) [102]. In another experiment, studentswere asked to read a passage about vehicle braking systems. Students who received passageswith words and pictures together explained the braking system better than the other groupin which words and pictures were presented separately [94]. In another experiment, Butcheret al. show that students in the group who had visual and verbal information presentedtogether performed better and had deeper transfer than the group who saw the informationseparately [22].

• Personalization principle: This principle states that “using conversational style rather thanformal style enhances learning [30, Chapter 8].” People make efforts to understand the instruc-tional material if it is presented in a way where they feel that they are in a conversation thanjust receiving the information. It is suggested to use “I,” “we,” “me,” “my,” “you,” and “your”in the instructional materials to enhance learning [95, Chapter 8]. In an experiment aimedat teaching arithmetical order-of-operation rules, students who received conversational stylemessages were engaged more and had better learning than the control group students [35].Experiments have also compared the formal and conversational information presentation ina computer based lesson on lightning formation. Results showed that students learned betterwhen the information was presented in a conversational style [103].

• Story-based agent environment principle: Agents are characters who help in guiding the usersthrough the learning process. These characters can be represented visually or verbally andcan be cartoon-like or real life characters. The story-based agent environment principle statesthat “using agents in a story-based content enhances user learning [104].” People tend to putin efforts to understand the materials if there is an agent who is guiding them in the learningprocess. Learning is further enhanced if the materials are presented within the context of astory [95, Chapter 8]. People learn from stories because stories organize events in a meaningfulframework and tend to stimulate the cognitive process of the reader [77, Chapter 11]. Hermanis an agent in the story based computer aided lessons on designing roots, stem, and leaves of

16

plants to survive in 8 different environments. Experiments showed that students in the groupwhich had Herman guiding them, outperformed the unguided group in terms of learningand right decision making [104]. Many such agents have been developed to help users learnbetter [88]. It was also found that existence of the agents influences learning while the features(cartoon like or real life) of the agents had little effect [148,151].

2.3.2 Measurement of learning

Real-world training should: (1) help a learner acquire new knowledge; (2) enable the learner toperform the skills learned in the long run; and (3) enable him/her to transfer the learning torelated and altered tasks [131]. These requirements can be used as a framework to measure theuser learning. There are many definitions for these measurements; for the work discussed in thisthesis proposal, I operationalize the measurements as follows:

• Knowledge Acquisition (KA): This is the ability of people to process and extract knowledgefrom the instructional materials. Users should be able to use the acquired knowledge to makea decision in a given situation [13, 89]. This is usually evaluated by testing the skills gainedjust after learning by asking learners to repeat or apply the knowledge that they have gained.

• Knowledge Retention (KR): There exists a large body of literature on quantifying retention;researchers have also created retention functions to describe the behavior of human mem-ory [128]. KR is defined as the ability to retain or recall the concepts and procedures givenby the content types when tested under the same or similar situations after a time period δfrom the time of acquisition of knowledge. Researchers have frequently debated the optimumδ to measure retention. I will use the time difference dimension (δ) to measure KR. Thishas been one of the most frequently used dimensions to measure KR [13, 128]. In this the-sis, I will measure retention where δ is anything more than one day. Researchers have useddifferent time periods ranging from 1 day to 20 days to test retention [13, 18, 92]. If testingis done within one day of training it is considered more of a test for knowledge acquisitionthan knowledge retention. I plan to measure retention at different time periods choosing asuitable time from literature. One way to move the knowledge gained through training tolong-term memory of the user is by frequent testing [30]. Therefore, I plan to send testingemails frequently to users to test their knowledge retention.

• Knowledge Transfer (KT): The definition of transfer and the measurement of transfer isheavily debated in learning science literature [20, 133, 137]. Researchers have also developeda taxonomy to classify and identify different types of transfers that are stated in the litera-ture [15,52]. Two types of transfers that are mostly discussed in the literature are immediate(near) transfer and delayed (far) transfer [50,98]. Researchers have emphasized that transfer-ability of learning is of prime importance in training. For the purpose of this thesis, transferis the ability to extend the learning in one instance of a phishing attack to another after atime period δ. As in retention, there is also considerable debate about the optimum δ tomeasure knowledge transfer. In this thesis, I will measure transfer at different time periodschoosing a suitable time from literature.

17

3 Past and present work contributing to this thesis

In this section, I will discuss some of my past and present work that contributes to my thesis work.Research presented in this section was or is conducted with other members of my research group.Discussing these research results serves several purposes: (1) it shows some preliminary results toindicate that user education helps users make better online trust decisions; (2) it shows that, I havethe necessary understanding of the literature in the different domains related to this thesis; and (3)it demonstrates that I have acquired necessary skills to complete the proposed work. The studydescribed in Section 3.1 helped me to understand the decision making process of the users andstrategies that they use in making their decision. I used this knowledge along with learning scienceprinciples to design instructions which were tested in a laboratory study (Section 3.2). I thenanalyzed the effectiveness of the existing online training materials among users and also analyzedthese training materials using learning science principles. Results from this study are discussed inSection 3.3. The study described in Section 3.4 evaluates the effectiveness of the training materialswhen presented in the form of an interactive game. The studies mentioned above tested usersimmediately after the training; but knowledge retention and knowledge transfer are the necessarymeasures for evaluating any training system. Currently, I am conducting a user study where I amtesting the effectiveness of the training materials to enhance knowledge retention and knowledgetransfer. This study is described in Section 3.5.

3.1 A Model of Trust in Phishing Scenario (MoTPS)

As described earlier, understanding the decision making process of users is an important first stepin helping them make better online trust decisions. We created a trust model to show the onlinetrust decision process of experts and non-experts in the area of phishing. Figure 4 shows thebasic trust model which can be modified to show the experts’ and non-experts’ model. This studyhelped us in understanding the signals (the information available to users about the states of theworld) that users use to make their decision and the actions (the set of things that a user may doin a given situation) that users take in a given situation. We use the expert decision process todesign instructional materials which can help non-experts make better decisions. To evaluate the

Figure 4: A Model of Trust in Phishing Scenario (MoTPS). Presents the generic model which can be modifiedto represent expert and non-expert model.

18

trust model, we collected data through an interview study; we interviewed 11 experts and 14 non-experts on their decision making strategies. During the interviews, we asked the subjects about theirdecision making strategies in the following scenarios: (1) receiving emails, (2) accessing websites,(3) downloading software, and (4) buying products online. We found significant differences betweenthe decision making process of experts and non-experts. We plan to use the lessons learned fromthis research to inform the design of the training system. We have already published a conferencepaper on the preliminary results from this study [83]; we are currently working on a journal versionof the paper. The trust model and the evaluation of the model will comprise one chapter of mythesis.

3.2 Protecting People from Phishing: The Design and Evaluation of an Em-bedded Training Email System

In this study, we discuss the design and evaluation of an embedded training (discussed further inSection 4.1) email system that teaches people about phishing during their normal use of email. Weconducted lab experiments contrasting the effectiveness of standard security notices on phishingwith two embedded training designs we developed. In this study, we tested the hypotheses: (1)people do not read the security notices that are sent by organizations, (2) users learn more effectivelywhen the training is embedded compared to security notices, and (3) users learn more effectivelywhen the instruction is in the form of a comic strip compared to graphics and text. In this study,we had three groups of ten participants each: security notices group, graphics and text group, andcomic strip group. Since we were interested in training non-experts, we recruited subjects who hadlower computer skills. The user study consisted of a think-aloud session in which participants playedthe role of “Bobby Smith,” an employee of Cognix Inc. who works in the marketing department.Participants were told that the study investigated “how people effectively manage and use emails.”They were told that they should interact with their email the way they would normally do in reallife. Figure 5 presents the comic strip design that we tested. We applied the principles discussedin Section 2.3 to the design of the instruction materials.

We had three groups in the study; the “notices” group, was shown typical security notices, the“graphics and text group” was shown the intervention, which describes the risks of phishing, showsa small screenshot of the training email, points out cues to look for in a phishing email, and outlinessimple actions that users can take to protect themselves [84, Figure 2], . The “comic strip” groupwas shown Figure 5 which conveys same information as the graphics and text intervention, but ina comic strip format. We performed pre and post tests to evaluate the effects of training amongparticipants. The number of participants falling for phishing attacks before and after training inthe security notices group was nine (90%), while the number of participants falling for phishingattacks in the comic strip group was ten (100%) before training and three (30%) after training.The difference between these two groups was significant (Chi-Sq = 23.062, P-Value = 0.001). Wealso compared data for the individual performance of the participants before and after training. Weobserved that 9 out of 10 participants (90%) in the security notices group clicked the first phishingemail and out of these 8 participants (89%) clicked on the final phishing email. In the graphics andtext group, 8 participants (80%) clicked on the first phishing email out of which 5 (63%) clickedon the final phishing email. In the comic strip group, 10 participants (100%) clicked on the firstphishing email out of which 3 participants (30%) clicked on the final phishing email. We found thatindividual performance of participants is significantly different between the security notices group

19

Figure 5: Earlier comic strip design. Presents information using the design principles discussed in Sec-tion 2.3.1.

and comic strip group (Chi-Sq = 18.245, P-Value = 0.001). Also, there was significant differencebetween the performance of participants in the graphics and text group and the comic strip group(Chi-Sq = 7.222, P-Value = 0.007). There was no significant difference between the performanceof participants in the notices group and the graphics and text group. The study results confirmedthe hypotheses presented earlier. We have already published a conference paper on the results fromthis study [84]. A part of a chapter of my thesis will be dedicated to this study.

In this study, we tested only the participant’s knowledge acquisition, we did not test knowledgeretention and transfer. I am currently working on a study to measure the knowledge retention andtransfer in the embedded training methodology (discussed in Section 3.5). In the study describedin this section, we showed that keeping the delivery channel (embedded training) the same andchanging the content (graphics and text vs. comic strip) has different effects on learning. We areconducting a study to test whether changing the channel but keeping the content same will haveany effect on learning (discussed in Section 3.5).

3.3 Teaching Johnny Not to Fall for Phish

Understanding the effectiveness of the existing online training materials is important while design-ing new instructional materials. Previous studies have not evaluated the quality of the trainingmaterials used in user studies or considered ways of designing more effective training materials. Inthis study, we wanted to analyze the existing online training materials that teach people how toprotect themselves from phishing attacks. This will help us in understanding the current, state-of-art training materials that are used for training users. We conducted a user study where wetested the effectiveness of the existing online training materials. We had two groups (control andtraining) of 14 participants each. Similar to the study described in Section 3.2, we recruited non-experts with low computer skills. After evaluating 25 of the existing online training materials on

20

phishing, we chose three of the most frequently cited training materials (eBay [39], FTC [44] andMicrosoft [100]). In this study, the participants were asked to gauge the legitimacy of some web-sites. They were told “Imagine you have received an email that has a link, and you click on thelink to find out if the website you are visiting is the real website or a fraudulent copy.” We used20 websites (10 pre-test and 10 post-test) for the study. After completing the pre-test websites,participants were given 15 minutes, during which the control group played a game like solitaire ordid mundane tasks. The training group viewed the above-mentioned training materials along withan introduction to URL [108].

We found that for the training group, there was a significant reduction in the false negative rateafter the training — from 0.40 to 0.11 (paired t-test: µ1=0.40, µ2=0.11, p = 0.01). There wasno statistically significant change in the false negative rate for the control group (paired t-test:µ1=0.47, µ2=0.43, p=0.29). The false positive rate remained virtually unchanged for the controlgroup, it increased from 0.31 to 0.41 in the training group. However, this increase is not statisticallysignificant (paired t-test: µ1=0.31, µ2=0.41, p=0.12). Total correctness is defined as the ratio of thenumber of correctly identified websites to the total number of websites shown to the participants.The total correctness for the control group changed from 0.59 in the pre test to 0.61 in the post-test, however, this change was not statistically significant. The total correctness of the traininggroup changed from 0.65 to 0.74. This change was marginally statistically significant (p=0.11).We found that the training materials were surprisingly effective when users actually read them.We then analyzed the training materials using instructional design principles from learning science,and provided some suggestions on how to improve training materials based on those principles. Wehave already published a technical report on the results from this study [85]. I plan to dedicatepart of a chapter of my thesis to this study.

3.4 Anti-Phishing Phil

There are three types of information presentation schemes: static, dynamic, and interactive [29,80].The study discussed in Section 3.2 used a static design. In dynamic presentation form, the trainingmaterials are presented in video or movie form. We wanted to understand the effectiveness oftraining when the instructions are presented using an interactive scheme. Hence, we conductedanother study to address the question of “How good is an educational game in training users aboutphishing URLs?” We created an interactive game called “Anti-Phishing Phil” in which Phil fish(the son) is taught by the father fish about different tips and cues to identify phishing websitesfrom the URLs. Figure 6 shows one of the screens in the game. The main focus of the game wasto teach users how to identify phishing URLs, where to look for cues in web browsers, and how touse search engines to find legitimate websites.

This Anti-Phishing Phil study had three groups of 15 participants each: (1) existing trainingmaterials group, (2) tutorial group, and (3) game group. The study setup was kept same asdiscussed in Section 3.3. The data for the existing training materials group was reused from thestudy described in Section 3.3. The tutorial group was asked to spend up to fifteen minutes readingan anti-phishing tutorial we created based on the Anti-Phishing Phil game. The tutorial included17 pages of color printouts of all of the between-round training messages and lists of the URLs usedin the game with explanations about which were legitimate and which were phishing, similar tothe games end-of-round screens. The game group played the game for fifteen minutes. This setup

21

Figure 6: Anti-Phishing Phil game screen. Phil, the small fish near the top of the screen, is asked to examinethe URL next to the worm he is about to eat and determine whether it is associated with a legitimate website or a phishing site. Phils father (lower right corner) offers some advice.

helped us to understand whether it is the interactiveness or the materials that provides betterlearning.

The results showed that post test false negative rates in all three groups decreased significantly fromthe pre-test values. For the existing training materials group, the false negative rate fell from 0.38to 0.12 (paired t-test: µ1=0.38, µ2=0.12, p = 0.01); for the tutorial group, it changed from 0.43 to0.19 (paired t-test: µ1=0.43, µ2=0.19, p < 0.03); for the game group, it changed from 0.34 to 0.17(paired t-test: µ1=0.34, µ2=0.17, p < 0.02). Post-test false positive rates decreased significantlyin the game group (paired t-test: µ1=0.30, µ2=0.14, p < 0.03). Combining false positive andfalse negatives we derived a measure for the total correctness. We found in the post-test thatthe game group performed better than the existing training material group (2 sample t test, p <0.02). We did not find the tutorial group’s improvement to be significant over the existing trainingmaterial group. In the context of this thesis, this research helps me understand how the interactiveinformation presentation scheme can be applied to phishing. We have already submitted a paperto a conference on the results from this study [136]. I plan to dedicate part of a chapter of thesisto this study.

3.5 Laboratory study I

I am currently conducting a study that has multiple goals: (1) to design an experiment to measureknowledge retention and knowledge transfer, (2) to measure the effects of learning in embeddedversus non-embedded training methodology, and (3) to measure the effect of presenting training

22

materials to educate users against materials that just increase the suspicion level for online inter-actions. Embedded methodology is the means in which the training materials are presented whenthe users fall for the phishing emails (as discussed in Section 3.2). In non-embedded methodology,the training materials are presented in the email itself. Increasing suspicion means just providinginformation regarding online frauds but not actually providing cues or strategies that can be usedfor making a better decision.

In this study, I plan to test the following hypotheses:

L1 - H1: Users learn more effectively when the training materials are presented whenusers fall for the attack (embedded) than when the training materials are sent by email(non-embedded).

L1 - H2: Users learn more effectively when the training is embedded and non-embeddedcompared to raising the suspicion.

L1 - H3: A user’s knowledge retention and knowledge transfer is more in the embeddedthan the non-embedded training methodology.

L1 - H4: A user’s knowledge retention and transfer is more in the embedded and thenon-embedded training methodology compared to raising the suspicion.

To test these hypotheses, I am conducting a study among three groups: (1) non-embedded group,where the participants get the training intervention emails with the instructional materials inthem; (2) embedded group, where the participants get the training intervention emails, but theinstructional materials are shown only when they click on the link in the email; and (3) suspiciongroup, where the participants get an email about phishing from a friend. I am using the comicstrip design which was improved from an earlier version (Figure 7) as the instructional materialsfor the non-embedded and the embedded group. The experimental setup is similar to the studydescribed in Section 3.2. I am showing participants 31 emails of the following types: legitimateemails, testing emails, training emails, and spam emails. I will have 15 users in each group for thestudy. I am carefully recruiting non-expert participants by posting fliers and online postings. Tomeasure retention and transfer this study is being conducted in two parts. I ask participants tocome back for a follow-up study after 7 days from the first part of the study. The follow up studyhas the same setup as the first part. I am conducting a pre test (before first part) and post test(after second part) survey to understand the change in behavior among participants.

Preliminary results show that participants in the non-embedded group spend less time in readingthe training materials compared to the participants in the embedded training group. There is alsopreliminary evidence that participants in the embedded group retain and transfer more knowledgethan participants in the non-embedded group. Most of the participants in the embedded groupmentioned that showing the training materials after clicking on the link in the email was effective.One of the participants mentioned, “I was more motivated to read the training materials since itwas presented after me falling for the attack.” We attribute this behavior to the learning-by-doingprinciple and showing immediate feedback principle. I plan to submit the results from this studyas a paper for a conference. I plan to dedicate a part of one chapter of my thesis to this study.

23

Figure 7: Latest comic strip design. This is an updated version of the design presented in Figure 5. Thisdesign was also developed by applying the principles discussed in Section 2.3.1. The top layer presentsthe activities of a phisher and the bottom layer presents the victim and the steps that the PhishGuru issuggesting.

4 Proposed Research

In this section, I will present the research that I propose to conduct in order to complete my thesis.The research proposed in this section builds on the results presented in Section 3.

4.1 Embedded training concept

Conventional wisdom says “training users does not work,” which is based on the assumption thatsecurity is a secondary task for users; therefore, users do not have an incentive to spend timereading the training materials. From the past research discussed in Section 3.2 and Section 3.3, Ihave shown that training materials are effective if users read them. The question then is how toget the users to read the training materials. In this section, I will discuss an approach to persuadeusers to read the training materials.

Education researchers believe that training can be most effective if the training materials can in-corporate the context or situation of the real world, work, or testing situation [10, 30]. Embeddedtraining is the methodology in which the training materials are integrated into the primary tasksthat users perform in their day to day lives. This training methodology has been widely applied intraining military personnel on new Future Combating Systems (FCS) [21,76]. Researchers believethat “embedded training is the ability to train a task using the associated operating system [soft-ware, machine, etc. that people use].” The philosophy behind embedded training is “to provideeffective training anytime, anywhere.” Embedded training can be classified into: fully embedded,where the training component is completely built into the primary system (task); appended embed-ded, where the training component is attached to the primary system when needed; and umbilical

24

embedded, which is the same as appended embedded training, but requires external hardware toconnect to the primary system [21, 27]. The design that I plan to adopt for the complete trainingsystem is a hybrid of the appended and the fully embedded systems, where the training system canbe installed on the primary system or used by other systems. I plan to write libraries and modulesthat can be used to integrate the training system that I plan to build with other systems. Studiesdiscussed in Section 3.2 and Section 3.5 are helping me iterating the functionalities of the completetraining system. Results from these studies are also helping me in iterating the intervention design.

There are two primary intervention points for an anti-phishing training system: email and website.I chose to focus on email rather than websites for three reasons. First, email is the main vectorfor delivering phishing messages to users. If I can prevent people from trusting phishing emails,they will not visit phishing websites. Second, anti-phishing websites [39, 44] require end-users toproactively visit them, limiting the number of people who will actually see these websites. Incontrast, an embedded training approach brings information to end users and teaches them overtime to differentiate between legitimate and illegitimate emails. Third, end users must alreadyhave some knowledge about phishing or other kinds of scams to seek out educational websites. Incontrast, embedded training (if distributed with standard email clients or sent by companies) worksfor experts as well as non-experts who are unaware of phishing by educating end-users immediatelyafter they have made a mistake. Studies have shown that providing immediate feedback enhanceslearning [7, Chapter 7], [92].

The embedded training system that I plan to develop works roughly as follows. From the pastresearch discussed in Section 3.2, I observed that 93% of the participants who clicked on linksprovided their personal information. Therefore, I want to use this as the intervention point fortraining. In this system, people are periodically sent training emails, perhaps from their systemadministrator or from a training company. Users can access these training emails in their inboxwhile they are checking their regular emails. These training emails look just like phishing emails,urging people to go to some website and log in. If people fall for the training email, that is, clickon a link in that email, we provide an intervention message that explains that they are at riskfor phishing attacks and give some tips for protecting themselves (as in Figure 7). Below theintervention message there will be a pointer (URL, but not as a link) to the game that I describedin Section 3.4. This approach of embedded training has two advantages: (1) it enables a systemadministrator or training company to continuously keep training people as new phishing methodsarise; and (2) it enables the users to get trained on semantic attacks without taking much timeout of their busy schedule. This system will have some type of training for all types of usersstarting from non-experts to experts; the system will be able to personalize the training accordingto the users. However, my initial focus will be to develop materials and methodologies to trainnon-experts. From now on I will refer to the system that I am developing as “The PhishGuru.”

The PhishGuru technique is close to the approach taken by the staged user interface which wasintroduced by Carroll et al. in the Training Wheels user interface [24]. In the training wheelsapproach, users are taken through a sequence of stages of system usage so that users do not makeany errors and they also understand the system. This methodology of shepherding the user has beenfound very effective in educating people about software or application usage [62]. This approach hasbeen adopted for security interfaces by Whitten [146, Chapter 4]. Others have also tried to createa training component that is embedded in a Microsoft excel spreadsheet [125]. These approachestrain users to operate software while the PhishGuru is aimed at changing the decision making

25

process of users and thereby changing their behavior.

4.2 HCI design methods

HCI methods play an important role in the design and evaluation of educational systems [55].Usability should be considered from the beginning of the design of educational systems and it shouldbe an integral part of all stages of design and implementation. The learning science principles, whenapplied using HCI design methods, enhance the effectiveness of an educational system [55]. In thissection, I will briefly describe the HCI approaches that I plan to use in my research.

Broadly, there are three stages in a system development process: designing, prototyping, and evalu-ating. I plan to use the iterative design approach, in which the design stage informs the prototyping,which in turn informs the evaluating stage, and the results from the evaluation are sent to the designof the system [107]. I describe how each of the three stages are applied in the PhishGuru.

1. Designing : From the study discussed in Section 3.1, I collected data regarding the signals,decisions, and decision process of experts and non-experts on different online activities. Thisstudy aided me in understanding the difference between the experts and non-experts with re-spect to their online trust decision making. I will use this knowledge to design the curriculum(refer to Appendix A) to train the non-experts. I have developed a draft curriculum which Iplan on iterating and making it comprehensive. I am also constantly iterating the interventiondesign discussed in Section 3.2. I plan to test the curriculum and the interventions through afew informal studies. I plan to test the curriculum by showing the production rules along withthe training content to non-experts to get feedback on the difficulty level of understandingthe instructions. I will the use this insights to iterate on the curriculum design. Similarly, Iwill also test the interventions among non-experts by asking them to provide feedback on thedesign of the intervention. I plan to use the comic strip design to create interventions for allinstructions. The study discussed in Section 3.3 helped me understand the landscape of ex-isting online training materials. The insights from this exercise helped me in designing bettertraining materials. Through these studies, I have also gained insights on the functionalitiesof the PhishGuru, and the essential components of the instructional materials that createsbetter learning.

2. Prototyping : Prototyping is a stage in which the requirements collected in the design phaseare converted into prototypes. For the studies discussed in Section 3.2 and Section 3.4, Ifirst created paper prototypes to get initial feedback from both the research team and actualusers. In developing the PhishGuru, I plan to create the paper prototypes (low fidelity) ofthe system and conduct informal studies with users to get feedback. Using this feedback, Iplan to do a HTML mockup of the system. The entire process of the paper prototype and theHTML design should not take more than 6 - 7 days for one iteration. Later, I will conductsome more informal user studies and then develop the system using JAVA, PHP, MySQLand Apache. I describe the components of the PhishGuru in detail in Section 4.5. I havedeveloped prototyping skills through the studies described in Section 3.

3. Evaluation: In this phase, the hypotheses developed while creating the system are tested.I will test my thesis statement through a few pilot studies, two lab studies, and a field

26

study. The planned laboratory user studies will have observation of users and think aloudsessions [57]. The user study that I am currently working on (discussed in Section 3.5) willbe one of the laboratory studies to test the thesis statement. I describe in detail the studythat I will conduct for evaluating the PhishGuru system in Section 4.6. I will also evaluatethe measurements discussed in Section 2.3.2. I have developed skills to evaluate systems andprototypes using different HCI methods through studies discussed in Section 3.

4.3 Instructional materials

In Section 2.3, I introduced the learning science principles and evaluation metrics from the learningsciences field. In this section, I will discuss how I will adopt these concpets in my designs. I planon using these design principles to gain insights on how to show the training materials to users. Allthe designs that I will create for this thesis will follow the instructional principles. For example,in Figure 7, instruction number 2, asks to “Type in the real website address into a web browser”along with an image of a browser (contiguity principle). I apply the personalization principle inthe comic strip design through “You’ve got mail !” and “I forged the address to look genuine.” Iuse the character “PhishGuru” as an agent in this design to provide the tips and instructions tothe users (agent environment principle). I have also used a story line where “The Phisher” is seendoing bad things and “The Victim” is taught how to avoid falling for phishing attacks (story-basedbackground). I found these principles effectively convey the training messages to the users. To addfun to the design and to increase the engagement of the users, I also plan on using comics stripas the template for the instructional materials. Historically comics have been suggested to be aneffective medium of instruction [60,124,138,153]. Comics have been applied to educate students inBiotechnology and Mathematics [127,153]. Researchers are also trying to provide primary schoolinginstructions through comic strips [63, 66]. But little work has been done in designing comic stripsfor technical training.

To address the question, what should the system present in the instruction to make learning effec-tive, I will use the data that I have collected through the studies mentioned in Section 3. Using thisdata, data from real world security experts, data from online materials, and data collected frominformal studies, I will create a curriculum of instructions that are reliable and easy to follow toavoid falling for phishing attacks. This curriculum will be used to train different users dependingon their level of expertise.

4.4 Learning algorithms

In this section, I describe the algorithms that I will develop to help the PhishGuru trace theknowledge that users are gaining, and rate the user according to their performance. Using thesealgorithms, the PhishGuru will provide personalized training.

1. Knowledge tracing algorithm: The goal of this algorithm is to model the changing knowl-edge [33] of the user using the PhishGuru. Knowledge tracing algorithm developed by Corbettet al. is a sophisticated probabilistic Bayesian algorithm for implementing mastery learningin cognitive tutors. This algorithm needs the estimated probabilities [33, Figure 4] of usersas input for tracing the knowledge. These probabilities are not yet available in the context of

27

Figure 8: A few production rules for phishing from the curriculum that I am building. Instructional contentwill be designed for each production rule and presented in the interventions.

phishing education. Therefore, I plan to modify the knowledge tracing algorithm to a simplis-tic proportion algorithm as discussed in this section. As discussed in Section 2.3, knowledgeis represented in the form of production rules in the human brain. Figure 8 presents a fewproduction rules that the PhishGuru will use to train non-experts. These rules are devel-oped using: the data that I collected from the study discussed in Section 3.1, the literatureavailable on phishing, and from members of our research group. I have started working on acomprehensive curriculum to train users about phishing. The rules presented in Figure 8 is asample from the curriculum. To make the curriculum more comprehensive, I am also seekinghelp from security experts.

Each production rule is tagged to specific instructional materials and an intervention email.For example, the production rule “NEVER CLICK ON LINKS: IF the email has a link init THEN never click on the link within the email” from Figure 8 will be tagged to Figure 7because the training message provides the instructions about the production rule. Also, theproduction rule is tagged to an intervention email like in Figure 2 which will persuade theuser to go to the training material. Using this curriculum, taxonomy of different phishingattacks, and the mental model of the experts, I plan to arrange the production rules in such amanner that the knowledge tracing algorithm will identify the next production rule that theuser has to be trained on. For example, the production rule NEVER CLICK ON LINKS willbe presented before any other production rule. Using this arrangement of production rules,the knowledge tracing algorithm can choose the next production rule from the comprehensivelist. I plan to group the production rules on the basis of the type and nature of the rule. Iforesee 5 to 6 high level categories of production rules and each category will contain 5 or 6rules. The PhishGuru will train users on one category and then move on to the next higherlevel category.

The knowledge tracing algorithm is employed in the PhishGuru to implement mastery learn-

28

ing. Mastery learning is a technique in which users are trained on a particular concept orprocedure until they have mastered it [19]. The PhishGuru will present the training materialsfor a particular production rule until the user has mastered the rule. I plan on using a knowl-edge score to decide whether the users have mastered the rule. Knowledge score is definedas the ratio of correctly identified phishing emails (sent by the PhishGuru) for a particularproduction rule to the total number of phishing emails (sent by the PhishGuru) received bythe user (Equation 1).

Knowledge score =Correctly identified phishing emails

Total number of phishing emails received(1)

The mastery criterion in the PhishGuru is a knowledge score of 0.80; this means, once the usercrosses the knowledge score of 0.80 for all the rules for a particular category, then the systemsends the training materials for the next category. This algorithm enables the PhishGuru totrain users individually according to their level of expertise and to personalize the trainingaccording to their performance. Some form of this personalization and individualization usingmastery learning has shown effective results [19,81,82].

This algorithm will also help in scaffolding the training materials. As discussed earlier, peoplelearn by connecting to the concepts and facts that are available in their memory [32]. Scaf-folding is a way of training, where users are taught new concepts and procedures, and usersconnect to or make use of their existing knowledge to learn these [33,62]. The curriculum thatI am currently developing carefully embeds scaffolding of knowledge into it. I will providenecessary declarative knowledge while training the users. I discuss the specific features of thealgorithms that I will implement in the PhishGuru, and the procedure to test the algorithmsin Section 4.5.1 and Section 4.6.1 respectively.

2. User rating algorithm: The goal of this algorithm is to rate the users according to theirknowledge and performance while using the PhishGuru. I plan to rate the users according totheir knowledge score and their scores from the tests (Equation 2).

User rating = f(knowledge score, scores from tests) (2)

Knowledge score will be obtained from Equation 1; the test score will be obtained as follows.I will collect information about the knowledge that the users have gained by asking questionsrelated to phishing emails and websites when they fall for phishing emails. Users will get tothese questions from a small fraction of training and testing emails that the PhishGuru sendsto the users. In total, I plan to send five emails a week to the users, of which two will betraining, two will be testing and one will be a link to questions. These questions will helpthe PhishGuru in gauging the users’ knowledge gain. The questions that I plan to ask fortesting the users will be aligned with the production rules that users have learned. The typesof questions that I will ask are multiple choice and explanation questions. The knowledgescore tests whether users can apply their knowledge (procedural) and the test score evaluateswhether users understand the concepts (declarative knowledge) without asking them to applyit. The results from these tests will be converted to a score and will be used in calculatingthe user rating. User rating will be calculated both at the individual level and the user grouplevel. This algorithm can also be applied to cluster users so that training can be personalized.I plan to show both the individual and aggregate scores to the user in a specific interface (e.g.

29

in the user group that you belong you are currently rated 12 of 17; the lower the numberthe better it is). Knowing the individual rating and the user group rating might create peer-pressure on the user to perform better and to improve the strategies and rules that theyuse in making their decision. Studies have shown that peer-pressure is one of the factors forchanging behavior [129].

4.5 The PhishGuru system

After reviewing the literature and doing an informal task analysis for embedded training systemin the context of phishing, I came up with the following requirements of the PhishGuru: provideinstructional materials; monitor and assess the users; provide feedback to the users; personalizeand contextualize the training; and make the training adaptive. I have incorporated these featuresin different modules of the PhishGuru. In this section, I describe the architecture of the PhishGurualong with the implementation details of the system.

4.5.1 The PhishGuru architecture

In this section, I describe the architecture of the PhishGuru system along with the functionalitiesof the modules (knowledge tracing, intervention, user rating, interpretation, user profiling, andinterface) of the system.

Figure 9 gives the architecture of the PhishGuru. The system works as follows. When the userscheck their email in their inbox, instead of directly going to their email server, they will connect toproxy server that will query the PhishGuru system for training interventions through the interfacemodule of the PhishGuru. If any intervention is available for the particular user, the PhishGuruwill provide all necessary information (discussed later in this section) to the proxy. The proxyalso queries the actual email server for users’ emails. While sending these emails from the actualemail server to users’ inbox, the proxy embeds the training email into the users’ email inbox. Theproxy can also be replaced by a plugin, which can connect to the PhishGuru to collect traininginterventions and place them in users’ email inbox. The users get to see the training emails intheir email inbox. In the process of training users, the PhishGuru tracks user behavior only for thetraining and testing emails. Training emails take the user to the instructional materials when theuser clicks on the link in the email. The testing email takes the user to the fake phishing websitethat I have hosted. The PhishGuru does not capture any personal information and the system doesnot track other emails of the users.

The following are the different modules of the PhishGuru:

• Knowledge tracing module: This module has the following functionalities: (1) ability to decideon the next intervention to be sent to the user; (2) curriculum; (3) user model; and (4)retraining and retesting time. As discussed earlier, the curriculum will be developed bycollecting data from real world security experts and it will be constantly updated. The usermodel will be updated by the output from the web server which is reported to the knowledgetracing module. This module incorporates the knowledge tracing algorithm discussed inSection 4.4; it calculates the knowledge score. Upon requests from the interpretation module,this module also adaptively decides which training intervention will be sent to the user next.

30

Figure 9: The PhishGuru architecture. Presents all the modules of the system. It also presents how themodules of the PhishGuru system will interact with other components in the architecture.

This module also keeps track of when the last training or testing email was sent to the user.According to the preference of the user, this module decides when the next training or testingemail will be sent to the user. Efforts involved in this module are: create the curriculum;create a knowledge tracing algorithm; create a function to decide on the next intervention;develop a methodology for updating the user model; and implement and test the module. Ialso plan on testing this module separately before building the complete system. The testingprocess of this module is discussed in Section 4.6.1.

• User rating module: As discussed earlier, users are rated according to the knowledge scorevalue and test scores. Applying the user rating algorithm, this module helps in calculatingthe test scores, converting the test scores to user rating, and providing feedback about theuser performance through the interface module. Efforts involved in this module are: createa function for calculating the user rating; create questions that will be asked to users tocollect further data about their learning; and implement and test the module. To developthis module, I will draw knowledge from recommendation and reputation systems [65, 112].I plan on testing this module separately before building the complete system. The testingprocess of this module is discussed in Section 4.6.1.

• Intervention module: This module stores all the email interventions (training and testingemails, and instructional materials) that will be used by the PhishGuru. Through the studydiscussed in Section 3.2 and 3.5, I have created multiple iterations of the interventions. I willuse the designs that are tested in these studies to develop interventions focussed on otherproduction rules discussed in the curriculum (Appendix A). Efforts involved in this moduleare: to create the instructional materials; to decide on the training and testing emails; and

31

to arrange the emails and materials in line with the curriculum.

• Interpretation module: The interpretation module plays an important role in the PhishGuru;this module connects the knowledge tracing, user rating and intervention modules to othercomponents of the system through the interface module. This module makes sure that all theother modules are doing what they are supposed to be doing.

• User profiling module: This module collects information about the user when the user startsusing the PhishGuru. Some information that I plan to collect is: how often to send thetraining interventions and what is the user’s knowledge about phishing (rate on a scale of 1to 7). The answers to these questions will be used to personalize the training when the usersstart using the PhishGuru.

• Interface module: The interface module will do the handshake functionality between thePhishGuru and the proxy. I will develop an API so that the PhishGuru system can interactwith other mail servers also.

• Web server : The web server tracks the users’ decisions about the training emails and testingemails (e.g. whether the users click on the link and go to the phishing website). The webserver will interact with the PhishGuru through the interface module.

4.5.2 Implementation

As described earlier, the system can be implemented in multiple ways. The PhishGuru may bepart of the large email server, or, the PhishGuru may exist as a separate system and the emailserver interacts with the system through the proxy. I plan to implement the PhishGuru with allits functionalities as an independent entity and test the system using a standard IMAP server (e.g.Cyrus or hMailSever). Microsoft Outlook and Thunderbird are different options for developing theplugin which can connect to the PhishGuru system (which users can install on their email client). Iwill consider developing the Thunderbird plugin. I will develop API for the PhishGuru so that anyother email server can also interact with the PhishGuru. I foresee using Java, PHP, MySQL andApache development environment. All the modules described in Section 4.5.1 will be implementedin this environment.

4.6 Evaluation

In this section, I describe the steps that I will take in evaluating the effectiveness of the methodologyand training materials that I develop. The evaluation of the PhishGuru will be focused on testingthe hypothesis:

Computer users trained using an embedded training system grounded inlearning science are able to make more accurate online trust decisions thanthose who read traditional security training materials distributed via emailor posted on web sites.

To find the effectiveness of the PhishGuru, I will measure user learning using the measurementsdiscussed in Section 2.3.2 and Section 4.4. To test my thesis statement, I plan on conducting a few

32

pilot and informal studies, two laboratory studies, and one field study. In this section, I describethe studies that I have planned in the context of this thesis.

4.6.1 Pilot studies

In this section, I describe the two main pilot studies that I will conduct to test the knowledgetracing and the user rating module. I also foresee a few other informal studies that I will conductto evaluate the different modules of the PhishGuru before I integrate all the modules of the system.As mentioned earlier, I will collect feedback from these pilot studies to improve the algorithms andthe design of the system. I will first test the knowledge tracing module and then the user ratingmodule.

• Pilot study I: The goal of this pilot study is to test whether the knowledge tracing algorithmis able to trace the users’ change in knowledge and whether it is able to select the righttraining intervention emails and training materials according to the user’s knowledge. Thiswill be a laboratory study where I will ask participants to access their hypothetical emailinbox. According to their knowledge and decisions that they make in the first few emails, theknowledge tracing algorithm will predict the training and testing emails. These interventionswill be sent to the users’ email inbox. I will ask the users to rate the relevancy of theseinterventions to the knowledge they have and the knowledge they gained from the PhishGuru.The more the correlation between the interventions sent by the knowledge tracing moduleand the users’ relevancy rating, the better is the module. I have provided the time-line totest this module in Section 5.

• Pilot study II: In this study, I plan to test whether showing users their ratings positivelyaffects user behavior compared to not showing the user rating. I will generate and show theuser rating according to the knowledge score and user’s response to the testing questions. Asin the previous module, I plan to conduct a laboratory study where users will be made tocheck the hypothetical email inbox. According to the decision that users make in the initialphase of the study and the response that they give in the testing questions, this module willgenerate the user rating and show the result to the user in a specific interface. I plan to testthe following hypothesis in this pilot study.

P1 - H1: Showing user rating to user positively affects user behavior compared tonot showing the user rating.

I will test this hypothesis by asking users what they feel when they see the user rating. I alsoplan on using simulation techniques to test the user rating module. The timeline for testingthis module is provided in Section 5.

4.6.2 Field-study

From the lessons learned through the first laboratory study, pilot studies and other informal studies,I plan on setting up a field study where I will: (1) measure the knowledge acquisition; (2) measurethe knowledge retention; (3) measure the knowledge transfer; and (4) test all algorithms andmeasurements described in Section 4.4 and Section 4.5. I will conduct a few pilot studies with

33

a few participants before making the study available to a large population. As in other studiesthat I have conducted, this study will be targeted at non-expert users. There will be two groupsin the study, the experimental group which will receive both training and testing emails, and thecontrol group which will only receive the standard security notice emails. I plan on recruiting usersthrough posting fliers and online postings. There will be an initial informed consent from the usersto be sent email interventions. I will recruit about 80 users for the study; participants will normallyaccess their emails in which I will embed the training interventions. The users will not find anydifference in using the PhishGuru than using their normal email inbox. Participants will be initiallycompensated with $20 for agreeing to use the PhishGuru. Participants will use the system for 8weeks and participants completing the study will enter raffle. Since this study will be done in thewild, there is a concern about how I will implement the phishing websites and send out phishingemails to participants, because this will be a trademark infringement of brands. There are multipleoptions that I am considering:

• To get permission from organizations to use their brands for the user study, I will discusswith a few organizations.

• The participants will asked to use a single machine (usually this is the case with non-experts)to access their emails; I can find the IP address of that particular machine and make onlyspecific IP addresses access the phishing websites that I have hosted. Another way is toembed a unique ID in the emails and participants can access the phishing websites that Ihave hosted only through clicking the link in the email. The embedded link will be availableonly for a few hours after it was sent out and will take the user to the website only once. Ifthe user tries to click on the link again he or she will be taken to the training intervention.The link will be available only if the users access it from their particular email account. Thiswill avoid users forwarding the email and others getting access to the website.

• Another approach is to embed the training link in the emails that PILFER [47] (an emailfilter that we have developed at CMU in the Supporting Trust Decision project) identifies asa phishing email. We can modify the link in the email to the link that we want the user toaccess, which will contain the training materials. We can also direct the user to the fakedphishing websites that we have created. This might avoid directly infringing trademarks ofthe brands.

To get users for the field study, many small organizations have volunteered to ask either theiremployees or customers to use the PhishGuru. I plan on revisiting and having discussions with theorganizations once I have the prototype of the system built. In any case, if I am not able to run thefield study because of the trademark issue, I will conduct a laboratory study with a similar setup.The timeline for testing this module is provided in Section 5.

4.7 Application to other semantic attacks

4.7.1 Other semantic attack

As described earlier, a semantic attack is defined as the attack in which users’ decisions are animportant aspect; till now, I discussed phishing, which is one type of semantic attack. I plan on

34

applying the lessons learned from the phishing case study to test whether the same methodologyand design principles can be applied in training for other semantic attacks like spyware. I also planto work on characterizing the semantic attacks for which the training methodology and principlescan be applied to get effective results. I will also work on finding the security attacks for whicheducation cannot be a solution, because it may be too difficult to train users on certain securityconcepts and attacks (e.g. PGP [147]). As for this proposal, I am considering testing one of thefollowing attacks (the choice of the other semantic attack is also dependent on the results fromLaboratory Study I and the field study):

1. Teaching users about spyware and how to avoid spyware related problems

2. Teaching users not to accept jobs through mule emails

3. Teaching users how opening an attachment in their emails can be dangerous

The application of learning science principles to another semantic attack will help me generalizethese principles, the designs, and ascertain whether the lessons learned from phishing can be appliedto other semantic attacks to train users.

4.7.2 Laboratory study II

One possibility of generalizing the results from phishing to other semantic attacks is by evaluatingthe training methodology to train users about opening an attachment from an email. I will developcurriculum for training users about attachments similar to the one discussed in Appendix A. Us-ing the curriculum, I will develop training interventions and instructions which will be evaluatedthrough a user study. As attachments are an essential way of transferring information, the instruc-tions cannot be like “Do not open attachments.” So I will work on designing instructions whichwill provide various strategies that users can follow in order to avoid falling for attacks throughattachments. I plan to conduct a laboratory study similar to the one discussed in Section 3.5 totest the effectiveness of the embedded training methodology in the context of training users aboutattachments. Through this laboratory study, I intend to show that the training methodology andprinciples that I used in the phishing scenario can be applied to other situations. The timelinefor testing this module is provided in Section 5. Also, Table 1 summarizes the goals for the userstudies planned towards completion of my thesis work.

5 Timeline

Figure 10 shows an approximate schedule of my research and writing towards completion of mydissertation. I have broken the schedule into four phases as shown in the Figure 10 and I planon completing my dissertation in Spring 2009. Below, I provide details about things I plan oncompleting during each phase.

• Phase I: Jan 07 – May 07

+ Planning and conducting the Laboratory study I (Section 3.5)

35

Table 1: Goals for the user studies that are planned towards completion of my thesis work

Study Goal(s)1. To test whether the knowledge tracing algorithm isable to trace the users’ change in knowledge.

Pilot study I2. To test whether the knowledge tracing algorithm isable to select the right training instructions.

Pilot study II 1. To test whether showing user rating to the userspositively affects user behavior compared to not show-ing the user rating.1. To measure knowledge retention and knowledgetransfer in the real-world situation.

Field-study2. To measure the overall effect of embedded trainingin the real-world situation.

Laboratory study II 1. To test the effectiveness of embedded training inthe context of another semantic attack.

+ Writing and doing the proposal

+ Deciding on the features for the PhishGuru

• Phase II: Aug 07 – July 08

+ Aug 07: Design of the PhishGuru

+ Sep 07 – Nov 07: Development of knowledge tracing module

+ Dec 07: Testing knowledge tracing algorithm

+ Jan 08 – Feb 08: Development, and testing of user rating, and testing module

+ March 08 – Integrating all modules

+ April 08: Pilot testing

+ May 08 – Aug 08

∗ Evaluation of the PhishGuru: field-study (Section 4.6.2)∗ Writing chapter 1 & 2

• Phase III: Sep 08 – Jan 09

+ Sep 08 – Nov 08

∗ Designing and developing materials for other semantic attack∗ Implement the embedded methodology for other semantic attack training∗ Writing chapter 3, 4 & 5

+ Dec 08 – Jan 09

36

Figure 10: Proposed schedule for research and writing my dissertation.

∗ Evaluation of the embedded methodology – laboratory study II (Section 4.7.2)∗ Writing chapters 6 & 7

• Phase IV: Feb 09 – May 09

+ Feb 09: Writing chapters 8 & 9

+ March 09: Drafts to committee

+ May 09: Defense

6 Outline of the thesis

• Chapter 1: Introduction

+ Phishing and its consequences

+ Empirical data on phishing

+ Motivation for developing an educational system

+ Summary of the thesis, including the thesis statement

• Chapter 2: Phishing

+ Introduction to phishing

+ Examples of different types of phishing attacks

+ Life cycle of phishing attacks

+ Countermeasures for phishing

37

• Chapter 3: Trust & Trust models

+ Trust in different literature

+ Online trust and credibility

+ A Model of Trust in Phishing Scenario (MoTPS)

+ MoTPS to inform the design of the PhishGuru

• Chapter 4: Learning science

+ Phishing and education

+ Instructional design principles

+ Measurements of learning

• Chapter 5: Preliminary studies

+ Protecting People from Phishing: The Design and Evaluation of an Embedded PrototypeTraining Email System

∗ Evaluation of the embedded training methodology∗ Evaluation of the training materials: Graphics / text & Comic strip

+ Teaching Johnny Not to Fall for Phish

∗ Evaluation of online training materials using learning science principles∗ Evaluation of online training materials among users∗ Evaluation of Anti-Phish Phil∗ Evaluation of knowledge retention and transfer for embedded training methodology

• Chapter 6: PhishGuru design and implementation

+ Design rationale for PhishGuru

+ PhishGuru architecture

+ Learning science design principles

+ Learning algorithms

+ PhishGuru implementation

• Chapter 7: PhishGuru evaluation

+ Study design

+ Study participants and protocol

+ Results and discussion

• Chapter 8: Other semantic attacks

+ Design rationale

+ System implementation

+ Evaluation

38

∗ Study design∗ Study participants and protocol∗ Results and discussion

• Chapter 9: Conclusion and future work

+ Discussion of all the results related to user education from this research

+ Generalized design principles that can be applied in other semantic attack situations

+ Future work on user education in the context of semantic attack

+ Limitations of this thesis

7 Conclusion

This proposal aims at addressing the conventional wisdom “training users [on semantic attacks]

does not work.” In addressing this, I look at ‘how’ and ‘what’ to teach users, which can prevent

them from falling for semantic attacks and to make training part of the users’ primary task. My goal

is not to make non-experts expert in security, but to train them so that they can make better online

trust decisions. I accept and argue that training should be thought of as a complementary solution

to the existing technological solutions for combating semantic attacks. This proposal formalizes the

integration between security user education and learning science methodologies. In this document,

first, I briefly reviewed the literature on phishing, trust models and learning science. In particular,

I reviewed a few of the learning science principles which will be applied as a starting point to

address the problem of user education on semantic attacks. Next, I discussed my past and present

research which lead to this thesis work. Later, as a case study, I proposed an embedded training

system (The PhishGuru) which will train users on phishing, a type of semantic attack. Then, I

provided a detailed description of the user studies and the field study that I plan to conduct to

evaluate this system. Later, I proposed to evaluate and extend the methodology developed for

phishing to another semantic attack. The design principles established in this thesis will be helpful

for researchers to develop systems that can train users in other online risky situations. Finally, I

provided an estimated schedule to complete my dissertation along with the tentative chapters.

39

A Appendix - Phishing training curriculum

• Training regarding the email

+ General / Thumb rules

∗ Never click on links with in emails∗ Type in the real website address into a web browser (Use the keypad not the mouse)∗ Find and call a real customer service center (Pick the phone and call)∗ Never give out personal information upon an email request∗ Always be wary of suspicious websites∗ Never trust strangers Be suspicious of emails from people you don’t know

+ Content of the email

∗ Fear not – Phishing emails come with threats or warnings (account verification,account suspension)

∗ Greed doesnt pay – Do not get attracted by emails stating get paid $10 for fillingout survey or you are the winner of 2 million dollars

∗ URL in the emails – Type in the real website address into a web browser (Use thekeypad not the mouse)

∗ Be suspicious of emails with grammatical errors or weird formatting∗ Action in the email – Usually phishing emails ask you to click the link in the email

and give out money, SSN, passwords, name, address, computer information andother sensitive information

∗ Logos of the brand or organization in the emails is of no importance. Anybody cancreate an email with the logo.

+ Trust

∗ Do I know the sender? – If suspicious use a different channel of communication(send a separate email or call) to verify

∗ Do I have an account with? – If you do not have an account with the organizationfrom where the email has come DELETE the email, DO NOT open it

∗ Am I expecting the email? – Notice carefully with the emails received unexpectedlyfrom unknown people

∗ Is the email sent to me or to many others (Mass emailing) – Usually the emailssent out to large number of people is some kind of spam or scam (look for the TOinformation usually blank or undisclosed)

∗ Is the email a forwarded email? Usually forwarded emails from unknown sourceasking for personal information is a scam

+ Others

∗ Log into your financial account regularly to check for various transactions∗ Be suspicious even if the emails give some personal information (eg. Name or first

part of credit card numbers); they are spear phishing emails∗ Sender information along with the subject line could help in making a decision

whether to open the email

40

+ Expert level

∗ Second time right – If suspicious that you are on phishing website, try giving wrongpassword, the website will let you in while the actual website will not

∗ Use search engine – Search for the brand name from the email to see whether it isa legitimate brand (as in the Anti-phishing Phil game)

∗ Last email server – Checking for the originating and the last email server can helpmake a better decision

• Training regarding the website

+ URL’s

∗ Look for https in the URL while giving personal information∗ IP based – As in game∗ Sub-domain – As in game∗ Similar domain or deceptive – As in game

+ Content of website

∗ Professionalism – Easy to copy the website pages to create a legitimate lookingwebsite

∗ Broken images – Usually broken images are good sign for spoofed website∗ Lock icon – This icon can be easily spoofed∗ Information requested – Unless and until it is a government website rarely organi-

zations ask for social security number on a website

+ Trust

∗ Have you interacted with the brand or organization in the URL before?∗ What do others (friends or user groups) say about the website in question?

References

[1] Account Guard. Retrieved Nov 3, 2006, http://pages.ebay.com/ebay toolbar/.

[2] Adams, A., and Sasse, M. A. Users are not the enemy. Communication ACM 42, 12(1999), 40–46. DOI=http://doi.acm.org/10.1145/322796.322806.

[3] Aldrich, C. Learning by Doing: A Comprehensive Guide to Simulations, Computer Games,and Pedagogy in e-Learning and Other Educational Experiences. Jossey-Bass, 2005.

[4] Aleven, V. An intelligent learning environment for case-based argumentation. Technology,Instruction, Cognition, and Learning (In press).

[5] Aleven, V., and Koedinger, K. R. An effective metacognitive strategy: learning bydoing and explaining with a computer-based cognitive tutor. Cognitive Science 26, 2 (2002),147 – 179.

41

[6] Anandpara, V., Dingman, A., Jakobsson, M., Liu, D., and Roinestad, H.Phishing IQ tests measure fear, not ability. Usable Security (USEC’07) (2007).http://usablesecurity.org/papers/anandpara.pdf.

[7] Anderson, J. R. Rules of the Mind. Lawrence Erlbaum Associates, Inc., 1993.

[8] Anderson, J. R., Boyle, C. F., Corbett, A. T., and Lewis, M. W. Cognitivemodelling and intelligent tutoring. Artificial Intelligence 42 (1990), 7–49.

[9] Anderson, J. R., Corbett, A. T., Koedinger, K. R., and Pelletier, R. Cognitivetutors: Lessons learned. The Journal of the Learning Sciences 4, 2 (1995), 167–207.

[10] Anderson, J. R., and Simon, H. A. Situated learning and education. Educational Re-searcher 25 (1996), 5–11.

[11] Anti-Phishing Working Group. Retrieved Jan 9, 2007, http://www.antiphishing.org/.

[12] Araujo, I., and Araujo, I. Developing trust in internet commerce. In Proceedings of the2003 conference of the Centre for Advanced Studies on Collaborative research (2003), IBMPress, pp. 1–15.

[13] Bahrick, H. P. Maintenance of knowledge: Questions about memory we forgot to ask.Journal of Experimental Psychology 108, 3 (September 1979), 296–308.

[14] Bank, D. ‘Spear phishing’ tests educate people about online scams. Newsarticle, Augurst 2005. http://online.wsj.com/public/article/SB112424042313615131-z 8jLB2WkfcVtgdAWf6LRh733sg 20060817.html?mod=blogs.

[15] Barnett, S. M., and Ceci, S. J. When and where do we apply what we learn? a taxonomyfor far transfer. In Psychological Bulletin (2002), vol. 128, pp. 612–637.

[16] Bhattacherjee, A. Individual Trust in Online Firms: Scale Development and Initial Test.Journal of Management Information Systems 19, 1 (Summer 2002), 211–242.

[17] Bishop, M. Education in information security. IEEE Concurrency 8, 4 (2000), 4–8.

[18] Bligh, D. A. What’s The Use of Lectures? Jossey-Bass, 2000.

[19] Block, J. H., and Burns, R. B. Mastery learning. Review of Research in Education 4(1976), 3–49.

[20] Bransford, J. D., and Schwartz, D. L. Rethinking transfer: A simple proposal withmultiple implications. In Review of Research in Education, A. Iran-Nejad and P. D. Pearson.,Eds., vol. 24. American Educational Research Association (AERA) Washington, DC, 2001,pp. 61 – 100.

[21] Burmester, G. M., Stottler, D., and Hart, J. L. Embedded trainingintelligent tutoring systems (ITS) for the future combat systems (FCS) commandand control (C2) vehicle. Tech. rep., Defense Technical Information Center, 2005.http://www.stottlerhenke.com/papers/IITSEC-02-ITSFCS.pdf.

42

[22] Butcher, K. R., and Aleven, V. Integrating visual and verbal knoweldge during class-room learning with computer tutors. To appear in Cognitive Science.

[23] Cabral, L. M. B. The Economics of Trust and Reputation: APrimer. Tech. rep., New York University and CEPR, May 2002.http://pages.stern.nyu.edu/ lcabral/reputation/Reputation June05.pdf.

[24] Carroll, J. M., and Carrithers, C. Training wheels in a user interface. CommunicationACM 27, 8 (1984), 800–806. DOI=http://doi.acm.org/10.1145/358198.358218.

[25] Cave, J. The economics of cyber trust between cyber partners. Retrieved Feb 20, 2005,http://www.foresight.gov.uk/Previous Projects/Cyber Trust and CrimePrevention/Reports and Publications/Economics of Trust Between CyberPartners/Economics%20of%20trust%20between%20cyber%20partners.pdf.

[26] Cavoukian, A., and Hamilton, T. The Privacy Payoff, How Successful Business BuildConsumer Trust. McGraw Hill Tyerson Limited, 2002.

[27] Cheikes, B. A., Geier, M., Hyland, R., Linton, F., Rodi, L., and Schaefer, H.-P.Embedded training for complex information systems. In Intelligent Tutoring Systems: 4thInternational Conference (1998), vol. 1452, Springer Berlin / Heidelberg, pp. 36–45.

[28] Chellappa, R. K., and Sin, R. Personalization versus Privacy: An Empirical Examinationof the Online Consumer’s Dilemma. Information Technology and Management. Vol. 6, No.2-3 (2005.). Retrieved Sept 13, 2005, http://asura.usc.edu/ ram/rcf-papers/per-priv-itm.pdf.

[29] Clark, R. C. Developing Technical Training: A Structured Approach for the Developmentof Classroom and Computer-Based Instructional Materials. Addison Wesley Publishing Com-pany, Beverly, MA, USA, June 1989.

[30] Clark, R. C., and Mayer, R. E. E-Learning and the science of instruction: provenguidelines for consumers and designers of multimedia learning. John Wiley & Sons, Inc.,USA, 2002.

[31] coates, R. Dumb users spread viruses. Retrieved Jan 24, 2007,http://www.silicon.com/software/security/0,39024655,39118228,00.htm.

[32] Committee on Developments in the Science of Learning and National ResearchCouncil. How People Learn: Bridging Research and Practice. National Academies Press,2000.

[33] Corbett, A. T., and Anderson, J. R. Knowledge tracing: Modeling the acquisition ofprocedural knowledge. User Modeling and User-Adapted Interaction 4, 4 (December 1994),253–278. DOI= 10.1007/BF01099821.

[34] Corbett, A. T., and Anderson, J. R. Locus of feedback control in computer-basedtutoring: impact on learning rate, achievement and attitudes. In CHI ’01: Proceedings of theSIGCHI conference on Human factors in computing systems (New York, NY, USA, 2001),ACM Press, pp. 245–252.

43

[35] Cordova, D. I., and Lepper, M. R. Intrinsic motivation and the process of learning:Beneficial effects of contextualization, personalization, and choice. Journal of EducationalPsychology 88, 4 (December 1996), 715–730.

[36] Corritore, C. L., Kracher, B., and Wiedenbeck, S. On-line trust: concepts, evolvingthemes, a model. Academic Press, Inc. 58, 6 (2003), 737–758.

[37] Cranor, L. F., and Garfinkel, S. Security and Usability: Designing Secure Systems thatPeople Can Use. O’Reilly, Sebastopol, CA, USA, Aug, 2005.

[38] Dhamija, R., and Tygar, J. The Battle Against Phishing: Dynamic SecuritySkins. In SOUPS ’05: Proceedings of the 2005 symposium on Usable privacy and se-curity (New York, NY, USA, 2005), ACM Press, New York, NY, pp. 77–88. DOI=http://doi.acm.org/10.1145/1073001.1073009.

[39] eBay. Spoof email tutorial. Retrieved December 30, 2006.http://pages.ebay.com/education/spooftutorial.

[40] Eberts, R. E. Handbook of Human-computer Interaction. Elsevier Science, 1997,ch. Computer-based Instruction, pp. 825–847.

[41] Egger, F. From Interactions to Transactions: Designing the Trust Experience for Business-to-Consumer Electronic Commerce. PhD thesis, Eindhoven University of Technology (TheNetherlands), http://www.ecommuse.com/research/publications/thesis.htm, 2003.

[42] Emigh, A. Online identity theft: Phishing technology, chokepoints and countermeasures.Tech. rep., Radix Labs, October 2005. http://www.antiphishing.org/Phishing-dhs-report.pdf.

[43] Evers, J. User education is pointless, October 2006. http://news.com.com/2100-7350 3-6125213.html.

[44] Federal Trade Commission. An e-card for you game. Retrieved December 30, 2006.http://www.ftc.gov/bcp/conline/ecards/phishing/index.html.

[45] Federal Trade Commission. How not to get hooked by aphishing scam. Consumer alert news. Retrieved Nov 7, 2006,http://www.ftc.gov/bcp/edu/pubs/consumer/alerts/alt127.htm.

[46] Ferguson, A. J. Fostering E-Mail Security Awareness: The West Point Carronade. EDU-CASE Quarterly, 1 (2005). http://www.educause.edu/ir/library/pdf/eqm0517.pdf.

[47] Fette, I., Sadeh, N., and Tomasic, A. Learning to detect phishing emails. Tech.rep., ISRI Technical report, CMU-ISRI-06-112. Carnegie Mellon University, June 2006.http://reports-archive.adm.cs.cmu.edu/anon/isri2006/CMU-ISRI-06-112.pdf.

[48] Financial Services Technology Consortium. Understanding and countering the phish-ing threat: A financial services industry perspective. Tech. rep., Financial Services TechnologyConsortium, 2005.

44

[49] Fogg, B. Persuasive Technology: Using Computers to Change What We Think and Do.Morgan Kaufmann, December 2002.

[50] Fong, G. T., and Nisbett, R. E. Immediate and delayed transfer of training effectsin statistical reasoning. In American Psychological Association Inc., vol. 120. Journal ofExperimental Psychology, 1991, pp. 34–45.

[51] frontier, M. Mailfrontier phishing IQ test. Retrieved Sept 2, 2006,http://survey.mailfrontier.com/survey/quiztest.html.

[52] Gagne, R. M., Foster, H., and Crowley, M. E. The measurement of transfer oftraining. Psychological Bulletin 45, 2 (1948), 97–130.

[53] Gartner. Gartner Survey Shows Frequent Data Security Lapses and Increased Cy-ber Attacks Damage Consumer Trust in Online Commerce. Retrieved Jan 9, 2007,http://www.gartner.com/press releases/asset 129754 11.html.

[54] Gefen, D. Reflections on the dimensions of trust and trustworthiness among online con-sumers. ACM Press 33, 3 (2002), 38–53. DOI=http://doi.acm.org/10.1145/569905.569910.

[55] Gilmore, D. J. The relevance of HCI guidelines for educational interfaces. Machine-Mediated Learning 5, 2 (1996), 119–133.

[56] Glaeser, E. L., Laibson, D. I., Scheinkman, J. A., and Soutter, C. L. Measuringtrust. The Quarterly Journal of Economics 115, 3 (2000), 811–846.

[57] Gomoll, K. The Art of Human-Computer Interface. Addison-Wesley Publishing Company,Inc. New York, 1992, ch. Some Techniques for Observing Users, pp. 85–90.

[58] Gordon, L. A., Loeb, M. P., Lucyshyn, W., and Richardson, R. CSI/FBI ComputerCrime and Security Survey. Report, Computer Security Institute, 2006.

[59] Gorling, S. The myth of user education. In Proceedings of the 16th Virus Bulletin Inter-national Conference (2006).

[60] Gruenberg, S. M. The comics as a social force. Journal of Educational Sociology 18, 4(December 1944), 204–213.

[61] Guerra, G. A., Zizzo, D. J., Dutton, W. H., and Peltu, M. Economics of Trustin the Information Economy: Issues of Identity, Privacy and Security. Tech. rep., OxfordInternet Institute, April 2003. http://www.oii.ox.ac.uk/resources/publications/RR1.pdf.

[62] Guzdial, M. Software-realized scaffolding to facilitate programming for science learning.Interactive Learning Environment 4, 1 (1995), 1–44.

[63] Harford County Public Schools. Comic books can inspire reluctant & advanced read-ers. http://www.hcps.org/academics/highlights/content/graphicnovels.asp.

[64] Hight, S. D. The importance of a security, education, training and awareness program,November 2005. http://www.infosecwriters.com/text resources/pdf/SETA SHight.pdf.

45

[65] Hofmann, T. Learning what people (don’t) want. In EMCL ’01: Proceedings of the 12thEuropean Conference on Machine Learning (London, UK, 2001), Springer-Verlag, pp. 214–225.

[66] Hutchinson, K. H. An experiment in the use of comics as instructional materials. Journalof Educational Sociology 23, 4 (December 1949), 236–245.

[67] Jackson, C., Simon, D., Tan, D., and Barth, A. An evaluation of extended val-idation and picture-in-picture phishing attacks. In Usable Security (USEC’07) (2007).http://usablesecurity.org/papers/jackson.pdf.

[68] Jagatic, T., Johnson, N., Jakobsson, M., and Menczer, F. Social Phish-ing. To appear in the Communications of the ACM . Retrieved March 7, 2006,http://www.indiana.edu/ phishing/social-network-experiment/phishing-preprint.pdf.

[69] Jakobsson, M. The human factor in phishing. In Privacy & Security of Consumer Infor-mation (2007). http://www.informatics.indiana.edu/markus/papers/aci.pdf.

[70] Jakobsson, M., Juels, A., and Ratkiewicz, J. Remote harm-diagnostics. Retrieved,Jan 14, 2007, http://www.ravenwhite.com/files/rhd.pdf.

[71] Jakobsson, M., and Myers, S., Eds. Phishing and Countermeasures: Understanding theIncreasing Problem of Electronic Identity Theft. Wiley-Interscience, 2006.

[72] James, L. Phishing Exposed. Syngress Publishing, Canada, November 10, 2005.

[73] Johnson, B. R., and Koedinger, K. R. Comparing instructional strategies for integratingconceptual and procedural knowledge. In Proceedings of the Annual Meeting [of the] NorthAmerican Chapter of the International Group for the Psychology of Mathematics Education(October 2002), vol. 1–4, pp. 969–978.

[74] Keinan, G. Decision making under stress: scanning of alternatives under controllable anduncontrollable threats. Journal of personality and social psychology 52, 3 (1987), 639–644.

[75] Kirda, E., and Kruegel, C. Protecting users against phishing attacks. The ComputerJournal 49, 5 (January 2006), 554–561.

[76] Kirkley, J. R., and et al. Problem-based embedded training: An instructionalmethodology for embedded training using mixed and virtual reality technologies. In In-terservice/Industry Training, Simulation, and Education Conference (I/ITSEC) (2003).http://www.iforces.org/downloads/problem-based.pdf.

[77] Klein, G. Sources of power : How people make decisions? The MIT Press Cambridge,Massachusetts The MIT Press, Cambridge, Massachusetts, London, England, February 1999.

[78] Klein Gary, Wolf Steve, M. L., and Caroline, Z. Characteristics of skilled optiongeneration in chess. Organizational Behavior and Human Decision Processes 62, 1 (April1995), 63–69.

46

[79] Koedinger, K. R. Toward evidence for instruction design principles: Examples from cog-nitive tutor math 6. Proocedings of the Annual Meeting, Norh American Chapter of theInternational Group for the Psychology of Mathematics Education 1 – 4 (2002).

[80] Koedinger, K. R., and Aleven, V. Exploring the assistance dilemma in eperiments withcognitive tutors. Manuscript in preparation.

[81] Kulik, C. L. C., Kulik, J. A., and Bangert-Drowns, R. L. Effectiveness of masterylearning programs: A meta-analysis. Review of Educational Research 60 (1990), 265–299.

[82] Kulik, J. A., Kulik, C. C., and Cohen, P. A. A meta-analysis of outcome studies ofkeller’s personalized system of instruction. American Psychologist 34 (1979), 307 – 318.

[83] Kumaraguru, P., Acquisti, A., and Cranor, L. Trust modeling foronline transactions: A phishing scenario. In Privacy Security Trust (2006).http://www.cs.cmu.edu/p̃onguru/pk aa lc pst 2006.pdf.

[84] Kumaraguru, P., Rhee, Y. W., Acquisti, A., Cranor, L., Hong, J., and Nunge, E.Protecting people from phishing: The design and evaluation of an embedded training emailsystem. To appear in the Conference on Computer Human Interaction (CHI) (November2006). http://www.cylab.cmu.edu/files/cmucylab06017.pdf.

[85] Kumaraguru, P., Sheng, S., Acquisti, A., Cranor, L. F., and Hong, J.Teaching johnny not to fall for phish. Tech. rep., Cranegie Mellon University, 2007.http://www.cylab.cmu.edu/files/cmucylab07003.pdf.

[86] Lee, J., Kim, J., and Moon, J. Y. What makes internet users visit cyber stores again?key design factors for customer loyalty. In CHI ’00: Proceedings of the SIGCHI conferenceon Human factors in computing systems (New York, NY, USA, 2000), pp. 305–312.

[87] Lininger, R., and Vines, R. D. Phishing: Cutting the Identity Theft Line. Indianapolis,Indiana, USA, 2005.

[88] Maldonado, H., Lee, J.-E. R., Brave, S., Nass, C., Nakajima, H., Yamada, R.,Iwamura, K., and Morishima, Y. We learn better together: enhancing elearning withemotional characters. In CSCL ’05: Proceedings of th 2005 conference on Computer supportfor collaborative learning (2005), International Society of the Learning Sciences, pp. 408–417.

[89] Mandl, H., and Levin, J. R. Knowledge Acquisition from Text and Pictures. North -Holland, 1989.

[90] Marsh, S. P. Formalising Trust as a Computational Concept. PhD thesis, 1994. cite-seer.ist.psu.edu/marsh94formalising.html.

[91] Mathan, S. A., and Koedinger, K. R. Artificial Intelligence in Education: Shapingthe Future of Learning Through Intelligent Technolgis. IOS Press, 2003, ch. Recasting theFeedback Debate: Benefits of Tutoring Error Detection and Correction Skills, pp. 13–20.

[92] Mathan, S. A., and Koedinger, K. R. Fostering the intelligent novice: Learning fromerrors with metacognitive tutoring. Educational Psychologist 40, 4 (2005), 257–265.

47

[93] Mayer, R. C., Davis, J. H., and Schoorman, D. F. An integrative model of organiza-tional trust. The Academy of Management Review. 1995, 3 (July, 1995), 709–734.

[94] Mayer, R. E. Systematic thinking fostered by illustrations in scientific text. Journal ofEducational Psychology 81, 2 (June 1989), 240–246.

[95] Mayer, R. E. Multimedia Learning. New York Cambridge University Press, 2001.

[96] Mayer, R. E., and Anderson, R. B. The instructive animation: Helping students buildconnections between words and pictures in multimedia learning. Journal of EducationalPsychology 84, 4 (December 1992), 444–452.

[97] McMillan, R. Consumers to lose $2.8b to phishers in 2006: Gartnersays fewer, but bigger, attacks will gain more for criminals, November 2006.http://www.infoworld.com/archives/emailPrint.jsp?R=printThis&A=/article/06/11/09/HNgartnerphishing 1.html.

[98] Merrienboer, J. V., de croock, M., and Jelsma, O. The transfer paradox : Effectsof contextual interference on retention andtransfer performance of a complex cognitive skill.Perceptual and motor skills 84 (1997), 784–786.

[99] Microsoft. Spear phishing: Highly targeted scams. Website, September 2006.http://www.microsoft.com/athome/security/email/spear phishing.mspx.

[100] Microsoft Corporation. Consumer awareness page on phishing. Retrieved Sep 10, 2006.http://www.microsoft.com/athome/security/email/phishing.mspx.

[101] Miller, R. C., and Wu, M. Fighting Phishing at the User Interface. O’Reilly (Aug,2005). In Lorrie Cranor and Simson Garfinkel (Eds.) Security and Usability: DesigningSecure Systems that People Can Use.

[102] Moreno, R., and Mayer, R. E. Cognitive principles of multimedia learning: The role ofmodality and contiguity. Journal of Educational Psychology 91 (1999), 358–368.

[103] Moreno, R., and Mayer, R. E. Engaging students in active learning: The case forpersonalized multimedia messages. Journal of Educational Psychology 92, 4 (December 2000),724–733.

[104] Moreno, R., Mayer, R. E., Spires, H. A., and Lester, J. C. The case for socialagency in computer-based teaching: Do students learn more deeply when they interact withanimated pedagogical agents? Cognition and Instruction 19, 2 (2001), 177–213.

[105] Mui, L., Mohtashemi, M., and Halberstadt, A. A computational model of trust andreputation. In Proceedings of the 35th Hawaii International Conference on System Sciences(2002).

[106] Mutz, D. C. Social trust and e-commerce, experimental evidence for the effects of socialtrust on individuals’ economic behavior. Public Opinion Quarterly 69, 3 (2005), 393–416.

[107] Myers, B. A. Overview of HCI design and implementation, October 1999.http://www.cs.cmu.edu/b̃am/uicourse/special/.

48

[108] MySecureCyberspace. Uniform resource locator (URL), 2007. RetrievedFeb 4, 2007, http://www.mysecurecyberspace.com/encyclopedia/index/uniform-resource-locator-url-.html.

[109] Neilsen, J. User education is not the answer to security problems, October 2004.http://www.useit.com/alertbox/20041025.html.

[110] Netcraft. Retrieved Nov 3, 2006, http://toolbar.netcraft.com/.

[111] New York State Office of Cyber Security & Critical Infrastructure Coor-dination. Gone phishing. . . a briefing on the anti-phishing exercise initiative for new yorkstate government. Aggregate Exercise Results for public release., 2005.

[112] Nichols, D. M. Implicit rating and filtering. In Proceedings of the 5thDELOS Workshop on Filtering and Collaborative Filtering, (1997), no. 98.http://www.comp.lancs.ac.uk/computing/research/cseg/projects/ariadne/docs/delos5.html.

[113] Patrick, A. S. Building trustworthy software agents. IEEE Internet Computing 6(6)(October 2002), 46–53. http://www.andrewpatrick.ca/cv/building trustworthy agents.pdf.

[114] Patrick, A. S., Briggs, P., and Marsh, S. Designing Systems That People Will Trust.O’Reilly (Aug, 2005), 75–100. In Lorrie Cranor and Simson Garfinkel (Eds.) Security andUsability: Designing Secure Systems that People Can Use.

[115] Patrizio, A. Vishing Joins Phishing as Security Threat, July 2006.http://www.internetnews.com/security/article.php/3619086.

[116] Pollitt, M. G. The economics of trust, norms and networks. Business Ethics - A EuropeanReview 11, 2 (2002), 119–128.

[117] Rahman, A. A., and Hailes, S. A distributed trust model. In NSPW ’97: Proceedings ofthe 1997 workshop on New security paradigms (New York, NY, USA, 1997), pp. 48–60.

[118] Ramzan, Z. Phishing attacks in and around april through september 2006. Tech.rep., Symantec, November 2006. http://www.symantec.com/avcenter/reference/phishing-stats.pdf.

[119] Reason, J. Human Error. Cambridge University Press, USA, October 1990.

[120] Resnick, P., and Zeckhauser, R. Trust among strangers in internet transactions: Em-pirical analysis of eBay’s reputation system. Draft Version for review by NBER workshop(2001). http://www.si.umich.edu/ presnick/papers/ebayNBER/RZNBERBodegaBay.pdf.

[121] Riegelsberger, J., and Sasse, M. A. Trustbuilders and Trustbusters: The Role of TrustCues in Interfaces to e-Commerce Applications. In 1st IFIP Conference on e-commerce,e-business, e-government i3e (October 2001).

[122] Riegelsberger, J., Sasse, M. A., and mccarthy, J. D. Shiny happy people buildingtrust?: photos on e-commerce websites and consumer trust. In CHI ’03: Proceedings of theSIGCHI conference on Human factors in computing systems (2003).

49

[123] Riegelsberger, J., Sasse, M. A., and McCarthy., J. D. The Mechanics of Trust: AFramework for Research and Design. International Journal of Human-Computer Studies 62,3 (2005), 381–422.

[124] Rifas, L. Educating with comics, June 2005. http://voice.aiga.org/content.cfm?ContentAlias= getfullarticle&aid=1142039.

[125] Ritter, S., and Koedinger, K. R. An architecture for plug-in tutor agents. Journal ofArtificial Intelligence in Education 7, 3-4 (1996), 315–347.

[126] Robila, S. A., and Ragucci, J. W. Don’t be a phish: steps in user education. InITICSE ’06: Proceedings of the 11th annual SIGCSE conference on Innovation and technol-ogy in computer science education (New York, NY, USA, 2006), ACM Press, pp. 237–241.DOI=http://doi.acm.org/10.1145/1140124.1140187.

[127] Rota, G., and Izquirdo, J. “comics” as a tool for teaching biotechnology in primaryschools. Electronic Journal of Biotechnology 6, 2 (August 2003), 85–89.

[128] Rubin, D. C., and Wenzel, A. E. One hundred years of forgetting : A quantitativedescription of retention. Psychological Review 103, 4 (1996), 734–760.

[129] Salovey, P., and Rothman, A. Social Psychology of Health. Psychology Press, 2003.

[130] Schank, R. C. Every curriculum tells a story. Tech. rep., Socraticarts.

[131] Schmidt, R. A., and Bjork, R. A. New conceptualizations of practice: Common principlesin three paradigms suggest new concepts for training. Psychological Science 3, 4 (July 1992),207–217.

[132] Schneier, B. Semantic attacks: The third wave of network attacks. Crypto-Gram Newslet-ter, October 2000. http://www.schneier.com/crypto-gram-0010.html#1.

[133] Schwartz, D. L., and Bransford, J. D. A time for telling. In Cognition & Instruction(1998), vol. 16, pp. 475–522.

[134] Scott, C. Interpersonal trust: A comparison of attitudinal and situational factors. HumanRelations 33, 11 (1980), 805–812.

[135] Sender Policy Framework. Sender Policy Framework. Retrieved Jan 21, 2007,http://www.openspf.org/.

[136] Sheng, S., Magnien, B., Kumaraguru, P., Acquisti, A., Cranor, L. F., Hong, J.,and Nunge, E. Anti-phishing phil: The design and evaluation of a game that teaches peoplenot to fall for phish. Under review.

[137] Singley, M., and Anderson, J. R. The Transfer of Cognitive Skill. Harvard UniversityPress, USA, May 1989.

[138] Sones, W. W. D. The comics and instructional method. Journal of Educational Sociology18, 4 (Dec 1944), 232–240.

50

[139] SpamAssasin. Spamassasin. Retrieved Sept 2, 2006, http://spamassassin.apache.org/.

[140] SpoofGuard. Spoofguard. Retrieved Sept 2, 2006,http://crypto.stanford.edu/SpoofGuard/.

[141] SpoofStick. Spoofstick. Retrieved Sept 2, 2006, http://www.spoofstick.com/.

[142] Stanford, J., Tauber, E. R., Fogg, B., and Marable, L. Experts vs. OnlineConsumers: A Comparative Credibility Study of Health and Finance Web Sites., 2002.http://www.consumerwebwatch.org/dynamic/web-credibility-reports-experts-vs-online.cfm.

[143] Tan, Y. H., and Thoen, W. An Outline of a Trust Model for Electronic Commerce.Applied Artificial Intelligence 14, 8 (2000).

[144] Tversky, A., and Kahneman, D. Judgment under Uncertainty: Heuristics and Biases.Science 185, 4157 (1974), 1124–1131.

[145] Tversky, A., and Shafir, E. The disjunction effect in choice under uncertainty. AmericanPsychological Society 3, 5 (September 1992), 305 – 309.

[146] Whitten, A. Making Security Usable. PhD thesis, Carnegie Mellon University, 2004.

[147] Whitten, A., and Tygar, J. D. Why Johnny Can’t Encrypt: A Usability Evaluationof PGP 5.0. O’Reilly (Aug, 2005), 669–692. In Lorrie Cranor and Simson Garfinkel (Eds.)Security and Usability: Designing Secure Systems that People Can Use.

[148] Woods, S., Hall, L., Wolke, D., Dautenhahn, K., and Sobral, D. An-imated characters in bullying intervention. eCircus. Retrieved Nov 4, 2006,http://homepages.feis.herts.ac.uk/ comqkd/Woods-iva03.pdf.

[149] Wu, M. Fighting Phishing at the User Interface. PhD thesis, MIT, 2006. Retrieved Nov 5,2006, http://groups.csail.mit.edu/uid/projects/phishing/minwu-thesis.pdf.

[150] Wu, M., Miller, R. C., and Garfinkel, S. L. Do Security Toolbars Actually PreventPhishing Attacks? Conference on Human Factors in Computing Systems (CHI) (2006).Retrieved Feb 10, 2006, http://www.simson.net/ref/2006/CHI-security-toolbar-final.pdf.

[151] Xiao, J., Stasko, J., and Catrambone, R. Embodied conversational agents as a uiparadigm: A framework for evaluation.

[152] Yahoo! DomainKeys: Proving and Protecting Email Sender Identity. Retrieved Jan 21,2007, http://antispam.yahoo.com/domainkeys.

[153] Yang, G. Comics in education. http://www.geneyang.com/comicsedu/index.html.

[154] Ye, Z. E., and Smith, S. Trusted paths for browsers. In Proceedings of the 11th USENIXSecurity Symposium (Berkeley, CA, USA, 2002), USENIX Association, pp. 263–279.

[155] Zhang, Y., Egelman, S., Cranor, L., and Hong, J. Phinding phish: Evaluating anti-phishing tools. In 14th Annual Network and Distributed System Security Symposium (2007).http://lorrie.cranor.org/pubs/ndss-phish-tools-final.pdf.

51