· Web viewSocial Engineering Toolkit Tests Had to download python ‘pefile’, and enter...

Social Engineering Toolkit Tests

Had to download python ‘pefile’, and enter ‘python setup.py install’ command in terminal, from with extracted folder from download.

Had to install ‘rar’ via apt-get, to use .rar file extension in SET.

Social-Engineering Attacks

Spear-Phishing (involves deception, masquerading as a trusted entity):

You can create a template (option 3), use your own payload (option 2), or use a wizard-style interface (option 1), which takes you through, step by step.

Option 1 - wizard-style interface (Testing with DLL hijacking (no. 1)):o A list of exploits to be chosen from, is shown:

o Testing with Wireshark, within the Kali Linux virtual machine, I was able to ping from the Windows host to it, without issue:

o Windows Host –

o Kali Linux VM –

o An IP address to send the payload to, is entered.

o Then an encoding for the payload is selected (I selected no. 16).

o You can also select a port to send the data to, on the host with the previously selected network interface/IP address. I left the default option set (HTTPS - port 443).

o Finally, you are able to use the vulnerability/exploit selected from the very beginning:

o I chose Microsoft Powerpoint 2010 (Option 8). You can then name the file (default: ‘openthis’), and choose its extension.

o ‘Rar’ was unavailable, so the application defaulted to a ‘.zip’ file.

o Then you are able to send the file out to a single email or multiple emails – the selected option was phishing, after all. I selected a single email.

o I then selected option 1, to use a pre-defined template (no. 6), and tested it using a fake gmail account: [email protected] Password setest019

o I chose option 1.

mailto:[email protected]

o My avast antivirus installation, caught the payload… :( At least it’s working properly.o After a retry, a gMail exception was caught, stating that the email

attachment/payload was caught, since its content presented “a potential security issue”, which was correct.

To get around this, I had to use payload option 2 (Custom Written Document)

I renamed the file to ‘test.xc’, and it was delivered via my normal Gmail account ([email protected]), to [email protected], as shown below:



The email itself:

I then parsed the text from the email (associated with the file ‘test.xc’) with JGAAP, and its inbuilt Stanford POS tagger. I also created four other payloads, using different email templates. In total, text from these options’ emails was used:

No. 9 No. 6 No. 3 No. 5 No. 7

Text from the emails of options 3, 6 and 9 were associated with authors’ names which were the email’s subject names, whereas the email text of options 5 and 7 was marked as unknown.

Using a Linear SVM or Gaussian (Radial Basis Kernel) SVM with the Stanford Part of Speech Tagger resulted in:

---------------------------------------------------------

test_xc.txt C:\Users\Luther\Desktop\SET data\test_xc.txt

Canonicizers: Normalize ASCII, Normalize Whitespace

EventDriver: Stanford Part of Speech tagginmodel : english-bidirectional-distsim

Analysis: Linear SVM

1. Order Confirmation 1.0

test_xc.txt C:\Users\Luther\Desktop\SET data\test_xc.txt



Analysis: Gaussian SVM

1. Order Confirmation 1.0

---------------------------------------------------------

The first file most closely matched the author (folder) containing that file, so this made sense.

---------------------------------------------------------

test2_xc.txt C:\Users\Luther\Desktop\SET data\test2_xc.txt




1. WOAAAA!!!!!!!!!! This is crazy... 2.0





1. WOAAAA!!!!!!!!!! This is crazy... 2.0

---------------------------------------------------------

The second file most closely matched the author (folder) containing that file, so this made sense.

---------------------------------------------------------





1. Dan Brown's Angels & Demons 3.0






---------------------------------------------------------

The third file most closely matched the author (folder) containing that file, so this made sense.

---------------------------------------------------------











---------------------------------------------------------

The fourth file most closely matched the third author (Dan Brown's Angels & Demons) Why? More tests needed?

---------------------------------------------------------











---------------------------------------------------------

The fifth file most closely matched the third author (Dan Brown's Angels & Demons) Why? More tests needed?

---------------------------------------------------------

Since the fourth and fifth files were both matched with the third author (Dan Brown's Angels & Demons), does this mean they are similar to each other?

I associated the fourth file with an author equal to its email’s subject name, and then just tested using the only unknown text file as the fifth file. The rest of the conditions were the same.

Results:

---------------------------------------------------------





1. Status Report 4.0





1. Status Report 4.0

---------------------------------------------------------

My hypothesis was correct, since the fourth file was matched most closely with the fifth file.

Spear Phishing – Option 3:

My next test was to compare text from an unrelated source, but same field (deception), to these SET generated email texts, to see if they were considered similar or not to the set of email texts used in the previous test. This was carried out using a number of techniques:

A Gaussian SVM A Linear SVM

o Each with the Stanford Part of Speech Tagger

I selected the text from the Brennan-Greenstadt Adversarial Corpus

(source: https://psal.cs.drexel.edu/index.php/Main_Page)

---------------------------------------------------------------------------------

Citations:

Michael Brennan, Sadia Afroz, and Rachel Greenstadt. 2012. Adversarial stylometry: Circumventing authorship recognition to preserve privacy and anonymity. ACM Trans. Inf. Syst. Secur. 15, 3, Article 12 (November 2012).

Michael Brennan and Rachel Greenstadt. Practical Attacks Against Authorship Recognition Techniques in Proceedings of the Twenty-First Conference on Innovative Applications of Artificial Intelligence (IAAI), Pasadena, California, July 2009.

---------------------------------------------------------------------------------

I used attacks classified by the corpus to obfuscation attacks, and compared to them to the SET generated attacks. These were the results:

a_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\a_obfuscation.txt

Canonicizers: none



1. Order Confirmati?on 1.0

https://psal.cs.drexel.edu/index.php/Main_Page

a_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\a_obfuscation.txt

Canonicizers: none



1. Baby Pics 5.0

b_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\b_obfuscation.txt

Canonicizers: none




b_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\b_obfuscation.txt

Canonicizers: none



1. Baby Pics 5.0

c_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\c_obfuscation.txt

Canonicizers: none




c_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\c_obfuscation.txt

Canonicizers: none



1. Baby Pics 5.0

d_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\d_obfuscation.txt

Canonicizers: none




d_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\d_obfuscation.txt

Canonicizers: none



1. Baby Pics 5.0

e_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\e_obfuscation.txt

Canonicizers: none




e_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\e_obfuscation.txt

Canonicizers: none



1. Baby Pics 5.0

f_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\f_obfuscation.txt

Canonicizers: none




f_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\f_obfuscation.txt

Canonicizers: none



1. Baby Pics 5.0

g_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\g_obfuscation.txt

Canonicizers: none




g_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\g_obfuscation.txt

Canonicizers: none



1. Baby Pics 5.0

h_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\h_obfuscation.txt

Canonicizers: none




h_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\h_obfuscation.txt

Canonicizers: none



1. Baby Pics 5.0

k_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\k_obfuscation.txt

Canonicizers: none




k_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\k_obfuscation.txt

Canonicizers: none



1. Baby Pics 5.0

m_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\m_obfuscation.txt

Canonicizers: none




m_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\m_obfuscation.txt

Canonicizers: none



1. Baby Pics 5.0

p_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\p_obfuscation.txt

Canonicizers: none




p_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\p_obfuscation.txt

Canonicizers: none



1. Baby Pics 5.0

s_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\s_obfuscation.txt

Canonicizers: none




s_obfuscation.txt D:\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_obfuscation\s_obfuscation.txt

Canonicizers: none



1. Baby Pics 5.0

Documents c, d and h were classified to match mostly with the 3rd author ‘Dan Brown's Angels & Demons’, with the other documents being classified to ‘Order Confirmation’, when using a Linear SVM. With a Gaussian SVM, all documents were classified to match ‘Baby Pics’ - author 5.

These were the results for the imitation corpus:

---------------------------------------------------------------------------------------------------------------------------------------------------

a_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\a_imitation.txt

Canonicizers: none



1. Baby Pics 5.0

a_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\a_imitation.txt

Canonicizers: none




---------------------------------------------------------------------------------------------------------------------------------------------------

b_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\b_imitation.txt

Canonicizers: none



1. Baby Pics 5.0

b_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\b_imitation.txt

Canonicizers: none




---------------------------------------------------------------------------------------------------------------------------------------------------

c_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\c_imitation.txt

Canonicizers: none



1. Baby Pics 5.0

c_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\c_imitation.txt

Canonicizers: none




---------------------------------------------------------------------------------------------------------------------------------------------------

d_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\d_imitation.txt

Canonicizers: none



1. Baby Pics 5.0

d_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\d_imitation.txt

Canonicizers: none




---------------------------------------------------------------------------------------------------------------------------------------------------

e_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\e_imitation.txt

Canonicizers: none



1. Baby Pics 5.0

e_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\e_imitation.txt

Canonicizers: none




---------------------------------------------------------------------------------------------------------------------------------------------------

f_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\f_imitation.txt

Canonicizers: none



1. Baby Pics 5.0

f_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\f_imitation.txt

Canonicizers: none




---------------------------------------------------------------------------------------------------------------------------------------------------

g_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\g_imitation.txt

Canonicizers: none



1. Baby Pics 5.0

g_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\g_imitation.txt

Canonicizers: none




---------------------------------------------------------------------------------------------------------------------------------------------------

h_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\h_imitation.txt

Canonicizers: none



1. Baby Pics 5.0

h_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\h_imitation.txt

Canonicizers: none




---------------------------------------------------------------------------------------------------------------------------------------------------

k_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\k_imitation.txt

Canonicizers: none



1. Baby Pics 5.0

k_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\k_imitation.txt

Canonicizers: none




---------------------------------------------------------------------------------------------------------------------------------------------------

m_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\m_imitation.txt

Canonicizers: none



1. Baby Pics 5.0

m_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\m_imitation.txt

Canonicizers: none




---------------------------------------------------------------------------------------------------------------------------------------------------

p_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\p_imitation.txt

Canonicizers: none



1. Baby Pics 5.0

p_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\p_imitation.txt

Canonicizers: none




---------------------------------------------------------------------------------------------------------------------------------------------------

s_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\s_imitation.txt

Canonicizers: none



1. Baby Pics 5.0

s_imitation.txt C:\SD\Dropbox\Coding\3rd Yr Project\Project Docs\SET data\attacks_imitation\s_imitation.txt

Canonicizers: none




When parsing using the Stanford POS tagger, all imitation documents, matched with the 5th author - ‘Baby Pics’, when using a Gaussian SVM, and all imitation documents apart from e, g and h, matched the 3rd author - ‘Dan Brown’s Angels and Demons’, when using a Linear SVM.

Documents e, g and h matched with ‘Order Confirmation’, when applying a Linear SVM to the imitation documents.

Overall, when either obfuscating or imitating, it was the case that the only matched document was no. 5 – ‘Baby Pics’, when using a Gaussian SVM at least, so there is no way to determine whether the chosen documents from the SET were more likely to be obfuscating their text, rather than imitating some text, and vice versa, using a Gaussian SVM.

In the case of the Linear SVM:

Obfuscation:o c, d and h were classified to match mostly with the 3rd author ‘Dan Brown's Angels & Demons’o All other documents (9) were classified as matching ‘Order Confirmation’

Imitation:o e, g and h matched with ‘Order Confirmation’o All others (9) matched with ‘Dan Brown’s Angels and Demons’

You could assume for a hypothesis, that since 75% of documents matched ‘Order Confirmation’ when obfuscating, you could research features of obfuscation, determine which are in use in the ‘Order Confirmation’ document, and then be able to classify other similar SET documents using that feature set.

o The same could be applied to the imitation side of things, as 75% of documents were classified to be closest to ‘Dan Brown’s Angels and Demons’.

The other funny thing is that each of the authors ‘Dan Brown’s Angels and Demons’ (3rd), and ‘Order Confirmation’ (1st) held the corresponding other quarter of document classifications, for either obfuscation and imitation, respectively.

o This means that if creating a distance measure, a lower weighting could be placed on features relating to obfuscation in the first case (3rd author), and imitation in the second (1st author).

o Maybe 75% of features could be used to detect obfuscation, and 25% imitation, for invoice/receipt/acknowledgement style documents, and vice-versa, for imitation/persuasion style documents.

Create test to determine whether SET documents have more of a focus on imitation or obfuscation.

o State features which determined thiso Maybe research distance measures

Start with own emails test against SET – ground truth Create distance measure with this

In future ask people to create emails for testing, which are in a similar vein the SET/Kali Linux generated ones.

Compare results to distance measure. See which corpora when applied to distance measure produces best

results over time If approx. half of corpora are suited to one distance measure, and

the approx. other half, another distance measure, maybe try combining the two, and adjusting weightings of metrics/features, to make sure a correct results is obtained for all classifications of corpora, within a certain threshold.

JGAAP terminology

(Source: http://evllabs.com/jgaap/w/index.php/JGAAP#Events)

Canonicizer – Preprocessors for raw document text. It means something that puts the raw text of a document into canonical form (e.g. changing characters to lowercase, removing punctuation)

Events – Singletons generated from the text of the document used to represent it to the learning algorithm (e.g. Words, Characters, Parts of Speech, Word BiGrams)

Cullers – Operate on the resulting events from processed documents. They remove events from those generated creating a set that meets its stipulations.

E.g.

Reduce to only the 50 most common events Reduce to the 100 least common events Reduce to only events that appear in every document.

Analysis – Analysis methods use the event based representations of documents to create models of their authors. It then uses these models to evaluate the most likely author of unknown documents from the pool of authors it knows about.

E.g. Support Vector Machines, or Nearest Neighbour.

JGAAP code

API is available, along with a command line interface (a Java class for each) Javadocs:

o Instructions for using the JGAAP API: First add documents both known and unknown (via addDocument) All other settings can be performed in any order which are:

setLanguage addCanonicizer addEventDriver – Used to generate a List of Events ordered in the

sequence they are found in the document addEventCuller addAnalysisDriver addDistanceFunction

Note: Of the settings only one EventDriver and one AnalysisDriver are required to run an experiment

o The execute method is then used to start the experimento Results are placed in unknown documents

To access them use the getUnknownDocuments method in the APIo The results can be retrieved as a List<Pair<String, Double>>

This is a sorted list from most likely to least likely author followed by a score generated based on your settings, using the getRawResult method

You can also get a Map of Maps of the raw results with the getRawResults method

http://evllabs.com/jgaap/w/index.php/JGAAP#Events

Type is: (Map<EventDriver, Map<AnalysisDriver, List<String, Double>>>)

They can also be retrieved as a string using either the getFormattedResult or getResult methods.

o For examples of how to use the API class see the com.jgaap.ui package for a GUI example

o or the com.jgaap.backend.CLI class for a command line example

JGAAP Command Line Interface

Done using the JGAAP Experiment Engine

(Source: http://evllabs.com/jgaap/w/index.php/Experiment_Engine)

The Experiment Engine is accessible through JGAAP's command line interface using the -ee flag and pointing it to a run file, i.e. java -jar jgaap.jar -ee ../experiment.run

A run file is a csv where the top row is a title for the experiment and each subsequent row is an experiment to be run.

A ‘Corpus File’ contains information on the authors and titles of documents that can be used in JGAAP to allow for the quick reproduction of experiments.

Corpus files can be generated within JGAAP after loading documents via the menu option“File>Batch Documents>Save Documents”

Standard:

Author,/path/to/file.txt,TitleAuthor,/path/to/other/file.txt,Title, /path/to/unknown/file.txt,Title

Example:

A,/home/ryan/example/sampleA-01.txt A,/home/ryan/example/sampleA-02.txt B,/home/ryan/example/sampleB-01.txt B,/home/ryan/example/sampleB-02.txt C,/home/ryan/example/sampleC-01.txt C,/home/ryan/example/sampleC-02.txt ,/home/ryan/example/test-01.txt ,/home/ryan/example/test-02.txt

Titles are optional. If you don’t assign one, a default title will be assigned.

http://evllabs.com/jgaap/w/index.php/Experiment_Engine

· Web viewSocial Engineering Toolkit Tests Had to download python ‘pefile’, and enter...

Documents

Transcript of · Web viewSocial Engineering Toolkit Tests Had to download python ‘pefile’, and enter...