Techniques against Web Anti-Automatization Bruno Ramos [email protected] H2HC II - 2005.

Techniques against Techniques against Web Anti-Web Anti-

AutomatizationAutomatization

Bruno RamosBruno Ramos

[email protected]@yahoo.com.br

H2HC II - 2005H2HC II - 2005

SummarySummary

ObjectivesObjectives Automatization x Anti-Automatization x Anti-

AutomatizationAutomatization Dog_Crawler x PHP_GUARDDog_Crawler x PHP_GUARD ConclusionConclusion DEMODEMO

ObjectivesObjectives

To present a new seek area on Web To present a new seek area on Web HackingHacking

Performance in the automatization Performance in the automatization processprocess

To generate new ideas of techniques To generate new ideas of techniques against anti-automatizationagainst anti-automatization

Automatization x Anti-Automatization x Anti-AutomatizationAutomatization

AutomatizationAutomatization Automatized SweepingsAutomatized Sweepings

What he is an automatized sweeper?What he is an automatized sweeper? Development of the automatized sweepingsDevelopment of the automatized sweepings Class of automatization toolsClass of automatization tools

Vulnerabilities TechniquesVulnerabilities Techniques Anti-AutomatizationAnti-Automatization

– Main used techniquesMain used techniques

AutomatizationAutomatization

ObjectiveObjective PrinciplesPrinciples

– CodificationCodification– AlgorithmsAlgorithms– ProcessProcess

Automatized SweepingsAutomatized Sweepings

What he is an automatized sweeper? What he is an automatized sweeper? – MirroringMirroring

The Robbery of copyrightThe Robbery of copyright Part of one has attacked of main-in-the-middlePart of one has attacked of main-in-the-middle

– SpideringSpidering Harvest of email for Spam listHarvest of email for Spam list Attack of social engineering on personal datasAttack of social engineering on personal datas To understand development techniquesTo understand development techniques To discover details of the application for exploration To discover details of the application for exploration

phasephase Map the structure of the applicationMap the structure of the application

– CGI ScanningCGI Scanning Probable administrative pages and Probable administrative pages and

directoriesdirectories Localization of archives and common Localization of archives and common

directoriesdirectories

– Brute ForcingBrute Forcing Based in dictionaryBased in dictionary List of archives and common directoriesList of archives and common directories Incremental Interaction on all the possible Incremental Interaction on all the possible

characterscharacters


– FuzzingFuzzing Buffer OverflowsBuffer Overflows Cross-site scriptingCross-site scripting SQL InjectionSQL Injection Difference enters the process of validation of Difference enters the process of validation of

client-side and server-sideclient-side and server-side


Development of the automatized sweepingsDevelopment of the automatized sweepings– 1 Generation 1 Generation

CGI scannersCGI scanners

– 2 Generation2 Generation SpideringSpidering MirroringMirroring Brute forcingBrute forcing

– 3 Generation3 Generation FuzzingFuzzing

– 4 Generation4 Generation Anti-Automatization? Anti-Automatization?


Class of automatization toolsClass of automatization tools– Web SpiderWeb Spider– CGI ScannerCGI Scanner– Brute ForceBrute Force– FuzzerFuzzer– Vulnerability ScannersVulnerability Scanners


Vulnerabilities TechniquesVulnerabilities Techniques

OWASP Top Ten Most Critical Web OWASP Top Ten Most Critical Web Application Security VulnerabilitiesApplication Security Vulnerabilities– Unvalidated Input Unvalidated Input – Broken Access ControlBroken Access Control– Broken Authentication and Session ManagementBroken Authentication and Session Management– Cross Site Scripting (XSS) FlawsCross Site Scripting (XSS) Flaws– Buffer OverflowsBuffer Overflows– Injection FlawsInjection Flaws– Improper Error HandlingImproper Error Handling – Insecure StorageInsecure Storage – Denial of ServiceDenial of Service– Insecure Configuration ManagementInsecure Configuration Management

Anti-AutomatizationAnti-Automatization

Blocking of HEAD requestsBlocking of HEAD requests Content-Type ManipulationContent-Type Manipulation HTTP Status CodesHTTP Status Codes Thresholds and TimeoutsThresholds and Timeouts Honeypot linksHoneypot links

Blocking of HEAD requestsBlocking of HEAD requests

Easy to implementEasy to implement Low ImpactLow Impact Used against:Used against:

1 Generation CGI Scanners1 Generation CGI Scanners 1 Generation Web Spiders1 Generation Web Spiders 1 Generation Fuzzers1 Generation Fuzzers

Content-Type ManipulationContent-Type Manipulation

Configuration in the ServerConfiguration in the Server Codification in the applicationCodification in the application Used against:Used against:

Mirroring SoftwareMirroring Software Web SpidersWeb Spiders 1 Generation vulnerability scanners1 Generation vulnerability scanners

HTTP Status CodesHTTP Status Codes

SimplicitySimplicity Control of the development Control of the development Used against:Used against:

FuzzersFuzzers Brute ForcersBrute Forcers CGI ScannersCGI Scanners Vulnerability ScannersVulnerability Scanners

Thresholds and TimeoutsThresholds and Timeouts

Frequency of solicitationsFrequency of solicitations Multiple solicitations Multiple solicitations Used against:Used against:

Web SpidersWeb Spiders Brute ForcersBrute Forcers CGI ScannersCGI Scanners Vulnerability ScannersVulnerability Scanners

Honeypot LinksHoneypot Links

Simple ConfigurationSimple Configuration Customizadas answersCustomizadas answers Used against:Used against:

Web SpidersWeb Spiders Mirroring SoftwareMirroring Software

Dog_Crawler x PHP_GUARDDog_Crawler x PHP_GUARD PHP_GUARDPHP_GUARD

Archetype to defeat the mechanism of Crawler? Archetype to defeat the mechanism of Crawler? Easily incorporated in an applicationEasily incorporated in an application Author: “Web Hacking – Attacks and Defense”Author: “Web Hacking – Attacks and Defense”

DOG_CrawlerDOG_Crawler Crawler with support the techniques against anti-Crawler with support the techniques against anti-

AutomatizationAutomatization Project in development that it needs new crazy ideas to Project in development that it needs new crazy ideas to

break others anti-automatization techniquesbreak others anti-automatization techniques Implemented using PerlImplemented using Perl It uses the mechanism of crawler of the LibwhiskerIt uses the mechanism of crawler of the Libwhisker

PHP_GUARD PHP_GUARD

TechniquesTechniques Enforces Strict Session ControlEnforces Strict Session Control Varying HTTP Response CodesVarying HTTP Response Codes Structurally Different HTML all the TimeStructurally Different HTML all the Time Generates Random HyperlinksGenerates Random Hyperlinks Generates Random HTML Authentication Generates Random HTML Authentication

FormsForms Ability to Slow Down Response Ability to Slow Down Response

Enforces Strict Session Enforces Strict Session ControlControl

set_session.phpset_session.php

<?php<?php

// begin a session// begin a session

session_start();session_start();

$_SESSION['begin'] = 1;$_SESSION['begin'] = 1;

?>?>

php_guard.phpphp_guard.php

// check the session status// check the session status

......

session_start();session_start();

if(!isset($_SESSION['begin'])) {if(!isset($_SESSION['begin'])) {

header("Location: /"); header("Location: /"); setcookie(session_name(), "", 0, "/"); setcookie(session_name(), "", 0, "/");

session_destroy();session_destroy();

exit;exit;

}}

......

Varying HTTP Response Varying HTTP Response Codes Codes


$dice = mt_rand(1, 100);$dice = mt_rand(1, 100);

if($dice < $SG_404_PROBABILITY) {if($dice < $SG_404_PROBABILITY) {

response_404();response_404();

}}

else {else {

$dice = mt_rand(1, 100);$dice = mt_rand(1, 100);

if($dice < $SG_302_PROBABILITY) {if($dice < $SG_302_PROBABILITY) {


}}

else {else {


}}

}}


function load_quote_array() {function load_quote_array() {

global $SG_QUOTE_ARRAY, global $SG_QUOTE_ARRAY, $SG_QUOTES_FILE, $DEBUG;$SG_QUOTES_FILE, $DEBUG;

static $quote_array, $flag = 0;static $quote_array, $flag = 0;

if(!$flag) {if(!$flag) {

$quote_array = $quote_array = file($SG_QUOTES_FILE);file($SG_QUOTES_FILE);

$flag = 1;$flag = 1;

}}

$SG_QUOTE_ARRAY = $quote_array;$SG_QUOTE_ARRAY = $quote_array;

}}



function response_404() {function response_404() {

header("HTTP/1.0 404 Not Found");header("HTTP/1.0 404 Not Found");

echo("<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n");echo("<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n");

echo("<html><head>\n");echo("<html><head>\n");

echo("<title>404 Not Found</title>\n");echo("<title>404 Not Found</title>\n");

echo("</head><body>\n");echo("</head><body>\n");

echo("<h1>Not Found</h1>\n");echo("<h1>Not Found</h1>\n");

echo("<p>The requested URL " . echo("<p>The requested URL " .

..

..

..




global $SG_QUOTE_ARRAY;global $SG_QUOTE_ARRAY;

$link = random_link($SG_QUOTE_ARRAY, "/");$link = random_link($SG_QUOTE_ARRAY, "/");

header("Location: " . $link);header("Location: " . $link);

}}

function random_link(&$list, $prefix) {function random_link(&$list, $prefix) {

$result = random_directory($list, $prefix) . random_word($list);$result = random_directory($list, $prefix) . random_word($list);

$result = random_extension($result);$result = random_extension($result);

$result .= random_querystring($list);$result .= random_querystring($list);

return($result);return($result);

}}

Structurally Differrent HTML Structurally Differrent HTML all the Timeall the Time



global $SG_QUOTE_ARRAY, $SG_OPENING_HTML, global $SG_QUOTE_ARRAY, $SG_OPENING_HTML, $SG_CLOSING_HTML;$SG_CLOSING_HTML;

global $SG_MAX_TEXT_LIMIT, global $SG_MAX_TEXT_LIMIT, $SG_MIN_TEXT_LIMIT;$SG_MIN_TEXT_LIMIT;

header("HTTP/1.0 200 OK");header("HTTP/1.0 200 OK");

// see how many quotes we have// see how many quotes we have

$quote_count = count($SG_QUOTE_ARRAY);$quote_count = count($SG_QUOTE_ARRAY);

// generate a random number// generate a random number

$limit = $quote_count;$limit = $quote_count;

if($limit > $SG_MAX_TEXT_LIMIT) {if($limit > $SG_MAX_TEXT_LIMIT) {

$limit = $SG_MAX_TEXT_LIMIT;$limit = $SG_MAX_TEXT_LIMIT;

}}

$random_number = $random_number = mt_rand($SG_MIN_TEXT_LIMIT, $limit);mt_rand($SG_MIN_TEXT_LIMIT, $limit);

// decide the HTML text containers// decide the HTML text containers

$opening_html = $SG_OPENING_HTML;$opening_html = $SG_OPENING_HTML;

$closing_html = $SG_CLOSING_HTML;$closing_html = $SG_CLOSING_HTML;

$rand_html = array_rand($opening_html, 1);$rand_html = array_rand($opening_html, 1);

$opening_format = $opening_html[$rand_html];$opening_format = $opening_html[$rand_html];

$closing_format = $closing_html[$rand_html];$closing_format = $closing_html[$rand_html];

$opening_block = "";$opening_block = "";

$closing_block = "";$closing_block = "";

// decide if we want to do HTML tables or not// decide if we want to do HTML tables or not

// 50% chance for throwing in tables.// 50% chance for throwing in tables.

$table_flag = mt_rand(0, 1);$table_flag = mt_rand(0, 1);

if($table_flag) if($table_flag)

$opening_block = "<TABLE>";$opening_block = "<TABLE>";

$closing_block = "</TABLE>";$closing_block = "</TABLE>";

$opening_format = "<TR><TD>";$opening_format = "<TR><TD>";

$closing_format = "</TD></TR>";$closing_format = "</TD></TR>";

}}

$form_flag = 0;$form_flag = 0; // 50% chance of throwing in an HTML form// 50% chance of throwing in an HTML form $print_form = mt_rand(0, 1);$print_form = mt_rand(0, 1);

$rand_keys = $rand_keys = array_rand($SG_QUOTE_ARRAY, array_rand($SG_QUOTE_ARRAY, $random_number);$random_number); $form_loc = mt_rand(0, count($rand_keys));$form_loc = mt_rand(0, count($rand_keys)); echo($opening_block . "\n");echo($opening_block . "\n"); for($i = 0; $i < count($rand_keys); $i++) {for($i = 0; $i < count($rand_keys); $i++) { echo($opening_format);echo($opening_format);

echo(quote_parse($SG_QUOTE_ARRAY[echo(quote_parse($SG_QUOTE_ARRAY[$rand_keys[$i]]));$rand_keys[$i]])); if($print_form && !$form_flag && $i == if($print_form && !$form_flag && $i == $form_loc) {$form_loc) { random_auth_form();random_auth_form(); $form_flag = 1;$form_flag = 1; }} echo($closing_format . "\n");echo($closing_format . "\n"); }} echo($closing_block . "\n");echo($closing_block . "\n");}}

Structurally Differrent HTML Structurally Differrent HTML all the Timeall the Time

Generates Random Generates Random HyperlinksHyperlinks


function random_link(&$list, $prefix) {function random_link(&$list, $prefix) {

$result = random_directory($list, $prefix) . $result = random_directory($list, $prefix) . random_word($list);random_word($list);

$result = random_extension($result);$result = random_extension($result);

$result .= random_querystring($list);$result .= random_querystring($list);


}}



function random_extension($str)function random_extension($str)

{{

global $SG_EXT_ARRAY;global $SG_EXT_ARRAY;

$ext_array = $SG_EXT_ARRAY;$ext_array = $SG_EXT_ARRAY;

$rand_key = array_rand($ext_array, 1);$rand_key = array_rand($ext_array, 1);

$result = $str . $ext_array[$rand_key];$result = $str . $ext_array[$rand_key];


}}

$param_value = random_extension($param_value);$param_value = random_extension($param_value); }} else {else { // choose a random word// choose a random word $param_name = random_word($list);$param_name = random_word($list); if(mt_rand(0, 1)) {if(mt_rand(0, 1)) { $param_value = mt_rand(0, 65535);$param_value = mt_rand(0, 65535); }} else {else { $param_value = random_word($list);$param_value = random_word($list); }} }} }} if(!$flag) {if(!$flag) { $result .= $param_name . "=" . $param_value;$result .= $param_name . "=" . $param_value; $flag = 1;$flag = 1; }} else {else { $result .= "&" . $param_name . "=" . $param_value;$result .= "&" . $param_name . "=" . $param_value; }} }} }} return($result);return($result);


function random_word(&$list)function random_word(&$list){{ $rand_key = array_rand($list, 1);$rand_key = array_rand($list, 1); $words = explode(" ", $list[$rand_key]);$words = explode(" ", $list[$rand_key]); $rand_key = array_rand($words, 1);$rand_key = array_rand($words, 1); $word = sanitize_alnum($words[$rand_key]);$word = sanitize_alnum($words[$rand_key]); return($word);return($word);}}

Generates Random HTML Generates Random HTML Authentication FormsAuthentication Forms


function random_auth_form() {function random_auth_form() {

global $SG_QUOTE_ARRAY, $SG_HIDDEN_FIELDS;global $SG_QUOTE_ARRAY, $SG_HIDDEN_FIELDS;

//$quote_array = load_quote_array();//$quote_array = load_quote_array();

generate_form_tag($SG_QUOTE_ARRAY);generate_form_tag($SG_QUOTE_ARRAY);

echo("<table>\n");echo("<table>\n");

generate_input_tag($SG_QUOTE_ARRAY, "text", 10);generate_input_tag($SG_QUOTE_ARRAY, "text", 10);

generate_input_tag($SG_QUOTE_ARRAY, "password", 10);generate_input_tag($SG_QUOTE_ARRAY, "password", 10);

$hidden_fields = mt_rand(0, $SG_HIDDEN_FIELDS);$hidden_fields = mt_rand(0, $SG_HIDDEN_FIELDS);

for($i = 0; $i < $hidden_fields; $i++) {for($i = 0; $i < $hidden_fields; $i++) {

generate_input_tag($SG_QUOTE_ARRAY, "hidden", 0);generate_input_tag($SG_QUOTE_ARRAY, "hidden", 0);

}}

generate_input_tag($SG_QUOTE_ARRAY, "submit", 0);generate_input_tag($SG_QUOTE_ARRAY, "submit", 0);

echo("</table>\n");echo("</table>\n");

generate_form_end();generate_form_end();

}}

DOG_CrawlerDOG_Crawler

Techniques against anti-Techniques against anti-automatization automatization

Test of Method HEADTest of Method HEAD Analyzes of ContentAnalyzes of Content Signature of ReplySignature of Reply Detection of Honeypots Links and FormDetection of Honeypots Links and Form Heuristic and RandomHeuristic and Random Distributed automatizationDistributed automatization

Test of Method HEADTest of Method HEAD

$ echo -e "HEAD / HTTP/1.0\n\n" | nc 192.168.1.1 80HTTP/1.1 406 Not AcceptableDate: Fri, 16 Sep 2005 05:27:00 GMTServer: Apache/1.3.31 (Unix) PHP/4.3.7Connection: closeContent-Type: text/html; charset=iso-8859-1

/HTTP\/*.* (200)/ig

If not to find the code of reply "200" the defense was detectedIf not to find the code of reply "200" the defense was detected

Analyzes of ContentAnalyzes of Content

$ echo -e "GET /index.gif HTTP/1.0\n\n" | nc 192.168.1.1 80HTTP/1.1 200 OKDate: Fri, 16 Sep 2005 12:00:56 GMTServer: Apache/1.3.31 (Unix) PHP/4.3.7Last-Modified: Wed, 14 Sep 2005 06:31:42 GMTETag: "47efb-732-4327c3ce"Accept-Ranges: bytesContent-Length: 1842Connection: closeContent-Type: text/htmlX-Pad: avoid browser bug<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"><html><head>…

– /=*([\w|\/|\.|\:]+.html)/ig– /=\"([\w|\/|\-]+.asp|.jsp|.php)/ig– /=\"([\w|\/|\-]+.gif)/ig– /=\"([\w|\/|\-]+.jpg)/ig– /=\"([\w|\/|\-]+.png)/ig– /=\"([\w|\/|\-]+.gif|.jpg|.png)/ig– /Content-Type: *([a-z&\/&\-]+)/i– /href=\"([\w|\/|\.|\:]+)/ig– /MIME-Version/– //(\%3C|<|\&lt)META*.content=*([a-z&\/&\-]+)/ig

Regular ExpressionsRegular Expressions

Analyzes of ContentAnalyzes of Content

Signature of ReplySignature of Reply

Detection of Honeypots Detection of Honeypots Links Links

<--! HREF="../honeypot.html"> --> <FONT COLOR="black"><HREF="../honeypot.html">escondido</A></FONT>

/(\%3C|<|\&lt)!--.*href=([\w|\/|\.|\:]+)*.-- (\%3C|>|\&gt)/ig

Honeypot linksHoneypot links

Link in the commentary detects HoneypotLink in the commentary detects Honeypot

HeuristicHeuristic

Heuristic MyopeHeuristic Myope

Algoritmo Míope(n, c, S, F) [Max {c(S) / S F}] Início Ordenar os elementos de E de forma que: C(s1) ≥ c(s2) ≥ ... ≥ c(sn) > 0; S := Ø; Para i = 1 até n fazer Se [S {si}] F então S := S {si}; Escrever {S, c(S) = ∑ c(s)}; Fim

RandomRandom

PERMUTE-BY-SORTING(A) n comprimento[A] for i 1 to n do P[i] = RANDOM(1, n3) ordenar A, usando P como chaves de ordenação return A

Xn+1 = KXn(módulo M)sendo- n = 1,2,3,...- x0 é um número aleatório inicial (semente), onde 0 < x0 < M;- K é número inteiro, tal que 0 < K < M- M = 10b + 1, onde b é quantidade de algarismos;

Random permutation of ArrangementsRandom permutation of Arrangements

Congruencial method MultiplicationCongruencial method Multiplication

Distributed automatizationDistributed automatization

WebServer DB

DB

Web app

Web app

Web app

Web app

1

2

3

4

0

[1] Gunter Ollmann – Second-order Code Injection Attacks http://www.ngssoftware.com/papers/StoppingAutomatedAttackTools.pdf[2] Saumil Shah - Defeating Automated Web Assessment Tools http://www.blackhat.com/presentations/bh-europe-05/BH_EU_05-Shah.pdf[3] SensePost - Revolutions in Web Server/Application Assessments http://www.blackhat.com/presentations/bh-europe-05/bh-eu-05-sensepost.pdf[4] http://www.owasp.org

SitesSites

DEMODEMO

Techniques against Web Anti-Automatization Bruno Ramos [email protected] H2HC II - 2005.

Documents

Transcript of Techniques against Web Anti-Automatization Bruno Ramos [email protected] H2HC II - 2005.