HTTP::Parser::XS - writing a fast & secure XS module
-
Upload
kazuho-oku -
Category
Technology
-
view
5.214 -
download
0
description
Transcript of HTTP::Parser::XS - writing a fast & secure XS module
HTTP::Parser::XSwriting a fast & secure XS module
Cybozu Labs, Inc.Kazuho Oku
Pros & Cons of XS
ProsFaster (well, not always, but…)
ConsDifficult and time-consumingHigher security risk
Mistakes in C programming leads to remote code injection
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 2
The Best Practice, in General
Don’t use XSUntil you hit performance bottleneck
Rewrite the bottleneck (only) using XS
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 3
HTTP::Parser::XS
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 4
Summary
An HTTP request parserDesigned and implemented to be
simple, fast & secureStatelessPSGI-compatible
Required or recommended by many Plack serversStarman, Starlet, Twiggy, etc.
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 5
Two-layered Approach
PicohttpparserHTTP request / response parser written in Chas its own test suitedoes not parse request / response content
request line (response line) and headers onlycopy-less
faster, lesser probability of security holes and memory leaks
HTTP::Parser::XSglue code to access picohttpparser from
perlMay 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 6
Speed and Complexity
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 7
reqs/sec. lines of code
HTTP::HeaderParser::XS
116,000 1,166
HTTP::Parser::XS 140,000 487
Plack::HTTPParser::PP 10,100 104
HTTP::Parser::XS is simple, and fastmost of the time is not spent in
picohttpparser, but in the glue code constructing hashrefpicohttpparser/trunk can handle >1Mreqs/sec.
Why is it a Stateless?
Faster and simpler than a stateful parserlower security risks
most HTTP requests / responses arrive in a single packet, anyway
if written optimally in C, the cost of re-parsing is smaller than storing headers into a perl hashref
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 8
Why is it Stateless? (cont’d)
Easy to determine the end of an multi-packet HTTP requestby looking for “\r\n\r\n” within the last
packet (and preceding three bytes)mainly as a countermeasure for Slowloris
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 9
Consistent Design
Functions take same arguments buf – points to current char buf_end – points to end of buffer *ret – error value returns pointer to the next char
or null on error (the reason will be stored in *ret)
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 10
const char* parse_http_version(const char* buf, const char* buf_end, int* minor_version, int* ret){ EXPECT_CHAR('H'); EXPECT_CHAR('T'); EXPECT_CHAR('T'); EXPECT_CHAR('P'); EXPECT_CHAR('/'); EXPECT_CHAR('1'); EXPECT_CHAR('.'); return parse_int(buf, buf_end, minor_version, ret);}
Macros
Consistent design is essential to heavy use of macrosGood abstraction ⇒
safe code
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 11
#define CHECK_EOF() \ if (buf == buf_end) { \ *ret = -2; \ return NULL; \ }#define EXPECT_CHAR(ch) \ CHECK_EOF(); \ if (*buf++ != ch) { \ *ret = -1; \ return NULL; \ }
const char* parse_http_version(const char* buf, const char* buf_end, int* minor_version, int* ret){ EXPECT_CHAR('H'); EXPECT_CHAR('T'); EXPECT_CHAR('T'); EXPECT_CHAR('P'); EXPECT_CHAR('/'); EXPECT_CHAR('1'); EXPECT_CHAR('.'); return parse_int(buf, buf_end, minor_version, ret);}
Micro-optimization (only in picohttpparser)
Reduce # of conditional branches, optimize for pipeline (27% faster)
Unroll loops (36% faster)
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 12
#define unlikely(x) __builtin_expect(!!(x), 0)
for (; ; ++buf) { CHECK_EOF(); if (unlikely((unsigned char)*buf <= '\r') && (*buf == '\r' || *buf == '\n')) goto EOL_FOUND;}
Release History of HTTP::Parser::XS
Current version: 0.07No security hole found since initial
releaseTwo memory leaks were found and
fixed
Please let me know if you find any security holes (especially the ones that lead to arbitrary code execution)
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 13
Conclusion – KISS
Keep it simple, stupidfor fast developmentsimple design leads to more secure codeuse perl whenever possible
simple operations (like tokenization) is worth converting to XS
complex operations (from handling of strings to database queries) are not so slow in perl
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 14