HTTPAll you need to
know
Who are you?
Gökhan Şengün
R&D Business Dev., New Product & Solutions Manager
www.gokhansengun.com
@gokhansengun
Aim• Better understand HTTP basics to debug problems
better• Know HTTP players to see the big picture• Know useful tools to do things faster
HTTP• HTTP is a stateless protocol. • How is being stateless like?• A stateless protocol does not require the server to retain
information or status about each user for the duration of multiple requests.
Http Proxy
Popular Http Proxies• Fiddler• Burp Suite• Browser Developer Tools (Embedded Proxy)
Demo – Bare Metal - Using Telnet
Demo – Browser Developer Tool
Demo – Fiddler
Demo – Burp Suite
Http Protocol – Important Parts
MethodsMethod Used for
GET Retrieve a resource
POST Create / Update a resource [Not Idempotent]
PUT Create / Update a resource [Idempotent]
DELETE Delete a resource
HEAD Retrieve a resource except the body
Response CodesCode Meaning
1xx Informative
2xx Success
3xx Requires Additional Action
4xx Client Error (It is your fault)
5xx Server Error (It is my fault)
Accept (Req)
MIME used for media-type. Client gives hint about the types that it understands well and preference.
Syntax:
• Accept: <MIME_type>/<MIME_subtype>
Examples:
• Accept: application/json, text/xml;q=0.9, */*;q=0.8
Content-Type (Req / Resp)
MIME used for media-type
Examples:
• Content-Type: text/html; charset=utf-8 • Content-Type: application/json• Content-Type: text/xml
Demo – Accept and Content-Type
Host (Req)
• Hints the web server about the domain name requested• Optionally includes port, default• HTTP: 80• HTTPS: 443
Examples:
• Host: www.gokhansengun.com• Host: localhost:8090
Connection (Req / Resp)
• Hint from both client and the web server about TCP connection• close: if either party for some reason wants to close• keep-alive: if either party want to keep open for further
requests• Persistent connection (default in HTTP/1.1
• RFC 2616 limits 2 connection per host, browsers have 6 now.
Examples:
• Connection: close• Connection: keep-alive
BTW: Http Pipelining• Only Idempotent
requests allowed (GET, HEAD)• Guess why?
• Has benefit only on high latency setups.
Accept-Languge (Req)
• Hint from client about its language preference
Examples:
• Accept-Language: en-US,en;q=0.8• Accept-Language: tr-TR, tr;q=0.9, en;q=0.8, *;q=0.5
Demo – Accept-Language
Accept-Encoding (Req)
• Hint from client about its encoding preference
Examples:
• Accept-Encoding: Accept-Encoding: gzip, deflate, sdch• Omit for non-encoding
Demo – Accept-Encoding
Referer (Req)
• Hint from client about the last page user navigated from.• Allows analytics, caching, logging
Examples:
• Referer: http://ads.xyz.com
User-Agent (Req)
• Hint from client about the type of client
Examples:
• User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36
Cache-Control (Req / Resp)
• Hint from server to all over the world about resource’s cache eligibility.
• Cache-Control: no-cache• Cache-Control: public• Cache-Control: private• Cache-Control: no-store• Cache-Control: max-age=300• Cache-Control: public, max-age=31536000
Post / Redirect / Get Pattern (1)• Problem (Multiple Post requests)
Post / Redirect / Get Pattern (2)
Post / Redirect / Get Pattern (3)• Solution
Demo – Mix
HTTP Players• Web Servers• Load Balancers• DDoS Protection and WAF Systems• Cache Server• CDN (Content Delivery Networks)• Cloudflare
Web Servers• Nginx• Apache• IIS
Load Balancers• Balance HTTP load between servers• Balance statefully (needs your SSL private key)• Cache responses• Alters requests and responses• Blocks, rate-limits requests• Does SSL-offloading (needs your SSL private key and
beneficial only if you have HW LB)
DDoS Protection Systems and WAF• Observes traffic (needs your SSL private key)• Detects malicious activity – several attacks• Blocks IP, IP Range• Redirects to No CAPTCHA or reCAPTCHA• Rate-limits requests
Cache Servers• Caches any type of HTTP responses from origion• Could be static file or reference data• Like very very simple KV store• Powerful if scripting allowed
Examples:
• Varnish• Nginx
CDN (Content Delivery Network)• Caches the content on the edges• Request does not enter your data center• Very very efficient
Cloudflare• CDN• Load Balancing (Cloud – Region Based through DNS)• DDoS• WAF• Rate Limiting• Website Optimization• Cache Header Optimization• AutoMinify• Aggressive Gzip• Automatic Content Caching
Cookies• Helps stateless HTTP protocol statefulness when
necessary,• Has restrictions in EU.
Types:
• Session Cookies• Persistent Cookies
Authentication and Tokens• Basic Authentication• Forms Authentication• Token Authentication
Session Cookie vs Token Auth
HTTP Security• Use SSL/TLS for transport layer security (HTTPS
everything)• Why?
• Set Cookies with HttpOnly• Avoid Cross Site Scripting
• Set Cookies with Secure• Avoid sending cookies in HTTP requests
• Use HSTS (HTTP Strict Transport Security) header• Instruct browser to comm only with HTTPS for a period of time• Avoid SSL-stripping attacks
HTTP Performance Measurement• Use Apache ab• Use Apache JMeter (blogs from
www.gokhansengun.com)• http://loader.io/• https://www.blazemeter.com/• Use APM (Application Performance Monitoring) tools• NewRelic, Dynatrace, Riverbed, App
Scaling HTTP• Use Cache Server• Use CDN• Cache Aggressively• Use DNS load balancing• Use SPA (Single Page Application) Technique• Minify and bundle JS / CSS
Questions?
Top Related