Web, FTP, and Proxy

48
Web, FTP, and Proxy Web, FTP, and Proxy

description

Web, FTP, and Proxy. Web Service. Web Service. Three major techniques in WWW (World Wide Web) System HTML – HyperText Markup Language Mark-up the text and define presentation effect by HTML Tags. http://www.w3.org/ HTTP – Hyper-Text Transfer Protocol - PowerPoint PPT Presentation

Transcript of Web, FTP, and Proxy

Page 1: Web, FTP, and Proxy

Web, FTP, and ProxyWeb, FTP, and Proxy

Page 2: Web, FTP, and Proxy

Web ServiceWeb Service

Page 3: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

3

Web ServiceWeb Service

Three major techniques in WWW (World Wide Web) System• HTML – HyperText Markup Language

Mark-up the text and define presentation effect by HTML Tags. http://www.w3.org/

• HTTP – Hyper-Text Transfer Protocol Communication method between client and server, both browsers and web server

s have to follow this standard. HTTPS – secured version

• URL – Uniform Resource Locator Describe how to access an object shared on the Internet Format

– Protocol :// [ [ username [ :password ] @ ] hostname [ :port ] ] [ /directory ] [ /filename ]

Page 4: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

4

Web ServiceWeb Service– The Client-Server Architecture– The Client-Server Architecture

Client-server architecture• Web Server: Answer HTTP request

• Web Client: Request certain page using URL

ClientBrowser

Web Server2. 以 HTTP 協定送出 Reque

st

4. 以 HTTP 協定回覆 Response

1. 以 URL 描述索取的資源位置向 Server 發送要

3. 從 URL 描述的位置將 HTML 文件取出並回覆給 Client

5. 接收到 HTML 後由 Browser 解析後根據 HTML 描述定義

將資料呈現出來

Page 5: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

5

Web ServiceWeb Service– The HTTP Protocol (1)– The HTTP Protocol (1)

HTTP: Hypertext Transfer Protocol• RFCs: (HTTP 1.1)

http://www.faqs.org/rfcs/rfc2068.html

http://www.faqs.org/rfcs/rfc2616.html (Updated Version)

• Useful Reference: http://jmarshall.com/easy/http/

• A network protocol used to deliver virtually all files and other data on the World Wide Web.

HTML files, image files, query results, or anything else.

• Client-Server Architecture A browser is an HTTP client because it sends requests to an HTTP serv

er (Web server), which then sends responses back to the client.

Page 6: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

6

Web ServiceWeb Service– The HTTP Protocol (2)– The HTTP Protocol (2)

• Clients: ※ Send Requests to Servers

Action “path or URL” Protocal– Actions: GET, POST, HEAD

– Ex. GET /index.php HTTP/1.1

Headers– Header_Name: value

– Ex.

From: [email protected]

(blank line) Data …

• Servers: ※ Respond to the clinets

Status:– 200: OK

– 404: Not Found

– …

– Ex. HTTP/1.1 200 OK

Headers– Same as clients

– Ex.

Content-Type: text/html

(blank line) Data…

Page 7: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

7

Web ServiceWeb Service– The HTTP Protocol (3)– The HTTP Protocol (3)

Example:nabsd [/home/chwong] -chwong- telnet nabsd.cs.nctu.edu.tw 80Trying 140.113.17.215...Connected to nabsd.cs.nctu.edu.tw.Escape character is '^]'.GET / HTTP/1.0Host: nabsd.cs.nctu.edu.tw

HTTP/1.0 200 OKContent-Type: text/htmlAccept-Ranges: bytesETag: "1897433431"Last-Modified: Tue, 29 May 2007 06:25:04 GMTContent-Length: 94Date: Tue, 29 May 2007 06:25:06 GMTServer: lighttpd/1.4.15X-Cache: HIT from nabsd.cs.nctu.edu.twVia: 1.0 nabsd.cs.nctu.edu.tw:80 (squid/2.6.STABLE13)Connection: close

<html><body> <a href="http://nabsd.cs.nctu.edu.tw/~chwong/docs/"> haha </a></body></html>Connection closed by foreign host.

status

Headers

Data

action

Headers

Page 8: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

8

Web ServiceWeb Service– The HTTP Protocol (4)– The HTTP Protocol (4)

Get vs. Post (client side)• Get:

Parameters in URLGET http://nabsd.cs.nctu.edu.tw/get.php?a=1&b=3 HTTP/1.1

No data content Corresponding in HTML files

– Link URL: http://nabsd.cs.nctu.edu.tw/get.php?a=1&b=3– Using Form:

<form method=“GET” action=“get.php”> … </form>

• Post: Parameters in Data Content

POST http://nabsd.cs.nctu.edu.tw/post.php HTTP/1.1 Corresponding in HTML files

– Using Form:

<form method=“POST” action=“post.php”> … </form>

Page 9: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

9

Web ServiceWeb Service– The HTTP Protocol (5)– The HTTP Protocol (5)

HTTP Headers:• What HTTP Headers can do?

[Ref] http://www.cs.tut.fi/~jkorpela/http.html

Content information (type, date, size, encoding, …)Cache controlAuthenticationURL RedirectionTransmitting cookiesKnowing where client come fromKnowing what software client use…

Page 10: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

10

Web ServiceWeb Service– Static vs. Dynamic Pages– Static vs. Dynamic Pages

Static vs. Dynamic Pages

• Technologies of Dynamic Web Pages Client Script Language

– JavaScript, Jscript, VBScript Client Interactive Technology

– Java Applet, Flash, XMLHTTP,AJAX Server Side

– CGI– Languages: Perl, ASP, JSP, PHP, C/C++, …etc.

Static vs. Dynamic

Page 11: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

11

Web ServiceWeb Service– Virtual Hosting (1)– Virtual Hosting (1)

Virtual Hosting• Providing services for more than one domain-name (or IP) in one w

eb server.

• IP-Based Virtual Hosting vs. Name-Based Virtual Hosting IP-Base – Several IPs (or ports) Name-Base – Singe IP, several hostnames

• Example (Apache configuration)

<VirtualHost 140.113.17.215:80>DocumentRoot /www/nabsdServerName nabsd.cs.nctu.edu.tw</VirtualHost>

<VirtualHost 140.113.17.221:80>DocumentRoot /www/tphpServerName tphp.cs.nctu.edu.tw</VirtualHost>

NameVirtualHost 140.113.17.215

<VirtualHost 140.113.17.215>ServerName nabsd.cs.nctu.edu.twDocumentRoot "/www/na"</VirtualHost>

<VirtualHost 140.113.17.215>ServerName sabsd.cs.nctu.edu.twDocumentRoot "/www/sa"</VirtualHost>

Page 12: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

12

Web ServiceWeb Service– Virtual Hosting (2)– Virtual Hosting (2)

Q: How Name-Based Virtual Hosting works?

A: It takes use of HTTP Headers.

% telnet cswproxy.cs.nctu.edu.tw 80Trying 140.113.235.111...Connected to cswproxy.cs.nctu.edu.tw.Escape character is '^]'.GET / HTTP/1.0Host: www.cs.nctu.edu.tw

HTTP/1.0 200 OKDate: Tue, 05 Jun 2007 13:50:34 GMT

…………

<html><head><title>NCTU -- CS</title><META HTTP-EQUIV="Pragma" CONTENT="no-cache"><meta http-equiv="refresh" content="0; URL=chinese/doc/index.html"></head></html>Connection closed by foreign host.

% telnet cswproxy.cs.nctu.edu.tw 80Trying 140.113.235.111...Connected to cswproxy.cs.nctu.edu.tw.Escape character is '^]'.GET / HTTP/1.0Host: www.csie.nctu.edu.tw

HTTP/1.0 200 OKDate: Tue, 05 Jun 2007 13:51:01 GMT

…………

<html><head><title>NCTU -- CSIE</title><meta http-equiv="refresh" content="0; URL=http://www.cs.nctu.edu.tw/">

Connection closed by foreign host.

Page 13: Web, FTP, and Proxy

FTPFTP

File Transfer Protocol

Page 14: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

14

FTPFTP

FTP• File Transfer Protocol

• Used to transfer data from one computer to another over the internet.

• Client-Server Architecture.

• Separated control/data connections.

• Modes: Active Mode, Passive Mode

• RFCs: RFC 959 – File Transfer Protocol RFC 2228 – FTP Security Extensions RFC 2640 – UTF-8 support for file name

Page 15: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

15

FTPFTP– Flow (1)– Flow (1)

Client

• Connect to server port 21 using port A.

• USER ####

• PASS ********

• PORT h1,h2,h3,h4,p1,p2

• Send some requestsget return data from p1*256+p2

• Quit

Server• Binding on port 21

• Accepts connection from client, output welcome messages.

• 331 User name okay, need password.

• 230 User logged in, proceed.

• 200 PORT Command successful.

• Binding source port 20, connect to client port p1*256+p2, send data.

• …

Page 16: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

16

FTPFTP– Flow (2)– Flow (2)

Example• Control

Connection

% telnet chonsilab.dyndns.org 21Trying 140.113.215.86...Connected to chonsilab.dyndns.org.Escape character is '^]'.220 Serv-U FTP-Server v2.5k for WinSock ready...USER test331 User name okay, need password.PASS test230 User logged in, proceed.PORT 140,113,17,215,39,19200 PORT Command successful.LIST150 Opening ASCII mode data connection for /bin/ls.226 Transfer complete.quit221 Goodbye!Connection closed by foreign host.

Page 17: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

17

FTPFTP– Flow (3)– Flow (3)

Example (contd.)• Retrieving Data

Client must bind the random port

%perl server.pl 100032007/06/06-13:16:08 MyPackage (type Net::Server::PreFork) starting! pid(4346)Binding to TCP port 10003 on host *Group Not Defined. Defaulting to EGID '1000 110 100 80 0 1000 1000'User Not Defined. Defaulting to EUID '1001'-rwxrwxrwx 1 user group 0 Sep 11 2005 AUTOEXEC.BAT-rwxrwxrwx 1 user group 209 Sep 11 2005 boot.ini-rwxrwxrwx 1 user group 213830 Mar 25 2005 bootfont.bin-rwxrwxrwx 1 user group 0 Sep 11 2005 CONFIG.SYSdrwxrwxrwx 1 user group 0 Apr 8 17:30 Documents and Settings-rwxrwxrwx 1 user group 0 Sep 11 2005 IO.SYS-rwxrwxrwx 1 user group 0 Sep 11 2005 MSDOS.SYS-rwxrwxrwx 1 user group 47772 Mar 25 2005 NTDETECT.COM-rwxrwxrwx 1 user group 304752 Mar 25 2005 ntldrdrwxrwxrwx 1 user group 0 May 21 23:30 Program Filesdrwxrwxrwx 1 user group 0 Aug 19 2006 RECYCLERdrwxrwxrwx 1 user group 0 Feb 16 2006 System Volume Informationdrwxrwxrwx 1 user group 0 May 28 16:45 WINDOWS

% cat server.pl#!/usr/bin/perl -w

package MyPackage;use strict;use base qw(Net::Server::PreFork);MyPackage->run(port => $ARGV[0]);

sub process_request { while (<STDIN>) { s/\r?\n$//; print STDERR "$_\n"; }}

Page 18: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

18

FTPFTP– commands, responses– commands, responses

Commands• USER username

• PASS password

• LIST Return list of file in current dir.

• RETR filename Retrieves (gets) file.

• STOR filename Stores (puts) file onto server.

• PORT h1,h2,h3,h4,p1,p2 Set to active mode

• PASV Set to passive mode

• DELE Remove file on the server.

• QUIT

Return Codes• First code

1: Positive Preliminary reply

2: Positive Completion reply

3: Positive Intermediate reply

4: Transient Negative Completion reply

5: Permanent Negative Completion reply

• Second code0: The failure was due to a syntax error

1: A reply to a request for information.

2: A reply relating to connection information

3: A reply relating to accounting and authorization.

5: The status of the Server file system

Page 19: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

19

FTPFTP– Active Mode vs. Passive Mode (1)– Active Mode vs. Passive Mode (1)

Active Mode• FTP client bind a random port (>1023) and sends the random port to FTP

server using “PORT” command.• When the FTP server initiates the data connection to the FTP client, it binds

the source port 20 and connect to the FTP client the random port sent by client.

• PORT h1,h2,h3,h4,p1,p2 Passive Mode

• FTP client sends “PASV” command to the server, make the server bind a random port (>1023) and reply the random port back.

• When initializing the data connection, the FTP client connect to the FTP Server the random port, get data from that port.

• PASV Server reply: 227 Entering Passive Mode (h1,h2,h3,h4,p1,p2)

※ IP:port (6bytes) h1,h2,h3,h4,p1,p2Ex. 140.113.17.215:45678 140,113,17,215,178,110

Page 20: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

20

FTPFTP– Active Mode vs. Passive Mode (2)– Active Mode vs. Passive Mode (2)

Active mode Passive mode

Page 21: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

21

FTPFTP– When FTP meets NAT/Firewall (1)– When FTP meets NAT/Firewall (1)

Firewall behavior• Generally, the NAT/Firewall permits all outgoing connection from internal

network, and denies all incoming connection from external network.

Problem when FTP meets NAT/Firewall• Due to the separated command/data connection, the data connections are

easily blocked by the NAT/Firewall.

Problem Cases:• Active mode, NAT/Firewall on client side.

Passive mode can solve this problem.

• Passive mode, NAT/Firewall on server side. Active mode can solve this problem.

• Both client side and server side have NAT/Firewall The real problem.

Page 22: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

22

FTPFTP– When FTP meets NAT/Firewall (2)– When FTP meets NAT/Firewall (2)

Active mode, NAT/Firewall on client side.• Passive mode can solve this problem.

Client ServerNAT/Firewall

PORT IP, port Y

Connect to port Y

BLOCKED

Active Mode

Client ServerNAT/Firewall

PASVreply IP, port Z

Connect to port ZPASS

Passive Mode

Page 23: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

23

FTPFTP– When FTP meets NAT/Firewall (3)– When FTP meets NAT/Firewall (3)

Passive mode, NAT/Firewall on Server side.• Active mode can solve this problem.

Client Server

NAT/Firewall

PORT IP, port Y

Connect to port Y

PASS

Active Mode

Client ServerNAT/Firewall

PASVreply IP, port Z

Connect to port ZBLOCKED

Passive Mode

Page 24: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

24

FTPFTP– When FTP meets NAT/Firewall (4)– When FTP meets NAT/Firewall (4)

Real Problem: Firewall on both sides.

• Solution: ftp-proxy running on NAT/Firewall

Client Server

NAT/Firewall

PORT IP, port Y

Connect to port Y

BLOCKED

Active Mode

NAT/Firewall

Client Server

NAT/Firewall

PASVreply IP, port Z

Connect to port ZBLOCKED

Passive Mode

NAT/Firewall

Page 25: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

25

FTPFTP– Security– Security

Security concern• As we seen, FTP connections (both command and data) are

transmitted in clear text.• What if somebody sniffing the network?

We need encryption.

Solutions• FTP over SSH

So called secure-FTP. Both commands and data are encrypted while transmitting. Poor performance.

• FTP over TLS Only commands are encrypted while transmitting. Better performance.

Page 26: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

26

FTPFTP– Pure-FTPd (1)– Pure-FTPd (1)

Introduction• A small, easy to set up, fast and secure FTP server

• Support chroot

• Restrictions on clients, and system-wide.

• Verbose logging with syslog

• Anonymous FTP with more restrictions

• Virtual Users, and Unix authentication

• FXP (File eXchange Protocol)

• FTP over TLS

• UTF-8 support for file names

Page 27: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

27

FTPFTP– Pure-FTPd (2)– Pure-FTPd (2)

Installation• Ports: /usr/ports/ftp/pure-ftpd

• Options

Page 28: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

28

FTPFTP– Pure-FTPd (3)– Pure-FTPd (3)

• Other options

• WITH_CERTFILE for TLS Default: /etc/ssl/private/pure-ftpd.pem

• WITH_LANG Change the language of output messages

Startup:• Add pureftpd_enable=“YES” into /etc/rc.conf

Page 29: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

29

FTPFTP– Pure-FTPd Configurations(1)– Pure-FTPd Configurations(1)

Configurations:• File: /usr/local/etc/pure-ftpd.conf

• Documents Configuration sample: /usr/local/etc/pure-ftpd.conf.sample

– All options are explained clearly in this file.

Other documents– See /usr/local/share/doc/pure-ftpd

nabsd [/usr/local/share/doc/pure-ftpd] -chwong- lsAUTHORS README README.MySQL THANKSCONTACT README.Authentication-Modules README.Netfilter pure-ftpd.pngCOPYING README.Configuration-File README.PGSQL pureftpd.schemaHISTORY README.Contrib README.TLSNEWS README.LDAP README.Virtual-Users

Page 30: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

30

FTPFTP– Pure-FTPd Configurations(2)– Pure-FTPd Configurations(2)

# Cage in every user in his home directoryChrootEveryone yes

# If the previous option is set to "no", members of the following group# won't be caged. Others will be. If you don't want chroot()ing anyone,# just comment out ChrootEveryone and TrustedGID.TrustedGID 0

# PureDB user database (see README.Virtual-Users)PureDB /etc/pureftpd.pdb

# If you want simple Unix (/etc/passwd) authentication, uncomment thisUnixAuthentication yes

# Port range for passive connections replies. - for firewalling.PassivePortRange 30000 50000

# This option can accept three values :# 0 : disable SSL/TLS encryption layer (default).# 1 : accept both traditional and encrypted sessions.# 2 : refuse connections that don't use SSL/TLS security mechanisms,# including anonymous sessions.# Do _not_ uncomment this blindly. Be sure that :# 1) Your server has been compiled with SSL/TLS support (--with-tls),# 2) A valid certificate is in place,# 3) Only compatible clients will log in.TLS 2

# UTF-8 support for file names (RFC 2640)# Define charset of the server filesystem and optionnally the default charset# for remote clients if they don't use UTF-8.# Works only if pure-ftpd has been compiled with --with-rfc2640FileSystemCharset big5# ClientCharset big5

Page 31: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

31

FTPFTP– Pure-FTPd Problem Shooting– Pure-FTPd Problem Shooting

Logs Location• In default, syslogd keeps ftp logs in /var/log/xferlog

• Most frequent problem pure-ftpd: (?@?) [ERROR] Unable to find the 'ftp' account

– It’s ok, but you may need it for Virtual FTP Account.

pure-ftpd: (?@?) [ERROR] Sorry, but that file doesn't exist: [/etc/ssl/private/pure-ftpd.pem]

– If you set TLS = 2, then this file is needed.

How to generate a pure-ftpd.pem?– See README.TLS

Page 32: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

32

FTPFTP– – Pure-FTPd ToolsPure-FTPd Tools

pure-*

pure-ftpwho• List information of users who use the FTP server now.

pure-pw• To create Virtual Users using PureDB

• man pure-pw

• See README.Virtual-Users

Page 33: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

33

FTPFTP– – PF: PF: Issues with FTP (1)Issues with FTP (1)

Reference: Reference: http://http://www.openbsd.org/faq/pf/ftp.htmlwww.openbsd.org/faq/pf/ftp.html FTP Client Behind the FirewallFTP Client Behind the Firewall

• Problem Clients cannot use active mode

• Use ftp-proxy Use inetd to start ftp-proxy man ftp-proxy

• In pf.conf nat-anchor “ftp-proxy/*” rdr-anchor “ftp-proxy/*” rdr on $int_if proto tcp from any to any port 21 -> 127.0.0.1 \ port

8021 anchor “ftp-proxy/*”

Page 34: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

34

FTPFTP– – PF: PF: Issues with FTP (2)Issues with FTP (2)

PF “Self-Protecting” an FTP ServerPF “Self-Protecting” an FTP Server • Problem

Clients cannot use passive mode

• Open holes so that clients can connect into the data channel

• In pf.conf pass in on $ext_if proto tcp from any to any port 21 keep state pass in on $ext_if proto tcp from any to any port > 49151 keep state

Page 35: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

35

FTPFTP– – PF: PF: Issues with FTP (3)Issues with FTP (3)

FTP Server Protected by an External PF Firewall Running NAFTP Server Protected by an External PF Firewall Running NATT • Problem

Clients cannot use passive mode

• Use ftp-proxy Need some flags of ftp-proxy man ftp-proxy

• In pf.conf nat-anchor “ftp-proxy/*” nat on $ext_if inet from $int_if -> ($ext_if) rdr-anchor “ftp-proxy/*” pass in on $ext_if inet proto tcp to $ext_ip port 21 flags S/SA keep state pass out on $int_if inet proto tcp to $ftp_ip port 21 user proxy flags S/SA ke

ep state anchor “ftp-proxy/*”

Page 36: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

36

FTPFTP– More Tools– More Tools

/usr/ports/ftp/pftpx• Another ftp proxy daemon

/usr/ports/ftp/lftp• A powerful functional client

• Support TLS

FileZilla• An FTP Client for Windows

• Support TLS

Page 37: Web, FTP, and Proxy

ProxyProxy

Page 38: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

38

ProxyProxy

Proxy• A proxy server is a server which services the requests of its clients by:

Making requests to other servers Caching some results for further same requests

• Goals: Performance Stability Central Control …etc.

• Roles: Forward Proxy Reverse Proxy

• Targets Web/FTP Pages TCP/IP Connections …etc.

RequestReply

Request

Reply

(using cached result)

Request

Replyclient

client

ProxyServer

OriginalServer

Page 39: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

39

ProxyProxy– The Forward Proxy– The Forward Proxy

Forward Proxy• Proxy the outgoing requests, for the reason of

Bandwidth saving Performance Central control

• When objects requested are In cache, return the cached objects Otherwise, proxy server requests object from origin server, then cache it and

return to client

RequestReply

Request

Reply

(using cached result)

Request

Replyclient

client

ProxyServer

OriginalServer

Page 40: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

40

ProxyProxy– The Reverse Proxy– The Reverse Proxy

Reverse Proxy• Proxy the incoming requests, for the reason of

Reducing Server Load (by caching) Load Balance Fault Tolerant

• Reverse proxy acts as the original server, accept incoming requests, reply corresponding result. SEAMLESS for clients!

Request

Replyclient

client

ReverseProxyServer

Server1RequestReply

Server1

InternetInternet

Page 41: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

41

ProxyProxy– SQUID– SQUID

A web proxy server & cache daemon.• Supports HTTP, FTP• Limited support for TLS, SSL, Gopher, HTTPS• Latest stable version: 2.6-STABLE13, 2007/5/11

Port install: /usr/ports/www/squid Startup:

• /etc/rc.conf squid_enable="YES" squid_config="/usr/local/etc/squid/squid.conf" squid_user="squid"

• /usr/local/etc/rc.d/squid start

Configuration Sample/Documents:• /usr/local/etc/squid/squid.conf.default

Page 42: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

42

ProxyProxy– SQUID Configuration (1)– SQUID Configuration (1)

Listen Port• Service Port

http_port 3128

• Neighbored Communication icp_port 3130

Logs• access_log

access_log /var/log/squid/access.log squid

• cache_log cache_log /var/log/squid/cache.log

• cache_store_log cache_store_log /var/log/squid/store.log

Page 43: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

43

ProxyProxy– SQUID Configuration (2)– SQUID Configuration (2)

Access Control• acl – define an access control list

Format: acl acl-name acl-type dataacl all src 0.0.0.0/0.0.0.0

acl NCTU srcdomain .nctu.edu.tw

acl YAHOO dstdomain .yahoo.com

acl allowhost src “/usr/local/etc/squid.squid.allow”

• http_access – define the control rule Format: http_access allow|deny acl-name

http_access allow NCTU

http_access allow allowhost

http_access deny all

Page 44: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

44

ProxyProxy– SQUID Configuration (3)– SQUID Configuration (3)

Proxy Relationship• Protocol: ICP (Internet Cache Protocol)

RFC 2186 2187, using UDP

• Related Configuration cache_peer hostname type http_port icp_port [options] cache_peer_domain cache-host domain [domain …] cache_peer_access cache-host allow|deny acl-name

Page 45: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

45

ProxyProxy– SQUID Configuration (4)– SQUID Configuration (4)

Cache Control• cache_mem 256 MB

• cache_dir ufs /usr/local/squid/cache 100 16 256

• cache_swap_low 93

• cache_swap_high 98

• maximum_object_size 4096 KB

• maximum_object_size_in_memory 8 KB

Page 46: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

46

ProxyProxy– SQUID Configuration (5)– SQUID Configuration (5)

Sample: Proxy Configuration

http_port 3128icp_port 3130

cache_mem 32 MBcache_dir ufs /usr/local/squid/cache 100 16 256

access_log /var/log/squid/access.log squidcache_log /var/log/squid/cache.logcache_store_log /var/log/squid/store.logpid_filename /usr/local/squid/logs/squid.pid

visible_hostname nabsd.cs.nctu.edu.twacl allowhosts src "/usr/local/etc/squid/squid.allow“http_access allow allowhostshttp_access deny all

Page 47: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

47

ProxyProxy– SQUID Configuration (6)– SQUID Configuration (6)

Sample: Reverse Proxy Configuration

http_port 80 vhosticp_port 3130

cache_mem 32 MBcache_dir ufs /usr/local/squid/cache 100 16 256

access_log /var/log/squid/access.log squidcache_log /var/log/squid/cache.logcache_store_log /var/log/squid/store.logpid_filename /usr/local/squid/logs/squid.pid

visible_hostname nabsd.cs.nctu.edu.twurl_rewrite_program /usr/local/squid/bin/redirect.placl cswww dstdomain csws1 csws2http_access allow all cswwwalways_direct allow cswww

Page 48: Web, FTP, and Proxy

Com

pu

ter C

en

ter, C

S, N

CTU

48

ProxyProxy– SQUID Configuration (7)– SQUID Configuration (7)

% cat /usr/local/squid/bin/redirect.pl

#!/usr/bin/perl

$|=1; # use non-blocking I/O

while(<STDIN>){ if (/^http:\/\/www\.cs\.nctu\.edu\.tw\/([^\s]*)/) { my $ran = int(rand(2)+1); print "http://csws$ran.cs.nctu.edu.tw/$1\n"; next; } print "\n";}