1

‘Dynamic’ Web Pages

• So far, we have developed ‘static’ web-pages, e.g., cv.html, repair.html and order.html.

• There is often a requirement to produce dynamic HTML content based on some data received from a web browser (the client).

• Think of how web search engines function …

2

Common Gateway Interface

• CGI is a simple, standard mechanism for: – Associating URLs with programs that can be run

by a web server (on behalf of a client).– A protocol (set of rules) for how the request is

passed to the external program.– How the external program sends the response to

the web server (which then sends the response to a client).

3

The CGI Mechanism

CLIENT

HTTP(WEB)

SERVER

CGI Program

HTTP request

HTTP response HTML

CGI invocation

4

CGI URLs

• There is some mapping between URLs and CGI programs provided by a web server. The exact mapping is not standardized (it differs from web server to web server).

• Typically:– URL requests that start with /CGI-BIN/, /cgi-bin/ or /cgi/, and so on, refer to CGI programs (i.e., not to static documents).

• CGI programs are often called ‘scripts’.

5

Request CGI program

• The web server sets some environment variables with information about the request.

• The web server starts up (or invokes) the CGI program.

• The CGI program gets information about the request from the environment variables. (In Perl, this information is available within the %ENV hash).

6

The role of STDIN and STDOUT

• Before starting the CGI program, the web server sets up ‘pipes’ so that the scripts’ input (STDIN) comes from the web server and its output (STDOUT) goes to the web server.

• In some cases part of the request can be read from STDIN (as opposed to %ENV) .

• Anything written to STDOUT by the CGI script is sent back to the web server, then forwarded to the client (i.e., the web browser). In this way, the user gets to see the output from the CGI script.

7

HTTP(WEB)

SERVERCGI Program

STDIN (input)

STDOUT (output)

EnvironmentVariables

8

Important CGIEnvironment Variables (Perl)

REQUEST_METHOD

$ENV{REQUEST_METHOD}

QUERY_STRING $ENV{QUERY_STRING}

CONTENT_LENGTH

$ENV{CONTENT_LENGTH}

9

Request Method: GET

• GET requests can include a query string as part of the URL:

GET /cgi-bin/finger?hollingd HTTP/1.0

RequestMethod

ResourceName

Delimiter

QueryString

10

/cgi-bin/finger?hollingd

• The web server treats everything before the ‘?’ delimiter as the resource name.

• In this case the resource name is the name of a program - finger.

• Everything after the ‘?’ is a string that is passed to the CGI program – “hollingd”.

11

HTTP Response and Headers:

• The CGI program can send back a HTTP status line (optional) to the web server:– print "HTTP/1.1 200 OK\r\n”;

• and headers (which are mandatory):– print Content-type: text/html\r\n”;– print “\r\n”;

12

To Emphasize• A CGI program doesn’t have to send a status line

(the HTTP server will do this for you if you don’t).

• A CGI program must always send back at least one header line indicating the data type of the content (usually text/html but text/plain is also popular).

• The web server will typically throw in a few header lines of it’s own (Date, Server, Connection). You do not need to worry about these (as they are added automatically).

13

Form Fields (with GET)

• Each field within an HTML form (<FORM>) has a name and a value (refer to your own order.html).

• The browser creates a query that includes a sequence of “name=value” sub-strings and sticks them together separated by the ‘&’ character.

14

Form Fields and Encoding

• For example: assume 2 fields - name and occupation.

• If user types in “Dave Holling.” as the name and “none” for occupation, the query would look like this (note: the URL encoding):

“name=Dave+Holling%2E&occupation=none”

15

HTML Forms

• Each HTML form includes a METHOD that determines what HTTP technique is used to submit the request (from the web browser to the web server).

• Each form includes an ACTION attribute that determines to where the request is made (i.e., used to identify the CGI script to send to).

16

An Example HTML Form

<FORM METHOD=GET

ACTION=http://localhost/cgi-bin/testcgi>

Name: <INPUT TYPE=TEXT NAME=name><BR>

Occupation: <INPUT TYPE=TEXT NAME=occupation><BR>

<INPUT TYPE=SUBMIT>

</FORM>

17

The Form Displayed in “Internet Explorer”

18

What a CGI Script will Receive

• The query (from the environment variable QUERY_STRING) will be a URL-encoded string containing the name/value pairs of all the form fields.

• The CGI must decode the query and separate the individual fields ….

• However, if the CGI module in Perl is used instead, this is done for you, so it couldn’t be easier.

19

HTTP Method: POST

• The HTTP POST method delivers data from the browser as the “content” of the request.

• The GET method delivered data as part of the URL.• When using forms, it’s generally better to use POST,

because:– there are limits on the maximum size of a GET query string

(environment variable), usually 256 bytes/characters.

– a POST query string doesn’t show up in the browser as part of the current URL (so it looks slightly better).

20

HTML Form using POST

• Simply set the form method to POST instead of GET:

<FORM METHOD=POST ACTION= … >

Everything else remains the same.

The web browser will take care of all the details...

21

CGI and POST

• If REQUEST_METHOD is a POST, the query is coming in STDIN. That is, the CGI script should read from STDIN to determine the input to process from the web server.

• The environment variable CONTENT_LENGTH tells us how much data to read.

• Again, when using Perl, this too is taken care of for us. So, once again, it could not be easier.

22

CGI Method Summary

• GET:– REQUEST_METHOD is “GET”.– QUERY_STRING is the query.

• POST:– REQUEST_METHOD is “POST”.– CONTENT_LENGTH is the size of the query

(in bytes).– query can be read from STDIN by the script.

23

Key Point (for Perl programmers)

• No matter which method chosen by the author of the web page, if Perl’s CGI technology is employed, the details are taken care of for you. This is cool.

• This helps to explain Perl’s dominance in this area. Estimates range from 50-70% of all Dynamic Web Page Creation on the WWW is performed by Perl CGI scripts.

24

CGI.pm

• A standard Perl module (which means it is already installed on pbmac.itcarlow.ie).

• Simply use the module at the top of your Perl CGI scripts, as follows:

#! /usr/bin/perl –w

use CGI;

25

Processing Names and Occupations.

• This script simply echoes back the data received from the web browser:

#! /usr/bin/perl -w

use CGI;

$q = new CGI;

print $q->header,

$q->start_html(‘Names and Occupations'),

$q->h1(‘You Sent:');

print $q->param(‘name’), “ works as ”,

$q->param(‘occupation’), “\n”,

$q->end_html;

26

Testing the CGI Script: Output Generated

Content-Type: text/html; charset=ISO-8859-1

<?xml version="1.0" encoding="utf-8"?>

<!DOCTYPE html

PUBLIC "-//W3C//DTD XHTML Basic 1.0//EN"

"http://www.w3.org/TR/xhtml-basic/xhtml-basic10.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" lang="en-US"><head><title>Names and Occupations</title>

</head><body><h1>You Sent:</h1> works as

</body></html>

27

Response Generated using GET

28

Response Generated using POST

29

Completely Dynamic Websites

• It is possible to build a completely dynamic website using the Perl CGI technology.

• In fact, is it possible to use the same CGI script to both display an HTML FORM and then subsequently process it.

• Consider the following CGI program based on the opening example from the CGI.pm documentation …

30

#! /usr/bin/perl

use CGI;

my $q = new CGI;

print $q->header, $q->start_html( 'A Simple CGI Example' ), $q->h1( 'A Simple CGI Example' ), $q->start_form, "What's your name? ", $q->textfield( 'name' ), $q->p, "What's the keyword combination? ", $q->checkbox_group( -name => 'words', -values => [ 'eenie', 'meenie', 'minie', 'moe' ], -defaults => ['eenie','minie' ] ), $q->p, "What's your favorite color? ", $q->popup_menu( -name => 'color', -values => [ 'red', 'green', 'blue', 'chartreuse' ] ), $q->p, $q->submit, $q->end_form, $q->hr;

31

if ( $q->param() )

{

print "Your name is: ",

$q->em( $q->param( 'name' ) ),

$q->p,

"The keywords are: ",

$q->em( join( ", ", $q->param( 'words' ) ) ),

$q->p,

"Your favorite color is: ",

$q->em( $q->param( 'color' ) ),

$q->hr;

}

$q->end_html;

32

http://localhost/cgi-bin/simple

33

Fill in Some Details, then Click ‘Submit’

34

Processed Results