CGI

Dynamic Web Content:

HTML Forms

CGI

Web Servers and HTTP

Duncan Temple Lang

Dept. of Statistics

UC Davis

Dynamic Content

We are all used to fetching pages from a Web server.

Most are prepared by a human and put on the Web site

for others to read. Each reader sees the same page.

The Web browser (client) can perform client-side

computations on the page to present it differently.

And the content can be dynamic via Javascript code in

the page that operates as the page is being rendered

(e.g. layout) and as it is being viewed (e.g. mouse

operations)

We, however, are interested in server-side dynamic

content.

Server-side Dynamic Pages

A request to the Web server returns a new, machine-

generated page,

e.g. google’ search result page,

travelocity’s ﬂight purchase page, ...

The server receives the request and executes some

code to create a new HTML document (or other type)

and returns that.

What makes this interesting is that the request can

specify inputs that govern how the page is created.

This is very similar to calling an R function with

different inputs and generating output in the form of an

HTML page

Client-Server components

We are in a client-server setup, like with relational

databases.

One application acts as a client and connects to the

server and sends a request; the server sends a reply.

Our client is a browser and speciﬁcally an HTML form

that provides ways to specify inputs to the server.

Our “server” is an R script on the Web server that gets

the inputs and generates an HTML page based these

inputs.

To provide dynamic access to R functionality via the

Web, you need to create both pieces.

Outline

Demo of an HTML form

The elements of HTML form processing

HTML forms,

CGI scripts & CGIwithR package.

Security issues.

Basics of HTTP - how this all works.

Details for project.

accounts, server, Web server directory layout

Basic Demo

A basic simulation to illustrate the Central Limit

Theorem.

User speciﬁes inputs for

the sample size,

the distribution from which to sample,

the number of repetitions,

the statistic to compute for each sample,

whether to create numerical and/or graphical

summary.

...

<form method="GET"

action="/cgi-bin/R.cgi/simulation.R">

....

.....

<option value="rnorm">Normal

<option value="rbinom">Binomial

<option value="rpois">Poisson

</select>

.....

Histogram: <input type="checkbox" name="output"

value="histogram" checked="checked">

Summary: <input type="checkbox" name="output"

value="summary" checked="checked">

<input type="hidden"

name="showAttribution" value="TRUE">

.....

</form>

Basics of HTML Forms

Create a form in an HTML document with

Specify the action which is the location (URI) of the

script to run when the form is submitted.

Also specify how the requested is to be submitted:

GET or POST.

Also add enctype attribute if necessary (more later).

<INPUT> Tags

The elements of the form are created via

<INPUT type=””>, <SELECT>/<OPTION> and

<TEXTAREA> tags in the HTML form

Different type values

TEXT, PASSWORD, CHECKBOX, RADIO,

SUBMIT, RESET, IMAGE, HIDDEN, FILE.

HTML Form Tutorial (http://www.w3schools.com/html/

html_forms.asp) & another: these describe the different

characteristics of these HTML elements to control their

appearance and behavior.

O’Reilly books: HTML - The Deﬁnitive Guide (chap. 8),

and CGI Programming.

Key Aspects of <INPUT>...

Each form element has a name attribute.

This is the name of the parameter that is sent in the

HTTP request.

The selected value(s) for the element is sent as the

value(s) for that parameter.

In our example, get

n = “3”, output = c(“histogram”, “summary”),

distribution = “rnorm”, statistic = “median”.

The method attribute of <form> tag speciﬁes how these

name-values are sent to the Web server in the request.

The CGI script

When a FORM is submitted, a request is sent to the

Web server which identiﬁes the URI as a script to run.

The Web server calls that script passing it the inputs.

This mechanism is called the Common Gateway Interface

- CGI

Typically, CGI scripts are located in the a cgi-bin area

of the Web server’s ﬁle system.

Only these can be run via a Web request.

The cgi-bin directory is not directly accessible via a

Web browser, but in a directory parallel to the Web

documents.

Script Programming Languages

The scripts can be written in any language, e.g. C, Perl,

Python, shell or R.

Each language has utilities that make it easier to

process the inputs into usable `arguments’.

In R, there is a package called CGIwithR that makes

using R scripts via CGI signiﬁcantly easier.

In the cgi-bin/ directory, we place the R.cgi

and .Rproﬁle from that package.

Then, we write our R script, say myCGI.R and put that

into the cgi-bin/ directory.

Accessing the script

To make use of this script (myCGI.R) in the HTML form,

specify the script as in

This runs myCGI.R via the shell script R.cgi.

Note R.cgi is an existing ﬁle, not a directory.

You don’t need to do anything to use it.

All printed output (to the console/screen) generated in

the R script is treated as HTML and displayed in the

resulting document in the client Web browser,

i.e. all output from cat() and print().

source("simulate.S") # Load work function for simulation.

ans = simulate(as.integer(formData$n), as.integer(formData$NumRep),

formData$statistic, formData$distribution)

cat("<h2>Sample distribution of", formData$statistic,

" for n =", formData$n, "from ", formData$distribution, "</h2>")

# Generate numerical summary if requested.

if("summary" %in% formData$output) {

invisible(capture.output(library(R2HTML)))

HTML(summary(ans), file = stdout())

}

# Generate density plot if requested.

if("histogram" %in% formData$output) {

webPNG("hist.png", type = "jpeg", graphDir = "../htdocs/tmp/")

dens = density(ans)

hist(ans, prob = TRUE, ylim = c(0, max(dens$y)), main = "Density of

sample values")

points(dens, col = "red")

img(src = "hist.png", graphURLroot = "/tmp/")

}

Accessing the form inputs

How does the R CGI script get the inputs from the

form?

When the script is run, there is already an R object -

formData - that is a named list containing the elements

of the form.

formData$n yields “3” - convert it to a number with

as.integer(formData$n)

For the output, there may be multiple entries (e.g.

“output” and “summary”) so formData$output would be a

character vector of length 2 if both are selected.

R2HTML Package

The R2HTML can convert many types of R objects into

HTML text.

Load it in the R script via the command

invisible(capture.output(library(R2HTML)))

This hides any messages the package annoyingly

produces when it is loaded.

Then, to output an R object as HTML, use

HTML(obj, ﬁle = stdout())

e.g. HTML(summary(x), ﬁle = stdout())

R2HTML

If you don’t like what it produces, look at the

documentation to customize it

Or, generate the HTML content yourself from the R

object.

The CGIwithR package provides some basic functions to

do this: img(), tag() & untag(), br(), lf(), mailto(), linkto().

Also, the XML package has facilities for creating HTML/

XML.

R Graphics in CGI

Can’t display graphics in a regular R window.

Want to create JPEG, PNG, GIF, TIFF ﬁles and insert

them inline into the HTML document via the

element.

2 issues: graphics device, ﬁle system.

Might use jpeg() or png() graphics devices in R, as usual,

but unfortunately, these typically require a connection

to an X11 server. So can’t use these.

Instead, use other approaches. GDD or external program

ghostscript.

Image Directories

When the R script is running, the current working

directory is the cgi-bin/ directory.

We don’t want to create the graphics ﬁle there,

especially since that is not part of the document tree

for the Web server and so cannot be seen by clients.

So we create a directory, say Rimages/, under the

htdocs/ directory and arrange to write the ﬁles there.

When we write the <img src=> element in HTML, we

need to specify <img src=”Rimages/myFile.png”>

But when creating the graphical display, we are in a

different directory and so must write to ../htdocs/

Rimages

CGIwithR - webPNG(), img()

Using CGIwithR, you can create graphics using

webPNG(“ﬁlename”) and insert the image into the

HTML content using img(“ﬁlename”).

There are two global variables that control where the

image is created and the URI in which it can be

referenced: graphDir and graphURLroot.

These are set in your cgi-bin/.Rproﬁle, so you don’t have

to worry about them.

2 Additional Examples

2 similar examples that share the same script and can

even use the same form (with redundant elements).

Illustrate techniques for getting dataset via HTML form.

Perform a 1-variable bootstrap.

User speciﬁes the sample values, the statistic, the

number of boostrap samples to generate and what

output to create (histogram and/or numerical summary).

In this case, the user provides data - 2 approaches.

insert into a <textarea name=”values”> element

upload a ﬁle via <input type=”ﬁle” name=”dataFile”>

Bootstrap Example

In the <textarea> example, the formData$values contains

the text from that element. We get the values via

data = scan(textConnection(formData$values))

In the case of the ﬁle upload, we have

formData$dataFile then contains the contents of that

ﬁle, and we can use scan(textConnection()) again.

Differences

While the two examples are very similar, and the same

R script can be used which “gets” the data from

different HTML elements, the underlying CGI mechanism

is very different.

The <textarea> version can be done with

<form action=”GET”...

The <input type=”ﬁle”> must be done with

<form action=”POST” enctype=”multipart/form-data”...

In the ﬁrst case, the data come as

URI?name=value&name=value&....

distribution=rnorm&n=3&statistic=mean&NumRep=1000&output=histogram&outp

ut=summary&R%3A%3Ascript=simulate.S&showAttribution=TRUE

In the second case,

they arrive as

-----------------------------823378840143542612896544303

Content-Disposition: form-data; name="values"

-----------------------------823378840143542612896544303

Content-Disposition: form-data; name="dataFile";

ﬁlename="bootData"

Content-Type: application/octet-stream

102

104

106

108

-----------------------------823378840143542612896544303

Content-Disposition: form-data; name="statistic"

median

-----------------------------823378840143542612896544303

Content-Disposition: form-data; name="NumRep"

1000

-----------------------------823378840143542612896544303

Content-Disposition: form-data; name="output"

histogram

-----------------------------823378840143542612896544303

Content-Disposition: form-data; name="output"

summary

-----------------------------823378840143542612896544303

Content-Disposition: form-data; name="R::script"

simulate.S

-----------------------------823378840143542612896544303--

In addition to the different formats, where the data are

located is different for GET and POST forms.

For action = GET, the data are given in an environment

variable named QUERY_STRING.

For action = POST, they are available from standard

input.

CGIwithR hides these details and just presents

formData.

Debugging CGI scripts

Difﬁcult since not running interactively.

Get the script running within a regular R session,

e.g. as a call to a function ﬁrst

Print diagnostic information to standard output so it

goes into the resulting document.

Or cat(“some message”, ﬁle = stderr()) which sends it to

the Web server’s log ﬁle.

Run

tail -f logs/error_log

in a separate window when submitting the request.

HTTP Error Log

[Mon Nov 28 06:02:02 2005] [error] [client 127.0.0.1] Error in cat(list(...), file, sep,

fill, labels, append) : , referer: http://localhost:9764/s141group1/htdocs/

bootstrapFile.html

[Mon Nov 28 06:02:02 2005] [error] [client 127.0.0.1] \targument 3 not yet handled by

cat, referer: http://localhost:9764/s141group1/htdocs/bootstrapFile.html

[Mon Nov 28 06:02:02 2005] [error] [client 127.0.0.1] Execution halted, referer: http://

localhost:9764/s141group1/htdocs/bootstrapFile.html

[Mon Nov 28 06:02:03 2005] [error] [client 127.0.0.1] File does not exist: /Users/