In the Python CGI programming, we will learn how we can run the Python Script on the web; we will learn how Python file can be executed as CGI Script and discuss the its configuration of web-browser to Python script run as a CGI. Also, we will learn the following topics:
- What is CGI?
- Web Browsing
- Configure Apache Web server for CGI
- Using the cgi Module
- CGI Architecture Diagram
- Running First Python Program as CGI
- Legends and syntaxes
- Debugging of Python CGI Script
- Functions of CGI
- Advantages of CGI
- Disadvantages of CGI
- Common Problems and Solutions
The Python Wiki. Welcome to the Python Wiki, a user-editable compendium of knowledge based around the Python programming language. Some pages are protected against casual editing - see WikiEditingGuidelines for more information about editing content. Python CGI programming. A typical HTML form. A typical CGI script. CGI script structure. Structure refinement. FieldStorage details. Other CGI niceties. Dealing with bugs. Debugging framework. Using persistent data. Plain files (G)DBM files. A simple lock solution. However, you can combine this technique with the cgi.escape. XML entities are different from, if related to, HTML entities. This page hints at the details: EscapingXml. Lee discusses still more refinements in implementation in this comp.lang.python follow-up. Probably easier if you write your cgi in python then call your python script from the cgi script. Update your script to separate the UI from the logic. Then it should be relatively easy to interface your script with the (python) cgi script. For python cgi reference: Five minutes to a Python CGI.
What is CGI?
The word CGI is the acronyms of the 'Common Gateway Interface', which is used to define how to exchange information between the web server and a custom script. The NCSA officially manages the CGI scripts.
The Common Gateway Interface is a standard for external gateway programs to interface with the server, such as HTTP Servers.
In simple words, it is the collection of methods used to set up a dynamic interaction between the server, and the client application. When a client sends a request to the webserver, the CGI programs execute that particular request and send back the result to the webserver.
The users may submit the information in web browser by using HTML<form> or <isindex> Champagne aoc map. element. There is a server's special directory called cgi-bin, where cgi script is generally stored. When a client makes a request, the server adds the additional information to request.
This additional information can be the hostname of the client, the query string, the requested URL, and so on. The webserver executes and sends the output to the web browser (or other client application).
Sometimes the server passes the data by a query string that is the part of the URL. A query string may not be conventionally a hierarchical path structure such as the following link.
Python provides the CGI module, which helps to debug the script and also support for the uploading files through a HTML form.
So here the question arises what does a Python CGI script output look like? The HTTPs server gives back the output as two sections separated by a blank line. The first section grasps the number of headers, notifying the client what kind of data is following.
Let's understand the following example of generate the minimal header section in the Python CGI programming.
Example -
In the above example, the first statement says that, the server that html code follows; the second blank line indicates the header is ended here. Let's understand the following example of generating the minimal header section in the Python CGI programming.
Example -
Web Browsing
Before understanding the CGI concepts, we should know the internal process of web page or URL when we click on the given link.
- The client (web browser) communicates with the HTTP server and asks for the URL i.e., filename.
- If the web browser finds that requested file, then it sends back to the client (web browser), otherwise it sends the error message to the client as error file.
- Here the responsibility of the web browser to display either the received file or an error message.
However, we can set a HTTP server so that whenever user requests in a particular dictionary, then it should be sent to the client; instead, it executed as a program and whatever the result is sent back for the client to display. This process is called the Common Gateway Interface or CGI and the programs are called CGI scripts. We can write the CGI programs as Python Script, PERL, Script, Shell Script, C or C++, programs, etc.
Configure Apache Web server for CGI
We need to configure the Apache Web server in order to run the CGI script on the server.
CGI Architecture
Using the cgi module
Python provides the cgi module, which consists of many useful built-in functions. We can use them by importing the cgi module.
Now, we can write further script.
The above script will stimulate an exception handler to show the detailed report in the web browser of occurred errors. We can also save the report by using the following script.
The above feature of the cgi module is helpful during the script development. These reports help us to debug the script effectively. When we get the expected output, we can remove this.
Previously, we have discussed the users save information using the form. So how can we get that information? Python provides the FieldStorage class. We can apply the encoding keyword parameter to the document if the form contains the non-ASCII character. We will find the content <META> tag in the <HEAD> section in our HTML document.
The FieldStorage class reads the form information from the standard input or the environment.
A FieldStorage instance is the same as the Python dictionary. We can use the len() and all dictionary function in the FieldStorage instance. It overlooks fields with empty string values. We can also consider the empty values using the optional keyword parameter keep_blank_values by setting True.
Example -
In the above example, we have used the form ['name'], here name is key. This is used to extract the value which is entered by the user.
We can use the getvalue() method to fetch the string value directly. This function also takes an optional second argument as a default. If the key is not present, it returns the default value.
If the submitted form data have more than one field with the same name, we should use the form.getlist() function. It returns the list of strings. Look at the following code, we add the any number of username field, which is separated by commas.
If the field is uploaded file, then it can be accessed by value attribute or the getvalue() method and read that uploaded file in bytes. Let's understand the following code if user upload the file.
Example -
Sometimes an error can interrupt the program while reading the content of the uploaded file (When a user clicks on Cancel Button or Back Button). FieldStorage class provides the done Panda sketch. attribute to set to the value -1.
If we submit the form in the 'old' format, the item will be instances of the class MiniFieldStorage. In this class, the list, file, and filename attributes are always None.
Generally, the form is submitted via POST and contains a query string with both FieldStorage and MiniStorage items.
Here, we are defining the FieldStorage attribute in the following table.
Attributes | Description |
---|---|
Name | It represents the field name. |
Filename | It represents Client side filename. |
File | It is a file(-like) object from which we can read data as bytes. |
Value | It is a string type value. Use for file uploads, reads the file and returns bytes. |
Type | It is used to display the content-type. |
Header | It is a dictionary type object which contains all headers. |
The FieldStorage instance uses the many built-in methods to manipulate the users' data. Below are a few FieldStorage's methods.
FieldStorage Methods:
Methods | Description |
---|---|
getfirst() | It returns the first value received. |
getlist() | It returns the list of the received values. |
getvalue() | It is a dictionary get() method. |
keys() | It is dictionary keys() method |
make_file() | It returns a readable and writable file. |
Running First Python File as CGI
In this section, we will discuss how can run the CGI program over the server. Before doing this, we must ensure that our system has the following configuration-
- Apache Server
- Python
If you already have the XAMPP server in your system then you can skip this part.
Installing the XAMPP server
XAMPP stands for cross-platform, Apache, MySQL, PHP, and Perl, and it provides the localhost server to test or deploy websites. Generally, it gives the two essential components for its installation, first is - Apache that creates a local server and MySQL, which we can use a database.
78,621 teddy bear drawing stock photos, vectors, and illustrations are available royalty-free. See teddy bear drawing stock video clips. Baby animals pencil drawing baby toys drawing sleeping teddy doll cute vector retro toy car cute bear drawing old toy airplane teddy bear illustration bear baby shower teddy illustration. Teddy bear sketch clip art.
Follow the below steps to install xampp.
Step - 1: Visit its official website (https://www.apachefriends.org/download.html) and download the latest version.
Step - 2: Once the download is complete, click on the run button.
Step - 3: Now, Click on the Next Button.
Step - 4: Next, it will display the xampp components. We can remove some of these but we will install all components of xampp, it won't affect our application.
Step - 6: Now, our set up is ready to install, start the installation by clicking the Next button.
It will take away on the webserver and start installing all packages and files.
Step - 7: Here, the download has completed. Select your desire language, and then, it will display the following dashboard of xampp.
Python Installation
To install the Python, visit out Python installation guide (https://www.javatpoint.com/how-to-install-python).
First CGI Program
We have created a new folder called example in xampp's htdocs folder. Then, we write a Python script, which includes the HTML tags. Let's see the following directory structure and the demo.py file.
demo.py
And, its directory structure as follow.
Type the localhost/example/demo.py into the web browser. It will display the following output.
Note - We need to start the Apache server then execute the CGI script. Our script demo.py will run on host 127.0.0.1 by default.
Let's understand another example of CGI script.
Example - 2:
Structure of a Python CGI Program
Let's understand the following structure of the program.
- The CGI script must contain two sections which separated by a blank line.
- The header must be in the first section, and the second section will contain the kind of data that will be used during the execution of the script.
Legend of Syntaxes
When scripting a CGI program in Python, take note of the following commonly used syntaxes.
HTML Header
In the above program, the line Content-type:text/htmlrnrn is a portion of the HTTP, which we will use in our CGI programs.
Sr. | Header | Description |
---|---|---|
1. | Content-type | It is a MIME string that is used to define the format of the file being returned. |
2. | Expires: Date | It displays the valid date information. |
3. | Location: URL | The URL that is returned by the server. |
4. | Last-modified: Date | It displays the date of the last modification of the resource. |
5. | Content-length: N | This information is used to report the estimated download time for a file. |
6. | Set-Cookies: String | It is used to set the cookies by using string. |
CGI Environment Variables
We should remember the following CGI environment variable along with the HTML syntax. Let's understand the commonly used CGI environment variables.
- CONTENT_TYPE - It describes the data and type of content.
- CONTENT_LENGHT - It defines the length of a query or information.
- HTTP_COOKIE - It is used to return the cookie, which is set by the user in the current scenario.
- HTTP_USER_AGENT - This variable is used to display the type of browser that the user is currently using.
- REMOTE_HOST - It is used to describe the path of the CGI scripts.
- PATH_INFO - This variable is used to define the path of the CGI script.
- REMOTE_ADDR - We can define the IP address of the visitor by using it.
- REQUEST_METHOD - It is used to make a request either via POST or GET.
Functions of Python CGI Programming
The CGI module provides the many functions to work with the cgi. We are defining a few important functions as follows.
- parse(fp = None, environ = os.environ, keep_blanks_values = False, strict_parsing = False) - It is used to parse a query in the environment. We can also parse it using a file, the default for which is sys.stdin.
- parse_qs(qs, keep_blank_values = False, strict _parsing = False) - While this is denigrated, Python uses it for urllib.parse.parse_qs() instead.
- parse_qsl(qs, keep_blank_value = False, strict_parsing = False) - This is also denigrated, and maintains of for backward-compatibility.
- parse_multipart(fb, pdict) - It is used to parse input of type multipart/form-data for file uploads. The first argument is the input file, and the second argument is a dictionary holding in the other parameters in the content-type header.
- parse_header(string) - It is used to parse the header. It permits the MIME header into the main value and a dictionary of parameters.
- test() - It is used to test a CGI script, and we can use it in our program. It will generally write minimal HTTP headers.
- print_form(form) - It formats a form in HTML.
- print_directory() - It formats the current directory in HTML.
- escape(s, quote = False) - The escape() function is used to convert characters '<', '>', and '&' in the string's to HTML safe sequence.
Debugging CGI Scripts
First, we need to check the trivial installation error. Most of the time, the error occurs during the installation of the CGI script. In the beginning, we must follow the installation instructions and try installing a copy of this module file cgi.py as a CGI script.
Next, we can use test() function from the script. Type the following code with a single statement.
Advantages of CGI Programming
There are various advantages of using CGI programming. Below are some of its advantages.
- They are language independent. We can use CGI programs with any programming languages.
- The CGI programs can work on almost any web server and the portable.
- They are portable.
- The CGI programs can perform both simple and complex tasks; means they are fairly scalable.
- The CGIs can increase the dynamic communication in the web applications.
- The CGIs can also be profitable, if we use them in development; they reduce the development costs and maintenance costs.
- The CGIs takes less time to process the requests.
Disadvantages of CGI
Consider the following disadvantages of CGI.
- CGI programs are too complex and hard to debug.
- When we initiate the program, the interpreter has to evaluate a CGI script in each initiation. The result is, creates a lot of traffic because are many requests from the side of the client-server.
- CGI programs are quite vulnerable, as most of them are free and easily available without the server security.
- CGI uses a lot of process time.
- During the page load, the data doesn't store in the cache memory.
- There are huge extensive codebases, most of it Perl.
Common Problems and Solutions
We can face problems during the implement the CGI script on the server. We have listed below the few common problems and their solutions.
- First of all, check the installation instructions. Most of the problems occur during the installation of server. Follow the installation guide properly.
- Check the HTTP server's log file. The tail -f logfile in a separated window may be valuable.
- In the CGI, it is possible to display the progress report on the client's screen of running requests. Most HTTP servers save the output from the CGI script until the script is finished.
- Before executing the file, check the syntax error in your script, following as python script.py.
- If the script does not have any syntax error then import the library such as import cgitb; cgitb.enable() to the top of the script.
- The absolute path must be included when importing the external program. The path is usually not set to a very useful value in a CGI script.
- When reading or writing external files, make sure that they can be read or written by the user under which your CGI script will be running. This is authorized user id where the script file in which the web server is running or some specified userid for a web server.
- It should be remembered that the CGI script must not set in set-uid It won't work on most systems, and also a security liability.
CGI stands for Common Gateway Interface. This interface is used byweb servers to process information requests supplied by a browser.Python has modules to allow programs to do this work. Theconvention used by many servers is to have the server programs thatsatisfy this interface end in ‘.cgi’. That is the convention usedbelow. All files below ending in ‘.cgi’ are CGI programs on a webserver, and in this chapter, they will all be Python programs(though there are many other languages in use for this purpose).These programs are often called scripts, so we will be dealing withPython CGI scripts.
You cannot run a .cgi file from inside Idle.
4.4.1. An Example in Operation¶
The web examples folder provides a simple web server, built into Python, that you canrun on your own computer. (It is also possible to set your computer upwith the right software to be a server for the Internet -that is totally unnecessary for this class.)
In an operating system file window, go to the folder with the wwwexamples. Depending on the setup of your operating system,there are several ways to start the local server that might work.
- Double click on
startServer.cmd
,which I have placed in the example www folder.If this does not work, try: - Right click on
localCGIServer.py
in the File Explorer window,and select Open With -> Python Launcher - If neither work, check if you need tomodify your Python installation,covered in Some Special Windows Instructions, and then try the
startServer.cmd
approach again.
You should see a console window pop up,saying “Localhost CGI server started”.This approach starts the
localCGIServer.py
server without monopolizing Idle.Once the server isstarted, leave the server console window thereas long as you want the local server running for that folder.Warning
Do not start the local server running from inside Idle.It will monopolize Idle.
Note
If the server aborts and gives an error message aboutspaces in the path, look at the path through the parent directoriesover this www directory. If any of the directory names have spacesin them, the local file server will not work.
In case of this error, either go up thedirectory chain and alter the directory names to eliminate spacesor move the examples directory to a directory that does not havethis issue.
For a very simple but complete example:
- Make sure you have the local server going.
- Open the web link http://localhost:8080/adder.html(preferably in a new window, separate from this tutorial).
- You see a web form. Follow the instructions, enter integers, and clickon the Find Sum button.You get back a web page that obviously used your data.
- Look at the local server console window. You should see a log of the activitywith the server.
We will end up completely explaining the web pages and .cgi fileneeded to accomplish this, allowing you to generalize the idea,but for now just see how it works.
First consider the rather involved basic execution steps behind the scene:
- The data you type is handled directly by the browser. Itrecognizes forms.
- An action instruction is stored in the form saying what to dowhen you press a button indicating you are ready to process thedata (the Find Sum button in this case).
- In the cases we consider in this tutorial, the action is givenas a web resource, giving the location of a CGI script on someserver (in our cases, the same directory on the server as thecurrent web page). It is a resource handled by the local server,when the URL starts with “http://localhost:8080/” followed by thename of the starting web file. All the URL’s you usefor this section and its exercises should start that way.
- When you press the button, the browser sends the data that youentered to that web location (in this case
adder.cgi
, in the samefolder as the original web page). - The server recognizes the web resource as an executable script,sees that it is a Python program, and executes it with the Python interpreter, using the datasent along from the browser form as input.
- The script runs, manipulates its input data into some results,and puts those results into the text of a web page that is theoutput of the program via
print
statements. - The server captures this output from the program and sends itback to your browser as a new page to display.
- You see the results in your browser.
Close the server window.
Test what happens if youtry to reload the web link http://localhost:8080/adder.html. You shouldget an error, since you refer to localhost, but youjust stopped the local server.
For the rest of this chapter, we will be wanting to use the localserver, so restart itin the example www folder,in a manner appropriate for your operating system:
- Windows: whatever worked when you started it the first time.
- Mac: Start the local server whatever way worked last time,either double clicking on
cgiServerScript
that you should have created,orright/control click onlocalCGIServer.py
in the Finder window,and select Open With -> Python Launcher
Now you can keep the local server goingas long as you want to run CGI scripts from the same folder.
If you ever want be have cgi scripts and supporting filesin a different folder, stop the server for any other folder first,and start it up in the folder where you have your materials.
4.4.2. A Simple Buildup¶
Before we get too complicated, consider the source code of a coupleof even simpler examples.
hellotxt.cgi
The simplest case is a CGI script with no input that justgenerates plain text, rather than HTML.Assuming you have your local server going, you can go to the linkhttp://localhost:8080/hellotxt.cgi. The code isin the www example directory,
hellotxt.cgi
, and below for youto read:The top line is what tells the operating system that this is a Python 3program. It says where to find the right Python interpreter to processthe rest of the script. This exact location is significanton a Unix derived server (like any Mac with OS X).In Windows the only thing important in the line is the distinctionbetween Python 2 and 3.If you leave the line thereas a part of your standard text, you have one less thing to thinkabout when uploading to a Unix server or running on a Mac.
The first print function is telling the server receiving thisoutput that the format of the rest of the output will be plaintext. This information gets passed back to the browser later. Thisline should be included exactly as stated IF you only want theoutput to be plain text (the simplest case, but not our usualcase).
The rest of the output (in this case just from one print function)becomes the body of the plain text document you see on your browserscreen, verbatim since it is plain text. The server captures thisoutput and redirects it to your browser.
hellohtml.cgi
We can make some variation and display an already determined htmlpage rather than plain text. Try the linkhttp://localhost:8080/hellohtml.cgi. The code is in the www exampledirectory,
hellohtml.cgi
, and below for you to read:There are two noteworthy changes. The first print function call nowdeclares the rest of the output will be html. This is astandard boilerplate line you will be using for your CGI programs. Theremaining print function call has the markup for an html page. Note thatthe enclosing triple quotes work for a multi line string. Otherthan as a simple illustration, this CGI script has no utility: Justputting the contents of the last print function in a file for astatic web page
hello.html
would be much simpler.now.cgi
One more simple step: we can have a CGI script that generatesdynamic output by reading the clock from inside of Python: Try thelink http://localhost:8080/now.cgi. Then click the refresh buttonand look again. This cannot come from a static page. The code is inthe www example directory,
now.cgi
, and below for you toread:This illustrates a couple more ideas: First a library module,
time
, is imported and used to generate the string for thecurrent date and time.The web page is generated like in
helloWeb2.py
, embedding thedynamic data (in this case the time) into a literal web page formatstring. (Note the embedded {timeStr}
.) Unlike helloWeb2.py
,this is a CGI script so the web page contents are delivered to theserver just with a print
function.adder.cgi
It is a small further step to get to processing dynamic input. Try fillingout and submitting the adder form one more time,http://localhost:8080/adder.html. This time notice the URL at thetop of the browser page when the result is displayed. You shouldsee something like the following (only the numbers should be theones you entered):
This shows onemechanism to deliver data from a web form to the CGI script thatprocesses it. The names x and y are used in the form (as we willsee later) and the data you entered is associated with those names.In fact a form is not needed at all to create such an association:If you directly go to the URLs
or
you get arithmetic displayed without the form. This is just a new inputmechanism into the CGI script.
You have already seen a program to produce this adder page frominside a regular Python program taking input from the keyboard. Thenew CGI version, adder.cgi, only needs to make a few modificationsto accept input this way from the browser. New features arecommented in the source and discussed below. The new parts are the
import
statement through the main
function, and the codeafter the end of the fileToStr
function. Read at least thesenew parts in the source code shown below:First the overall structure of the code:
- To handle the CGI input we import the
cgi
module. - The main body of the code is in a
main
method, followinggood programming practice. - After the definition of
main
come supporting functions, eachone copied from the earlier local web page version,additionWeb.py. - At the end is the new, boilerplate cgi wrapper code for
main()
. This is code that you can always just copy. I chose toput the initialprint
function here, that tells the server htmlis being produced. That mean themain
method only needs toconstruct and print the actual html code. Also keep the finaltry
-except
block that catches any execution errors in theprogram and generates possibly helpful trace information that youcan see from your browser. (How to write such error catching code ingeneral is not covered in this introductory tutorial,but you can copy it in this situation!)
The
main
function has three sections, as in the local web pageversion: read input (this time from the form, not the keyboard), process it, andgenerate the html output.- Reading input: The first line of main is a standard one (for you tocopy) that sets up an object called
form
that holds the CGIform data accompanying the web request sent by the browser. Youaccess the form data with statements like the next two that havethe pattern:variable=form.getfirst(
nameAttrib,
default)
If there is a form input field with namenameAttrib, its valuefrom the browser data is assigned to variable. If no value isgiven in the browser’s data for nameAttrib,variable is set equal to default instead.In this waydata associated with names given by the browser can be transferred toyour Python CGI program,and if there is no such name in a form feeding this cgi program,the program does not immediately bomb out.In this program the values associated withthe browser-supplied names, ‘x’ and ‘y’, are extracted. I useslightly verbose Python variable namesthat remind you that all values from thebrowser forms are strings. - The
processInput
function that is passed the inputparameters from whatever source, is exactly the same as inadditionWeb.py
, so we already know it works! - Output the page. In a CGI script, this is easier than with thelocal web pages: just print it - no need to save and separatelydisplay a file! The server captures the “printed” output.
This program can now serve as a template for your own CGI scripts:The only things you need to change are
- The lines in
main()
thatget the input from a web form, using the names from the inputtags in the form, and assigning the string values toPython varibles. - In main call
processInput(..)
with all the data from the form thatyou need to pass as parameters toprocessInput
. - The heading of the definition of
processInput
need to fit with the actual parameterspassed by the call in main(). Furthermore theprocessInput
part can be written andtested earlier with a Python program in the format of the ..Web.pyprograms that we have discussed. - While this is the only Pythoncode, you still need to create an output web page template,andrefer to it as the parameter of
fileToStr
. Again, if you testedyour logic using an earlier ..Web.py program, just use the same output web template!
A stripped down skeleton, to start a cgi program from, with comments about needed changes is in
skeletonFor.cgi
. If you do not want to start your code by modifyingan existing example cgi program, then you might start by copyingskeletonFor.cgi
.You can always start Idle a way that you have before, like on an existing .py file.Then deal with editing .cgi files from inside Idle.
For a Mac, See the Chapter 4 part of Some Special Mac Instructions for a way to start Idle on a .cgi fileeasily, after some initial work the first time.
On Windows, the easiest thing may be to use IdleOnWindows.cmd.
You will want to open and save .cgi files in Idle.Then dialog windows in both Windows and on a Mac have the same ideas,but different labels. There is a file type filter field for both.It is labeled “Format” on a Mac. On Windows, for saving,it is labeled “Save as Type”, and for opening a file, there is a fieldit at the lower right, unlabeled.
If you want a file ending in .cgi, change the filter field to All files (*).For saving, be sure to enter the full file name you want, with the extension, .cgi.
If you forget and do not change the file filter when saving, a ”.py” will be added to your file name.Then you can rename it in an operating system file folder.
In the followingdiagram, now we have discussed both the top regular Python sequence using the top, the bottomcgi specific sequence, and the common part in the middle, as shown.In both cases input data gets processed into the content of aweb page that goes to a browser. For any major application the main workis in the processing in the middle. Since that part is shared in bothapproaches, it can be tested with a simple Python program,before the starting and ending stepsof the input and output flow are changed for the cgi client/server model.
The only part that still needs details explained is for web forms.Before going on to that, it is time to talk about handling errorsin the CGI scripts we have been discussing.
The programs discussed here follow the simple pattern:
get all input data; process input data; output results all together.
This pattern can be used both in a testing
..Web.py
version andin a dynamic cgi version, using a processInput
function common to both.Clearly more complicated patterns are often neededwhen input, processing, and output are mixed in a different way,but for these examples andfor the chapter exercises, this is all we need.
4.4.3. Errors in CGI Scripts¶
Before you start running your own CGI scripts on the local server,it is important to understand how the different kinds of errors thatyou might make will be handled.
If you write a regular Python program, even one that produces a web page,you can write the code and run it in Idle, andIdle will display all the kinds of errors.
With a CGI script, you can still use Idle to write your code,but you cannot run the cgi code in Idle, and errors show up in threedifferent places, depending on their type:
You are encouraged to check for syntax errorsinside Idle, by using the menu sequence Run -> Check Module.If you fail to do this andtry running a script with a syntax error, the error trace appearsin the console window that appears when you start the localserver – better to have it fixed before then,while you are still editing.If you want an illustration, you might try changingadder.cgi, making an error like
imporcgi
, and try usingadder.html with the flawed script. (Then fix it and try again.)If you find it in Idle, you can jump to the line where the error was detected.
The error trace for execution errors isdisplayed in your web browser, thanks to the final boilerplate codewith the
try
-catch
block at the end of the CGI script.If you omit that final boilerplate code, you completely lose descriptivefeedback: Be sure to include that boilerplate code! You can alsosee an illustration now. Get an error by introducing the statement:in the main function and run it.(Then take it out.)
Your work can cause an issue inside the local server,not directly in the Python execution. Some errors arecommunicated to the browser, but not necessarily all.Other errors appear in the log generated in the local server’s window.You could have a file named wrong, for instance,in any operating system.
Also, on a Mac or in Linux, where the CGI script needsto be set as executable, an error with a non-executable CGI scriptonly shows up in this log in the local server window. You should be able to fix this error by running the
CGIfix.py
program,in the same folder.processInput
function.Here is an outline for client/server program testing,emphasizing errors to be conscious of and avoid.If you have problems, please actively check out this whole list:
- You are encouraged to add your files into the examples www folder.If you really want to work in a different folder,you will want to copy a number of support files into that folder:CGIfix.py and localCGIServer,py, plus dumpcgi.cgi if you want to testweb forms separately.For Windows: startServer.cmd. On a Mac be sure to run CGIfix.py,to generate the folder-specific version of cgiServerScript.
- If you want an easy environment to test a fancy
processInput
function,embed itin a regular Python program, as we have discussed for ..Web.py programs.Then you can test it normally in Idle.This will also allow you to make sure the web template, that you refer toin yourprocessInput
function, is in a legitimate form,with substitutions only for local variables. - You can code a CGI script in idle, but not run it.Be sure to save it with the suffix
.cgi
,not.py
. Do not run it it Idle. The only testing you can do in Idleis for syntax, using the menu sequence Run -> Check Module orits keyboard shortcut. Do run that test. - At the end of your CGI script, make sure you include the boilerplatecode that catches execution errors.
- Remember to run CGIfix.py in the same folder as a precaution toclean things up, particularly with a new file on a Mac.
- Make sure your local server is going, and that all the files youreference are in the same folder as the local server.
- Make sure you test your page by starting it in your web browser with aURL starting
http://localhost:8080/
. If you load a web pagedirectly from your file system by mistake,it will not cause an obvious error -the dynamic actions will just not take place. If you are not paying attention,this can happen and be very confusing! - If things do not work, remember the three places to check for errors.After initially checking for syntax errors in Idle,the remaining errors might be displayed on the output page in the browser(if you confirmed that you have the try-catch boilerplate code at the end).If that does not totally explain things,remember to check the server console window, too.
We have not covered web forms yet, but rather than bite off toomuch at once, this is a good time to write your own first CGIscript in the following exercise.
4.4.3.1. Quotient.cgi Exercise¶
* Modify Quotient Web Exercise and save it as a CGI script
quotient.cgi
in the same directory where you havelocalCGIServer.py
and your output html page template from quotientWeb.py
.Make quotient.cgi
take its input from a browser, rather than thekeyboard. This means merging all the standard CGI code fromadder.cgi
with the processInput
function code from yourquotientWeb.py
.Please keep the same web form data names, ‘x’ and ‘y’, as in adder.cgi, sothe main method should not need changes from adder.cgi. Remember totest for syntax errors inside Idle, and to have the local webserver running when you run the CGI script in your browser. Sincewe have not yet covered web forms, test your CGI script by enteringtest data into the URL in the browser. In particular use linkshttp://localhost:8080/quotient.cgi?x=24&y=56 andhttp://localhost:8080/quotient.cgi?x=36&y=15.These URLs depend on you keeping the same data names, x and y, in the form.After trying theselinks, you can edit the numbers in the URL in the browser to seedifferent results.4.4.4. Editing HTML Forms¶
This section is a continuation ofIntroduction to Static Pages in Kompozer, intended for users running Kompozer.A parallel development, without Kompozer, that may be simpleris in HTML Source Markup.
This section is about HTML editing using Kompozer – not about Python.HTML forms will allow user-friendly data entry for PythonCGI scripts. This is the last elaboration to allow basic webinteraction: Enter data in a form, submit it, and get a processedresult back from the server.
The initial example, adder.html, had a form with only text input fieldsand a submit button inside.You will not need anything with further kinds of markup for the exercises.
Now open
adder.html
in Kompozer. Switch to the Source view. This isa short enough page that you should not get lost in the sourcecode. The raw text illustrates another feature of html: attributes.The tag to start the form contains not only the tag code form,but also several expressions that look like Python assignmentstatements with string values. The names on the left-hand side ofthe equal signs identify a type of attribute, and the stringvalue after the equal sign gives the corresponding string value for theattribute. The tag for many kinds of input fields is input
.Notice that each field includes name
and value
attributes.(So there is some confusion possible here from the fact the valueis the name of a particular attribute,and all attributes have string as their value. I will try to use the phrase “string value” consistently for the second usage.)See that the ‘x’ and ‘y’ that are passed in the URL by the browsercome from the name attribute given in the HTML code for the correspondingfields!Kompozer and other web editors translate your menu selections intothe raw html code with proper attribute types. This high leveleditor behavior is convenient to avoid having to learn and debugthe exact right html syntax! On the other hand, using pop-up fieldediting windows has the disadvantage that you can only see theattributes of one field at a time,and likely need mutiple mouse clicks, plus typing.Particularly if you want tomodify a number of name or value attributes, it is annoying thatyou need a number of mouse clicks to go from one field to the next.
If you only want to modify the string values assigned toexisting attributes like
name
and value
, it may be easier to do in the sourcewindow, where you can see everything at once.(Or use a separate html source editor, as described inHTML Source Markup.)Making syntax errors in the html sourcein not very likely if the only changes that you makle to tagsare data inside quoted value strings.The action URL is a property of the entire form. To edit it inKompozer via the GUI, right click inside the form, but not on any fieldelement, and select the bottom pop-up choice, Form Properties. Thenyou see a window listing the Action URL and you can change thevalue to the name of the CGI script that you want to receive theform data. When you create your own web form, I suggest you makethe initial action URL be dumpcgi.cgi. This will allow you to debugyour form separate from your CGI script. When you have tested thatyour web form has all the right names and initial values, you canchange the action URL to your CGI script name (like quotient.cgi),and go on to test the combination of the form and the CGI script!
If you are modifying a previous working form, it is likely easierto use the source view,and just replace the value of the form’s action attributeto the new .cgi file name.
Optionally, you can test various input methods by opening and playing withhttp://localhost:8080/commonFormFields.html. (Make sure your localserver is still running!)
To allow easy concentration on the data sent by the browser, thisform connects to a simple CGI script
dumpcgi.cgi
, that justdumps and labels all the form data to a web page.Press the submit button inthe form, and see the result. Back up from the output to theprevious page, the form, and change some of the data in all kindsof fields. Submit again and see the results. Play with this untilyou get the idea clearly that the form is passing on your data.If you want to create a form and input fields using the Kompozer GUI,open this same file, the wwwexample
commonFormFields.html
, in Kompozer. The static text inthis page is set up as a tutorial on forms in Kompozer. Read thecontent of the page describing how to editat least forms with text input fields and submit buttons.The other fancier fields are optional – not needed for the chapter exercises.Read the static text about how to editindividual fields, and change some field parameters, save the fileand reload it in your browser, and submit again. If you change thename or value attributes, they are immediately indicated in thedumped output. If you change things like the text field size, itmakes a change in the way the form looks and behaves. You canreturn to the original version: An extra copy is saved in
commonFormFieldsOrig.html
.Now we have discussed the last piece, web forms,in the diagram for the comparison of generating web pages dynamicallyby a regular Python program or a server CGI script:
Note the last three Python videos do not directly corresponding to a singleplace in the Tutorial text. Instead they go through the entire processfor web based programs from the beginning.Video 4.4.4b creates a
birthday.html
web form looking forward tobirthday.cgi
of video 4.4.4d.In the middle video 4.4.4c creates birthdayWeb.py
, testing the processfunction and output template to be used in birthday.cgi
.4.4.4.1. QuotientWeb Form Exercise¶
* Complete the web presentation for quotient.cgi ofQuotient.cgi Exercise by creating a web form
quotient.html
thatis intelligible to a user and which supplies the necessary data toquotient.cgi
.Be sure to test the new form on your local server,using the URL http://localhost:8080/quotient.html. Remember thatyou must have the local server running first. You must have all theassociated files in the same directory as the server program youare running, and you cannot just click on quotient.html in a filebrowser. You must start it from the the URLhttp://localhost:8080/quotient.html, that specifically refers tothe server localhost.
4.4.4.2. Dynamic Web Programming Exercise¶
* Make a simple complete dynamic web presentation with a CGIscript that uses at least three user inputs from a form. Thesimplest would be to just add three numbers instead of two. Callyour form
dynamic.html
. Call your CGI script dynamic.cgi
. Call anoutput template dynamicTemplate.html
. Remember the details listedin the previous exercise to make the results work on localhost.After the server is started and you have all the files, go tohttp://localhost:8080/dynamic.html to test it a few times.Summary starts with the overall process forcreating dynamic web pages.
4.4.5. More Advanced Examples¶
This is an optional section.
One of the advantages of having a program running on a publicserver is that data may be stored centrally and augmented andshared by all. In high performance sites data is typically storedin a sophisticated database, beyond the scope of this tutorial. Fora less robust but simpler way to store data persistently, we canuse simple text files on the server.
The www example page
namelist.html
uses namelist.cgi
to maintain afile namelist.txt
of data submitted by users of the page. You cantest the program with your local Python server. It is lessimpressive when you are the only one who can make changes!The source code is documented for those who would like tohave a look.You also may want to look at the source code of the utility scriptyou have been using,
dumpcgi.cgi
. It uses a method of gettingvalues from the CGI data that has not been discussed:This method returns a list of values associated with a name fromthe web form. The list many have, 0, 1, or many elements. It isneeded if you have a number of check boxes with the same name.(Maybe you want a list of all the toppings someone selects for apizza.)
Both
dumpcgi.cgi
and namelist.html
add an extra layer of robustnessin reflecting back arbitrary text from a user. The user’s text mayinclude symbols used specially in html like ‘<’. The functionsafePlainText
replaces reserved symbols with appropriatealternatives.The examples in earlier sections were designed to illustrate theflow of data from input form to output page, but neither the htmlor the data transformations have been very complicated. A moreelaborate situation is ordering pizza online, and recording theorders for the restaurant owner. You can tryhttp://localhost:8080/pizza1.cgi several times and look at thesupporting example www files
pizza1.cgi
, pizzaOrderTemplate1.html
,and the simple pizzaReportTemplate.html
. To see the report, theowner needs to know the special name owner777
. After orderingseveral pizzas, enter that name and press the Submit button again.This CGI script gets used in two ways by a regular user: initially,when there is no order, and later to confirm an order that has beensubmitted. The two situations use different logic, and the scriptmust distinguish what is the current use. A hidden variable is usedto distinguish the two cases: when pizza1.cgi is called directly(not from a form), there is no
pastState
field. On the other handthe pizzaOrderTemplate1.html
includes a hidden field namedpastState
, which is set to the value 'order'
. (You can confirm thisby examining the end of the page in Kompozer’s source mode.) The CGIscript checks the value of the field pastState
, and varies itsbehavior based on whether the value is 'order'
or not.The form in pizzaOrderTemplate1.html has radio buttons and checkboxes hard coded into it for the options, and copies of the dataare in pizza1.cgi. Keeping multiple active copies of data is not agood idea: They can get out of sync. If you look at the source codefor pizzaOrderTemplate1.html, you see that all the entries for theradio button and check box lines are in a similar form. In thebetter version with altered files
pizza.cgi
andpizzaOrderTemplate.html
(that appears the same to the user), thebasic data for the pizza options is only in one place in pizza.cgi
,and the proper number of lines of radio buttons and check boxeswith the right data are generated dynamically. To do the dynamic generation,templates for an individual html line with a size radio buttonis in the source code, and it is used repeatedly to generate multiple lines, each witha different size and price embedded into the format stringfrom the program data. These lines are joinedtogether and placed as one entity into the html form template.A similar procedure is done with the toppings and check-boxes.Python Cgi Iis
The logic of using the hidden input field is futher outlinedin HTML Source Markup.
Python Cgitb
A further possible elaboration would be to also allow therestaurant manager to edit the size, cost and available topping data online,and store the data in a file rather than having the data hard coded inpizza.cgi, so if the manager runs out of a topping, she can removeit from the order form. This change would be a fairly elaborateproject compared to the earlier exercises!