Assignments and Lectures: Weeks 8 - 11
Posted 22 Oct 2005 by herman

Reading Assignment, Programming Assignment(s), and news about Lectures.

Reading Assignment

For week 8 (the week of the midterm), I covered some material on content services from Chapter 8 of the Content Networking text; you aren't asked to read this chapter. Starting week 9, we read from the other text, Network Distributed Computing. Week 9's lectures covered Chapter 1; see margin notes here, which tell you what part of the chapter to skip. For weeks 10-11, read Chapter 2 of the NDC book.

Lectures for Week 9 (24-28 October)

Instead of lectures, there will be discussions about the upcoming homework, described below. The discussions will be directed by the teaching assistants at least on Monday (section 1) and Tuesday (section 2); if you would like more discussions, please ask the TA at the first discussion. Additional replies to this article on the website, authored by the TAs, may contain announcements about this.

Programming Assignment

Finally, this assignment uses what you've learned about Python to start the path to web services. This assignment will combine apache, python, and mysql. Using all of these things can be intimidating with no experience. I have tried to break things down into manageable tasks and give you working examples. You have five tasks and suggested deadlines. Instead of making one final due date for the entire assignment, it is best to make incremental progress, so I have the following list. Although some of the deadlines may be distant, you should read over the assignment and related documentation soon, so you can discuss it with the TAs and ask questions about it (Teachers have less sympathy for students who delay getting started).

  1. [Wednesday 26 October] Create a web page.
    This is a trivial assignment, except for the fact that you have to create a web page on a new machine outside of the department's cluster. You'll create the web page on a machine called alacran (see alacran.html for basic information). I manually created accounts for everyone on this machine, but for security reasons I am not putting the account names and passwords here. The course TAs can provide this information. In order to do this assignment, you need to contend with the logistics of remotely logging in (changing your password would be an excellent idea), building a directory, perhaps transferring a file for the web page, and other things. We'll know your assignment is done when we can make a URL to your web page and see it on a browser. Call your web page file demo.html and it doesn't need to have any fancy content.
  2. [Friday 28 October] Some SQL.
    Build a MySQL table, add rows of data, and use a query to display some (but not all) of the table's data. Due on Friday is a file, called sample.sql, which contains the command lines you used to build the table, add some data, and query it. Put the sample.sql file into your directory on alacran. For documentation on MySQL, read mysql.html.
  3. [Monday 31 October] Program.
    Write a Python program that adds data to your MySQL database on alacran. Call this program newdata.py and put it in your directory on alacran (so that the TAs can find it there). See mysql.html for information about Python and MySQLdb, which is installed on alacran.
  4. [Monday 31 October] Second Program (same due date)
    Write a Python program that queries your MySQL database and returns table data. Call this program query.py and put it in your directory on alacran.
  5. [Wednesday 2 November] Dynamic HTML
    Write a Python program that is invoked by the web server in response to an HTTP request (typically a browser with the appropriate URL). Your program should return HTML so that a browser will show a web page. To do this, use the ModPython module: see modpython.html for some information about this. Your program will of course be in your public_html directory, where the TA can see it. You have a choice about what style to use, but please call your program simplehtml.py.
  6. [Friday 4 November] Dynamic HTML + Database
    Write a Python program that receives parameters from an HTTP request (either as CGI parameters or as POST data) and uses the parameters to operate on your database. A rudimentary program will just show simple data and a more complex program would update tables as well as displaying data from the database to the browser. This part of the homework is competitive: higher scores for more interesting functions. Call your main program mainhtml.py (you may use several programs as part of this part of the assignment, depending on how you do it).
Good luck and check back here for announcements and updates.
Some "Mini Lectures" on ModPython, MySQL, posted 23 Oct 2005 by herman
Here are three presentations I made to show, interactively, how such things as ModPython and MySQL are used to do the homework.
  1. ModPython

  2. MySQL

  3. MySQLdb
Introduction to HTML language and MySQL Reference Manual, posted 24 Oct 2005 by asignori
You can find a very nice introduction to the HTML language at this address

         http://www.make-a-web-site.com/making1.html

The MySQL v4.1 Reference Manual can be found at the address

         http://dev.mysql.com/doc/refman/4.1/en/
Copy Permissions, posted 31 Oct 2005 by herman
Some students correctly point out that if a Python program is in the public_html directory and permissions are set to allow public read/execute access, then what's to keep your homework from being copied?

Actually, copying of the first parts of the homework (building a simple web page, basic SQL commands) isn't much an issue; and the points for those parts will be minimal anyway. Where things become more crucial is in the later parts (especially parts 5 and 6). My recommendation is the following:

Keep permissions on your Python programs set to private, except for testing them. So, to test a program simplehtml.py you change the permissions to:

chmod og+rx simplehtml.py
(of course, this might be in the "publisher" or "cgi" subdirectory of public_html). After testing, remove the permissions by the command:
chmod og-rx simplehtml.py
This method isn't perfect: there is a chance that your work could be copied while you are testing it, and before you remove permissions. However the chances of this happening are small and the TAs will be looking extra hard for any signs of cheating on the later parts of this assignment. Remember especially that on part 6, creativity will be rewarded with extra points and part 6 has the most potential points of any part of the homework. The points on the assignment will be
  1. Part 1 (web page): One Point.
  2. Part 2 (sample.sql): Two Points.
  3. Part 3 (newdata.py): Two Points.
  4. Part 4 (query.py): Two Points.
  5. Part 5 (simplehtml.py): Two Points.
  6. Part 6 (mainhtml.py): Fifteen Points maximum.
Note that because Part 6 is essentially scored by competitive ranking, a working program -- one with no faults but rather simple, and only using the minimal features of modpython -- might only score ten points, so that a more advanced program can be rewarded for the additional effort.

Question and Answer, posted 1 Nov 2005 by herman
Question

A student writes the following question:

Hi Professor,

I've figured out a good deal about how to call programs, but I'm still a little unclear about getting data from the user. I created a form for the first and last name of a person, and added a link to an "Enter" program. This program would take the data in the first and last name fields, and add it to the table. My question, is how do you pass data into a program? How will I get the data from the text fields when the user hits the enter link?

Answer

Suppose you are using the publisher handler. An example is found here: http://www.modpython.org/live/current/doc-html/hand-pub-intro.html.

Notice how the first parameter to "say" is the req parameter. This is a reference to something called the "Request Object", described here: http://www.modpython.org/live/current/doc-html/pyapi-mprequest.html.

One attribute of the request object is 'form', (it would be known as req.form in the example), described here: http://www.modpython.org/live/current/doc-html/node92.html. It states that the 'form' attribute refers to an object in the Field Storage class: http://www.modpython.org/live/current/doc-html/pyapi-util-fstor.html. The description states that it can be used like a dictionary, presumably giving you text associated with named fields. Actually, it isn't literally a dictionary type, but it strongly resembles one. I found a function to convert a FieldStorage object to a true dictionary (suppose X is the FieldStorage object):

def cgiFieldStorageToDict( X ):
   """Get a plain dictionary, rather than the '.value' system used by
    the cgi module."""
   newdict = {}
   for key in X.keys():
       newdict[ key ] = X[ key ].value
   return newdict
As you can see, all that's different between the FieldStorage object and a Python dictionary is that instead of saying X[key] to get the value associated with "key", you say X[key].value.

It's best to experiment a little to learn how forms and the FieldStorage objects work. I did this by printing the FieldStorage object, printing its keys (for instance "print X.keys()" and so on), and watching what came back in the browser's window. Incrementally, I learned how the FieldStorage object contains various field data in a form.

* * *

But what if you don't want to use the publisher handler, what if you want to stick with CGI, how do you access form data in that case? Well, read http://www.modpython.org/live/current/pyapi-util.html, and you will see that you can create an instance of the FieldStorage class, given the Apache request object. Problem is, the cgihandler doesn't give you access to the request object! (No wonder they say using publisher is easier.)

However, true CGI execution of Python is documented in the Python standard library: http://www.python.org/doc/2.4.2/lib/module-cgi.html; in true CGI (and also in the modpython simulation of CGI), you just invoke FieldStorage() with no parameters to obtain the FieldStorage object containing form data: http://www.python.org/doc/2.4.2/lib/node471.html (that documentation possibly assumes you know something already about CGI, such as documented here: http://hoohoo.ncsa.uiuc.edu/cgi/). So what would a CGI python program look like, as a test? Maybe the simplest would be something like this:

#!/usr/bin/python
import cgi
f = cgi.FieldStorage()
print "Content-Type: text/html\n\n"
print "<html><body>"
print f
print "</body></html>"
Of course my example is silly, but you could learn from it depending on exactly how your CGI program is invoked (depending on the HTML, the browser might be doing a POST as a result of entering form data). In that case, the FieldStorage would be interesting, and varying the above example you could discover how it works in more detail.

Putting Everything Together

Actually, the publisher handler does make things a little simpler, though the syntax of the URL to access the publisher handler Python programs takes a bit of study before you get used to it. The one thing missing from the description above is the browser side of things. To get the browser to submit a form, you'll need the appropriate type of HTML, which in turn could depend on being generated from a Python script, and so on. But some things you need to put together yourself for this assignment.

Home PageHelp PageView ArticlesView Public DocumentsLogin, do stuff (record diaries)List Group MembersList of Projects