Homeworks

The technical homeworks will add up to the project component of the course. Different parts of the project will be described and completed as homeworks. The main idea of the project is to write software that can analyze a substantial amount of data and produce various reports, possibly answering queries submitted online (via a web browser). In order to do this, we need a source of data. The dataset for the project is a file of consumer complaints, which has been compiled by the United States Department of Transportation. See the DOT Complaint page for some details.

Fifth Project Homework due Thursday 6 December 11:59pm (by midnight)

The last homework for the project is to create a webserver, using the BaseHTTPServer class from the Python library, and make pages that queries, show graphs, have links to other pages from your server, and more. The queries and graph generation are all finished, thanks to earlier homeworks. Your main task will be to create HTML files, use templates and other ideas to make your server.

This homework has no unit tests. Grading will be more subjective, as it will be tested by running your server and finding out what it can do. The basic things we will look for are listed here:

Technical Notes:

Optional Work. For the highest possible score, go beyond the basics. To do this, make another page on your server that shows, for a given year, month and day, locations on a Google map of the complaint incidents. You will need to use the UsCityLocation.csv, templates (format() with substitution) into CGI-style keywords on a URL, as shown by example in hw5demo.py; also you will use the eventplaces() function which is found in hw5util.

Still to Come. The dropbox has not been created. The zip file hasn't been fully debugged (yet).

Frequently Asked Questions

  1. I just don't get it. Some students have said, even after looking at Simple Web Servers and the example hw5demo.py program (inside hw5base.zip) that the assignment isn't clear. To make a much simpler demonstration, try looking at (and running) the file simple5.py -- which has comments explaining the steps. The logic of this simpler demonstration is straightforward, but not so elegant and does not use the full set of features possible for the assignment.

  2. How do I get started? Do not attempt to write a program for the entire assignment, all at once. Instead, first copy a test program from the Simple Web Servers page. Make sure you can run a test with a simple web server, between the server and a browser. Change the port number from 8000 to some random number of your choosing between 8001 and 16000 -- once you have a port number, use it for all your work. Then download the hw5base.zip (right-click and "Download as") to someplace on your desktop. Then you will need to unzip, or extract the contents of this archive, or zipped folder. If you haven't done this before, please ask a TA for instructions on how to do this. (For submitting your homework, you will also need to know how to do the reverse process, putting your work into a zipped folder.) Once you have extracted the zipped folder, modify the hw5util.py program to use a different port number than 8000: make it the number you will use in your testing, as you did with the simple web server. Then try running the hw5util.py program. In a browser, you can try testing it with some URL like http://localhost:8825/manuf or http://localhost:8825/map (read the code in hw5util and try to figure out how the URL drives decisions made in the server program). This should get you started, since you will have some skill at starting and testing a server.

  3. How do I make progress? Try making an HTML file, after studying the HTML Primer page. You make the HTML file using an editor, like Gedit on Linux or Notepad on Windows. It is just an ordinary text file, but ending with the file extension of ".html". Such a file can be viewed with a web browser, for example. Then adapt either hw5util.py or the simple web server to display the page through your modified server -- in the program, it will need to read the HTML file as a string and use the same or similar statements as you find in the example programs. Try some experiments where you use the format() method to change fields like "{0}" that you put in the HTML file. The point is that you gradually learn that a server can create strings, from a file, from other sources if you like, that represent a web page. After doing this, try the IMG tag or a link to make your page(s) more interesting. Finally, see if you can use functions from the previous assignment contained in the hw5util module to create an image file, which you then display on your page. Bit by bit, learn the techniques needed for completing the assignment.

  4. Is this program good enough to get maximum score? Some students would like to have us look at their work, judge it right away (before grading), then have an opportunity to change the program before submitting. No, we will not do this.


Fourth Project Homework due Friday 16 November 11:59pm (by midnight)

To have any hope of completing this assignment, you need to download six files. Right click on the links below, and use "save as" to put these files into the same folder, the folder you will use for doing the homework. It is important that all these files be in the same folder.

The fourth project homework builds on the third homework. Your task is to write four missing functions. Download homework4.py to see the test cases and some explanations of what is needed. This file also has a solution to Homework 3 in it (please use these solutions in your work for this homework). For this homework, you will need to use data files, DOT1000.txt and Top5000Population.txt --- You can download these from the page on DOT Complaint files.

For this homework, you need some extra modules. The first additional module is ezplot.py which, in turn, needs the MatPlotLib Package to be installed. The MatPlotLib Package has been installed for Python2.7 on all the CS Department Machines, so you don't need to install this yourself (but if you are preparing the homework on your own machine, you may need to install it, though some Python distributions have MatPlotLib built-in). The docstring at the beginning of homework4.py has several suggestions of useful techniques to complete the homework. Some examples of these techniques, and some explanation of ezplot, will be part of a class lecture.

Although homework4.py has unit tests, they aren't enough to know whether your functions are correct. You should look at Homework4 Images to see examples of what kind of graphs your functions should produce. The homework asks you to write four functions:

  1. plotEventFreq --- this function should just call eventfreq (which is provided for you in homework4.py), passing along the year and month arguments; the result from eventfreq is a list, from which it's easy to extract the x-values and y-values to be graphed. Note that for plotEventFreq, the x-values graphed are dates (date objects in the pairs that eventfreq returns).
  2. plotManufTop10 --- this function is also quite simple. It just gets the result of manuftop10 (also provided in homework4.py) and then extracts x-values and y-values to be plotted, using the ezplot.barplot function. The x-values are manufacturer names (strings) whereas the y-values are number of complaints.
  3. pagesub --- is the only function you write that doesn't graph anything. The pagesub function will be useful later, when writing web server.
  4. plotPopEvent --- is the most challenging function to write. Each point plotted represents a (city,state) pair. The x-values are number of complaints and the y-values are (city,state) population sizes. The data is obtained from calling fieldict("DOT1000.txt") and calling citypop() to get population numbers. Both fieldict() and citypop() functions are provided for you already in homework4.py, so please use them. The results that plotEventFreq displays are graphed using ezplot.corplot.

Submitting Your Work

Edit the downloaded homework4.py program, writing the functions needed. Test your functions until the unit tests all work (or as many as you can). Once this is done, save the file and then upload the revised homework3.py to the ICON drop box for this new homework. Make sure you click on the button to actually put the file into the dropbox (this can be tricky). If you submit homework3.py and then find some improvement to make, you can submit it again (there's no penalty for this). Please do not submit zip files, rar files, or the data files. We have already the data files to run your program!

Late Work

You can submit your homework late, but there will be a penalty of at least 15 percent, and after two days, the penalty rises to 50 percent. After solutions to the homework have been published on this website, even late submissions get no credit.


Third Project Homework due Wednesday 7 November 11:59pm (by midnight)

The third project homework builds on the last problem of the second homework. Again, your task is to write three missing function. Download homework3.py to see the test cases and some explanations of what is needed. This file also has a solution to Homework 2 in it. For this homework, you will need to use two data files, DOT1000.txt and Top5000Population.txt. You can download these from the page on DOT Complaint files. The docstring at the beginning of homework3.py has many suggestions of useful techniques to complete the homework.

Notes/Concerns by Students as of 5 November:

Grading Note: unlike previous homeworks, comments will be part of scoring the homework. In cases where the functions don't work, but the pseudocode shown in comments is well reasoned, that will be taken into account. Feel free to modify the docstrings already given in the functions in homework3.py to add clarity and explanation to your work. Other comments, variable names and techniques for making your code understandable can follow recommended style conventions from Chapter 20.

Submitting Your Work

Edit the downloaded homework3.py program, writing the functions needed. Test your functions until the unit tests all work (or as many as you can). Once this is done, save the file and then upload the revised homework3.py to the ICON drop box for this new homework. Make sure you click on the button to actually put the file into the dropbox (this can be tricky). If you submit homework3.py and then find some improvement to make, you can submit it again (there's no penalty for this). Please do not submit zip files, rar files, or the data files. We have already the data files to run your program!

Late Work

You can submit your homework late, but there will be a penalty of at least 15 percent, and after two days, the penalty rises to 50 percent. After solutions to the homework have been published on this website, even late submissions get no credit.


Second Project Homework due 31 October 11:59pm (by midnight)

Your second project homework uses the DOT Complaint files. Your task is to write three missing functions. Download homework2.py to see the test cases and some explanations of what is needed. You will also need to download two files from the DOT Complaint page. It will be helpful to study the example Python program, also shown on the DOT Complaint page -- it shows how to read a file, line by line, in an iteration.


First Project Homework due 3 October 11:59pm (by midnight)

Your first project homework is to write missing functions in a Python program. Download homework1.py to see the test cases (review the lecture on unit testing to understand the style of how functions are tested). In grading this homework, we will look at how many functions you successfully got working, how many test cases passed, and whether you used the style of Python programming in this first part of the course. No loops are allowed in your answers (that is, you cannot use techniques that will be covered later in the course). This homework is to enforce your learning about list comprehensions, dictionary comprehensions, and string methods. TAs in discussion sections will have practice examples and demonstrate using Idle or other ways to write programs, save the files, and test the work.

Submitting Your Work

Edit the downloaded homework1.py program, writing the functions needed. Test your functions until the unit tests all work (or as many as you can). Once this is done, save the file and then upload the revised homework1.py to the ICON drop box for this new homework. Make sure you click on the button to actually put the file into the dropbox (this can be tricky). If you submit homework1.py and then find some improvement to make, you can submit it again (there's no penalty for this).

Late Work

You can submit your homework late, but there will be a penalty of at least 15 percent, and after two days, the penalty rises to 50 percent. After solutions to the homework have been published on this website, even late submissions get no credit.

Restrictions

Two intents of this homework are (1) to start working on a project, with full details to come later, and (2) to get practice using string methods, comprehensions, and possibly recursion -- those will be on the next exam. However, some students asked if the homework could be done in some other way and still get full credit. Here are some specific questions and answers:


Attendance Homework due by 4 September 11:59am (just before noon)

This isn't really a technical homework, but it is required that we take attendance within the first ten days of the course. This will be done by having every student submit some file through the ICON dropbox feature. You complete the homework simply by logging on to ICON and uploading some (hopefully small) file to the dropbox. If you have any questions on how to do this, the first discussion section on Tuesday 28th August will demonstrate the dropbox and answer questions.

There won't be any credit for this homework, it will just be used for completing an attendance form.

Why should you do this?. If you fail to submit this homework, then we will assume you aren't attending, and mark the form appropriately. Keep the following in mind: students for whom academic activity could not be verified were required to repay an average of $1,900 in federal financial aid. So this turns out to be important.

How to submit to dropbox. Find 22C:016 in ICON, then navigate to dropbox on the menu bar (listed horizontally toward the top of the page). Click on that, and you should see a list of Dropbox Folders. One of these is the attendance homework, so click on Attendance Homework, and there will be a page with an Add a File option. Clicking on this will bring up yet another button, where you can choose a file to upload and complete this homework.

That's it!

Homeworks (last edited 2014-05-25 18:26:50 by localhost)