Training labels: train-labels.idx1-ubyte
gzipped (train-labels.idx1-ubyte.gz) or
zipped (train-labels.idx1-ubyte.zip)
Note that to write a successful program you must make some sense of the file formats
of these databases.
To help some, a small java application is provided which extracts an arbitrary
characters as a .ppm file. (More on the ppm format later.) Also included are some
unix shell scripts which call the java program appropriately. For example,
one of them pipes the output directly into the graphical viewing program xv.
Even if you are not a java programmer, you might get some insights on how to read from the
MNIST database. Here are the files:
- MNISTImageFile.java -- provides access methods to the image database
- MNISTLabelFile.java -- provides access methods to the label database
- MNISTtoPPM.java -- uses the two classes above to read and print a number
- viewtest -- a shell script which calls MNISTtoPPM to display a test number
- viewtrain -- a shell script which calls MNISTtoPPM to display a training number
Want to download MNISTTools these tools? Great. Here they are
tarred and gzipped (MNISTTools.tgz) or
zipped (MNISTTools.zip)
To run this code, you must have the java environment installed on your
machine. But the best way to use this code is to simply print it
out and look at it. Hopefully it will help you write your own drivers
to read from the MNIST database and print the characters.
Note: You are not required to use this code. This is only to point you in
the right direction.
Part I -- A Perceptron Learning Program (75%)
Choose two numbers between 0 and 9 (at random if you feel up to it...)
Train a single perceptron to learn to signal +1 when it encounters
either of those two numbers and -1 when it encounters any other
number.
You should follow these guidelines for Part I:
- For a training set, randomly select 2000 images from the training database
- For a testing set, use the first 500 images from the testing
database
- You should randomize your weights between -1 and 1
- Try to get as good performance out of your network as you
can by varying learning rate and bias. Note that since weights
are randomized, you might get lucky on one trial or another.
You'll need to run the system a few times for different settings
in order to get a feel for how they work
- As a starting point, try setting the learning rate, c, at .01
- As a starting point, setting the bias (-threshold) at 0.0
You should turn in the following for Part I:
- All source code
- A brief overview of how you implemented your project,
including what two numbers you chose to learn and the parameters which
worked the best.
- A graph of error over time (%-failed on y axis, time on x axis).
How many data points? Well, you should test for error using the
testing set "as often as is necessary" to get a good picture of
learning.
- An animation of the weightspace as it changes over time.
This will allow you to watch learning progress. We suggest
writing the weights matrix fairly often to a commin graphical file format
called portable pixmap (.ppm). From there you can translate into
.gif and finally to animated gif. Here is more
information on creating a good animation. (Note: if you can't get
this to work, print out graphs of the weight matrix as it changes
for a minor loss of credit.)
- Except for the animation, all material should be printed out.
The animated gif can be emailed to Doug (email at bottom of this
page).
Part II -- Two Questions to Hand In (12.5% Each)
You should turn in for Part II brief answers to these questions:
- (a) The perceptron may be used to perform numerous logic
functions. Demonstrate the implementation of the basic binary logic
functions AND, OR, and COMPLEMENT. (b) A basic limitation of the perceptron is that it cannont implement
the EXCLUSIVE OR function. Explain the reason for this limitation.
- Learning is like politeness: too much of it reverses the intended effect.
(a) Explain why over-learning can hamper the performance
of a network. (b) Suggest a method for guarding against such failure.
Part III -- Required Extra Work for Pairs (part of Part II)
Hint: it's not that much extra work. And your instructors
strongly suggest that you work in pairs.
Train the network to discern between a single number between 0 and 9
and the rest of the numbers in the training set.
- Turn in the same graphs you turned in for Part I.
- Explain the differences in
performance between this single number case and the two-number case
from Part I.
- Briefly describe how you divided the work.
deck@unm.edu
Last modified: Mon Aug 31 16:57:32 MDT 1998