Assignment 3

CS 265/571 Advanced Programming Tools and Techniques
Instructors: Jeremy Johnson and Kurt Schmidt
Due date: Wed. Mar. 2 at 9:00 am

Introduction

The purpose of this assignment is for students to learn effective techniques for debugging and testing programs. In particular, they should review and think about the material in Lecture 7 (chapters 5 and 6 of the text).

In the first part of the assignment students will be given a buggy version of binary search and they are to systematically test it, looking for cases where the program is correct and where it is incorrect, and then debug it. Students will need to write a main program to read the input array and test the binary search program.

In the second part of the assignment, students will develop a test suite and script, to test the csv programs from chapter 4 of the text. This should be useful for the final project. The specifications for csv should be used to develop the test suite to make sure that the different possibilities are tested. A key point is to automate the testing so that it is easy to run the entire suite when changes are made to the program (regression testing). This will be illustrated by having students run the suite for all three implementations of the csv programs in chapter 4.

Problems

Part 1

1. Write a main program to test the function lookup.  The program should prepare a sorted input array and call lookup with different keys.  Systematically test your program for input arrays of size 4 (this means testing the four cases where the key is in the array and the 5 cases where the key is not in the array - i.e. where the key is smaller than the first element in the array, the key is between the first and second elements in the array and so on). List all of the cases where lookup was correct and where it was incorrect. You should incorporate these tests into your main program so that you do not have to enter them by hand. Document the tests you are performing.
2. Add print statements to lookup show the values of low and high each time through the loop. use the trace of low and high to determine where the bugs occur.
3. Modify the lookup function (store it in lookup2.c) to correct the bugs found in the previous step and rerun your tests from part 2.
4. Write a boolean function called issorted to check whether an array is sorted. Use an assert statement to verify that the input array passed to lookup is sorted. What would happen if the array was not sorted? Incorporate this into a second version of lookup.
5. Write a boolean function inarray (use linear search) to test if an element is not in an array. Use this to verify that the result of lookup is correct. Incorporate this into a second main program (main2.c) that verifies that lookup succeeded for all of the test cases.

Part 2

1. Develop a suite of test cases for the csvgetline programs that thoroughly tests CSV inputs (refer to the specifications for cases to consider). For each test case indicate what the correct response should be.  You should prepare a file, test.csv, with the test cases, and a file out.csv with the correct output.  The test cases should cover different lines with allowable csv format as specified on page 93 of the text.  Provide sufficient cases to thoroughly test the csvgetline programs from chapter 4.
2. Write a script (bash or perl) to test csvgetline main program using the test cases that were developed. Build in a check of the result so that the script can determine if the test was successful or not.  The check should simply say whether the test passed or failed.  You can perform this test by comparing the output of running the csvgetline program on the test input with the expected output.  It would be helpful, but is not required, to indicate which lines of the test failed or passed.
3. Run your script for all three versions of csvgetline. Provide the output so that the tests that passed and failed are indicated. Do versions two and three pass all tests? Are there tests for which version one failed?

What to hand in

Place your solution in a directory called A3. Your directory should contain a file called README that describes all of the files in A3 and summarizes what you did. Make sure to indicate if any of the problems were not done or are not working.

Create two subdirectories called part1 and part2 respectively to contain your solutions to Part 1 and Part 2 of the assignment.
In Part 1, include

• main1.c  (first test program)
• lookup1.c  (version of lookup obtained from buggy.c)
• log1 (summary from running the tests in main1.c.   I.e. which tests succeeded and which failed, and if a test failed, indicate how it failed, i.e. returned the wrong index, returned -1 when it was not supposed to, did not return, crashed etc.)
• lookup1p.c  (version of lookup obtained from buggy.c with print statements)
• log2 (summary of running the tests in main1.c with the lookup1p.c.  Indicate what caused the bugs and how they were corrected.)
• lookup2.c  (corrected version of lookup1.c with assertion to check if the input is sorted).
• main2.c  (modify main1.c to include a test to verify that lookup succeeded).

• test.csv  (input file containing your test suite)
• test.out  (expected output from running the main csvgetline programs from chapter 4 with test.csv as input)
• runtest  (script to run csvgetline main programs on test.csv and compare the output to test.out.  The script may be written in perl or bash and should print either "Test Passed" or "Test Failed".  The script should take as an argument the name of the csvgetline program that is being tested.

Create a gzipped tar file containing a directory called A3 (use the command tar to do this) with your solutions.   If you created a directory called A3 with all of the files for your assignment 2 solution, you can use the command:

$ tar -zcvf A3.tar.gz A3

to create the desired gzipped tar file (the convention is to use the .tar extension for tar files and the .gz extension for gzipped files).  See man tar of info tar for more information about tar.

All assignments should be submitted using webct. You should submit your gzipped tar file.