There are three different aspects of this project which can be assessed for success. The first of these is ``does the technique work to produce CAI scripts and if so, how well?'' We have found that this approach does indeed produce usable CAI lessons. One concern we had was how well this would work with a lab of 20 or more students all using the system at once. With the server code (including the CGI scripts) running on a SPARCStation 2, we served two labs sections in the first semester. The larger of the two labs had 20 students in it. We did not observe and no students reported any delays in response by the server.
The one place where the general technique could be improved upon is the short answer question. Here it is not unusual for a student to ask what they are doing wrong only to find that the answer they have given is technically correct but was not considered as one of the possibilities during development. We don't yet know if these different answers will be common across semesters. If so, then we only need to catalog them for a couple of semesters and add them to the existing CGI scripts. If not, then we need to investigate how to better evaluate the short answer responses.
While we have seen that this technique works to produce usable CAI lessons as seen by the students, the issue of how usable it is for the instructor is another matter. As is illustrated by the code fragments presented here, this is not a simple technique to use. Indeed, one could not expect faculty outside of Computer Science to even attempt to develop CAI lessons directly in Lex. Even for Computer Science faculty experienced in using Lex, this technique is hardly inviting.
A mechanism for more naturally expressing the server's actions during the student-computer interaction is called for. This is the focus of the next phase of this research. Our current plan is to develop a superset of HTML which will include directives to identify question types, correct answers and responses for correct and wrong answers. A translator will be developed to take this lesson script and build the set of HTML and Lex files necessary to provide the interaction. One potential difficulty in the use of such a tool outside our department is the platform of choice for our faculty. Most of the faculty of our institution use the Macintosh, but all of our development has been carried out in a UNIX environment. To encourage use by other faculty, our intent is to provide an e-mail robot that will take the HTML superset scripts and process them using the translator building the full lesson they can then access from web browsers running on Macs in their offices.
One remaining question to ask about such as project is, ``was it successful in it's educational goals?'' At the time of this writing, we are not yet finished with the first semester of use. However, some preliminary impressions can be related. Even though we are assigning fewer programming assignments as part of the lab sessions, the quality of student programs is at least as good as we've seen in the past. It appears to us that the students are more engaged and less prone to frustration than had been true in the past. Some of the students have reported that they like using the CAI lessons and some have even taken to reporting typographical errors they find. Our overall impression is that this approach does indeed provide the same or higher level of understanding and competence with a lower investment of effort on the part of the student. While instructor investment is quite high in developing the lessons, there seems to be a lower demand on instructor time during the labs sessions.