CS 485 Fall 2005
Topics in Artificial Intelligence: Search Techniques

For this assignment, you are to implement at least one evolutionary approach to solving some classic difficult problems.  You are required to implement one solution; any extra solutions will earn credits toward your final exam grade.  The algorithms are not difficult to implement, but may need to run for a very long time, so it is recommended you start early and let 'em rip!

I. Classic genetic algorithm:

Boolean Satisfiability:

Using a genetic encoding for the Logic Puzzle we examined in class:

Suggestion:  This problem can be modeled as having 75 variables – one for each combination of person and attribute (college, degree class, degree program), e.g., variables BCA, BCH, BKA, etc. for (Bernice/Canterbury, Bernice/Charterhouse, Bernice/King Alfred’s, …).  Thus many clauses of the following form arise naturally:

(BCA or BCH or BKA or BPW or BSM) means (Bernice is in Canterbury OR Bernice is in Charterhouse OR …)

Note, however, that it is necessary to encode the fact that only one assignment of (BCA or BCH or BKA or BPW or BSM) can be TRUE at any one time.  This gives rise to many more clauses.  Furthermore, since only one of the students can be at each school, there are many more clauses.

One acceptable shortcut would be to use 0’s and 1’s for TRUE/FALSE, and use constraints of the form (BCA + BCH + BKA + BPW + BSM = 1), which considerably reduces the number of clauses.

Notice that the other facts give rise to Boolean clauses, e.g., “One girl was studying Archaeology, the other was at Charterhouse College” yields

(BA or SA) and (BCh or SCh) and (!BA or SCh) and (!SA or BCh)

...

Formulate and solve this using the approach for Boolean SAT described in the text.

II. Permutation Problems:                 

A. Traveling Salesman Problem:

Several approaches for formulating the TSP and solving it using an evolutionary approach are discussed in the text. We discussed the following three approaches for this problem in class. Choose one of these:

i. single parent -> multiple offspring . Represent a tour as a permutation, e.g, [2 5 4 6 1 3 7], which means that the tour begins with city 2 and proceeds through each of remaining cities in order, then returns to city 2. Choose a parent from the the initial population and use it to produce offspring by selecting random "start" and "stop" points in the permutation, then reversing the order of the cities. For instance, [2 5 | 4 6 1 | 3 7] produces [2 5 | 1 6 4 | 3 7] after reversal. Do this for several random sets of start/stop points. This has a distinct advantage over the "crossover" approach as each offspring is a permutation and therefore is a candidate tour. (Note, it is not possible that it will have subcycles, since that would require a city to appear more than once.) Do this for many parents selected randomly according to fitness.

Notes:

• when generating the "start" and "stop" points randomly, there is a possibility you will have start > stop. If this happens, you should "wrap around" the vector far enough so that stop > start, reverse the order of the items between them, then rotate them back around to the original position. Remember that the tour has the same cost regardless of where it starts, since it returns to its starting point.
• The Standard Template Library provides many routines for manipulating vectors in this way; e.g., if we make the following C+ declarations:
    vector<int> tour ;
    vector<int>::iterator p,q ;
  The following routines prove useful:
  random_shuffle(tour.begin(),tour.end()) ; // randomly permutes elements of tour
  reverse(p,q) ; // reverses order of elements between iterators p and q
  rotate(tour.begin(),p,tour.end()) ; rotates vector so that item pointed at by iterator p is at the front

ii. two parents -> multiple offspring.

a. Two permutations can be composed in a variety of ways discussed in the text. One way is to choose a random cutoff point k in the first vector and create the offspring by copying the first k elements of the first vector, then removing those elements from the second vector and copying the remaining n-k elements from the second vector in order. Once again, the offspring will be a permutation and so can be added to the population. It retains some of the cities in succession from one of the parents, and retains the remaining cities in order (although not necessarily contiguous) form the other parent.

b. Another way to combine two parent tours is to make a list for each city {1..n} of the cities adjacent to it in each tour. The total number of cities adjacent to city i will either be 2, 3, or 4. Select an initial city for the new tour by choosing a city with only 2 neighbors (if one exists). Select one of those neighbors as the next city, by choosing the one with the least neighbors of its own. Continue this process until completing a tour. Once again, this method produces a permutation and so is a candidate solution. It also is a reflection of its parents, because each edge in the new tour is an edge from one of the parent tours. Note that there is no guarantee that this method will actually produce a complet tour without getting "stuck", however.

For each of the Traveling Salesman Problems, you may use the examples from the previous assignment and from the text.

B. N Queens Problem:

The problem of placing N chess queens on a chessboard in mutually non-threatening positions is well known. Note that a solution may be expressed as a permutation, where the i-th element of the permutation signifies the position of the queen in row i. For instance, for the 4-Queens problem, the permutation [4 1 2 3] means "put queens in positions (1,4), (2,1), (3,2), and (4,3). Note that this is not a valid solution, however, since the queens in rows 2 and 3 threaten each other (as do others).

Using any of the methods described above for producing offspring for permutations, solve the 9-Queens problem with an evolutionary approach.

III. Numerical Optimization:

The classic optimization problem of finding the maximum value of a function in a given region can be quite difficult. Iterative approaches have the disadvantage of compounded rounding error, which are not suffered by evolutionary approaches.

i. two parents -> multiple offspring: One encoding method is simply to use the 64-bit floating-point representation for each variable, and employ standard crossover, mutation, etc. as a classic genetic algorithm.

ii. single parent -> multiple offspring: Another method, described in the text (Section 6.3), is to select a parent and generate offspring from it by adding a Gaussian random variable to each independent variable in the function. This ensures that the offspring will not deviate greatly from the parent.

Selecting the next generation can be done by simply choosing the n elements with highest value of the objective function. Another approach is to hold "tournaments" between randomly selected pairs of elements (e.g., choose two elements and random and keep the one with the higher objective function value). This has the advantages of selecting higher performing elements while retaining the diversity of some lower-performing elements. It also guarantees that the best performing solution remains in the next generation.

Converting between bit representations and floating-point values can be done by using a union construct in C/C++. which allows elements to be referred to by more than one type. (See sample program, provided without warranty.) Bit manipulation is also greatly facilitated using bitset objects from the standard template library. A test program using bitset objects is provided (also without warranty!) for your assistance.