CS 583: Introduction to Computer Vision

Fall 2009


Project 1: Homography

    
Assigned: 10/05/2009
Due: 10/18/2009 5:59PM

[Synopsis] [Skeleton Code] [TODO] [Turn In] [Extra Credits] [Resources]

Synopsis

In this project, you will implement a program to (1) rectify an image (Image Rectification) (2) and superimpose one whole image or a part of an image into another (Image Composite) using manually selected correspondences on planar surfaces in the images. Along the way, you will learn how to compute homographies and how to use them to warp images. To start your project, you will be supplied with some test images and skeleton code you can use as the basis of your project, a sample solution executable you can use to compare with your program.

The requirements of the assignment are:

  1. Take your own images (10pts)
  2. Manually select corresponding points
  3. Compute homography (15pts)
  4. Warp the images
    1. Image Rectification: Compute fronto-parallel images (15pts)
    2. Image Composite: superimpose one image into another (15pts)
  5. Implement masking (5 pts)
  6. Implement linear blending (10 pts)

Skeleton Code and Test Images

Start by downloading and looking at the skeleton code. The skeleton code is written in Processing and the executable runs on tux.cs.drexel.edu. You are required to develop your program based on this skeleton code by filling in the empty functions. Of the necessary libraries (matrix-toolkits-java and ini4j) only ini4j is installed on TUX. You should refer to this page for information about how to set up required libraries on TUX or your own machine. You can develop on other platforms, but please avoid any platform dependent code. Note that the core should be written by yourself -- you cannot call a pre-existing function (say a library function) that computes the homography for you!

Use these images to test your code. (Note that they should be placed in the "data" directory.) For rectification, compute a fronto-parallel image of the side wall of the building. You may manually select the four corners of the wall and compute the homography such that the corners are mapped to the four corners of a rectangle. For image composite, compute the homography and warp face.jpg into the display of the computer in laptop.jpg In this case the 4 corners of face.jpg will map to the 4 corners of the display in laptop.jpg. Note that more points can be used and, in general, the computation of homography becomes more accurate by using more than 4 points. Compare your results with what the example program generates to check your code.


TODO

The skeleton code uses matrix-toolkits-java and ini4j. Information about these packages and how to set them up can be found at this page. Please be sure that you can run a simple application using these libraries as soon as possible.

The skeleton code consists of the following functions. The sentences in bold face describe what you will have to do and implement.

  Take/Load Images

Use a digital camera to take images. Plan before you take the images. Note that homography is a transformation of points on a plane to another plane. Use your creativity!

  Pick and Read Correspondences (readPoints)

Use an image viewer, e.g. gimp, to manually select corresponding points in the images. Write down the image coordinates shown in your viewer to a text file. Note that you will need at least 4 points to compute the homography. In general, using more points will make the result more accurate. Use readPoints to read in the text files.

  Compute Homography (computeH)

Compute the homography given the corresponding points by implementing computeH. This function should take two Nx2 matrices, where N is the number of points (image coordinates of corresponding points) as the input and return a 3x3 matrix (homography). Note that you will also need the inverse of the homography matrix. You can compute the inverse in various ways; you can either add a flag as an argument to computeH and take an inverse inside or use computeH with swapped inputs (it is up to you).

  Warp Image (warpImage)

Implement warpImage. This function should take the homography and its inverse as the input as well as the image to be warped and return the warped image. In order to correctly warp the image, the program must first automatically determine the size of the resulting warped image. Implement and use newImageSize that computes the bounding box of an image after applying the homography.

The actual computation of the warped image should be done using inverse-warping. Inverse-warping will return non-integer image coordinates. You must implement bi-linear interpolation to compute the pixel values for these non-integer image coordinates. You will have to implement these as separate functions called from warpImage, which are named applyHomography and getColorBilinear in the skeleton code, respectively.

  Rectify Image (rectify)

Implement rectify which takes one image file and two corresponding point list files as the input and returns the rectified image. You should be able to do this by simply calling the functions mentioned above. Once you finish coding up to this function, run your program on the rectification test image and see whether the rectification result matches the result you saw in class. This will help debuging your code.

  Composite Images (composite)

Implement composite which takes two image files and two corresponding point list files as the input and returns the image composite. This function should call the aforementioned functions as well as another function that actually does the composite (you might want to name it compositeImages (skeleton is not provided)).

  Main (draw)

Implement draw such that the final executable Homography will read homography.ini and perform the different jobs indicated by the values within it. For example, the file below (given in the skeleton code) will create both of the images shown at the top of this page: 
[Rectification example]
sourceImage = drexel.jpg
sourcePoints = drexel.source
targetPoints = drexel.target
outputFile = drexel-rectified.jpg


[Composition example]
sourceImage = face.jpg
sourcePoints = face.source
targetImage = laptop.jpg
targetPoints = laptop.target
outputFile = face-composition.jpg

Turn In

Please email a link to a final webpage before the deadline. The website must include the following
  • These links:
    1. The main pde file (Homography.pde).
    2. If you added files, explain and provide links to the additional files.
    3. A link to the 'data' directory which should contain your images and your updated homography.ini file
  • At least three examples of image rectification: the test image and two images that you took. For each example, show the original image and the rectified image side-by-side. Explain what you made fronto-parallel in text beneath each example. Each of these images on the web page should be in relatively low-resolution which is linked to the full-resolution image.
  • At least three examples of image composite: the test images and two sets of images that you took. For each example, show the original two images and the result side-by-side. Explain your artifact in text beneath each example. Each of these images on the web page should be in relatively low-resolution which is linked to the full-resolution image.
  • An applet export that runs the jobs specified in your homography.ini file. Note that you will have to sign all of the jars exported. See this thread.
  • A short description of what worked well and what didn't. If you tried several variants or did something non-standard, please describe this as well. If you did the items in the extra credit list, clearly state which one you did and how you did it.
Again, you can work on any machine/platform, but please avoid any platform dependant code.

You will need to submit all your code as well as at least three examples including the provided test images for both image rectification and image composite (that means you need two examples each for rectification and composite that uses images that you took!). Additionally, submit whatever you have done from the extra credits list.


Extra Credits

  • Implement blending. The boundary after image composite will be visible in the destination image. Implementing linear blending or alpha blending is a requirement not an extra credit but you may earn extra credits by implementing more sophisticated methods such as
    • Laplacian pyramid blending (10pts) (read Burt and Adelson's paper)
    • Even more sophisticated blending methods (15pts) (for example, Poisson Image Editing)
  • Make planar mosaics. Take three or more photographs while pointing the camera to different directions of the scene. Try to keep the position of the camera the same for all images (the COP has to be the same as explained in class). Also, try to overlap the fields of view significantly (more than 40%). Then, make an image composite of one entire image with another. After this, warp another to the resulting image composite. Iterate till all the images are used and warped into one large image composite (an image mosaic). (15pts)
  • Do something cool and make interesting artifacts. (points to be determined by instructor)

Resources