Lab 7: Evaluation of the Logic-based Software Failure Detection

Your goal is to evaluate how well the logic-based software detection technique works. The evaluation process involves calculating the false positive rate (i.e., false alarm) and false negative rate (i.e., false detection). To calculate the false positive rate, use the detection-predicate created in the fifth lab along with the pre-Trojan version of the java application. To calculate the false negative rate, use the Trojan version of the java application from the previous lab.

The false positive rate is determined by executing the java application (pre-Trojan version). The execution can be automatically scripted, but you should not use the test suite, which was used for learning the detection predicate. After executing the application, calculate and record the false positive detection rate. Namely, how many times the detection predicate evaluated to FALSE even though it was operating normally.

The false negative detection part of the evaluation uses the Trojan version of the java application. The detection predicate should determine whether the application is operating normally or abnormally. Calculate and record the false negative rate by executing the malware features as well as the non-malware features of the application. Calculate how many times a malicious feature was executed but not detected by the detection predicate.