H&E Histology Color Normalization Instructions QuPath Integration

We have created a workflow, in which color normalization will be run from QuPath, calling on MATLAB to do the calculations and corrections.

Setting up QuPath

The first step you will need to complete is "Option #2: Control MATLAB from inside QuPath." The instructions have been copies from the QuPath Wiki (but reproduced below).

Starting with 2016b, MATLAB comes with a MATLAB Engine API for Java. This makes it possible to send variables from within QuPath to a running MATLAB session, manipulate them within MATLAB if necessary, and pull back the results if required.

This section describes how to get started with this method of connecting QuPath and MATLAB. It is written to provide QuPath-specific help to complete the general setup steps described on the MATLAB website.

Add MATLAB libraries to QuPath's Java Library Path

To enable QuPath to find MATLAB's required native libraries, it is essential to ensure that the java.library.path is set appropriately when QuPath is started.

The easiest - albeit somewhat hackish - way to do this is to modify the QuPath.cfg file to include this under JVMOptions.

On macOS QuPath.cfg is found under QuPath.app/Contents/Java/QuPath.cfg.

Remember, to 'see inside' a QuPath.app (or any similar .app file) on macOS, right-click on it and choose Show Package Contents.

An example of a (i.e. my) modified QuPath.cfg file is shown below:
[Application]
        app.name=QuPath
        app.mainjar=QuPathApp.jar
        app.version=0.0.3
        app.preferences.id=QuPathAppID
        app.mainclass=qupath/QuPath
        app.classpath=qupath/qupath-core-0.0.3.jar:qupath/qupath-core-awt-0.0.3.jar:qupath/qupath-core-processing-0.0.3.jar:qupath/qupath-core-processing-awt-0.0.3.jar:qupath/qupath-extension-ij-0.0.3.jar:qupath/qupath-extension-input-0.0.3.jar:qupath/qupath-extension-opencv-0.0.3.jar:qupath/qupath-extension-openslide-0.0.3.jar:qupath/qupath-extension-pen-0.0.3.jar:qupath/qupath-extension-script-editor-0.0.3.jar:qupath/qupath-gui-fx-0.0.3.jar:qupath/qupath-processing-ij-0.0.3.jar:qupath/qupath-processing-opencv-0.0.3.jar:jars/commons-math3-3.6.1.jar:jars/controlsfx-8.40.12.jar:jars/flowless-0.4.5.jar:jars/groovy-2.4.7.jar:jars/groovy-jsr223-2.4.7.jar:jars/gson-2.7.jar:jars/ij-1.51g.jar:jars/jfxtras-common-8.0-r5.jar:jars/jfxtras-menu-8.0-r5.jar:jars/jinput-2.0.6.jar:jars/jpen-2-150301.jar:jars/jutils-1.0.0.jar:jars/logback-classic-1.1.7.jar:jars/logback-core-1.1.7.jar:jars/opencv-3.1.0.jar:jars/openslide-3.4.1.jar:jars/packager.jar:jars/reactfx-2.0-M4u1.jar:jars/richtextfx-0.6.10.jar:jars/slf4j-api-1.7.20.jar:jars/undofx-1.2.jar:jars/wellbehavedfx-0.1.1.jar
        app.runtime=$APPDIR/PlugIns/Java.runtime
        app.identifier=QuPathAppID
        
        [JVMOptions]
        -Djava.library.path=/Applications/MATLAB_R2016b.app/bin/maci64:.:../MacOS
        
        [JVMUserOptions]
        
        [ArgOptions]
        
The only line that has been added is
-Djava.library.path=/Applications/MATLAB_R2016b.app/bin/maci64:.:../MacOS
        
/Applications/MATLAB_R2016b.app/bin/maci64 is the path to the MATLAB libraries, while the :.:../MacOS added at the end is to ensure that the original contents of QuPath's library path are retained. This is essential to ensure QuPath can find its own native libraries!

Note: On Windows, try replacing the colon : with a semi-colon ;.

The contents of the library path (either before or after modifying it) can also be seen under Help → System info - however it cannot be changed here.

Set path to the MATLAB engine

The next step is comparatively straightforward, and can be done from within QuPath.

Start up QuPath, and run Extensions → MATLAB → Set path to MATLAB engine.

A file chooser dialog should open up, from which you can select the MATLAB engine path (a file called engine.jar), as described here.

On my computer, this is found under

/Applications/MATLAB_R2016b.app/extern/engines/java/jar/engine.jar

Start a shared MATLAB session

Now start up a MATLAB session, and run the following command at the command prompt:
matlab.engine.shareEngine
        
This isn't strictly necessary, but it means that QuPath will start interacting with the MATLAB session that you started - and you will be able to query which variables it sent there etc.

If you omit this line, QuPath can use the MATLAB engine to start up its own instance each time it is needed - but this will be considerably slower, and harder to see quite what is happening.

Write Groovy scripts to exchange data

Finally, you should now be able to write Groovy scripts to send data to MATLAB, and also to pull back the results.

The first example to try is shown below, and also found within Extensions → MATLAB → Groovy MATLAB samples → MATLAB Hello
/**
         * Demo script to check QuPath can talk to MATLAB in a friendly way.
         *
         * This requires the MATLAB Engine, available with MATLAB R2016b,
         * and setup as described at https://github.com/qupath/qupath/wiki/Working-with-MATLAB
         *
         * @author Pete Bankhead
         */
        
        // Import the helper class
        import qupath.extension.matlab.QuPathMATLABExtension;
        QuPathMATLAB = this.class.classLoader.parseClass(QuPathMATLABExtension.getQuPathMATLABScript())
        
        // Get the MATLAB engine
        // Currently, it's essential to make sure close() is called -
        // so the try/finally is important.
        // Without it, there's a risk of an error that will prevent reconnection...
        QuPathMATLAB.getEngine(this)
        try {
            QuPathMATLAB.putVariable('a', 'Hello from QuPath!')
            QuPathMATLAB.eval('disp(a);')
        } finally {
            QuPathMATLAB.close()
        }
        
Running this script from within QuPath's script editor should result both in a friendly message being printing, and also a variable a appearing within the running MATLAB session.

Other - arguably more interesting - example scripts, which require images to be open, are provided within the Extensions → MATLAB → Groovy MATLAB samples menu.

Example of using MATLAB from QuPath

The screenshot above shows an example of calling MATLAB's k-means clustering applied to features calculated using QuPath's cell detection and (smoothed) feature computations.

Using the QuPath Script

This script will color normalize your images. You will get obtain results similar to this.

H&E Normalization Normalized Image

A histological H&E stained slide before (left) and after (right) normalization.

Setup QuPath and MATLAB per #2 here (copied above)
Download the attached zip (QuPathColorNormalization.zip) and unzip to your MATLAB Directory.
Open QuPath
Open an image in QuPath (File→ Open) (see Figure 1)
Select a Region of Interest using the rectangle tool in QuPath (see Figure 2)
Open the Automate menu, and then select "Show script editor" (see Figure 3)
Load the "QuPathColorNormalization.groovy" script (or copy it in)
With the script editor selected, select the "run" menu (see Figure 4)
Select "Run" from the Run menu, and the script should begin. (see Figure 5)
- If running, you should see "INFO: Starting parallel pool (parpool) using the 'local' profile …
A GUI will load with the selected ROI (see Figure 6)
- Use the view to highlight the structures in the thumbnail
- Select the values for Lumen, Nuclei, Stroma, and Cytoplasm
  - You need AT LEAST 1 selection per above structure
- When finished with selection, click the "Done" button and move the GUI to the background
  - In its current iteration, it will not close automatically
After selection of color values per structure, the rest of the script will run automatically
At the end of the workflow, a dialog box will appear to ask where to save the color normalized image (see Figure 7)
Voila, you now have a color normalized image!