Model Evaluation

The Model Evaluation tool currently provides a selection of cell-level measures for quantifying overall and per-class agreement.

Table of Contents

Tool Parameters
1. Two-Map Comparison Measures
2. Three-Map Comparison Measures
Usage Instructions

Tool Parameters

See Data Preparation instructions before using the Model Evaluation tool.

Parameter	Description	Requirements
Actual Raster Layer	The real-world raster corresponding with the same timestep as the simulated raster.	A raster data layer containing positive integer values.
Simulated Raster Layer	The simulated raster being compared with the real-world data.	A raster data layer containing positive integer values.
(Optional) Actual Previous Raster Layer	The real-world raster corresponding with the previous timestep. This is used to determine which locations have undergone changes in reality.	A raster data layer containing positive integer values.
(Optional) Exclusion Mask Raster Layer	A raster data layer indicating which areas should be excluded from the analysis.	A reclassified data layer containing binary values, where 1 = areas to include in the evaluation and 0 = areas to exclude.
Report Name (*.html)	The name of the Model Evaluation report file that will be generated in the default project folder.	String

Two-Map Comparison Measures

Two-map comparisons are

Option	Description
Accuracy Measures	Overall Accuracy, Number of Agreeing Cells, Number of Disagreeing Cells
Class-Level Accuracy Measures	Overall accuracy calculated at the class-level
Kappa Statistics	Kappa, Kappa Histogram, Kappa Location, Cross Tabulation Table

Three-Map Comparison Measures

To activate the Three-Map Comparison options, you must specify an “Actual Previous Raster Layer.”

Three-map comparison measures include

Option	Description
Change Accuracy Measures	Overall accuracy, Change forecasting accuracy, Persistence forecasting accuracy
Change Error Measures	Total number of cells compared, Hits, Misses, Incorrect Changes, False alarms, Correct rejections, Total error, Error due to Quantity (EQ),Error due to Allocation (EA), Excluded cells
Figure of Merit	Figure of Merit, Producer’s Accuracy, User’s Accuracy

Model Evaluation Tool Overview

Usage Instructions

Step 1. Provide file paths to the actual and simulated raster data layers

Model Evaluation Required Inputs

Step 2. (Optional) Provide a file path to the actual previous raster data layer

Model Evaluation Previous Raster Input

Step 3. (Optional) Provide a file path to an exclusion raster data layer

Model Evaluation Exclusion Mask Input

Step 4. Select the measures you would like to calculate

Check the boxes beside the Two-Map Comparison Measures and (if applicable)

Model Evaluation Selected Metrics Inputs

Step 5. Verify the tool parameters

The tool verification will check that the Data Preparation guidelines are met, file paths are valid, and that other inputs contain no errors.

Model Evaluation Verify Inputs

If the Model Evaluation tool verification fails, you may have a data alignment issue. Please see the instructions on Data Preparation.

Step 6. Run the Model Evaluation tool and open the report file

The Model Evaluation tool will output the HTML file to your project’s default directory. You may need to refresh the ArcGIS Pro Catalog to see that the file has been added. An example Two-Map Comparison report section is shown below with respect to a binary modeling example.

Model Evaluation Report

Metrics affected by divisions by zero will be displayed as -9999 or NaN in the Model Parameter Report tables.