Lab 6 Accuracy Assessment

Goal and Background

The main goal of lab 6 was to assess the unsupervised and supervised LULC maps created in labs 4 and 5. Using accuracy assessment is a necessary step in outputting LULC maps out to the public and potential users of the maps. If the accuracy of LULC map is too low it cannot be used and then the producer may have to redo the map(s). To assess the map ground reference testing was used. This lab introduced how to collect and use these tests. Ground reference tests use a reference image to compare pixels of the reference image to the LULC image.

Methodology

Part 1 of the lab was to assess the unsupervised LULC map from lab 4, the image being eau_Chipp2000rcF.img. The reference image, eau_chip05.img was added to a second viewer in ERDAS Imagine. This reference image is a high resolution image of the Chippewa and Eau Claire Counties study area. The reference image is an image from 2005 like the LULC map. To begin the accuracy assessment navigation to the Accuracy Assessment window must be done. It is found under the Raster tab through the Supervised option. The Accuracy Assessment window is opened through this path. To choose the image that is to be assessed it must be selected from the proper folder. This being the lab 4 folder. Input of the eau_Chipp2000rcF.img was done. This makes sure that it is the proper image. The title of the Accuracy Assessment window should change from “No Title” to the image title. To choose the reference image for comparison there is the Select Viewer icon. By clicking this icon a pop up will appear which tells the user to click in the reference image. Clicking the reference image selects it for assessment. Anywhere in the image can be clicked. The next step is to change the reference points’ colors. This was done by clicking View then Change Colors. Points without a reference will be white and points with a reference will be yellow. Adding random points was the next step. The path for this is Edit then Create/Add Random Points. The number of points that were needed was 125 points. Making sure Stratified Random was clicked is integral. Only the 5 classes from the LULC map were used. The process for selecting these classes is found in Select Classes in the Add Random Points window. A minimum of 15 points for each class is selected. This places random points all over the reference image. To make it easier to compare points only 10 points were assessed at one time. To show only 10 points select the points and then View and then Show Current Selection. This process is done until all 125 points were referenced. To reference the points found the point on the map and classify it as 1 of the 5 classes (water, forest, agriculture, urban/built-up, or bare soil). This referencing of the LULC maps was also done for the Supervised classification map. The process for doing this is exactly the same except for which classification to use. Once all the referencing was done classification accuracy report was made. This report has the user’s accuracy, the producer’s accuracy, the kappa statistic, and the overall classification. The accuracy assessment shows how many pixels were classified correctly out of random selection.

Figure 1: Showing ground reference points

Figure 2: Showing each reference point as a LULC class.

Figure 3: The original accuracy assessment report

Results

Below is the accuracy assessment report of the unsupervised classification method. One can see that the accuracy of this is too low for proper use.

Unsupervised

Class	Water	Forest	Agriculture	Urban/built up	Bare soil	Row Total
Water	15	0	0	0	0	15
Forest	1	33	11	1	0	46
Agriculture	0	2	24	1	1	28
Urban/built up	1	1	9	6	0	17
Bare soil	0	2	15	0	2	19
Column Total	17	38	59	8	3	125

Overall classification accuracy = 64%                                                        Overall Kappa Statistics = 0.52

Producer’s accuracy (Omission Error)                                                      User’s accuracy (Commission Error)

Water = 88.24%                                                                                                Water = 100.00%

Forest = 86.84%                                                                                                                Forest = 71.74%

Agriculture = 40.68%                                                                                       Agriculture = 40.68%

Urban/built-up = 75.00%                                                                              Urban/built-up = 35.29%

Bare soil = 66.67%                                                                                            Bare soil = 10.53%

Here is the supervised classification accuracy assessment. Like the unsupervised accuracy assessment, the supervised is also too low for use.

Supervised

Class	Water	Forest	Agriculture	Urban/built-up	Bare Soil	Row Total
Water	2	0	0	0	0	2
Forest	0	6	0	0	0	6
Agriculture	0	48	11	0	2	61
Urban/built-up	1	0	2	2	1	6
Bare Soil	0	13	30	2	4	50
Column Total	3	67	43	4	7	125

Overall classification accuracy = 20%                                                        Overall Kappa Statistics = -0.02

Producer’s accuracy (Omission Error)                                                      User’s accuracy (Commission Error)

Water = 66.67%                                                                                                Water = 100.00%

Forest = 8.96%                                                                                                  Forest = 100%

Agriculture = 25.58%                                                                                       Agriculture = 18.03%

Urban/built-up = 50.00%                                                                              Urban/built-up = 33.33%

Bare soil = 57.14%                                                                                            Bare soil = 8.00%