The challenges of classifying high resolution imagery stems primarily from two attributes. First, the classification system must be vastly more detailed than traditional classification systems because the spectral response of each pixel is associated with very specific earth and man-made materials. There is no inherent generalization as in low resolution imagery. Typically, we are not interested in such high levels of classification detail and some degree of generalization is desired in the end. Secondly, while the small pixel size dictates a fairly high degree of classification specificity, the spectral resolution of high resolution sensors is limited to only four bands thereby limiting one's ability to discriminate among materials. Such low spectral resolution makes it difficult to distinguish many earth and man-made materials. Thus, a careful balance must be sought between the number of classes defined and the ability to discriminate among them. Failing to strike the correct balance means that you can end up with classes that have broad spectral characteristics so they cannot be reasonably distinguished from other classes. In the end, we may require several classes of asphalt road, but still find it rather difficult to distinguish some of these road classes from rooftops of structures that are comprised of very similar materials.

In this activity, imagery over the same target from three different sensing systems is classified to illustrate the different capabilities of data from these systems. These data also illustrate some of the challenges one faces in classifying high resolution imagery of an urban environment. We classify a portion of a Landsat ETM+ scene, Advanced Thermal Land Applications Sensor (ATLAS) scene and a QuickBird-2 scene. The acquisition dates and resolution of these data sets is given in the table below.

ATLAS was flown on board a Lear 23 jet aircraft operated by NASA Stennis Space Center. The ATLAS is a 15-channel multispectral scanner that incorporates the bandwidths of the Landsat TM and ETM+ with additional bands in the middle reflective infrared and thermal infrared range. ATLAS data were collected at an altitude of approximately 5,032 m above mean terrain, which resulted in an image spatial resolution of 10 m. Images from each north-south trending flight line were rectified with approximately 100 ground control points each.

eCognition Professional (v. 4.2) was used for segmentation and classification in this activity. Figure 1 is an example of a software dialog box that allows the image analyst to specify scale and homogeneity criteria. In addition, each image band can be weighted individually as to their importance in influencing the outcome of the segmentation processes. Here, it would be desirable to reduce the weight of relatively noisy bands or bands that have little spectral contrast. On the other hand, a band with noise may help to differentiate classes and therefore be desirable if the noise is restricted to particular classes.

The notion of image "objects" is by its very nature a function of scale and thus highly dependent on image resolution. After all, objects are a collection of pixels that differentiate from their surroundings. In traditional classification with coarse resolution imagery (30 m to 1 km), we tend to think of objects as equivalent to classes, such as woodland, water, cropland. As resolution increases, objects tend to differentiate from classes. For example, objects might be represented by buildings, which taken collectively, might be classified as a commercial district. At even higher resolution, objects dissolve into materials with different spectral properties. Although this scale may appear more "realistic" to the human brain, the resolution may be too high to obtain meaningful objective interpretation through image processing. Multiscale segmentation offers a way to circumvent this problem by grouping pixels into objects of different size that can be interpreted. In addition, the objects can be further clustered into meaningful groups that can be classified.

Below is a subset of imagery for the same spatial domain from Landsat ETM+, ATLAS, and Quickbird. The spatial resolution of the QuickBird image (532 x 609) is approximately an order of magnitude greater than the ETM image (45 x 51). In this image of an urban residential area, the layout of roads can be seen in the ATLAS image at 10 m resolution. We can interpret from the patterns we see along the roads that there are houses. It is unclear to us from the ATLAS image alone whether the somewhat "noisy" pattern of houses is due to insufficient resolution of possibly tree canopy obscuring portions of each house. In the center of the image, we can also detect a large building surrounded by green space and associated with a large oval. It is not difficult to interpret this as a school and track. In the full size version of the Quickbird image, the level of detail is much greater and we can readily see a baseball field between the school building and track and tennis courts. Not only is the outline of individual houses also visible, but the shadow cast by the houses is also discernible. In contrast, very little information is interpretable from the ETM image. Because the scale of discernible objects is much greater, a much larger portion of image is required to recognize features and objects.

Figure 3 shows a portion of QuickBird imagery showing a school and associated athletic fields surrounded by a residential area. Figure 4 is the resulting segmentation. The objective of this initial segmentation is to restrict the size of the segments to closely correspond to the size of primary objects of interest, in this case individual houses. In many cases, the houses are comprised of more than one segment, but increasing the segment scale even slightly caused much of the adjacent yard or shadows surrounding the house to be included in the segments. Thus, a delicate balance was established. On the other hand, creating such small segments means that large contiguous surfaces of grass or forest seem to have an excessive number of segments. This situation can be remedied utilizing a multiresolution classification based segmentation. Secondary segmentation can cluster first order segments of similar attributes into larger groups. Segments can be classified at the appropriate scale and then combined for the final classified image.

The ETM image is comprised of 181 x 173 pixels covering approximately 5 x 5 km. The six visible and infrared bands at 28.5 m resolution were used on the classification. The coarse resolution thermal band was not used in the classification. Figure 6 shows the resulting classification. The built up areas in this image could not be well discriminated in terms of objects. Thus, they tend to get lumped together in one large Urban/Transportation/Commercial/Industrial class. Residential density also could not be discerned from areas with either higher density or simply larger trees. Woodland was the only Undeveloped class and image resolution was insufficient and the woodland stands too small to distinguish deciduous from evergreen trees. Nonetheless, if your primary interest was in distinguishing highly developed commercial land from residential and non-residential, ETM data would be adequate for the task.

The ATLAS image is comprised of 312 x 342 pixels covering 3.1 x 3.4 km. Technically speaking, ATLAS has higher spectral resolution than ETM as well as better spatial resolution. However, Band 9 of ATLAS was not functioning, and there is very little difference among the five thermal channels between 8.2 and 12.5 µm. In order to maximize the information among these channels, one could perform a decorrelation stretch on three of these channels and use the result in the classification. At the higher resolution, the tree canopy in the residential area has a more significant influence on the spectral properties of the area (Figure 7). It is incorrect to consider these areas as woodland and "undeveloped." But an attempt was made to carry forward as many of the classes from the ETM classification as possible for sake of comparison. It would be best to impose an area scalar on this class to identify truly undeveloped land. At this resolution, major roads several lanes wide are detectable as are houses and buildings. The segmentation scale used, however, was too coarse to define individual houses for classification.

The QuickBird image is comprised of 532 x 609 pixels covering 1.28 x 1.46 km. At a resolution of 2.4 m, very small features can be segmented as image objects. However, because QuickBird imagery is limited to four channels, there is not much spectral information to permit a high degree of differentiation. Thus, there is a delicate balance between object size and spectral fidelity. This QuickBird classification was performed at multiple scales. The first order classification is based on the initial or level 1 segmentation in which individual houses and trees were defined and classified. These were then clustered in a second order or level 2 classification (Figure 8). The result is that individual buildings were differentiated based on the type and color of roofing material and several types of road materials were differentiated. In addition, although deciduous and evergreen trees could be distinguished at level 1, a scalar was imposed such that if both types occurred within a segment at level 2, then a Mixed Woodland class could be identified. In classifications of this type, it is conceivable that any number of quantitative parameters could subsequently be determined, such as quantifying the number of houses, or number of house of a certain size. One could also perhaps estimate the total length of roadways within the image. Certainly, the advantage of any image based classification is the manner in which the computer is utilized to yield quantitative information about the classified scene.

• Discriminating between light colored asphalt and gray roofed houses. These are largely comprised of the same materials (asphalt shingles or tar paper sheeting on building roofs) and thus yield a similar spectral signature. Classifying asphalt is sometimes further complicated at such high resolution when there is an abundance of automobiles on the road, in parking lots or parked on the shoulder in a residential area. Asphalt is also difficult to classify at high resolution when shadows from tall features (buildings, trees) cast shadows on the road or parking lot. These shadows also have different characteristics depending on the transparency of the obstruction (building vs. tree).

• Transitions between classes are always difficult, but are much more common in high resolution imagery. The problem is exacerbated by the fact that, with the exception of hyperspectral data, high resolution sensors have low spectral resolution. One of the most commonly encountered transitions pertains to discriminating between grass and bare soil. The problem is even greater in winter or early spring imagery with senesced grass. Confusion is common for this leaf-off scene because Bermuda grass is the most commonly used variety for residences, commercial property and golf courses. Public parks are typically covered with a fescue variety of grass that remains green year-round. In March, the Bermuda is still senesced and, where it is thin or sparse, it may be spectrally indistinguishable from sand. In this image, the albedo of sand does not appear to be as high as expected and may, in fact, be wet thereby contributing to the confusion with grass.

• In wintertime imagery, there is often also difficulty in discriminating between senescent grass and deciduous trees, largely because significant contribution to overall radiance from the underlying grass passes through the tree crown when there are no leaves.

• Mixed woodland-the level 1 segmentation scale is so small that individual trees or small clusters are segmented. Consequently, segments with mixed trees are uncommon unless the segment size is increased or multiscale segmentation-based classification is used.