
		AT&T Laboratories Cambridge

DART: Image Analysis

Image Analysis

Retrieval of still images independent of any associated textual information requires a language for describing image content . This language should be able to describe everyday objects and be intuitive enough to permit user definition of queries for retrieving archived images. It is highly desirable that the computer be able to decompose an image into elements of such a language unsupervised.

The first necessary step towards developing such a language is to develop a stable image segmentation scheme. Computers will not necessarily see images in the same way we do but it is desirable that the machine's perception be as close to our own as possible. Towards this goal work has been done on evaluating existing segmentation algorithms and developing new ones.

Approaches to a priori image segmentation

There follows a brief walk through a selection of the methods for image segmentation evaluated here.

Edge-based segmentation

A multi-scale, multi-colour Canny type edge detector was written and used to extract strong edges from colour images. The obvious and well known problem with edge detectors is that the edge map they extract is not complete and many of the edges in it correspond to surface markings or shadows rather than object boundaries. We try to address these issues with saliency filtering and Voronoi regions. Check out some sample output from our edge-based segmentation methods.

Internal region properties: Colour and Texture

It is necessary to define internal properties for regions. Typically regions will have both texture and colour properties associated with them.

Texture representation is an active topic of research within the DART project. Classically texture representations were divided between statistical and filter bank approaches, but the two have been to some extent been unified by Mumford. The choice of filters in a filter bank indicates a prejudice towards certain types of image structure.

A discrete non-linear filter that responds to oriented image roughness has been developed. The texture feature has been termed string . String is defined to have 8 possible orientations and is either present at a pixel or not. Check out some sample output from our string-based texture analysis methods.

Locally smooth regions

The final form of a texture model that will be usable for both segmentation and retrieval has not yet been determined. With this in mind a smooth region finding routine was developed. Smoothness is often determined through the power spectrum of the outputs of a bank of Gabor or wavelet filters. Filters with onmi-directional domains of support will fail near smooth/rough boundaries. A non-linear anisotropically supported smoothness filter was developed. Check out some sample output from our smoothness filter in action. Note that a side-effect of the smooth-region detection is a very aesthetically appealing transformation of photographic images into what look like charcoal drawings.

Shape clustering

Repeated structure is a common feature of many images. Examples of repeated structure might include patterns on fabric, brick walls, bird feathers or leaves. The nature of the imaging process (from 3D to 2D) and random variation in natural objects means that any repeated structure understanding routine must be able to handle modest variation in shape and appearance of structuring elements.

It is desirable that the system be able to physically extract the archetypal structuring element or elements. Correlation based methods cope poorly with brightness gradients and size or orientation changes.

Pairwise geometric histograms may be used to represent the shape of the boundaries of small closed regions. This shape representation is independent of orientation. These histograms may be clustered and groups of similar shaped regions extracted. In this case small closed region boundaries were extracted from the isobrightness contours of an image. The clustering routine used in this case required pre-specification of the number of cluster centres. Check out some sample output from our approach to shape clustering.

Cluster based coloured texture representation

This work is in its initial stages but looks very promising. It is intended to combine the repeated structure detection methods with a direct local colour clustering routine to give a novel coloured texture representation which will provide natural a priori image segmentation.