A vertex-based representation of objects in an image

Novel polygon evolution models are introduced in this paper for capturing polygonal object boundaries in images which have one or more objects that have statistically different distributions on the intensity values. The key idea in our approach is to design evolution equations for vertices of a polygon that integrate both local and global image characteristics. Our method naturally provides an efficient representation of an object through a few number of vertices, which also leads to a significant amount of compression of image content. This methodology can effectively be used in the context of MPEG-7. We also propose usage of the Jensen-Shannon criterion as an information measure between the densities of regions of an image to capture more general statistical characteristics of the data.


INTRODUCTION
Emerging multimedia applications such as searching for multimedia material of one's interests over large databases quickly reveal a need for efficient access and representation of visual objects in a scene.An efficient description of an object is through its contour representation.This representation may later be compactly stored and provide a significant amount of compression of the multimedia content.Several techniques such as in [I.21 for shape coding entailed first extracting object boundaries and later finding a polygonal approximation of the extracted contour.The locations of the resulting polygon vertices are also encoded for further data compression.
We propose in this paper.a technique that directly captures a polygonal representation of objects in images.Our approach.hence.produces vertex locations of objects in the scene in one shot rather than performing this search for meaningful locations in sweral steps.Our method, which provides a polygon as the resulting object description, can also be efficiently used for visual image retrieval or indexing based on an object's shape captured by a few vertices.This relates our technique to MPEG-7 framework which focuses on multimedia content [3] , Objects are delineated (and approximaled) by moving vertices of a polygon according to some energy criterion.Specifically.we dwelop a fast numerical approximation of an information-theoretic measure which drives the flow of polygon vertices on the basis of the prevailing intensity distributions of regions enclosed by the polygons in an image.This is. as we later show.achieved by a system of coupled Ordinary Differential Equations (ODES).In contrast to the region-based active contours framework [4-91.where continuous curves are utilized.our approach offers significant advantages over them by providing a very compact description par-

Anthony kzzi
Georgia Institute of Technology School of Electrical Eng.Atlanta, GA 30332 ticularly for simply connected target shapes.Similarly.our approach differs from the early snakes [IO1 in that the number of vertices that are involved in our flows are remarkably lower than the number of contour points used by the snake-based methods.
An energy functional over regions delineated by a number of polygons are defined in terms of an information measure among the distributions of the related regions.The so-called generalized Jensen-Shannon divergence measure Ill.121 not only provides a more general description of the properties of the regions than only mean, and variance as the representative statistics.but also gives a natural extension to a distance criterion among a finite number of distributions over the regions.We are then able to utilize multiple polygons moving over the image to capture several meaningful regions.
In the remainder of the paper.we first derive the polygon evolution equations.then introduce the information-theoretic criterion for evolution of a single contour or multiple contours.Some simulation results are presented followed by the conclusions.

POLYGON EVOLUTION MODELS
In this section, we present gradient flows designed to move polygon vertices with the goal of parsing an image domain into meaningfully different regions.A general form of an energy functional over a region R which is to capture various objects can be written as a contour integral where N denotes Ihe outward unit normal to C (the boundary of R). ds the Euclidean arclengthelement, and where F = (F', F') is chosen so that V .F = f.It can be shown 171 that a gradient flow for C with respect to E may be written as The key to our approach is to consider a closed polygon V as the contour C , and with a fixed number of vertices, say n E N, { V I , .., V"} = ( ( z ( , y i ) , i = 1,. ..n}.We may parameterize

C b y p E [O,n] as
where lp] denotes the largest integer which is not greater than p. a n d w h e r e L ( t , A , B ) = ( 1 -t ) A +tB parameterizesbetween 0 to 1 the line from A to B with constant speed.Note that the indices of V should be interpreted as modulo n so that V o and 0-7803-7622-6/02/$17.00 Q2002 IEEE I -896 IEEE ICIP 2002 V ,, dcnote the same vertex (recall C is a closed curve).Finally, note that C approximation of the entropy functional is found in 1141 upon substituting a first-order approximate density in is defined almost everywhere (where p # bJ) by CPb, Using the above parameterization C , ( p , V ) , we obtain the first variation of the energy functional E as

."=In ~( L @ -I ~J , V I ~I , V I ~I + , ) )
X ( C -2 J(v i p j + l -v i p j ) ) dP, yielding a minimization of E by way of a gradient descent flow given by a set of differential equations for each vertex v k as where Nk,r-, (resp.Nk+l,k) denotes the outward unit normal of edge and which in the interest of space, we defer the proof of to [13].These ODE'S are solved simultaneously for each vertex of a polygon.Note how the information through the functional f is being integrated along adjacent edges for a verlex.In addition to a small number of vertices, their well-separated locations clearly distinguish our proposed approach from the snake-based methods.
Upon obtaining a polygonal evolution model for a generic energy functional.we next choose a "good" criterion that captures the underlying statistical properties of each region delineated by a contour (particularly a polygon) and defines a metric among them.This may be achieved by an information divergence measure that defines a distance among probability densities.On account of its properties [12l.we pick as our energy functional the Jensen-Shannon criterion which, in its general form may be written for N probability densities (with prior probabilities.a = (a,, . . ., U N ) such that E,"=, a; = 1) as where p i ( [ ) denotes the probability density of pixel mlues in the ith region, and H is the Shannon entropy.One of the many features of the Jensen-Shannon divergence is that different weights may be assigned to the distributions involved and according to their importance.Estimation of a probability density and its entropy may be carried out in a variety of ways.we hence adopt a first order approximation of a density which achieves a maximum entropy solution.which is in turn used in approximating the entropy expression as proposed in (141.
We assume a scalar random variable (.v.) ( : 3 C W + R on the intensity values of a region delineated by an active polygon.and the information available on the density of the r.v. 5 is given by LP(E)Gj(OdF=uj, for i = L . ., r n , where the estimates uj are the expectations E { G j ( ( ) } of m different independent functions {Gi(.)} of (.Note that there is no model assumption for the random variable ( , however, the distribution which has the maximum entropy and which is also compatible with the measurements in Eq. (3) is sought 115.161.A simple where H ( v ) = $(l + log(2rr)) is thc cntropy of a standardized Gaussian variable.

EVOLUTION OF A SINGLE CONTOUR
With a single contour, the image domain il is split into two regions.namely a region inside the contour.call it R,.and a region outside the contour, n\&.Measurements U, (resp.v j ) for region R, (resp.R J , are given by withl&l=SR.dx.(/QII=Sn,R"ddz:).z

= ( x , Y ) .
Exploiting the approximation in Eq. (4).we define an energy functional whose optimization yields the evolution of our active conlour based on a mutual information measurc defined in Eq. ( 2).The new energy functional for two regions, is
The image domain will be partitioned into "distinct" regions whose characteristics are well-separated as long as the measurements in a region successfully capture the statistical characteristics of the data.For instance.when images can be approximated by constants.the mean intensity over each region is an adequate measurement, i.e., the first moment of the T.V. is utilized.(G(x) = x ) .For Gaussian densities, a second order moment.the variance.is utilized.In order to capture statistical characteristics for nowGaussian densities.we use the functions per as an odd function to measure asymmetry (analogous to the third moment as a measure of skewness).and

G2(() = 161, or e d a t 2 as choices ofeven functions (analogous to
the fourth moment as a measure of sparsity, bimodality.relative to a Gaussian density).
The choice of prior probabilities for each density, i.e. the co- efficients a1,al in Eq.( 7) leads to different gradient descent flows for a contour.If equal priors are assigned (a, = az = 0.5). the gradient flow for a continuous contour C is found as where f is the speed of the contour.We note that this is a generalized form of the data term of the flow in 191.
In an attempt to normalize the effective contribution to the divergence measure from a region with respect to the ratio of the number of pixels in its population to the overall number of pixels, we assign priors that are proportional to the ratio of area of each region to the total area of the image domain.In this case, we arrive at the following gradient descent flow f whose energy functional is indeed a generalized form of the energy functional in [SI.For case of wolving polygon vertices.each vertex flow is given by substituting the total image force f into Eq.(l) and solving the ODE'S simultaneously.
Note that to avoid degeneracies.such as edges intersecting each other.we dweloped a reguiarizer which balances the tendency of edges getting too close in order to better satisfy the optimization problem.The issue of the regularization.the choice of initial number of vertices, and adaptive removal and addition of vertices are detailed in [13l.
Two sample simulation results that demonstrate the evolution of a single polygon over an image domain are given in Fig. 1 and Fig. 2. (The progression of the flow is left-right, top-bottom.)In Fig. 1, the polygon vertex locations marked on the region enclosed by them are also shown at the same snapshot moments.Note the significant compression of the image content in both cases to merely a few vertices.

EVOLUTION OF MULTIPLE CONTOURS
The generalization of the above development is also readily achiwed as we show next.Suppose there are two active contours whose inner regions are denoted by &, and &, and their common exterior by R,,,.The measurements.i.e. statistics. in these respective regions are denoted by u j , u j , w j , j = 1,. . ., m for m different measurements.with respective prior probabilities a,, 02, a3 I a1 + 02 + a3 = 1.The energy functional for three densities can be written as

ai a2(uj-uj12 +a1 a s ( ~j -w j ) ~ +a2 as(uj-wj)'] (IO)
This form of energy functional may easily be extrapolated to N active contours, thus partitioning the image domain into N + 1 regions.Observe that the first-order approximations to both the densities and the corresponding entropies of the regions.lead to a measure that computes a weighted sum of the mutual information measure (i.e.distances between their statistics) between all pairwise combinations of the regions.We illustrate the use of ternary wolutions in Fig. 3, and Fig. 4. (The progression of the flow is left-right, top-bottom.)Two contours move in such a way to maximize the approximate Jensen-Shannon divergence between densities of the three regions, namely 9 : inside the contour C,: &: inside the contour C,: R,,,: the complement of 9 U &.The resulting two polygons show that the gain is again two-fold segmentation of the target objects.and their description in terms of a small number of vertices.

CONCLUSIONS
We proposed novel flows which capture polygonal objects in images.We proposed the application of a Jensen-Shannon criterion as a distance among the regions delineated by polygons, and utilized an approximate implementation of the criterion.The resuit of our methodology provides a significant compression of image content for a variety of purposes such as image retrieval by shape templates.shape coding.content-based description of the multimedia data which are the core functionalities of MPEG-7.
The ternary case entails the derivation of a gradient descent flow for each of the two active contours C,. and C,.Taking the first variation of the energy functional in Eq.(lO) w.r.t.contour C, and C . .yields a gradient descent flow for each contour (here we assumed constant priors), and the image forces are given by m Gj(1)uj l h l fv=C[-al m(uj -u j ) +a2 aa(ujw j ) ] j = * where xu. and xv are the characteristic functions over the regions &, and &.