Title:
Characterizing the Conceptual Coherence of Computing Applications Using Ontological Excavation
Characterizing the Conceptual Coherence of Computing Applications Using Ontological Excavation
Author(s)
Hsi, Idris
Advisor(s)
Editor(s)
Collections
Supplementary to
Permanent Link
Abstract
We are investigating metrics for measuring the usefulness of computing applications relative to a
specific use context. We define "usefulness" as "the extent to which an application's features
succeed in assisting a set of users to achieve a set of goals, relative to the amount of effort
required to engage those features." We define a feature as a user-accessible behavior or service
implemented by a computing application. Computing applications embody and operationalize a
set of concepts that correlate to concepts in the domain of the user. The degree to which the
application’s concepts and the user's concepts agree is its conceptual fitness. We believe that
applications with high conceptual fitness to a particular use context will also be perceived as
useful by the users in this context. We have chosen in this work to study the problem of
measuring conceptual fitness from the application side using conceptual coherence as our unit of
interest.
Conceptual coherence is an attribute of conceptual integrity, described by Fred Brooks as the
property of a system designed under a unified and coordinated set of design ideas. It is the
property of a computing application that measures the degree to which that application's
concepts are tightly related. In previous work, we established that applications have core
concepts – concepts that are essential to defining that application's features. An application will
have a low conceptual coherence if it possesses a disproportionate number of non-core
(peripheral) concepts. We intend to show that the conceptual coherence of an application
determines its perceived usefulness to its users, and features with only tangential relationships to
an application are less likely to be used and reduce that application's conceptual coherence.
The set of concepts and relationships contained in an application can be said to be its ontology.
We have developed methods for the black-box reverse engineering (excavation) of a computing
application's ontology from the user interface and use techniques from the user interface and use
techniques from graph theory to identify the core concepts of an application and its teleons –
tightly connected functional subgroups within the ontology. We have also developed a technique
called use case silhouetting which measures the ontological coverage, the number of concepts
activated by a use case or set of use cases, and the relative importance of a concept to a set of use
cases as a first approximation of conceptual fitness. We have applied these techniques to four
small applications: the Windows 95/98 CD Player, the Palm Pilot Scheduler, Microsoft Notepad,
and the Protocol Calculator / Calendar.
We propose to perform two exploratory studies and one confirmatory one. Our first exploratory
study will excavate and analyze the ontologies from three large systems – Microsoft Powerpoint
2000, Microsoft Word 2000, and Yahoo Instant Messenger 5.5. Our second study will obtain use
cases from an independently written instruction manual (the "for Dummies" series). We will use
these to develop use case silhouettes on our excavated obtained from Dr. Joanna McGrenere in
her study on adaptable interfaces. We will show that the user preferences expressed by her
subjects correlate to data obtained from our ontological analysis and use case silhouettes of
Word.
Sponsor
Date Issued
2005
Extent
845447 bytes
Resource Type
Text
Resource Subtype
Technical Report