|
|
Questions and Summary of John Hummel and Keith Holyoak. 1997. Distributed Representations of Structure: A
Theory of Analogical Access and Mapping.
discussed by Dan Bauer
Hummel and Holyoak present a computational theory of analogy formation which
they have implemented in a model called LISA (Learning and Inference with
Schemas and Analogies). Their central motivation is to integrate two major
processes of analogy formation-- memory access and structural mapping-- while
preserving both flexibility and sensitivity to structure.
Analogy makes use of complex relations and structures which emerge from
recombination of simpler elements, but also requires flexiblity and
generalization. Traditional symbolic systems maintain structure but are
inflexible; connectionist systems are just the reverse. Past hybrid models
have lacked a natural interface between the two. The LISA model attempts to
reconcile the two approaches and unify access and mapping with both structure
sensitivity and flexiblilty.
In this summary I will focus primarily on the architecture of LISA, since I
believe a concrete picture of the implementation will make it easier to
understand the theory's broader ramifications. The model has relatively few
components, but it is complex because of its subtle and dynamic behavior, and I
will make some simplifications for conceptual clarity.
OVERVIEW OF THE MODEL
LISA can be divided roughly into two interacting systems: a "working memory"
(WM) and "long-term memory" (LTM). LTM is a layered network of "structural"
units, and the bottom structural layer connects to WM's single layer of
semantic units.
Concepts and relations (in LTM) are represented as trees of structural units of
three types: propositions, subpropositions, and objects/predicates.
Propositions (like "John loves Mary") are broken into subpropositions which
isolate the roles of each agent or patient (like "John loves" and "Mary is
loved"). These in turn are broken into objects ("John", "Mary") and predicates
("loves", "is loved"). There may also be additional levels of propositions
within objects ("Sam knows John loves Mary").
Each proposition tree in LTM is a potential "analog"-- the source or target of
an analogy.
The semantic units of WM connect to and allow distributed representations of
each object or predicate at the bottom of an LTM proposition tree. The more
similar two objects/predicates are, the more semantic units they will share.
WM also includes a set of "mapping" links between LTM structure units of the
same type (eg. predicate-predicate, proposition-proposition).
The basic dynamics of the system are as follows:
Activity starts in LTM, in a particular proposition unit chosen as the target
analog, or "driver". Flashes of activity spread alternately down various
competing branches of the driver's structure units and activate patterns of
semantic units in WM. These semantic units activate "similar" objects and
predicates, and activation spreads back up competing branches of other
"recipient" analogs. Recipients which are most strongly activated ("retrieved"
from LTM) at any moment are considered the best "source" analogs for the
original target.
When structure units of the same type are active concurrently, the WM "mapping"
weight between them strenghtens; when structure units are uncorrelated, the
connecting weight is weakened.
After one branch of the target (eg. "John loves") has been active for some time
and brought out the best source analog (eg. "Bill likes") it shuts off, and
another target branch ("Mary is loved") takes over and spreads activation to a
new recipient ("Susan is liked"). Over time, as the components of the target
("John loves Mary") map to analogous components of the source ("Bill likes
Susan"), WM builds up a flexible but structure-preserving analogy.
MAJOR THEORETICAL ISSUES
While some limitations of the LISA model appear to be flaws, many actually
parallel and therefore illuminate certain limitations of human reasoning,
including processing order and memory constraints.
--Processing Order and Coherence: The quality of the analogical mappings in
LISA depends on the order in which the target/driver's proposition and
subproposition branches are activated. Stronger mappings result when
successive propositions tend to overlap in their objects and predicates, and
when important propositions appear early and often. These are some of the same
qualities which make written text coherent, suggesting that the human mind
prefers the same types of sequences. In fact, LISA's default strategy is to
select propositions in the order and frequency in which they appear in a text
describing the target analog.
--Memory Constraints: In LISA, the set of structure and semantic units which
are active at any particular moment can be considered "active memory" which
intuitively corresponds to the system's "current thought". The number of
structure trees which can be active at any moment (called the "phase set") can
be modified, and this "phase set" determines the size of active memory. The
greater the active memory, the more subpropositions which can be interrelated,
and the more complex the analogy which can be extracted.
This seems to parallel the role of active memory in human reasoning.
--Representational Flexibility (N-ary constraint): Previous analogy-mapping
models (like ACME & SME) were able to map N-ary predicates only onto other
N-ary predicates In LISA, because source analogs emerge indirectly from the
semantic distributed representations, there can be reasonable mappings between
predicates with different numbers of arguments.
=====================
MY RESPONSE:
In many respects, LISA embodies an elegant unification of access and mapping.
I find particularly appealing its treatment of mapping as an emergent
correlation of activity rather than brute-force predicate matching.
LISA seems to benefit from many of the advantages of distributed
representations while maintaining the ability to portray complex structures.
Nevertheless, perhaps because I do not fully understand the model, I am left
with an uneasy concern:
1) The authors claim on p436: "...the structure units do not directly encode
meaning. Rather, they work together to impose particular patterns of synchrony
on the semantic units; it is only the latter which encode meaning."
I would argue: Where does the meaning in the semantic units come from, if not
from the structure units? A set of semantic units mean "dog" only because they
are attached to the Dog structure unit, and because some of them are also
attached to the Animal, Bark, and Tail structure units.
Imagine an unanticipated set of semantic units which together suggest a viable
concept (e.g. animal + small + cute +...) but which are not explicitly grouped
by any structure unit. This concept could never take on a role in an analog
without an explicit structural unit to babysit it! The "meaning" could never
participate in the retrieval and mapping process. Based on this, I would say
the semantic units have meaning only because of the structures.
And if meaning is introduced only through explicit structures, then no
meanings, mappings, or analogies can emerge which have not been implicitly
anticipated and loaded into the model.
2) This is really the same point seen from a different direction:
It is true that LISA's mappings are not limited to predicates with the same
number of arguments. However, there are plenty of other intuitive
correspondences which do not seem to be mappable. For example, can
IsHungry(John) map onto Wants(Food, John)? Neither Wants nor Food alone shares
much semantic similarity with IsHungry. but even if their meanings could be
combined, how would Hungry map simultaneously onto both object and predicate?
It seems that for two potential analogs to map onto each other, their
representations must be arranged in advance; Wants(Food,John) must be
re-represented as Hungry(John), or the reverse. In this case, LISA's
representations are already implicitly loaded with analogous structures!
Therefore it is not surprising that LISA finds the analogies.
I believe, if I understand the model correctly, that LISA's dependency on
predicate structures actually undermines most of the strength of the
distributed semantic representations, and that it still suffers many of the
same flaws of traditional symbolic systems.
=======================
Dan Bauer
Cognitive Science, UCSD
dsbauer@cogsci.ucsd.edu
Questions and Summary of Hofstadter's review of Mental Leaps
discussed by Misty Karin
Cognitive Science, UCSD
Summary of Douglas Hofstadter's review of
Holyoak & Thagard _Mental Leaps: Analogy in Creative Thought_ (in _AI
Magazine_ fall 1995 issue) and its relation to LISA.
This critique by Hofstadter discusses the 1994 book _Mental Leaps_ and the
analogy-making model, ACME, which is discussed in the book. His main
complaint cuts right to the core of HolyoakUs treatment of analogy. For
Hofstadter, the gist, or "essence," of a situation is the basis for its
representation and is the connecting element in analogy. Extracting the gist
of a situation is the key to making an analogy. Hofstadter pointed to a
complete lack of focus on the essence of the situation in _Mental Leaps_ and
charged that Holyoak and Thagard failed to recognize the gulf between human
perception of a situation which has no hard and fast distinctions and the view
of a situation as a set of predicate calculus formulas full of distinct
objects, attributes, and relations.
Holyoak and Thagard emphasize the presence of a one-to-one correspondence
between the elements of analogous situations. They also see causal
relationships as a central anological connection. But this focus on causality
overlooks the importance of the complex situational themes present in the
gist. Also, causality is ubiquitous to all events - thus it is a meaningless
tool for classification. Hofstadter instead points to a one-to-one
correspondence at the level of the gist, not the elements, of analogous
situations.
ACME was designed to make analogies by mapping formalized situations onto
each other. It worked by mapping analogies based on the structural similarity
of situational predicates. ACME did not have any semantic knowledge of the
elements being mapped together. The information given to the model was a
formalization of the gist already extracted and translated into meaningless
symbols:
Saddam launched the Gulf War by invading Kuwait.
president-of(Saddam, Iraq) -> p(S, I)
invade(Iraq, Kuwait) -> i(I, K)
Hitler launched W.W.II by occupying Austria.
furher-of(Hitler, Germany) -> f(H, G)
occupy(Germany, Austria) -> o(G, A)
The gist of these situations is a countryUs leader acting to start a war.
This gist is pre-extracted for ACME. All the model does is play a matching
game with the symbolic elements it is fed.
With their new analogy-maker, LISA, Hummel and Holyoak use a connectionist
hybrid network to incorporate semantics into their analogical model. The
architecture of LISA is described in Dan's summary, so I will not duplicate
the detailed discription here. The major improvement over ACME is that
predicate elements are now connected to semantic units which supply a
rudimentary meaning to the proposition. A pair of analogous propositions will
share many semantic units, and infrences can be made about one proposition
from the semantic relationship with its analog.
Hummel and Holyoak claim that LISA solves the problem of maintaining
structure in a distributed representation, but they fail to address the major
criticism posed by Hofstadter in his critique of the earlier model. LISA
still uses formalized predicates, and still focuses on correspondences between
propositional elements. Hofstadter would claim thatthey are still modeling a
theory of analogy-making which ignores both gist extraction and concept
formation.
Questions for clarification:
There is an example of analogical inference given:
Analog 1: father(Abe, Bill) Analog 2: father(Adam, Bob)
brother(Charles, Abe) brother(Cary, Adam)
uncle(Charles, Bill)
LISA was able to infer the predicate uncle(Cary, Bob). Is the model
flexible enough to make the inference if the predicate arguments are not laid
out so neatly for the system?
What if analog 2 were given as father (Adam, Bob); Brother (Adam,
Cary)?
What if the propositions for analog 2 were given as mother, sister,
aunt? This is a simple analogy for us, but how would LISA handle it?
Hummel & Holyoak (1996) state that "analogical reasoning plays an
important role in schema induction." Yet the schema is basically the same as the
gist which Hofstadter holds as the basis for analogy. Could you address this
conflict between your views of analogy?
|
