LINGUISTICA COMPUTAZIONALE - Springer978-0-585-35958-8/1.pdf · Linguistica computazionale is...

22
LINGUISTICA COMPUTAZIONALE

Transcript of LINGUISTICA COMPUTAZIONALE - Springer978-0-585-35958-8/1.pdf · Linguistica computazionale is...

LINGUISTICA COMPUTAZIONALE

LINGUISTICA COMPUTAZIONALE

VOLUME IX· X

CURRENT ISSUES IN COMPUTATIONAL

LINGUISTICS: IN HONOUR OF DON WALKER

EDITED BY: ANTONIO ZAMPOLLI, NICOLETTA CALZOLARI,

MARTHA PALMER

Springer-Science+Business Media, B.V.

Linguistica computazionale is published two issues per year. Subscriptions should be sent to the Publisher Giardini editori e stampatori in Pisa, via delle Sorgenti 23, 1-56010 Agnano Pisano (Pisa), 1taly. Tel. +39-50-934242 . Fax +39-50-934200. Postal current account n. 12777561.

This book is co-published by Giardini editori e stampatori in Pisa and Kluwer Academic Publishers. Kluwer Academic Publishers incorporates the publishing programmes of D. Reidel, Martinus Ni­jhoff, Dr W. Junk and MTP Press.

© Springer Science+Business Media Dordrecht 1994

Originally published by Giardini editori e stampatori in Pisa in 1994

Softcover reprint ofthe hardcover Ist edition 1994

All rights reserved No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without written permission.

Library of Congress Cataloging-in-Production Data available

ISBN 978-0-7923-2998-5 ISBN 978-0-585-35958-8 (eBook) DOI 10.1007/978-0-585-35958-8

Royalties will be made available by Kluwer Academic Publishers to the Association of Computation­al Linguistics Don and Betty Walker International Student Fund.

CONTENTS

Preface IX

Introduction XVII

Donald Walker: A Remembrance XXI

SECTION 1- THE TASK OF NATURAL LANGUAGE PROCESSING

K. SPARCK }ONES, Natural Language Processing: A Historical Review 3

J. RoBINSON, On Getting a Computer to Listen 17

B. GRosz, Utterance and Objective: Issues in Natural Language Communication 21

M. KING, On the Proper Place of Semantics in Machine Translation 41

G. G. HENDRIX, E. D. SACERDOTI, D. SAGALOWICZ, }. SLOCUM, Developing a Natural Language Interface to Complex Data 59

K. KUKICH, K. McKEoWN, J. SHAw, J. RoBIN,}. LIM, N. MoRGAN, J. PHILLIPS, User-Needs Analysis and Design Methodology for an Automated Document Generator 109

SECTION 2- BUILDING COMPUTATIONAL LEXICONS

B. BOGURAEV, Machine-Readable Dictionaries and Computational Linguistics Research 119

R. A. AMSLER, Research Toward the Development of a Lexical Knowledge Base for Natu-ral Language Processing 155

R. J. BYRD, Discovering Relationships among Word Senses 177

E. HAJICOVA, A. RosEN, Machine Readable Dictionary as a Source of Grammatical In-formation 191

S. PIN-NGERN CoNLON, J. DARDAINE, A. D'SouzA, M. EVENS, S. HAYNES, J. S. KIM, R. STRUTZ, The liT Lexical Database: Dream and Reality 201

J. L. KLAVANS, Visions of the Digital Library: Views on Using Computational Linguistics and Semantic Nets in Information Retrieval 227

B. T. ATKINS, J. KEGL, B. LEVIN, Anatomy of a Verb Entry: from Linguistic Theory to Lexicographic Practice 237

N. CALzoLARI, Issues for Lexicon Building 267

N. IDE, J. LE MAITRE, J. V~RONIS, Outline of a Model for Lexical Databases 283

L. LEVIN, S. NIRENBURG, Construction-Based MT Lexicons 321

VII

P. SGALL, Dependency-Based Grammatical Information in the Lexicon 339

H. ScHNELLE, Semantics in the Brain's Lexicon - Some preliminary Remarks on its Episte-mology 345

SECTION 3 -THE ACQUISITION AND USE OF LARGE CORPORA

D. E. WALKER, The Ecology of Language 359

D. BIBER, Representativeness in Corpus Design 377

C. M. SPERBERG-McQUEEN, The Text Encoding Initiative 409

W. A. GALE, K. W. CHURCH, D. YAROWSKY, Discrimination Decisions for 100,000-Dimensional Spaces 429

S. ARMsTRONG-WARWICK, Acquisition and Exploitation of Textual Resources for NLP 451

S. HocKEY, The Center for Electronic Texts in the Humanities 467

N.J. BELKIN, Design Principles for Electronic Textual Resources: Investigating Users and Uses of Scholarly Information 479

SECTION 4- TOPICS, METHODS AND FORMALISMS IN SYNTAX, SEMANTICS AND PRAGMATICS

A. K. JosHI, Some Recent Trends In Natural Language Processing 491

J. R. HoBBS, J. BEAR, Two Principles of Parse Preference 503

M. NAGAO, Varieties of Heuristics in Sentence Parsing 513

R. JoHNSON, M. RosNER, UD, yet another Unification Device 525

J. FRIEDMAN, D. B. MoRAN, D. S. WARREN, Evaluating English Sentences in a Logical Model 535

M.S. PALMER, D. A. DAHL, R. J. SCHIFFMAN, L. HIRSCHMAN, M. LINEBARGER, J. DOWD-ING, Recovering Implicit Information 553

C. L. PARis, V. 0. MITTAL, Flexible Generation: Taking the User into Account 569

Y. WILKS, Stone Soup and the French Room 585

VIII

Preface

With this volume in honour of Don Walker, Linguistica Computazionale con­tinues the series of special issues dedicated to outstanding personalities who have made a significant contribution to the progress of our discipline and maintained a special collaborative relationship with our Institute in Pisa. I take the liberty of quoting in this preface some of the initiatives Pisa and Don Walker have jointly promoted and developed during our collaboration, because I think that they might serve to illustrate some outstanding features of Don's personality, in particular his capacity for identifying areas of potential convergence among the different scientific communities within our field and establishing concrete forms of coop­eration. These initiatives also testify to his continuous and untiring work, dedi­cated to putting people into contact and opening up communication between them, collecting and disseminating information, knowledge and resources, and creating shareable basic infrastructures needed for progress in our field.

Our collaboration began within the Linguistics in Documentation group of the FID and continued in the framework of the !CCL (International Committee for Computational Linguistics). In 1982 this collaboration was strengthened when, at CO LING in Prague, I was invited by Don to join him in the organization of a series of workshops with participants of the various communities interested in the study, development, and use of computational lexica. I was unable to participate in the first workshop organized by Don at SRI in 1983, because I was involved in a CETIL meeting 1 at the same time in Luxembourg, where the suggestion of holding a second workshop in Europe was accepted. So, Don, Nicoletta Calzolari and I, together with Loll Rolling (CETIL promoter) and Juan Sager (CETIL President), organized the workshop "On Automating the Lexicon" in 1986 in Grosseto, sponsored by the CEC, the Council of Europe, ACL, A/LA, ALLC and EURALEX. The objective of this workshop was to survey research efforts, current practices, and potential developments in work on the lexicon, machine­readable dictionaries, and lexical knowledge bases, with special consideration of the problems created by working in a multilingual environment. The brief was to recommend directions for future activities. The participants were chosen to bring together, for the first time, representatives of all those working on the lexicon: lexicologists, grammarians, semanticists, lexicographers, computational linguists, artificial intelligence specialists, cognitive scientists, publishers, lexical software producers, translators, terminologists, and representatives of funding agencies and of professional associations. The final recommendations, transmitted to the CEC

1 CETIL was a CEC Committee of Experts in Language and Information.

IX

and widely distributed, could be summarized as follows: 2

• To establish procedures for creating multifunctional databases from the in­formation contained both implicitly and explicitly in those traditional dic­tionaries that exist in machine-readable form.

• To develop computational tools for more efficient handling of lexical and lexicographical data, and to provide 'workstation' environments within which these tools may be used by lexicologists and lexicographers.

• To explore the possibility of creating multifunctional lexical databases capa­ble of general use, despite divergences of linguistic theories and differences in computational and applicational frameworks.

• To study the possibility of linking lexical databases and large text files, in both monolingual and multilingual contexts, in order to determine the most effective ways of exploiting the relationships among the various lexical ele­ments.

Don dedicated an unbelievable amount of time, effort, and care in preparation of this workshop. Don, Nicoletta and I were in touch nearly every day for more than a year. I remember this, with joy, as one of the happiest and most fruitful periods of my working life. During the organization of the workshop, and as we went along establishing relationships with the participants invited to it, we had the feeling that a new research paradigm was emerging more and more clearly every day. In Pisa, we found here a confirmation of our work on texts, reusable lexica, and related tools, despite the almost complete lack of interest in the major trends in contemporary computational linguistics. Don found continual confirmations for his intuition of the multi-disciplinary centrality of the lexicon, and for the necessity of a vast organization to provide the various communities, represented in the workshop, with basic linguistic resources.

The Grosseto Workshop is now recognized as marking the starting-point of a new phase in the field of computational linguistics. This phase is character­ized by an increasing number of fresh initiatives, particularly at the international level, whose aim is to further the development of the scientific, technical, and organizational conditions conductive to the creation of large multifunctional lin­guistic resources, such as written and spoken corpora, lexicons and grammars. An updated version of the proceedings of the Grosseto Workshop, edited by Don, Nicoletta and me, will appear soon (Walker et al, 1994). Don put a lot of effort into collecting and updating the articles, and we are extremely grateful that he was able to finish this task.

Don was particularly active in promoting actions whose purpose was to realise the strategic vision that emer·ged during the workshop. He organized two follow­up workshops, one in New York in 1986, the other, in 1987, in Stanford, in the framework of the Linguistic Summer School of the LSA.

2 See Zampolli, 1987, for the complete text of the recommendations.

X

He also organized a panel on linguistic resources at the 1988 Pisa Summer School on Computational Lexicography and Lexicology, explicitly designed to con­tribute to the goals identified during the Grosseto Workshop. Unfortunately, the day before the panel was due to meet, Don notified me that he was not able to come, because his illness had just been discovered and he was about to have his first operation. I still remember that phone call with deep emotion. Don encour­aged me to carry on with the work regardless. The way in which, during the five years of his illness, he was able to continue working with energy and enthusiasm, guiding and encouraging us in the various initiatives we were promoting together, and always planning new activities, was for me a miraculous example of what devotion to one's discipline, community and ideals can really achieve.

The day after the Grosseto Workshop, I set up a group (Hans Uszkoreit, Nicoletta Calzolari, Bob Ingria, Bran Boguraev) to explore the feasibility of con­structing large-scale linguistic resources, explicitly designed to be multifunctional, i.e. capable of serving, through appropriate interfaces, a wide variety of present and future research and applications. A crucial and controversial problem was to what extent it was possible, as well as desirable, to make linguistic resources, at least within certain limits, "polytheoretical", i.e. usable in different linguis­tic frameworks. Don immediately intervened and secured for the group, initally supported by our Institute, the help of the ACL. He was enthusiastic about the goal of this group, (the so-called "Pisa - Group"), which was extended to in­clude outstanding representatives of some major linguistic schools, and (I quote Don's words) "investigated in detail the possibility of a polytheoretical represen­tation of the lexical information needed by parsers and generators, such as the major syntactic categories, subcategorization and complementation. The common representation sought was one that could be used in any of the following theoret­ical frameworks: government and binding grammar, generalized phrase structure grammar, lexical functional grammar, relational grammar, systemic grammar, dependency unification grammar and categorial grammar" (Walker et al, 1987}.

After the Grosseto Workshop, we wrote a report together for the CEC DC­XIII, suggesting a large two-phase programme: a one-year phase, to define the methods and the common specifications for a coordinated set of lexical data bases and corpora for the European languages, and a second three-year phase for their actual construction (Zampolli, Walker, 1987}. This programme, it seems, is fi­nally being realized now, even if organizational and financial problems have made its progress slower than thought.

The first step taken by the CEC consisted in a feasibility study: the ET-7 Project, building on the encouraging results of the "Pisa-Group" work. This project was launched by the CEC with the aim of recommending a methodol­ogy for the concrete construction of shareable lexical resources. Since different theories use different descriptive devices to describe the same linguistic phenom­ena and yield different generalizations and conclusions, ET-7 proposed the use of the observable differences between linguistic phenomena as a platform for the exchange of data. In particular, the study has assessed the feasibility of some basic standards for the description of lexical items at the level of orthography, phonology/phonetics, morphology, collocation, syntax, semantics and pragmatics.

XI

(Heid, McNaught 1991} The second major step is now underway, and is represented by the establish­

ment of projects focussing on the notion of standards. Two examples in the field of lexical data are the CEC ESPRIT project MULTILEX, whose objective was to devise a model for multilingual lexicons (Khatchadourian and Modiano, 1994}, and the EUREKA project GENELEX, which concentrates on a model for mono­lingual generic lexicons (Antoni-Lay, Francopoulo and Zaysser, 1993). In the area of textual corpora the CEC sponsored the NERC study aiming at defining the scientific, technical and organizational conditions for the creation of a Net­work of European Reference Corpora, and at exploring the feasibility of reaching a consensus on agreed standards for various aspects of corpora building and analysis (see NERC Final Report, 1993}. The European Speech community had indepen­dently organized an outstanding standardization activity, coordinated through the ESPRIT Project SAM (Fourcin, Gibbon, 1993). Feeling it necessary to coordi­nate their activities, the representatives of these various standardization projects formed an initial, preparatory group. Enlarging this group, the CEC established the EAGLES project in the framework of the LRE programme. This project aims to provide guidelines and de facto standards, based on the consensus of the major European projects, for the following areas: corpora, lexica, formalism, assessment and evaluation and speech data. (Elsnews Bulletin, 2(1}, 1993). The project also encompasses an international dimension, influenced by Don's suggestions, which includes: the support of the European participation in the TEl; the preparation of a survey of the state-of-the-art in Natural Language and Speech Processing, jointly sponsored by the NSF and the CEC; the preparation of a Multilingual Corpus (MLCC) intended to support Cooperation with similar ARPA sponsored initiatives; and the exploration of possible strategies for international cooperation and coordination in the field of linguistic resources.

We are now on the verge of the third step needed to complete the programme suggested in our 1987 report to the CEC: to define and provide a common set of basic multifunctional reusable linguistic resources for all the European languages, available in the public domain, and to create an infrastructure for the collection, dissemination and management of new and existing resources. The second task is taken care of, at the experimental level, by the new LRE-RELATOR project (Elsnews Bulletin, 2(5}, 199.'1). The MLAP call issued recently by the CEC DG­XIII, gives us the hope that the first task will be finally undertaken within the 4th Framework Resear·ch Programme of the CEC.

Given his interest for· making linguistic resources shareable, and people and disciplines cooperate, the deep involvement of Don in standardization efforts is not surprising. His participation, from the very beginning, in the process of set­ting up the Text Encoding Initiative, is yet another testimony to his capacity to overcome (often artificial) disciplinary boundaries and of his determination in realizing his strategic vision of the future. Nancy Ide, President of the ACH, in November 1987 held a brainstorming workshop of representatives of the ACH and ALLC at Vassar to explore the desirability and feasibility of standards for encod­ing and exchanging te1:ts in the field of humanities. Don, informed of this event, contacted Nancy and was also invited together with Bob Amsler. I remember my

XII

emotion on hearing Don {in the evening I had gathered Don round the fire together with Susan Hockey and Nancy Ide, to discuss the possibilities of cooperation of the three Associations} announcing that he would ensure the support of the ACL to a joint initiative with the ACH and ALLC. Those of us old enough, like me, to have lived through the entire history of linguistic data processing, will probably understand my reaction. In the '50s and early '60s, activities such as machine translation, document retrieval, lexical text analysis for humanities, statistical linguistics, quantitative stylistics, which later on were separated by scientific and organizational barriers, were linked by frequent contacts and recognized each other as "poles" of a disciplinary continuum, linked together by an interest in computa­tional processing of large "real" texts and language data. In the second half of the '60s, computational linguistics, in an effort to define its disciplinary identity, and under the influence of the generative paradigm in linguistics and of the embryonic AI, became progressively detached from the interest and work on "real" language data. The occasions for contact with the community of humanities computing became increasingly rare, despite the efforts of some institutes, which refused this separation, and continued to consider the computational processing of real lan­guage data as a unifying goal to which the various disciplines could contribute relevant know-how, methods and tools.

Reversing this trend, Don went on to play a central role within the TEl, where he contributed his capacities to mediate, to organize and balance our efforts, and to envision the future. Despite our tremendous loss, we are determined to realize his vision of the TEl as a bridge between standardization activities in various con­tinents. He also recognized the areas of potential synergies between computational linguistics and literary and humanistic computing, and identified the benefits po­tentially deriving from mutual cooperation. He promoted, in particular through the friendships established in the framework of TEl with Susan Hockey {chairman of the ALLC), various events jointly sponsored by the ACH, ACL, ALLC, and above all, he actively contributed, from the initial steps to the establishment of the CETH (Center for Electronic Texts in Humanities). · Don's interest in linguistic resources (conceived as a fundamental infrastruc­

ture for natural language and speech processing) and standards (conceived as a means for allowing exchanges and cumulative efforts) naturally converged with his vision of "Language, information and knowledge .... combined in human efforts after communication" 3 and with the profound humanism that pervaded all his activities. At the basis of his capacity to open up lines of communication between peoples and communities and to propose new collaborative initiatives, of his will to create the best working conditions for everyone, of his continuous and untiring work in the promotion of scientific development in our sector, considered by him to be inseparable from the development of people, lie, I believe, not only his desire to open new perspectives maximizing synergies and interdisciplinary fertilization, but also and above all his genuine, profound interest in people. In the various activities of his long career, he was always inspired by a sense of help, and by a deeply democratic and humanistic conception of research and its management.

I would like to relate a personal anecdote which revealed to me Don's strong

3 See Don's article in this volume.

xm

values. One evening, on the occasion of a reunion of the "Pisa Group", I invited Don to my house, with Annie Zaenen, to listen to some music, and at a certain moment I put on the records of "The Weavers Reunion at Carnegie Hall" of 1953 and 1963. I was surprised to see that Don became more and more moved as each song played, and I asked him why. He told me that the Weavers remained, at the beginning of his scientific career, during the worst period of the cold war, one of the most visible symbols of democracy and freedom of culture in his country, the basic values that had inspired his life.

To close, I would like to thank, in the name of our journal Linguistica Com­putazionale, all those who have contributed to the realization of this issue, moved by admiration, gratitude and love for Don and his work. The Scientific Council of the ILC, in its role as editorial board of the journal, has approved and sustained my proposal to dedicate this number to Don Walker. Nicoletta Calzolari, Susan Hockey, James Pustejovsky and Susan Armstrong have signed with me the invi­tation to the contributors. As co-editor of this volume, Martha Palmer expended substantial scientific and organizational effort in collecting and preparing these articles, for which our heartfelt thanks. We are grateful as well to Carolyn Elken, Tarina Ayazi and Antonietta Spanu for their precious assistance.

Antonio Zampolli

I only wish to express my deep gratitude to Don, with very simple words, and of my many different memories of him just touch on one or two of those which perhaps mean most to me. I strongly feel the need to thank Don for what I received from him as a friend, as a fellow linguist, and as a person.

As a friend he treated me as I later realised he treated others whom he trusted: he helped me, he encouraged me, he gave me the right opportunities in the right moments, and I always knew that I could rely on him, at any moment. And this is a ver·y rare gift indeed. We worked together in a wonderful way, as only friends can do.

As a linguist, we all know what a great contribution he made to our field and in how many different ways, both big and small: moving things forward, putting people together, encouraging newcomers, understanding in advance what would be important in the coming years, and acting concretely to make things happen. Among his many talents, he was a great builder.

As a person, I really must express my immense admiration for the way in which, once he knew of his illness and right up to the last days, he not only engaged in his own very private but constant battle but was constantly and coura­geously able to talk about the future with others, to make plans, to act towards the achievement of results that he knew that he personally would never see. And always with a smile and a kind light in his eyes. This for me was the clearest sign of his greatness. This memory will remain with me forever.

Nicoletta Calzolari

XIV

References [1] Antoni-Lay, M.H., G. Francopoulo, L. Zaysser, "A Generic Model for Reusable

Lexicons: The Genelex Project", Literary and Linguistic Computing, 8 (4), Oxford University Press, 1993.

(2] "RELATing to Resources", Elsnews Bulletin, 2 (5), 1993.

[3] "EAGLES Update", Elsnews Bulletin, 2 (1), 1993.

(4] Fourcin, A., D. Gibbon, "Spoken Language Assessment in the European Context", Literary and Linguistic Computing, 8 (4), Oxford University Press, 1993.

[5] U. Heid, J. McNaught (eds.), "Eurotra-7 Study: Feasibility and Project Definition Study on the Reusability of Lexical and Terminological Resources in Computerised Applications", Eurotra-7 Final Report, Stuttgart, 1991.

(6] Khatchadourian, H., N. Modiano, "Use and Importance of Standard in Electronic Dictionaries: The Compilation Approach for Lexical Resources", Literary and Lin­guistic Computing, 8 ( 4), Oxford University Press, 1993.

(7] LRE, "Technical Report Document", Unpublished document available from R. Cen­cioni, CEE- DGXIII, Luxembourg.

(8] NERC Final Report, Istituto di Linguistica Computazionale, Pisa, 1993.

(9] Walker, D., A. Zampolli, N. Calzolari, "Toward a polytheoretical Lexical Database", Istituto di Linguistica Computazionale, Pisa, 1987.

(10] Walker, D., A. Zampolli, N. Calzolari (eds.), On Automating the Lexicon: Research and Practice in a Multilingual Environment, Proceedings of a Workshop held in Grosseto, Oxford University Press, Oxford, 1994.

(11] Zampolli A., Walker D., "Multilingual Lexicography and Lexicology; New Direc­tions", Unpublished paper presented to CGXII-TSC of the CEC.

(12] Zampolli A., "Perspectives for an Italian Multifunctional Lexical Database", Stud­ies in honour of Roberto Busa S.J., A. Zampolli, A. Cappelli, C. Peters (eds.), Linguistica Computazionale, Pisa, 6, 1987, 301-341.

[13] Zampolli A., "Technology and Linguistic Resources", in M. Katzen (ed.), Scholarship and Technology in the Humanities, British Library Research, London, 1991.

XV

Introduction

This collection of papers is dedicated to Don Walker, a kind, gentle soul, and a man of intelligence and vision whom we have been privileged to know and to work with. The technical content of these papers clearly reflects his guidance and direction: his understanding of the relevance of theories from related disciplines such as linguistics, psychology, and philosophy; and his dream of the future availability of electronic docu­ments and the potential of this as a resource for corpus-based analysis. This led to one of his most interdisciplinary roles, as a primary mover in the push to standardize the acquisition and tagging of electronic text, an endeavor that brought together people from fields as distinct as publishing, the humanities and computer science. His wide-spread interests are mirrored in the rich and diverse collection of papers in this volume which portray many different approaches and many different styles. These papers co-exist in harmony here, united by the goal of paying tribute to Don as well as actualizing part of his dream.

Anyone perusing this book will want to start with Karen Sparck Jones's entertaining and insightful history of natural language processing. Not only does she aptly sum up the driving forces behind particular developments in the field, but her history provides an appropriate context in which to appreciate the rest of the book. Karen sees the initial impetus for the field beginning with the push to perform automatic machine translation. It has been followed by three separate phases of development, each of which can be characterized by a distinct methodological approach, as researchers have explored the incorporation of techniques and theories from other disciplines. The first of these was focused on building knowledge based natural language systems, the second one, the grammatico-Iogical phase, was aimed at identifying an appropriate granularity of logical formalisms for representing syntactic and semantic information, and finally, our current phase, based on making the best possible use of the vast quantities of text that are now available electronically. In spite of the differences, she sees common themes running through all four phases, such as the balancing of generalizable principled methods with the "ad-hoc particularization necessary for specific applications," "the relative emphasis to be placed on syntax and on semantics" and an attempt to measure "the actual value of the results, especially when balanced against pre- or post- editing requirements." She also sees a certain inevitability in the field's return to machine translation as one of its preeminent application areas, and the subsequent interest in cross-linguistic studies.

Our first section, The Task of Natural language Processing, begins with Karen's paper, which is followed by three classic articles highlighting some of the most intractable problems in the field, the solutions to which still lie tantalizingly out of reach, and which transcend any particular methodological approach: Jane Robinson describes the complexities inherent in understanding spoken language; Barbara Grosz discusses the importance of shared models for effective communication and the distinction between an utterance and the objective of its speaker; and Maghi King explores linguistic evidence

XVII

supporting the presence or absence of semantic universals. The difficulties inherent in the task of natural language processing have not daunted many intrepid system builders, and this section concludes with two different real-world applications, both of which have had to make reference to the balancing of the principled methods exemplified by the preceding papers and "ad-hoc particularization." The first application is a classic database question answering system developed by Gary Hendrix, et al., in the late 70's under Don's leadership. It is followed by a current, state of the art, documentation generation system from Don's more recent colleagues at Bellcore, Karen Kukich, et al.

The sections that follow focus more on the particular modules and components that these large-scale systems are comprised of, with special emphasis being placed on lexical issues and deriving infonnation automatically from dictionaries and corpora. Our second section, Building Computational Lexicons, clearly presents the need for large-scale lexicons, and addresses the question of "How can we build the lexicons we need in the time available?" Almost every one of these papers discusses the difficulty of augmenting dictionary entries, whether machine-readable or not, with the infonnation required by a natural language application, and the necessity of finding automatic methods for this augmentation. The section begins with a comprehensive overview of the field by Bran Boguraev that provides a useful context for the rest of the papers. It is followed by a paper from Bob Amsler describing the automated derivation of a lexicon from newspaper articles. The subsequent papers examine the advantages and drawbacks of different ways in which Machine Readable Dictionaries (MRDs) can be used for the automatic derivation of infonnation for a computational lexicon. The first of these, a paper by Roy Byrd, focuses on adequately representing polysemous word senses, relying as much as possible on infonnation that is automatically derived from machine readable dictionaries. Then Eva Hajicova discusses a system that obtains grammatical infonnation about verb argument structure from machine readable dictionaries. This is followed by a detailed description of an actual implementation, the impressive liT lexicon project headed by Martha Evens. The Judy Klavans paper presents proposals for incorporating a semantic net derived from MRDs into information retrieval systems. Sue Atkins, Judy Keg! and Beth Levin carefully examine the elements of a verb's meaning that are reflected in its syntactic alternations, and how they are and are not captured by typical dictionary entries. Nicoletta Calzolari 's paper, ten years after the Boguraev paper and after much work on deriving part of the information needed for a computational lexicon from MRDs, puts forward a number of issues which anyone aiming at building a very large multifunctional lexicon for NLP should pay attention to. The paper by Nancy Ide proposes a novel data base scheme especially designed for computational lexicons. The Sergei Nirenberg and Lori Levin paper introduces constructions, a class of linguistic phenomena that is especially difficult to deal with on a word by word basis. The paper by Petr Sgall discusses in detail the underlying theoretical approach for including syntactic infonnation into lexical entries. This section concludes with an intriguing discussion of a new approach to representing semantic infonnation by Helmut Schnelle.

This brings us to our third section, The Acquisition and Use of Large Corpora which is concerned with making the best possible use of the vast quantities of text that are now available electronically, clearly situated in Karen's statistical phase. The paper by Don Walker gives a broad overview of the field. It is followed by a paper by Doug Biber which deals with corpus design, a very important issue in the collection of a corpus. C.

XVIII

M. Sperberg-McQueen discusses guidelines and standards in text representation. The paper by Bill Gale, Ken Church, and David Yarowsky describes significant advances in the application of statistical word probabilities to diverse problems in disambiguation. Both the Susan Armstrong-Warwick and Susan Hockey papers are aimed at the task of acquiring and disseminating text in electronic form. This section ends with a paper that discusses ways in which the corpora can be used, with Nick Belkin looking at principled ways of making on-line data available to scholars.

We end the book with a section that returns to the theme of system building, and discusses the integration of separate components, and issues involved in applying these techniques to other languages: Topics, Methods, and Formalisms in Syntax, Semantics and Pragmatics. Syntax has been an area of concern from the beginning, when the early work on machine translation revealed the necessity of valid syntactic information, " ... long-distance dependencies, the lack of a transparent word order in languages like German, and also the need for a whole-sentence structure characterization to obtain properly ordered output, as well as a perceived value in generalization, led to the de­velopment of autonomous sentence grammars and parsers." In addition to highlighting the field's undying interest in parsing, this section also illustrates the more recent move towards cross-linguistic studies. We begin with a topical paper by Aravind Joshi that explores the applicability of statistical data derived from large corpora to the problem of syntactic parsing. He presents a hybrid approach which incorporates statistical data into a more conventional grammar based parsing system, signaling a recent trend towards the integration of statistical information with more traditional approaches. The next three papers are also primarily concerned with syntax, in a more and more global sense. While the Hobbs paper focuses on strategies for the syntactic parsing of English, the Nagao paper examines the applicability of English grammars and parsing algorithms to the processing of Japanese. The Rosner paper describes a state of the art formalism based on feature unification that can be applied cross-linguistically. The rest of this section concentrates on integrating syntactic information with other sources of informa­tion, such as semantics and pragmatics. The Joyce Friedman and Dave Warren paper describes their implementation of Montague grammar, which subsequently served as the springboard for much of the grammatico-logical phase. It is followed by the Martha Palmer, et al., paper, presenting a later implementation that used a less formal semantic representation as the basis for an integration of semantics and pragmatics. The following more recent paper, also a pragmatics paper, is by Cecile Paris and Vibhul Mittal, and describes a system implementation that requires a user model, in particular knowledge of the user's expertise, to appropriately tailor text generation. Finally, we end this section where we began, with a look at the use of statistical data derived from large corpora, this time for the comprehensive task of machine translation. Yorick Wilks discusses the IBM statistically based machine translation system which he portrays as having fallen somewhat short of its original goals. Similarly to Joshi, Wilks argues in favor of a hybrid approach that will incorporate statistical methods into more traditional systems. This is a positive note on which to end, showing that we can continue to build on the work of the past, incorporating first linguistics, then artificial intelligence and formal logic, and now even statistical methods, into our natural language processing systems, making them richer, more robust, and more reliable, and bringing them yet another step closer to that ultimate goal, of someday really understanding what a human is trying to communicate.

XIX

The process of putting the volume together has been very revealing of Don 's personal touch, the inimitable stamp of courage and gallantry that was so uniquely his. Every author was kind and gracious to deal with, eager to make a contribution and to do everything possible to ease the task of the editors. And every author could relate some incident, some little story, some way in which Don had put them in his debt; a debt they delighted in repaying. This personal view of Don comes across very poignantly in Don Walker: A Remembrance by Jerry Hobbs and Barbara Grosz which follows this introduction. We can say unanimously, with the contributors to this volume, and his colleagues from Bellcore, "Our greatest debt is to Don Walker, whose incredible energy first brought us together and inspired us to make this practical [natural language] application a reality. His legacy will fire the imagination and influence the practice of computational natural language processing for ages to come."

In addition we would like to express our appreciation to all of the authors and to the many people who worked tirelessly to put this volume together in record time, and to an unusually flexible publisher, Giardini. We could never have made our deadline without the generous help of many University of Pennsylvania graduate students and staff who volunteered for proofreading and copyediting, such as Julie Bourne, Brett Douville, Dania Egedi, Nobo Komogata, Libby Levison, Michael Niv, Joseph Rosen­zweig, Matthew Stone and Mike White. We would especially like to thank Tilman Becker, Alexis Dimitriadis, B. Srinivas, and David Yarowsky who worked into the small hours of the morning to resolve all of our latexing conundrums. Aravind Joshi very generously provided the necessary computing and printing resources. Allison Anderson did a sterling job as copy editor with an impossible overnight turnaround, and finally, our heartfelt gratitude goes to Carolyn Elken and Tarina Ayazi, who kept track of all the papers, and made it all happen. Carolyn, in particular, made latexing and copyediting changes to almost every paper, in a prodigious effort on her part that single-handedly enabled us to finish on time.

Martha Palmer May, 1994

XX

Donald Walker: A Remembrance

with contributions from

Robert Amsler, Woody Bledsoe, Barbara Grosz, Joyce Friedman, Eva Hajicova, Jerry R. Hobbs, Susan Hockey, Alistair Holden, Martha Palmer,

Fernando Pereira, Jane Robinson, Petr Sgall, Karen Sparck Jones, Wolfgang Wahlster, and Arnold Zwicky

Don Walker had a vision of how natural language technology could help solve people's problems. He knew the challenges were great and would require the efforts of many people. He had a genius for bringing those people together.

For this introduction, a number of people who had known Don over the years were asked to write reminiscences. Though each person's story differed, a striking commonality emerged. It is remarkable how often Don was present at the key juncture in people's careers, and, in his understated, soft-spoken, low-key way, he did just the right thing for them. What Don did almost always involved bringing people together.

There is a story by Jorge Luis Borges in which someone travels all over India attempting to discover the nature of a very wise man, through the subtle but profound influence he had had on the people he met. Reading the reminiscences of Don and seeing the impact he had on people's lives reminded us of that story.

Don organized research teams. He was often instrumental in matching people with positions in laboratories located far from his own as well as in groups he managed. Several people attribute to Don significant help in starting their careers or in finding the funding that supported their most productive period of research. He often gave essential advice or provided the key opportunity that led to a fruitful new direction in someone's career. Don knew who was doing everything, how to get in touch with them, and how to facilitate the appropriate actions that matched the person with the opportunity.

This was first evidenced in the group he built at Mitre, after a sojourn at Rice University and a year at MIT. A psychologist by training, Don was one of the first to recognize the relevance of contemporary MIT linguistics to computational processing of language and in 1963 at MITRE he found Air Force support for that application. The work at MITRE was on the problem of formalizing grammars for computer ap­plication to obtain "deep structures" of grammatical sentences by means of inverse transformations-transformational grammar. It resulted in a large computer program, described in internal MITRE documents, and presented to the 1965 AFIPS conference. The group he assembled was strikingly interdisciplinary, especially for its time. At one stage or another, it included a large number of linguists, philosophers, psychologists, mathematicians, and computer scientists. Linguistics students from MIT played a key

XXI

role, often spending their summers at MITRE. Among them were many young scholars such who have since become leaders in their fields, and remain grateful to Don for his help when they were starting out. Some of the people who worked at Mitre during this period were Ted Cohen, Tom Bever, Paul Chapin, Ted Cohen, Bruce Fraser, Joyce Friedman, Mike Geis, Dick Glantz, Ted Haines, Barbara Hall (Partee), Steve Isard, Dick Jeffrey, Jackie Mintz, Stan Peters, Peter Rosenbaum, Haj Ross, lim Scanlon, Sandy Shane, Howard Smokier, and Arnold Zwicky.

Don was an amazing project leader at MITRE. He encouraged the widest range of explorations, gently but firmly critiqued everything everyone did, and somehow coped with the corporation and military brass. Yet he was always self-effacing and almost apologetic about his accomplishments.

Don went on from Mitre to SRI, where in the 1970s he built the natural language group there that continues today as one of the premier natural language research groups in the world. Barbara Grosz recalls how he gave her her first AI job, even though she had yet to pick a thesis topic, let alone finish a Ph.D. In doing so, he took a risk of a magnitude that she fully appreciated only years later when she was hiring research associates. It was not a unique gamble; Don was often credited at Mitre with identifying good people before their reputations were widely established and he continued this process at SRI.

Jerry Hobbs recalls meeting Don at the ACL conference in Boston in 1975. Don greeted him so enthusiastically that he applied for a job at SRI. Little did he know that Don greeted everyone that way.

Don was also a wonderful group leader at SRI, not least because he so thoroughly integrated the personal and professional. Don appreciated and cared about the whole person of each member of the group, and nurtured them. They were as much family as research group, working hard and arguing hard, but appreciating one another and truly a team. They bonded closer in crises in a way few groups achieve. Don knew his people and led them to work together through the best and the worst. He demonstrated that one need not compromise personal warmth or care for individuals to have a top-rate research group. He also continued to provide guidance for members of his group even after they were in different labs. He often worked behind the scenes in many places and at many crucial junctures to make sure that their work was recognized. Don would never admit to these efforts, but he smiled in a certain way when asked. Asking was the closest one could come to saying thanks, and the smile the only sign that he'd welcomed the appreciation.

Don not only brought people together; he continued to bring disciplines together. In forming the SRI natural language group, he again built up an interdisciplinary group of researchers long before "cognitive science" was a phrase, and long before the need for interdisciplinary approaches to the study of language was taken for granted. He firmly believed that interdisciplinary work was the only way to go; to succeed at building the kind of computer systems they were after, AI researchers would have to be informed by linguistics, philosophy, psychology, and sociology. He constructed an environment that freed the people in his group to concentrate on their research, and he communicated his own excitement and enthusiasm for what could be done in this new interdisciplinary, unpredictable field where human language and computers meet.

Don played a central role in the organization and operation of the Association for Computational Linguistics (ACL), the International Conference on Computational

XXII

Linguistics (Coling), the International Joint Conference on Artificial Intelligence (U­CAI), and the,American Association for Artificial Intelligence (AAAI). He was Sec­retaryffreasurer for most of them, some for many decades. Expressing a theme that appeared again and again in the reminiscences, Wolfgang Wahlster, Conference Chair of IJCAI-93, said, "Don guided me like a father through the complex UCAI world." This feeling is one every ACL and IJCAI official through the years has shared.

When Don took over from Hood Roberts as Secretary-Treasurer of ACL, the organi­zation was relatively small. It grew, and Don's job grew with it, in size and complexity. Over the years, ACL officials have had a glimmer of how many concerns Don had to deal with and how much he did, and they have been vaguely aware that there were all sorts of other things he was managing behind the scenes. But only now have the scale and variety of these responsibilities become apparent. In a report about her recent visit to Don and Betty's house in Bernardsville, Karen Sparck Jones said, "Betty told me that in looking for houses when they moved to New Jersey one of the important considerations was that there had to be a good big basement for the ACL office. Sitting in it and talking with Don and Betty about the gritty details of ACL's finances was a direct encounter with what running something like ACL has implied for Don and thus how much he has contributed to building it up to the first-class society it is." Evident in all this is not only that Don did an enormous amount of work keeping the ACL show on the road and improving its act all the time, but also that he had continual concern and respect for all the different parties in the ACL and for the ACL's wider interests in the community and, especially, the international community.

Don was Program Chair of the first UCAI and General Chair of the next IJCAI. Alistair Holden,- who was General Chair of the first IJCAI, reports that as the General Chair of the second conference Don "organized IJCAI as it is today; an independent body governed by trustees who come from the international AI community .... [He] provided the continuity and organization that have led to the present IJCAI success." As Secretary-Treasurer of IJCAI for many years, Don provided invaluable guidance to successive Conference Chairs and Boards of Trustees. Recently, the Principles of Knowledge Representation and Reasoning conference, incorporated March 1993, adopted a governing structure that closely mirrored IJCAI's. It is no small tribute to Don that a structure he formulated more than twenty years ago is being copied today.

In recognition of Don's many contributions to the IJCAI organization and his very great service to the international AI community, the IJCAI, Inc. Board of Trustees renamed the IJCAI Distinguished Service Award at IJCAI-93. The award is now the Donald E. Walker Distinguished Service Award. Daniel G. Bobrow, the first recipient of the newly named award and a former President of AAAI, remarked on accepting the award that Don had provided a model of service which everyone in the field could look up to.

Don 's well-known organizational skills led Raj Reddy to invite him to participate in formative discussions of the AAAI (on the way to IJCAI-79) and designated him the first Secretary-Treasurer. Don's knowledge of financial procedure was invaluable in those early days, as was his considerable memory for the details of the charter, California law, and resolutions passed in Council meetings. Don was so absolutely dependable and trustworthy that it was not until well after his term was up that anyone thought that the treasury of the AAAI needed any protection from an incumbent Treasurer.

XXlll

Don was always fair; he carried his sense of what was right and fair to the complex discussions of arrangements between AAAI and IJCAI; his considerable efforts and negotiation skills were important to establishing a good working relationship between the two organizations.

A listing ofDon's service roles does not make apparent the depth of his contribution, for that goes far beyond the day-to-day management he often provided. Don's service mattered so much because it was always in the interests of a larger goal that he cared passionately about. When he participated in the organization of a new conference or the founding of an organization, it was not for its own sake (or because he didn'talready have enough irons in the fire), but because he thought it was necessary to further an important research goal. The goals, as well as the conferences and organizations, were always integrative. IJCAI brings together researchers from around the world and across the fields of AI; Don constantly reminded the Trustees and Advisory Board of the importance of safeguarding both dimensions of its diversity. Co ling is likewise international, and, with much guidance from Don, ACL has also evolved into an international body. Having helped to foster the now-flourishing European Chapter of the ACL, he looked toward a Chapter on the Pacific Rim. Don was always inclusive, never exclusive.

Don brought people together in other ways. He organized a fund and talked to the right people to give Eastern Europeans the right to visit their colleagues abroad. Petr Sgall remembers his first long personal conversation with Don at the romantic, medieval Hungarian castle of Vise grad. "Our conversation touched perhaps everything from text retrieval to topical issues of world politics, but Don 's main concern was to find maximally effective ways of overcoming the disadvantages of researchers in Communist countries. I then saw that Don 's effort towards this aim were of much larger dimensions than what I could imagine. Don's conception of international cooperation went far beyond generous support for individual stays. For example, thanks to his initiative, such valuable sources of information as the journal Computational Linguistics reaches dozens of colleagues who, due to the unfavorable conditions in their countries, would not otherwise be able to get it."

The computational linguistics community in Eastern Europe benefitted greatly from Don's efforts toward their integration into the international research community, espe­cially the initiation of the international fund which gave them all the advantages of ACL membership. Eva Hajicova recalls, "When Don called me at the beginning of 1982 from California ... and asked me whether such an action would be good for us, I just could not believe my ears. It lasted a while before I realized the depth of his offer and became able to express our gratitude and to think of the technicalities."

In tribute to Don's efforts to include as many people as possible in the computational linguistics endeavor he found so exciting, and his and Betty's longstanding contributions to the ACL, the ACL Executive Committee established the Don and Betty Walker Student Fund. This fund enables students to attend ACL meetings. His legacy will live on, bringing together people who never had the good fortune to know him.

In recent years, Don's research and his organizational efforts took a new tum. Corpus-based natural-language processing, which at present may be the fastest-growing and most exciting area of computational linguistics, owes a particular debt to Don 's vision and leadership. He recognized the importance of working on real natural-language corpora and other linguistic knowledge resources before that was in vogue, and he

XXIV

developed and sustained successful efforts to organize the collection and annotation of databases for corpus-based research. In particular, he played crucial leadership roles in the ACL Data Collection Initiative, which has now made available a variety of corpora to the international research community, and in the Text Encoding Initiative, which has brought together international organizations concerned with textual research to create a standard for encoding machine-readable text. Through the Text Encoding Initiative, Don became interested in humanities computing and its approaches to working with large textual databases. He soon realized how much that community and computational linguistics could benefit from interaction with each other and in his characteristic way went on to foster those interactions. His influence and support can also be seen in the European Corpus Initiative, the Linguistic Data Consortium, and the Consortium for Lexical Research, all of which are actively contributing to the variety and quantity of corpora and lexical resources available to the research community in natural-language and speech processing, linguistics and the humanities. All of this was in the service of a long-term vision he referred to as "The Ecology of Language", that is, the attempt to characterize the contexts in which people use language, and thus in which natural language technology can be made useful to people. This theme is reflective of his aims during his whole career.

We can't resist one more anecdote that shows that he was able not only to bring people together, but, at least in one instance, to keep them together for a few more hours. Paul Martin held a large party to celebrate receiving his Ph.D., and after midnight it was getting noisy enough that the neighbors called the police. When they came, Don told Paul not to worry. He'd handle it. He went out to talk to them, in his inimitable, reasonable, soft-spoken manner. No one knows what he said, but the police left and the party continued.

The delight that Don took in his work and in the people he worked with was infectious. Many times he would try to share that delight with friends who were strangers to the field. His face would light up; his hands would orchestrate his attempts to make them understand the wonder of it all. Often he would search in vain for a word that could convey how he felt about the field, the people, and the ideas, and he would end up saying something like "It's just so ... so ... " and fall back with a sigh to "elegant", or "amazing", extending his hands as if to shape the elusive message.

The ancient Chinese reserved a special place in heaven for people who built bridges. Don Walker built bridges.

XXV