[Mechanical Translation and Computational Linguistics, vol.10, nos.3/4, September and December 1967] Automatic Determination of Parts of Speech of English Words

by Lois L. Earl,* Lockheed Palo Alto Research Laboratory, Palo Alto, California

The classifying of words according to syntactic usage is basic to language handling; this paper describes an algorithm for automatically classifying words according to thirteen commonly used parts of speech: noun, adjective, verb, past verb, adverb, preposition, conjunction, pronoun, interjection, present participle, past participle, auxiliary verb, and plural or collective noun. The algorithm was derived by a computerized study of the words in The Shorter Oxford English Dictionary. In its operation it utilizes a prepared dictionary of around nine hundred words to assign parts of speech to special or exceptional words. Other words are split into affix and kernel parts and assigned a part of speech on the basis of the part-of-speech implications of the affixes and the length of the remaining kernel. An accuracy of 95 per cent is achieved from the point of view of inclusive part of speech, where inclusive part of speech is defined as that string which contains all the parts of speech attributed to the word by the dictionary but which may also contain one or two more parts of speech.

Introduction

This paper describes the development and details of a procedure for automatically assigning part-of-speech characteristics to English words, largely from graphemic considerations. The development of the algorithm began with the observation of Dolby and Resnikoff1 that the parts of speech associated with one-syllable words are frequently noun (or noun and adjective) and verb, while the parts of speech associated with multisyllable words are usually noun and adjective only. Develop- ment of a working part-of-speech algorithm required the study of exceptions to this general rule so that analytical subrules and exception lists sufficient to identify automatically all such exceptions could be derived. Two analyses were utilized for the isolation and study of exceptions: (1) Exhaustive sorts of a 73,582-word dictionary on magnetic tape were used to separate words consistent with the general rule from those words that were not and to classify them. (2) Computer analysis of possible part-of-speech implica- tions of affixes was carried out on the same dictionary. The algorithm developed utilizes a prepared dictionary of around nine hundred words and an affix list of less than two hundred entries.

Parts of Speech Assigned and Their Abbreviations

The tape dictionary used for both analyses contained 73,582 words, with part-of-speech and word-status in-

*I wish to thank J. L. Dolby and H. L. Resnikoff, who have acted as consultants on Office of Naval Research contract Nonr 4440(00), which supported this research.

formation from The Shorter Oxford English Dictionary (SOX)2 and Webster's Third New International Dic- tionary (MW3).3 The tape dictionary is reliable in most respects, since it was made from punched cards transcribed directly from the dictionaries, verified by different personnel, and spot-checked periodically dur- ing the process. Nevertheless, errors did occur, par- ticularly in the recording of part-of-speech information which was not always understood by the keypunchers. The parts of speech recorded are as follows:

Noun N Adverb AV Pronoun PN Adjective AJ Preposition PR Interjection IJ Verb VB Conjunction CJ Past verb PV

In addition, the category "other" (OT) was used when- ever the dictionary gave some part of speech other than the nine listed above. Participles, numerals, arti- cles, and collective nouns mainly comprise OT.

The algorithm was designed to assign these same nine parts of speech (excluding OT) with the addition of four more which were unfortunately subsumed under OT: present participle (PA), past participle (PP), auxiliary verb (AX), and plural or collective noun (NP). The category "noun" was changed to the category "noun-or-adjective" (NA) on the grounds that nearly all nouns can act as adjectives under some circumstances. Thus, although the algorithm attempts to distinguish words usable only as adjectives from those usable either as nouns or adjectives, it does not try to distinguish words usable only as nouns from those usable as either nouns or adjectives. Collective nouns will be assigned the string NA and NP to show possible use with either singular or plural verbs. Al-

53

though a dictionary may show additional or fewer parts of speech for participial forms, their use (or lack of use) as nouns, adjectives, or verbs was considered implicit in the participle assignment, and no attempt was made to further partition the categories PA or PP. Thus, present participles are implicitly possible nouns, adjectives, or in a verb phrase, and past participles are implicitly adjectives, past verbs, or in a verb phrase. An attempt was made to identify participles which have any other special usages and to identify irregular past tense and past participial forms.

Like a dictionary, the algorithm is designed to indi- cate all the possible parts of speech for a word. That is, a part-of-speech string is assigned to each word, represented here by writing the part-of-speech abbrevi- ations contiguously. For example, a word assigned the part-of-speech string AJ VB is a word that can act as an adjective or as a verb.

Design Plan

As a starting point in the design of a part-of-speech algorithm, three basic rules were postulated:

Rule A: The part-of-speech string associated with a word containing only one vowel string in its kernel will be NA VB, where a kernel will be defined as a word stripped of its affixes. Similarly, the part-of-speech string associated with words with multivowel string kernels will be NA.

Rule B: The part-of-speech string associated with a word ending in ed will be PP, and with a word end- ing in ing will be PA. All PP will also be considered PV. An NA classification will be changed to NP for all words ending in single s.

Rule C: The part-of-speech string associated with a word ending in ly will be AJ AV.

Rule A is basically a refinement of the original Dolby-Resnikoff1 hypothesis and depends on the Dolby- Resnikoff definition of a legal vowel string. This rule also depends on the existence of an operational defini- tion of affixes.4,5 Rules B and C are a recognition of the most consistently used and meaningful suffixes of English.

A goal of 95 per cent accuracy was set for the algorithm. To reach that goal, three steps were de- cided upon:

Task 1: Tabulation of the exceptions to Rules B and C.

Task 2: Tabulation of special-purpose words, with part-of-speech PR, CJ, PN, or IJ, which are not covered by Rules A, B, or C.

Task 3: Modification of Rule A as much as neces- sary to achieve 95 per cent accuracy, using a study of affixes, or a tabulation of exceptions, or both, as a means to this end.

The first two tasks could be accomplished by sorting the dictionary on magnetic tape, as mentioned in the Introduction, although it may be of interest that not all of the necessary data handling could be accomplished with a generalized sort routine. The 7094 SORT was used in conjunction with special-purpose routines. The implementation of Tasks 1 and 2 is described in this paper; then the implementation of Task 3, which is more involved, is summarized with references for those who wish to pursue the details.

Dictionary Studies

TASK 1: EXCEPTIONS TO RULES B AND C

According to Rule B, all words ending in ed, ing, or single s should be categorized OT, for participle or noun-plural. All words violating this rule were listed and examined. Because many obscure and specialized words are listed in the dictionaries, it was decided that only words in standard usage would be included in exception lists. This reduced the list of Rule B excep- tions somewhat, and further reduction was accom- plished by removing the words ending in as, is, ous, and us whose part of speech would be properly in- ferred from these suffixes (see Task 3). Fortunately, many words ending in ing which are not participles could be removed because their actual parts of speech (usually NA, as for pudding) are subsumed under the participle heading. Classifying them as present parti- ciples is correct from the point of view of an "inclusive" part-of-speech string because present participles can be used as nouns or adjectives. (By an "inclusive" part- of-speech string is meant that string which is sure to contain all the parts of speech attributed to the word by either dictionary, but which may also contain one more or, rarely, two more parts of speech. Since use of inclusive part of speech becomes necessary in Task 3, its justification will be considered when Task 3 is discussed.) Similarly, words ending in ed which are not marked OT but are marked either AJ or VP are correctly classified past participle, from an inclusive viewpoint. All remaining ed and ing words, generally NA ed words and VB or AV ing words, are given in Table 1 along with the s-ending exception words. There are 104 words in this table, which is an exhaustive list.

Just as there are ed, ing, and s-ending words which are exceptions to Rule B, there are also some parti- ciples, past tense verbs, and plural or collective nouns which are exceptions because they cannot be recog- nized from s, ing, or ed endings. When all such words were listed from the dictionary, there were 1,380 entries, a very long list, since the goal of automatic determination of part of speech presupposes as small a dictionary as possible. From the list of 1,380 words, all irregular participles and past tense verbs have been

54 EARL

listed in Table 2 (145 words). The rest of the words (1,235) included numerals, obscure collective nouns (e.g., herb, scrub), words which become collective only when s is added (e.g., geriatric), and some errors in judgment by the keypuncher. From this heterogene- ous group, sixty were selected as reasonably common collective nouns and were listed in Table 3. Since the list is subjective, it may have to be augmented from experience, but it is believed to be adequate to main- tain the goal of 95 per cent accuracy.

DETERMINATION OF PARTS OF SPEECH 55

In investigating exceptions to Rule C, adverbs with additional parts of speech of PR, CJ, PV, IJ, PN, and OT were ignored in order to avoid duplication of words with those in lists compiled in Task 2. Within this limitation, all words were extracted from the dic- tionary which, though ending in ly, were not adverbs or, conversely, though not ending in ly, were adverbs. Contrary to expectations, there was a large number of such words (slightly over 1,500). Many of these words were judged rare, or rare in the usage in ques- tion (e.g., dog-fly as NA, or dash, pi, rife, smell, thistle as AV); others could be predicted by an ex- tension of the affix lists, to be discussed later. In ac- cordance with the philosophy of maintaining a rela- tively short exception list without sacrificing too much accuracy, this list of 1,500 words has been arbitrarily reduced to a list of 361 of the common words which are exceptions to Rule C, as shown in Table 4. In addition, there are many non-ly adverbs which occur in Table 5.

56 EARL

TASK 2: TABULATION OF SPECIAL-PURPOSE WORDS WHICH ARE NOT COVERED BY RULES A, B, OR C

For Task 2, a subset of the dictionary was prepared containing all the words which: (1) have at least one standard meaning corresponding to a part of speech other than NA, VB, AJ, or AV (the parts of speech assigned by Rules A, B, C), (2) have all "irregular" entries removed (fragments, etc.), and (3) have all words ending in ed, ing, or s removed (the suffixes covered by Rule B). By extracting from this subset all words with standard meaning corresponding to a part of speech PR, CJ, IJ, PN, or OT, we should get an exhaustive list of those structural, special-pur- pose words which are so important in a mechanized handling of English.

Table 5 shows the 253 function words so extracted.

DETERMINATION OF PARTS OF SPEECH 57

The words are listed in groups according to number of syllables and are arranged alphabetically from the end of the word. Note that Table 1 lists the eighteen func- tion words ending in s or ing. This list is otherwise

58 EARL

theoretically complete, but because of a misunderstand- ing by keypunchers in the original creation of the dictionary, some important pronouns were not so clas- sified in the MW3 part-of-speech designations and are therefore missing from the list (I, your, his, we, them, our, us, their, they). Similarly, some important auxiliary verbs were not so classified in the SOX part-of-speech designations and are therefore missing (am, is, are, was, were, be, will). Also, the word as has been lost in the sorting process. No other significant omissions have been noted, but are possible, since checking of

the tape dictionaries was not exhaustive. For the con- venience of the reader, the words in Tables 1 through 5, plus the words given here, have been alphabetized and given in Table 6.

The parts of speech given in Tables 1 through 5 were taken from the tape dictionary and have not been verified in the dictionaries themselves. Particular care should be taken in the use of Table 2, which seems to have many errors in the omission or intrusion of the PV and PP codes.

DETERMINATION OF PARTS OF SPEECH 59

60 EARL

TASK 3: MODIFICATION OF RULE A USING A STUDY OF AFFIXES

Rule A is based upon a general observation and is good for only a simple majority of words. The business of Task 3 is to discover if it is possible, by considering prefixes and suffixes, to convert this general rule to a more precise rule, adequate for 95 per cent of English words. As a first step, a formal and reproducible defi- nition for affixes was developed, as is described in The Nature of Affixing in Written English* and Structural Definition of Affixes in Multisyllable Words.5 Then, the extent of correlation between affixes and part of speech was investigated, both for the formally defined affixes and for others listed in Modern English Usage.6 This investigation is described in "Part-of-Speech Implica- tions of Affixes"7 but can be summarized here.

All words with part of speech AV, PR, PN, NP, IJ, PA, PP, VP, and CJ can be automatically assigned part of speech by reference to the word lists in Tables 1 through 4, followed by application of Rules B and C for words not in these lists. "Part-of-Speech Implica- tions of Affixes"7 was therefore concerned only with words whose part-of-speech string contained the ele- ments NA, AJ, and VB, which allows the five possible combinations VB, NA, AJ, NA-VB, AJ-VB. NA-AJ is considered equivalent to NA. Attempts to establish a 95 per cent correlation between the part-of-speech string of a word and its affixes failed. However, it was noted that the correlation was closer for four- to seven- syllable words than for two- to three-syllable words and that a very good correlation could be obtained for all words between an "inclusive" part-of-speech string and the affixes. Thus, in some cases determining the affixes and counting vowel strings lead to an abso- lute identification of the part of speech of a word, but in other cases identification is to a more inclusive set. For example, an NA or a VB may be classified as NA-VB, or an AJ may be classified as an NA. Such a classification is justifiable on the following grounds: (1) A primary use of part-of-speech information is in automatic syntactic analysis. It is the natural task of a syntactic analysis program to choose among several possible parts of speech, and it is easier to do so than to supply a missing part of speech. (2) Dictionaries

are very reliable in the information explicitly given, but implications inferred from the absence of informa- tion are less reliable. Thus, the inclusive part-of-speech string assigned by the algorithm may in some cases be more correct than the more limited one assigned by a particular dictionary. In our experience with the SOX and MW3 dictionaries, we found many instances of non-agreement; usually one was more inclusive than the other.

In "Part-of-Speech Implications of Affixes,"7 the re- sults of the correlation study are given for seventy-two prefixes and eighty-seven suffixes. Implications are of the form NA or NA-VB, or VB or AJ. For example, the four s-ending suffixes mentioned in the discussion of Task 2 carry the following part of speech implica- tions :

is NA-VB as NA ous AJ us NA

For forty-one of the affixes, the part-of-speech implica- tion changes with the length of the word, from NA-VB for two- and three-syllable words to NA for four- to eight-syllable words.

Later a correlation was made for other affixes which seemed to be likely candidates for reducing the excep- tion lists by aiding in the identification of adverbs or in the identification of words ending in ed which are not past participles. Though not operationally defined, these affixes are of practical importance and are there- fore listed here, with their part-of-speech implications:

Prefixes POS Suffixes POS

north NA AV seed NA south NA AV weed NA west NA AV like NA AV a- AJ AV wise AJ AV

ward NA AV wards NA AV

-fly NA -bed NA -deed NA -feed VB -tenths NA

DETERMINATION OF PARTS OF SPEECH 61

Testing and Evaluation Rules A, B, and C, the exception lists, and the prefix and suffix implications reported in Reference 7 formed the basis of a part-of-speech algorithm, which has been programed on the IBM 7090 and is being im- plemented on the IBM 360/30. In the program, a word whose part of speech is to be determined is first checked against the exception lists, which yield a part- of-speech string for words which match. For all other words, the word is separated into kernel and affix parts, and the part-of-speech implication of the affixes is looked up and applied to the word. For any word without affixes or whose affixes do not have an impli- cation, Rule A is applied to obtain the part-of-speech assignment. There are some complications involved in some of these steps, particularly in separating a word into kernel and affix parts and in assigning parts of speech on the basis of affixes. The logic used by the program for these steps is given in Figure 1.

To summarize the logic briefly, we can say that affixes are stripped from the word one at a time, with prefixes given a limited priority over suffixes other than ed. Thus, the word exceptional becomes first ex-cep- tional, then ex-ception-al, and finally ex-cep-tion-al. The criterion by which an affix sequence was accepted was for most affixes the same as that given in Reference 7; simply stated, this means that the affix was accepted if the remaining kernel was a reasonable syllable or syllables, determined by examining the consonant and vowel strings. Some affixes were designated as trans- formational and were subject to additional constraints or modifications. For example, s is a suffix only at the end of a word and when not preceded by another s. The implications of the outermost affixes were used in assigning parts of speech, and the priority indicators were set to use suffix implications, if any, in preference to prefix implications, in accordance with the findings of Reference 7.

To test the algorithm, five hundred words were chosen at random from the tape dictionary, 2,3 and the parts of speech assigned by the algorithm were com- pared with those given in the dictionary. If dialectal, obsolete, archaic, and rare words causing errors are removed, and if program errors are corrected, results are as follows:

No. of Words Category in Category

Assigned POS matches dictionary POS ................. 271 Extra POS assigned ................................................. 196 Missing POS .......................................................... 16 POS does not match at all—error ............................ 8 Total sample ........................................................... 491

This shows that 95.1 per cent of the words were as- signed the correct inclusive part of speech and 55.2 per cent were assigned parts of speech exactly coin-

ciding with those assigned by the dictionary. Thus, the goal of 95 per cent is just achieved.

It is interesting to consider how little the affix impli- cations have improved the results for this sample. Taking the first 192 of the five hundred alphabetized words and applying the original Rules A, B, and C only, twenty words are shifted into the exact-match category and twenty-five words shifted from the exact- match category, for a net loss of five words, where two of these go into the error category. Six words are added to the words with missing part of speech, while two words are taken out of the category. Thus, the total loss is four more words into the missing category and two more words into the error category, or about a 3 per cent loss from the point of view of inclusive part of speech. Rule A, it will be remembered, requires the removal of affixes from the kernel of the word. If this kernelizing of the word is omitted, there is about a 13 per cent loss from the point of view of inclusive part of speech, indicating that the fact that a word is affixed is more important in predicting part of speech than what the affix is (the affixes ing, ed, ly, and s excepted). Nevertheless, using the implications of affixes is a refinement in an area where refinement is sorely needed.

It might be interesting at this point to evaluate the two original premises—that one-syllable words are large- ly noun-verb and that all other words are largely noun only.1 Although the tape dictionary does not provide a syllable count, it does provide a count of the number of legitimate vowel strings; final e is not to be consid- ered legitimate. To test the first premise, the standard one-vowel-string words in the tape dictionary were divided into two sections, those which were NA-VB (and only NA-VB) and those which were not (the OT category was ignored). There were 2,520 words in the NA-VB category and 1,925 words with more or fewer parts of speech than NA-VB. The 1,925-word list includes the 132 one-vowel-string members of the word-class with parts of speech PR, CJ, IJ, PN, and PV listed in Table 4. Discounting these 132 function words, then, the first premise is true for 2,520 out of 4,313 cases, or about 58 per cent. To get 95 per cent of the one-vowel-string words assigned as in the dic- tionary, most of the 1,793 non-NA-VB words would have to be in an exception dictionary. However, since most of these are NA, from the point of view of in- clusive part of speech, the NA-VB rule for one-vowel- string words is quite good, giving results very close to those obtained in the five-hundred-word random sample of all words (55 per cent exactly matching dictionary, 95 per cent giving correct inclusive part of speech). Note that these statistics hold for one-vowel-string words and that the statistics for one-syllable words would differ somewhat.

The second premise has not been directly tested, but may be inferred from the five-hundred-word

62 EARL

DETERMINATION OF PARTS OF SPEECH 63

64 EARL

DETERMINATION OF PARTS OF SPEECH 65

66 EARL

random sample, since we have just proved that the one-syllable words (there are forty-six in the sample) do not affect the results substantially. In its general form the second premise is accurate about 70 per cent of the time, as is reported in Reference 1. In its modi- fied form, as stated in Rule A and tested by our five- hundred-word sample, it is accurate for only about 55-60 per cent of the cases, but is good for about 90-95 per cent of the cases from the point of view of in- clusive part of speech, with something less than 5 per cent variation, depending on whether part of speech implications of affixes are used.

Summary

The net result of the part-of-speech studies is an algorithm which, used in conjunction with a dictionary of less than one thousand words and an affix list of less than two hundred, gives a correct "inclusive" part of speech for 95 per cent of a five-hundred-word random sample and which should do better on textual material. The dictionary is derived from an exhaustive compilation of words which the algorithm is not capable of handling. Such words are adverbs, function words, participles, or collective nouns not recognized by the program or, conversely, words so classified which should not be. The number of words in the exhaustive list is 3,163, of which less than one-third were selected for the dictionary. However, all of the function words

with parts of speech other than NA, AJ, VB, or AV have been included, as have all of the irregular past verbs and past participles and the more commonly used adverbs and collective nouns. The omitted words are mainly less common adverbs and collective nouns, and they comprise only about 3 per cent of the total 73,582-word dictionary.

Received March 6, 1967

References

1. Dolby, J., and Resnikoff, H. "On the Structure of Written English Words," Language, Vol. 40 (April-June, 1964), p. 2.

2. The Shorter Oxford English Dictionary on Historical Principles (3d ed., revised with addenda; Oxford: Clar- endon Press, 1959).

3. Webster's Third New International Dictionary of the English Language (Springfield, Mass.: G. C. Merriam Co., 1961).

4. Resnikoff, H., and Dolby, J. "The Nature of Affixing in Written English," Mechanical Translation, Vol. 8 (June and October, 1965), pp. 84-89.

5. Earl, L. L. "Structural Definition of Affixes in Multi- syllable Words," Mechanical Translation, Vol. 9 (June, 1966), pp. 34-43.

6. Fowler, H. W. A Dictionary of Modern English Usage, rev. and ed. Sir Ernest Gowers (2d ed.; New York: Oxford University Press, 1965).

7. Earl, L. L. "Part-of-Speech Implications of Affixes," Mechanical Translation, Vol. 9 (June, 1966), pp. 38-43.

DETERMINATION OF PARTS OF SPEECH 67