He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.
-
Upload
brodie-nute -
Category
Documents
-
view
223 -
download
0
Transcript of He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.
![Page 1: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/1.jpg)
He Said, She Said: Gender in the ACL Anthology
Adam Vogel and Dan JurafskyStanford University
![Page 2: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/2.jpg)
Gender in Computational Linguistics
• Well known gender imbalance in computer science– In 2008, women granted 20.5% of PhDs [CRA,
2008]• Linguistics departments are close to parity– In 2007, women granted 57% of PhDs [LSA, 2008]
• What about computational linguistics?
![Page 3: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/3.jpg)
Gender Studies Methodologies
• Previous studies utilize:– University enrollment/graduation– Job placement– Professional society membership
![Page 4: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/4.jpg)
Gender Studies Methodologies
• Previous studies utilize:– University enrollment/graduation– Job placement– Professional society membership
• Corpus based approach using publications:– Overall population– Publication counts – Authorship order– Topic models by gender
![Page 5: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/5.jpg)
ACL Anthology Network
• 13,000 papers• 12,000 authors– Not marked for gender
• 1965 – 2008– We only use data from 1980 onwards
[Radev et al, 2009]
![Page 6: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/6.jpg)
Determining Gender by Name
• Broad background of ACL authors makes automatic assignment difficult – “Jan” in Europe vs. US– “Weiwei” in Chinese
• Some names are poorly formatted or missing first names– H. Murakami– ukasz– The LOLITA Group
![Page 7: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/7.jpg)
Determining Gender by Name
• Automatic approaches:– Unambiguous first names from US census data– Morphological markings in Czech and Bulgarian– Lists of unambiguous Indian and Basque names
• Hand labels:– Help from ACL authors in China, Taiwan, and Singapore– Personal knowledge or website photos
• Remaining: 2048 names– Baby name website: www.gpeters.com/names/
• Unknown: 761 names
![Page 8: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/8.jpg)
Female: 3359 Male: 8573 Unknown: 761 (26.7%) (67.5%) (6.0%)
![Page 9: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/9.jpg)
![Page 10: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/10.jpg)
![Page 11: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/11.jpg)
Population Conclusions
• Female authorship increased from 13% in 1980 to 27% in 2007– Using best fit lines: 19.4% -> 29.1%– 50% relative increase!
• Male authorship decreased from 79% to 71%
![Page 12: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/12.jpg)
Population Conclusions
• Female authorship increased from 13% in 1980 to 27% in 2007– Using best fit lines: 19.4% -> 29.1%– 50% relative increase!
• Male authorship decreased from 79% to 71%
Next: how prolific are men and women?
![Page 13: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/13.jpg)
For 1st authored papers: Female 27% Male: 71% Unknown: 2%
![Page 14: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/14.jpg)
![Page 15: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/15.jpg)
![Page 16: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/16.jpg)
![Page 17: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/17.jpg)
Publication Count Conclusions
• The most prolific authors are male• Men have on average been in the field longer• Men and women have comparable publication
output per year
![Page 18: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/18.jpg)
Publication Count Conclusions
• The most prolific authors are male• Men have on average been in the field longer• Men and women have comparable publication
output per year
Next: what do men and women write about?
![Page 19: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/19.jpg)
Latent Dirichlet Allocation (LDA)
![Page 20: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/20.jpg)
• Generate 100 topics using LDA• Throw out 27 junk topics, yielding 73
substantive topics• Label topics based on their term distributions• Find topics with biggest difference between
men and women:
LDA for AAN
![Page 21: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/21.jpg)
Topic Calculations
Probability of a topic for a gender
Documents with 1st author gender g
![Page 22: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/22.jpg)
Topic Calculations
Probability of a topic for a gender and year
Documents with 1st author gender g written in year y
![Page 23: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/23.jpg)
speaker utterance act hearer belief proposition acts beliefs focus evidence
Sandra Carberry
![Page 24: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/24.jpg)
prosodic pitch boundary accent prosody boundaries cues repairs speaker phrases
Mari Ostendorf
![Page 25: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/25.jpg)
question answer questions answers answering opinion sentiment negative trec positive
Soo-Min Kim
![Page 26: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/26.jpg)
dialogue utterance utterances spoken dialog dialogues act turn interaction conversation
Diane Litman
![Page 27: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/27.jpg)
class classes verbs paraphrases classification subcategorization paraphrase frames acquisition
Anna Korhonen
![Page 28: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/28.jpg)
topic summarization summary document news summaries documents topics articles content
Ani Nenkova
![Page 29: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/29.jpg)
resolution pronoun anaphora antecedent pronouns coreference anaphoric definite reference
Renata Vieira
![Page 30: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/30.jpg)
students student reading course computer tutoring teaching writing essay native
Jill Burstein
![Page 31: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/31.jpg)
Topic Conclusions
Women published relatively more papers in:– Speech Acts + BDI– Prosody– QA + Sentiment Analysis– Dialog– Acquisition of Verb Subcategorization– Summarization– Anaphora Resolution– Tutoring Systems
![Page 32: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/32.jpg)
dependency dependencies head czech depen dependent treebank structures
Joakim Nivre
![Page 33: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/33.jpg)
search length size space cost algorithms large complexity pruning efficient
Kenneth Church
![Page 34: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/34.jpg)
proof logic definition let formula theorem every defined categorial axioms
Mark Hepple
![Page 35: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/35.jpg)
grammars parse chart context-free edge edges production symbols symbol cfg
Mark-Jan Nederhof
![Page 36: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/36.jpg)
label conditional sequence random labels discriminative inference crf fields
Ryan McDonald
![Page 37: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/37.jpg)
unification constraints structures value hpsg default head grammars values
James Kilbury
![Page 38: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/38.jpg)
probability probabilities distribution probabilistic estimation estimate entropy
Mark Johnson
![Page 39: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/39.jpg)
semantics logical scope interpretation logic meaning representation predicate
Jerry Hobbs
![Page 40: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/40.jpg)
Topic Conclusions
Men published relatively more papers in:– Categorial Grammar– Dependency Parsing– Algorithmic Efficiency– Parsing– Discriminative Sequence Models– Unification Based Grammars– Probability Theory– Formal Computation Semantics
![Page 41: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/41.jpg)
Conclusion
• Approximately 50% increase in the proportion of female authors since 1980
• Men and women have similar publication rates
• Gender labels for names available for download:http://nlp.stanford.edu/projects/gender.shtml
![Page 42: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/42.jpg)
Acknowledgements
• Thanks to Chu-Ren Huang, Olivia Kwong, Heeyoung Lee, Hwee Tou Ng, and Nigel Ward for helping to label names for gender
• Thanks to Chris Manning for helping to assign topic names
• Thanks to Steven Bethard and David Hall for creating the topic models
![Page 43: He Said, She Said: Gender in the ACL Anthology Adam Vogel and Dan Jurafsky Stanford University.](https://reader036.fdocuments.us/reader036/viewer/2022062320/56649c6d5503460f9491f6ed/html5/thumbnails/43.jpg)