KOSEN sogoriko 0313 h1 - 東京工業大学Title KOSEN_sogoriko_0313_h1 Created Date 3/27/2014 9:26:28 AM
Methods Problem - 東京工業大学
1
藤 板屋 ⼀層し 夜な夜な 花桜 枝 侭 清かなり 折る ⽂無し 寒けし 彦星 花 元 初む ⾹る 朝明 真⽊ 散らす ⻑閑けし 籬 桜 菅原 傾く 明け⽅ 主 氷 ⼀年 氷柱 辺 菊 誤つ 井⼿ ⼥郎花 勝ち 有明⽅ 払ふ 合ふ 匂ふ 鴛鴦 村雲 明けし 宵 川瀬 東雲 凍る 宿す 烏羽⽟ 桃 狭筵 実 盛る 窓 九重 秋萩 移ろふ 異異に 赤し 柳 後ろめたし 藤袴 伏⾒ 暗し 更く 移す 隠す 昇る 先づ 早 撫⼦ 吉野⼭ 卯花 明く 鴨 盛りなり 仄かなり 夏 ⼭吹 涼し ⼾ 閉づ ⽊末 漏る 独り 徒⼈ 橋姫 ⾹ 梅 桜花 ⻑⽉ 四⽅ 垣根 ⼋重 夜半 橘 冬 疾し 弥 通ひ路 咲く 触る ⼭桜 0 1 2 3 4 0 1 2 PCA 1 (29.73%) PCA 2 (12.81%) -1.0 -0.5 0.0 0.5 1.0 Similarity 花‘hana’ (flower) 夏 ‘natsu’ (summer) Figure 3: PCA of words similar to 花 ‘hana’ (flower) and 夏 ‘natsu’ (summer). JADH 2017: "Creating Data through Collaboration" @ Doshisha University, September 11-12, 2017 Relationships between Flowers in a Word Embedding Space of Classic Japanese Poetry Bor Hodošček, Osaka University [email protected] Hilofumi Yamamoto, Tokyo Institute of Technology [email protected] Introduction Word embedding methods such as Word2Vec (Mikolov et al., 2013; Le and Mikolov, 2014) have been shown effective in extracting semantic knowledge from large corpora. Problem Can word embeddings trained on the Hachidaishu encode enough semantic information to find subordinate words via their superordinate concept? Quantify the relationship between the content of a word and its word embedding vector. Examine the possibility of word embedding spaces to explain the semantic relationships between classical Japanese poetic terms. Materials Hachidaishū: classical Japanese poem anthologies compiled under decree by Emperors (ca., 905―1205), comprising approximately 9,500 poems and 159,183 tokens (Source: KokkakaitanNijūichidaishū database published by NIJIL). Methods In order to examine the notable relationships between ‘ka’ (fragrance), ‘chiru’(fall), we look at the cosine similarity scores between terms in the word embedding space generated by Word2Vec. Conclusion Word embeddings allowed us to extract specific subordinate words based on the superordinate concept of classical terms → when the distance between two terms such as ‘tachibana’ (orange) and ‘natsu’ (summer) is close enough, the superordinate concept A indicates the subordinate concept a. References Katagiri, Yoichi (1983) Utamakura utakotoba jiten (Dictionary of poetic vocabulary), Vol. 35 of Kadokawa shojiten, Tokyo: Kadokawa Shoten. Le, Quoc V. and Tomas Mikolov (2014) “Distributed Representations of Sentences and Documents,” CoRR, Vol. abs/1405.4053, URL: http://arxiv.org/abs/1405.4053. Mikolov, Tomas, Kai Chen, Greg Corrado, and Jeffrey Dean (2013) “Efficient Estimation of Word Representations in Vector Space,” CoRR, URL: http://arxiv.org/abs/1301.3781. Mizutani, Sizuo (1983) Goi (Vocabulary), Vol. 2 of Asakura Nihogo Shin-Kōza, Tokyo, Japan: Asakura Shoten. Řehůřek, Radim and Petr Sojka (2010) “Software Framework for Topic Modelling with Large Corpora,” in Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, pp. 45–50, Valletta, Malta: ELRA, May, http://is.muni.cz/publication/884893/en. Rodd, Laurel Rasplica and Mary Catherine Henkenius (1984) Kokinshū - A Collection of Poems Ancient and Modern, Boston MA USA: Cheng and Tsui Company. Yamamoto, Hilofumi (2007) “Waka no tame no Hinshi tagu zuke shisutemu / POS tagger for Classical Japanese Poems,” Nihongo no Kenkyu / Studies in the Japanese Language, Vol. 3, No. 3, pp. 33–39. Each poem is tokenized into lemma forms by kh (Yamamoto, 2007) which divides poem texts into tokens using a classical Japanese dictionary. 50-dimensional skip-gram model with negative sampling, context window covering the whole poem using Gensim 2.3.0 (Řehůřek & Sojka, 2010). We could therefore verify that it allows us to extract the concrete name from its superordinate concept. Results ‘ka’ (fragrance) is related to ‘ume’ (plum) (replicating Mizutani, 1983). Falling flowers denote ‘sakura’ (cherry) and not ‘ume’ (plum); ‘sakura’ (cherry) relates to chiru (fall), which indicates that people at the time lamented falling sakura (falling cherry blossom petals) (replicating p. 84 in Katagiri, 1983). Subtracting tachibana out from the summer vectors reveals a vector space devoid of relationships between natsu (summer) and hana (flower). These relational expressions (summer + flower; summer + flower - tachibana) reproduce our current understanding of the relationships between flowers and seasons as well as some emotions associated with them in the word embedding space. 藤 交ふ 下⽔ 重なる ⽴⽥ 挿頭す 麓 霞 み吉野 彼 著し 峡 花桜 花橘 枝 ます ⼭辺 折る ⽂無し 蔭 桂 ⽐ふ 花 勝つ 元 初む ⾹る 散らす ⻑閑けし 縁 籬 千種なり 遠 春 霞む 其れ 憧る 主 隣 ⼀年 遣す 辺 菊 緑 誤つ 惜しむ 誘ふ 井⼿ ⼥郎花 ⽩雲 幣 ⾊⾊ ⼭⾥ 軒端 常夏 鶯 標む ⽩菊 標す 匂ふ 咎む ⽩妙 御雪 ⻘葉 別きて 急ぐ 古⾥ 桃 実 浅緑 盛る 鴬 べらなり 返す返す ⾒す 九重 梢 秋萩 紛ふ 春⾬ 移ろふ 移り⾹ 増す 芽 異異に 柳 春霞 後ろめたし 卯の花 藤袴 随に 移す 隠す 打ち付けなり 常磐 先づ 棚引く 瓶 藤浪 吉野⼭ 撫⼦ 卯花 ⼋重桜 残す 盛りなり 若⽊ 家苞 ⼭吹 ⽊⽊ 始む 訪ふ ⽊末 杉 映る 押し並べて 徒⼈ 三輪 頼る ⾹ 撓なり 梅 桜花 奥⼭ 夙 ⽩河 尋ぬ 四⽅ 垣根 狩る ⼋重 ⾊⾹ 谷 散る 驚く 橘 何れ 疾し 宿 桜 春風 掘る 続く 植う 咲く ⾄る 交じる 触る 尋む 千種 ⼭桜 -1 0 1 2 3 4 0 1 2 3 PCA 1 (29.73%) PCA 2 (12.81%) -1.0 -0.5 0.0 0.5 1.0 Similarity Access the dataset online using Google's Embedding Projector Figure 1: PCA of word embedding space (4157 words × 50 dimensions) filtered to include only top 100 similar words for each of ume and sakura (150 total). Similarity is represented by the difference in similarity scores between ume and sakura, scaled to [-1, 1]. 梅 ‘ume’ (plum blossom) 桜 ‘sakura’ (cherry blossom) (values below denote top 5 cosine similarity scores) Figure 2: PCA of words similar to ⾹ ‘ka’ (fragrance) and 散る ‘chiru’ (fall). 藤 交ふ 下⽔ 峰 ⽴⽥ 麓 み吉野 薄し 花橘 花桜 枝 ⼭辺 庭 折る ⽂無し ⽐ふ 元 柞 散らす ⻑閑けし 縁 籬 桜 主 ⼀年 辺 菊 誤つ 誘ふ ⽊ ⼥郎花 井⼿ 幣 紅葉葉 ⼭⾥ ⼭桜 常夏 ⽩菊 匂ふ 咎む 御雪 別きて 古⾥ 桃 実 盛る ⼿向く 九重 梢 秋萩 紛ふ 春⾬ 移ろふ 移り⾹ 芽 異異に 後ろめたし 藤袴 春霞 随に 移す 隠す 打ち付けなり 先づ 撫⼦ 吉野⼭ 紅葉づ 残す 盛りなり ⼭吹 ⽊末 押し並べて 徒⼈ 錦 ⾹ 梅 桜花 奥⼭ ⽩河 垣根 狩る ⼋重 谷 散る 橘 何れ 春風 植う 咲く 触る ⾄る -1.0 -0.5 0.0 0.5 1.0 Similarity ⾹ ‘ka’ (fragrance) 散る ‘chiru’ (fall) Among summer flowers, the tachibana (the flower of orange) is very famous.
Transcript of Methods Problem - 東京工業大学
藤
板屋
⼀層し
夜な夜な
花桜
枝
侭
清かなり
折る
⽂無し
寒けし
彦星
花
元
初む
⾹る
朝明
真⽊
散らす
⻑閑けし
籬
桜
菅原
傾く
明け⽅ 主
氷
⼀年
氷柱
辺
菊
誤つ井⼿
⼥郎花
勝ち有明⽅
払ふ
合ふ
匂ふ
鴛鴦
村雲
明けし
宵
川瀬
東雲
凍る
宿す
烏羽⽟
桃
狭筵
実
盛る
窓
九重
秋萩
移ろふ
異異に
赤し
柳
後ろめたし
藤袴
伏⾒
暗し
更く
移す隠す
昇る
先づ
早
撫⼦
吉野⼭卯花
明く鴨
盛りなり
仄かなり
夏
⼭吹
涼し
⼾
閉づ ⽊末
漏る独り
徒⼈
橋姫
⾹
梅
桜花
⻑⽉
四⽅
垣根
⼋重
夜半
橘
冬
疾し
弥
通ひ路
咲く
触る
⼭桜
0
1
2
3
4
0 1 2
PCA 1 (29.73%)
PC
A 2
(12
.81%
)
-1.0
-0.5
0.0
0.5
1.0
Sim
ilari
ty
花‘h
ana’
(flow
er)
夏 ‘natsu’ (sum
mer)
Figure 3: PCA of words similar to 花 ‘hana’ (flower) and 夏 ‘natsu’ (summer).
JADH 2017: "Creating Data through Collaboration" @ Doshisha University, September 11-12, 2017
Relationships between Flowers in a Word Embedding Space of Classic Japanese Poetry
Bor Hodošček, Osaka [email protected]
Hilofumi Yamamoto, Tokyo Institute of [email protected]
IntroductionWord embedding methods such as Word2Vec (Mikolov et al., 2013; Le and Mikolov, 2014) have been shown effective in extracting semantic knowledge from large corpora.
ProblemCan word embeddings trained on the Hachidaishu encode enough semantic information to find subordinate words via their superordinate concept?
Quantify the relationship between the content of a word and its word embedding vector.
Examine the possibility of word embedding spaces to explain the semantic relationships between classical Japanese poetic terms.
MaterialsHachidaishū: classical Japanese poem anthologies compiled under decree by Emperors (ca., 905―1205), comprising approximately 9,500 poems and 159,183 tokens (Source: KokkakaitanNijūichidaishū database published by NIJIL).
Methods
In order to examine the notable relationships between ‘ka’ (fragrance), ‘chiru’(fall), we look at the cosine similarity scores between terms in the word embedding space generated by Word2Vec.
ConclusionWord embeddings allowed us to extract specific subordinate words based on the superordinate concept of classical terms → when the distance between two terms such as ‘tachibana’ (orange) and ‘natsu’ (summer) is close enough, the superordinate concept A indicates the subordinate concept a.
ReferencesKatagiri, Yoichi (1983) Utamakura utakotoba jiten (Dictionary of poetic vocabulary), Vol. 35 of Kadokawa shojiten, Tokyo: Kadokawa Shoten.
Le, Quoc V. and Tomas Mikolov (2014) “Distributed Representations of Sentences and Documents,” CoRR, Vol. abs/1405.4053, URL: http://arxiv.org/abs/1405.4053.
Mikolov, Tomas, Kai Chen, Greg Corrado, and Jeffrey Dean (2013) “Efficient Estimation of Word Representations in Vector Space,” CoRR, URL: http://arxiv.org/abs/1301.3781.
Mizutani, Sizuo (1983) Goi (Vocabulary), Vol. 2 of Asakura Nihogo Shin-Kōza, Tokyo, Japan: Asakura Shoten.
Řehůřek, Radim and Petr Sojka (2010) “Software Framework for Topic Modelling with Large Corpora,” in Proceedings of the LREC 2010 Workshop on New Challenges for NLP
Frameworks, pp. 45–50, Valletta, Malta: ELRA, May, http://is.muni.cz/publication/884893/en.
Rodd, Laurel Rasplica and Mary Catherine Henkenius (1984) Kokinshū - A Collection of Poems Ancient and Modern, Boston MA USA: Cheng and Tsui Company.
Yamamoto, Hilofumi (2007) “Waka no tame no Hinshi tagu zuke shisutemu / POS tagger for Classical Japanese Poems,” Nihongo no Kenkyu / Studies in the Japanese Language, Vol. 3,
No. 3, pp. 33–39.
Each poem is tokenized into lemma forms by kh (Yamamoto, 2007) which divides poem texts into tokens using a classical Japanese dictionary.
50-dimensional skip-gram model with negative sampling, context window covering the whole poem using Gensim 2.3.0 (Řehůřek & Sojka, 2010).
We could therefore verify that it allows us to extract the concrete name from its superordinate concept.
Results‘ka’ (fragrance) is related to ‘ume’ (plum) (replicating Mizutani, 1983).
Falling flowers denote ‘sakura’ (cherry) and not ‘ume’ (plum); ‘sakura’ (cherry) relates to chiru (fall), which indicates that people at the time lamented falling sakura (falling cherry blossom petals) (replicating p. 84 in Katagiri, 1983).
Subtracting tachibana out from the summer vectors reveals a vector space devoid of relationships between natsu (summer) and hana (flower). These relational expressions (summer + flower; summer + flower - tachibana) reproduce our current understanding of the relationships between flowers and seasons as well as some emotions associated with them in the word embedding space.
藤
交ふ
下⽔
重なる
⽴⽥
挿頭す
麓霞
み吉野
彼
著し
峡
花桜
花橘
枝
ます
⼭辺
折る
⽂無し
蔭
桂
⽐ふ
花
勝つ
元
初む
⾹る散らす
⻑閑けし
縁
籬
千種なり遠
春
霞む
其れ
憧る
主
隣
⼀年
遣す
辺
菊緑
誤つ
惜しむ
誘ふ
井⼿
⼥郎花
⽩雲
幣
⾊⾊
⼭⾥
軒端
常夏
鶯
標む⽩菊
標す
匂ふ
咎む
⽩妙
御雪⻘葉
別きて
急ぐ
古⾥
桃
実
浅緑
盛る
鴬
べらなり
返す返す
⾒す
九重
梢
秋萩
紛ふ
春⾬移ろふ
移り⾹
増す
芽
異異に
柳
春霞
後ろめたし
卯の花
藤袴随に
移す隠す打ち付けなり
常磐
先づ
棚引く
瓶
藤浪
吉野⼭
撫⼦
卯花
⼋重桜
残す
盛りなり
若⽊
家苞
⼭吹⽊⽊
始む
訪ふ⽊末
杉
映る
押し並べて
徒⼈三輪頼る
⾹
撓なり
梅
桜花
奥⼭
夙
⽩河
尋ぬ
四⽅
垣根
狩る⼋重
⾊⾹
谷
散る
驚く
橘
何れ
疾し
宿桜
春風
掘る
続く
植う
咲く
⾄る
交じる
触る
尋む
千種
⼭桜
-1
0
1
2
3
4
0 1 2 3
PCA 1 (29.73%)
PC
A 2
(12
.81%
)
-1.0
-0.5
0.0
0.5
1.0
Similarity
Access the dataset online usingGoogle's Embedding Projector
Figure 1: PCA of word embedding space (4157 words × 50 dimensions) filtered to include only top 100 similar words for each of ume and sakura (150 total). Similarity is represented by the difference in similarity scores between ume and sakura, scaled to [-1, 1].
梅 ‘ume’ (plum blossom)
桜 ‘sakura’ (cherry blossom)
(values below denote top 5 cosine similarity scores)
Figure 2: PCA of words similar to ⾹ ‘ka’ (fragrance) and 散る ‘chiru’ (fall).
藤
交ふ
下⽔
峰
⽴⽥
麓
み吉野
薄し
花橘
花桜
枝
⼭辺
庭
折る
⽂無し
⽐ふ
元
柞散らす
⻑閑けし
縁
籬
桜
主
⼀年
辺
菊
誤つ
誘ふ
⽊
⼥郎花
井⼿
幣
紅葉葉
⼭⾥
⼭桜
常夏
⽩菊
匂ふ
咎む
御雪
別きて
古⾥
桃
実
盛る
⼿向く九重
梢
秋萩
紛ふ
春⾬移ろふ
移り⾹
芽
異異に後ろめたし
藤袴
春霞随に
移す隠す打ち付けなり
先づ撫⼦
吉野⼭
紅葉づ
残す
盛りなり
⼭吹
⽊末
押し並べて
徒⼈
錦
⾹
梅
桜花
奥⼭
⽩河
垣根
狩る⼋重
谷
散る
橘
何れ
春風
植う
咲く
触る
⾄る
-1.0
-0.5
0.0
0.5
1.0
Sim
ilari
ty
⾹ ‘
ka’
(fra
gran
ce)
散る
‘chiru’ (fall)
Among summer flowers, the tachibana (the flower of orange) is very famous.
