NIPS 2016 Supervised Word Mover's Distance

http://qiita.com/tmshn/items/3ccc5d84daa23a98d4be

Supervied Word Mover's DistanceG. Huang, C. Guo, M.J. Kusner, Y. Sun, K.Q. Weinberger, F. Sha, In NIPS 2016.

‣https://papers.nips.cc/paper/6139-supervised-word-movers-distance

‣https://channel9.msdn.com/Events/Neural-Information-Processing-Systems-Conference/Neural-Information-Processing-Systems-Conference-NIPS-2016/Supervised-Word-Movers-Distance

‣https://github.com/gaohuang/S-WMD

‣

Supervied Word Mover's DistanceG. Huang, C. Guo, M.J. Kusner, Y. Sun, K.Q. Weinberger, F. Sha, In NIPS 2016.

‣https://papers.nips.cc/paper/6139-supervised-word-movers-distance

‣https://channel9.msdn.com/Events/Neural-Information-Processing-Systems-Conference/Neural-Information-Processing-Systems-Conference-NIPS-2016/Supervised-Word-Movers-Distance

‣https://github.com/gaohuang/S-WMD

‣

[Mikolov et al., '13]

Paris - France + Italy = Rome.

Paris

FranceRome

Italy

[Rubner et al., '98]

Icon designed by Freepik from Flaticon http://www.flaticon.com/free-icon/truck-cleaner-snow_96129

T i

=

db

T da

j

T da

ji

=

db

[Kusner et al., '15]X ∈ Rd×n

da, db

[Kusner et al., '15]X ∈ Rd×n

da, db

[Kusner et al., '15]X ∈ Rd×n

da, db

L2

[Kusner et al., '15]X ∈ Rd×n

da, db

L2

[Kusner et al., '15]X ∈ Rd×n

da, db

L2

O(p3 log p) O(p)

p da, db

O(dp) d L2

A dA(x, y) = d(Ax, Ay)

A

A

k[Goldberger et al., '05]

k

P(i j)

A1 A2A

pij

A1 A2A

A2

A1

i

j

k

[Huang et al., '16] A w

k

[Huang et al., '16] A w

k

‣ w0 = 1

‣ w0 = 1

‣ A0 = WCD NCA

‣ w0 = 1

‣ A0 = WCD NCA

A0

‣ w0 = 1

‣ A0 = WCD NCA

A0

A0, w0

k L2

k L2

k L2

P(a's NN is b) b

O(p3 log p) O(p2)

O(p3 log p) O(p2)

k

L2

[Huang et al., '16] Supervised Word Mover's Distance G. Huang, C. Guo, M.J. Kusner, Y. Sun, K.Q. Weinberger, F. Sha. Supervised Word Mover's Distance. In NIPS 2016. https://papers.nips.cc/paper/6139-supervised-word-movers-distance

[Mikolov et al., '13] word2vec T. Mikolov, K. Chen, G. Corrado, J. Dean. Efficient Estimation of Word Representations in Vector Space. In ICLR 2013 Workshop. https://arxiv.org/abs/1301.3781

[Rubner et al., '98] Eearth Mover's Distance Y. Rubner, C. Tomasi, L.J. Guibas. A Metric for Distributions with Applications to Image Databases. In ICCV 1998. http://ieeexplore.ieee.org/abstract/document/710701/ http://ai.stanford.edu/~rubner/papers/rubnerIccv98.pdf

[Kusner et al., '15] Word Mover's Distance M.J. Kusner, Y. Sun, N.I. Kolkin, K.Q. Weinberger. From Word Embedding To Documents Distances. In ICML 2015. http://www.jmlr.org/proceedings/papers/v37/kusnerb15.html

[Goldberger et al., '05] Neighborhood Component Analysis J. Goldberger, S. Roweis, G. Hinton, R. Salakhutdinov. Neighborhood Component Analysis. In NIPS 2005. https://papers.nips.cc/paper/2566-neighbourhood-components-analysis

Word Mover's Distance

http://yubessy.hatenablog.com/entry/2017/01/10/122737

http://www.slideshare.net/kentonozawa75/from-word-embeddings-to-document-distances

Earth Mover's Distance

http://aidiary.hatenablog.com/entry/20120804/1344058475