Joint Representation Learning for Multi-Modal …Joint Representation Learning for Multi-Modal...
Transcript of Joint Representation Learning for Multi-Modal …Joint Representation Learning for Multi-Modal...
![Page 1: Joint Representation Learning for Multi-Modal …Joint Representation Learning for Multi-Modal Transportation Recommendation Hao Liu1, Ting Li2, Renjun Hu3, Yanjie Fu4, Jingjing Gu5,](https://reader033.fdocuments.us/reader033/viewer/2022042410/5f281e619323d963c078c66d/html5/thumbnails/1.jpg)
Joint Representation Learning for Multi-Modal Transportation Recommendation
Hao Liu1, Ting Li2, Renjun Hu3, Yanjie Fu4, Jingjing Gu5, Hui Xiong1∗ 1The Business Intelligence Lab, Baidu Research
2National University of Defense Technology, Changsha, China 3SKLSDE Lab, Beihang University, Beijing, China
4Missouri University of Science and Technology, Missouri, USA 5Nanjing University of Aeronautics and Astronautics, Nanjing, China
Present by: Dr. Hao Liu
![Page 2: Joint Representation Learning for Multi-Modal …Joint Representation Learning for Multi-Modal Transportation Recommendation Hao Liu1, Ting Li2, Renjun Hu3, Yanjie Fu4, Jingjing Gu5,](https://reader033.fdocuments.us/reader033/viewer/2022042410/5f281e619323d963c078c66d/html5/thumbnails/2.jpg)
Emerging user requirements
High route planning decision cost across multiple transportation modes
Increasing activity radius
Complex travel context
Diversified transportation choices
Personalized and context-aware intelligent route planning Mul$-ModalTransporta$onRecommenda$on
![Page 3: Joint Representation Learning for Multi-Modal …Joint Representation Learning for Multi-Modal Transportation Recommendation Hao Liu1, Ting Li2, Renjun Hu3, Yanjie Fu4, Jingjing Gu5,](https://reader033.fdocuments.us/reader033/viewer/2022042410/5f281e619323d963c078c66d/html5/thumbnails/3.jpg)
Related Work
RouteRecommenda$on
• Liuetal.[1]discussedgenera$ngmul$-modalshortestroutesbasedonheterogeneoustransporta$onnetwork.
• MPR[2]andTPMFP[3]minesthemostpopularroutesandthemostfrequentpathsfrommassivetrajectoriesontheroadnetwork,respec$vely.
• Rogersetal.[4]considerspersonalpreferencetoimproverouterecommenda$onsquality.
NetworkEmbedding
• Metapath2vec[5]studiesnetworkembeddinginheterogeneousnetworks.
• Yaoetal.[6]andWangetal.[7]leveragesnetworkembeddingforregionfunc$onprofiling.
• Fengetal.[8]andZhaoetal.[9]appliesnetworkembeddingonPOIrecommenda$ons.
![Page 4: Joint Representation Learning for Multi-Modal …Joint Representation Learning for Multi-Modal Transportation Recommendation Hao Liu1, Ting Li2, Renjun Hu3, Yanjie Fu4, Jingjing Gu5,](https://reader033.fdocuments.us/reader033/viewer/2022042410/5f281e619323d963c078c66d/html5/thumbnails/4.jpg)
Trans2vec: Multi-Modal Transportation Recommendation Architecture
OD profiling
POI KG
User profiling
Multi-modal data
User
Modes
OD
Real time ETA
Station service
User profile
Context sensing
Trans2vec
Multi-modal transportation graph
construction
Joint representation learning
Online recommendations
![Page 5: Joint Representation Learning for Multi-Modal …Joint Representation Learning for Multi-Modal Transportation Recommendation Hao Liu1, Ting Li2, Renjun Hu3, Yanjie Fu4, Jingjing Gu5,](https://reader033.fdocuments.us/reader033/viewer/2022042410/5f281e619323d963c078c66d/html5/thumbnails/5.jpg)
Multi-Modal Transportation Graph Construction
• Amul&-modal transporta&on graph is a heterogeneous undirectedweighted graph𝐺=(𝑉,𝐸), where 𝑉=𝑈∪𝑂𝐷∪𝑀 is a set of heterogeneous nodes, and 𝐸= 𝐸↓𝑢𝑚 ∪ 𝐸↓𝑜𝑑𝑚 ∪ 𝐸↓𝑢𝑢 ∪ 𝐸↓𝑜𝑑𝑜𝑑 isasetofheterogeneousedgesincludinguser-modeedges 𝐸↓𝑢𝑚 ,OD-modeedges 𝐸↓𝑜𝑑𝑚 ,user-useredges 𝐸↓𝑢𝑢 andOD-ODedges 𝐸↓𝑜𝑑𝑜𝑑 .
Office toIndustrial
CBD to MallResidentialto MallResidential
to Office
Users
Transportmodes
OD pairs
Car
Taxi
BusBike
Walk
Anillustra$veExampleofMul$-modalTransporta$onGraphTravelevents
ResidentialIndustrial
MallCBD
![Page 6: Joint Representation Learning for Multi-Modal …Joint Representation Learning for Multi-Modal Transportation Recommendation Hao Liu1, Ting Li2, Renjun Hu3, Yanjie Fu4, Jingjing Gu5,](https://reader033.fdocuments.us/reader033/viewer/2022042410/5f281e619323d963c078c66d/html5/thumbnails/6.jpg)
• Analogizetraveleventstosentencesandrandomwalks,inordertolearnlow-dimensionalrepresenta$onsofusers,ODpairs,andtransportmodes.
The Base Model
sigmoid Embedding of user Embedding of mode Embedding of OD
User-mode-ODembedding:
EmbeddingwithNega$vesampling:
![Page 7: Joint Representation Learning for Multi-Modal …Joint Representation Learning for Multi-Modal Transportation Recommendation Hao Liu1, Ting Li2, Renjun Hu3, Yanjie Fu4, Jingjing Gu5,](https://reader033.fdocuments.us/reader033/viewer/2022042410/5f281e619323d963c078c66d/html5/thumbnails/7.jpg)
Anchor Embedding
Pairwisetransportmoderelevancematrix
Problem
Ø thereareonlyseveral(e.g.,5inourcase)
transportmodenodeswhereastherearea
largenumberofusernodesandODnodes.
ü eachnodeisassignedadiscrimina$ve
embeddingthatreflectsitsinherentcontext
informa$on.
Solution
![Page 8: Joint Representation Learning for Multi-Modal …Joint Representation Learning for Multi-Modal Transportation Recommendation Hao Liu1, Ting Li2, Renjun Hu3, Yanjie Fu4, Jingjing Gu5,](https://reader033.fdocuments.us/reader033/viewer/2022042410/5f281e619323d963c078c66d/html5/thumbnails/8.jpg)
• Thechoiceoftransportmodeishighlyinfluencedbythecharacteris$csofusers
• e.g.,age,sex,mar$al
• User-userrelevance:
• Userconstraints:
Modeling User Relevance
Beyondtravelpreference:fined-graineduserprofileatBaidu
User attribute vector
![Page 9: Joint Representation Learning for Multi-Modal …Joint Representation Learning for Multi-Modal Transportation Recommendation Hao Liu1, Ting Li2, Renjun Hu3, Yanjie Fu4, Jingjing Gu5,](https://reader033.fdocuments.us/reader033/viewer/2022042410/5f281e619323d963c078c66d/html5/thumbnails/9.jpg)
• Distanceandtravelpurpose(e.g.,home-work,home-commercial)areamongthemostinfluen$alfactorsforchoosingtransportmodes
• ODrelevence:
• ODconstraints:
Modeling OD Relevance
ODheatmap
![Page 10: Joint Representation Learning for Multi-Modal …Joint Representation Learning for Multi-Modal Transportation Recommendation Hao Liu1, Ting Li2, Renjun Hu3, Yanjie Fu4, Jingjing Gu5,](https://reader033.fdocuments.us/reader033/viewer/2022042410/5f281e619323d963c078c66d/html5/thumbnails/10.jpg)
Joint Learning & Online Recommendations
• Overallobjec$ve:
• Thescoreofeachmodeiscomputedby:
![Page 11: Joint Representation Learning for Multi-Modal …Joint Representation Learning for Multi-Modal Transportation Recommendation Hao Liu1, Ting Li2, Renjun Hu3, Yanjie Fu4, Jingjing Gu5,](https://reader033.fdocuments.us/reader033/viewer/2022042410/5f281e619323d963c078c66d/html5/thumbnails/11.jpg)
Experiments – Objectives & Data Sets
Table1.DataSta$s$cs
• TheoverallperformanceofTrans2Vec
• Theparametersensi$vity• Thetransportmoderelevance• TherobustnessofTrans2Vec
Objec$ves• BEIJINGandSHANGHAI• ProducedbasedonthemapqueriesanduserfeedbacksontheBaiduMap,
• TimewindowApril1,2018-August20,2018.
Datasets
![Page 12: Joint Representation Learning for Multi-Modal …Joint Representation Learning for Multi-Modal Transportation Recommendation Hao Liu1, Ting Li2, Renjun Hu3, Yanjie Fu4, Jingjing Gu5,](https://reader033.fdocuments.us/reader033/viewer/2022042410/5f281e619323d963c078c66d/html5/thumbnails/12.jpg)
Experiments – Overall Results
Table2.Overallperformance
• Logis$cregression• L2R[10]• PTE[11]• Metapath2Vec[5]
• NDCG@5,• Theweightedprecision(PREC)• Recall(REC)• F1
Evalua$onmetrics Baselines
![Page 13: Joint Representation Learning for Multi-Modal …Joint Representation Learning for Multi-Modal Transportation Recommendation Hao Liu1, Ting Li2, Renjun Hu3, Yanjie Fu4, Jingjing Gu5,](https://reader033.fdocuments.us/reader033/viewer/2022042410/5f281e619323d963c078c66d/html5/thumbnails/13.jpg)
Experiments – Parameter Sensitivity
EffectofdonBEIJING EffectofkonBEIJING
Effectof𝛽↓1 onBEIJING Effectof𝛽↓2 onBEIJING
![Page 14: Joint Representation Learning for Multi-Modal …Joint Representation Learning for Multi-Modal Transportation Recommendation Hao Liu1, Ting Li2, Renjun Hu3, Yanjie Fu4, Jingjing Gu5,](https://reader033.fdocuments.us/reader033/viewer/2022042410/5f281e619323d963c078c66d/html5/thumbnails/14.jpg)
Experiments – Robustness Check
GroupbyusersonBEIJING GroupbyodsonBEIJING
• Wetesttheperformanceonfoursubgroupsofusers(resp.ODpairs)• Groupusers(resp.ODpairs)byK-means
• TheperformanceisstableindifferentgroupsofusersandODpairs.
![Page 15: Joint Representation Learning for Multi-Modal …Joint Representation Learning for Multi-Modal Transportation Recommendation Hao Liu1, Ting Li2, Renjun Hu3, Yanjie Fu4, Jingjing Gu5,](https://reader033.fdocuments.us/reader033/viewer/2022042410/5f281e619323d963c078c66d/html5/thumbnails/15.jpg)
%
Faster than bus & drive
%
Cheaper than taxi
Multi-Modal Transportation Recommendation on Baidu Map Multi-Modal Transportation Recommendation on Baidu Map
![Page 16: Joint Representation Learning for Multi-Modal …Joint Representation Learning for Multi-Modal Transportation Recommendation Hao Liu1, Ting Li2, Renjun Hu3, Yanjie Fu4, Jingjing Gu5,](https://reader033.fdocuments.us/reader033/viewer/2022042410/5f281e619323d963c078c66d/html5/thumbnails/16.jpg)
Multi-Modal Transportation Recommendation on Baidu Map References
[1]Liu,L.2011.Datamodelandalgorithmsformul&modalrouteplanningwithtransporta&onnetworks.Ph.D.Disser-ta$on,TechnischeUniversitatMunchen.[2]Chen,Z.;Shen,H.T.;andZhou,X.2011.Discoveringpopularroutesfromtrajectories.[3]Luo,W.;Tan,H.;Chen,L.;andNi,L.M.2013.Find-ing$meperiod-basedmostfrequentpathinbigtrajectorydata.InProceedingsofthe2013ACMSIGMODinterna-&onalconferenceonmanagementofdata,713–724.ACM.[4]Rogers,S.,andLangley,P.1998.Personalizeddrivingrouterecommenda$ons.InProceedingsoftheAmericanAssoci-a&onofAr&ficialIntelligenceWorkshoponRecommenderSystems,96–100.[5]Dong,Y.;Chawla,N.V.;andSwami,A.2017.metap-ath2vec:Scalablerepresenta$onlearningforheterogeneousnetworks..[6]Yao,Z.;Fu,Y.;Liu,B.;Hu,W.;andXiong,H.2018.Rep-resen$ngurbanfunc$onsthroughzoneembeddingwithhu-manmobilitypaoerns.InIJCAI,3919–3925.[7]Wang,H.,andLi,Z.2017.Regionrepresenta$onlearningviamobilityflow.InProceedingsofthe2017ACMonCon-ferenceonInforma&onandKnowledgeManagement,237–246.ACM.[8]Feng,S.;Cong,G.;An,B.;andChee,Y.M.2017.Poi2vec:Geographicallatentrepresenta$onforpredic$ngfuturevis-itors.InAAAI,102–108.[9]Zhao,S.;Zhao,T.;King,I.;andLyu,M.R.2017.Geo-teaser:Geo-temporalsequen$alembeddingrankforpoint-of-interestrecommenda$on.InProceedingsofthe26thin-terna&onalconferenceonworldwidewebcompanion,153–162.Interna$onalWorldWideWebConferencesSteeringCommioee.[10]Burges,C.J.2010.Fromranknettolambdaranktolamb-damart:Anoverview.Technicalreport.[11]Tang,J.;Qu,M.;andMei,Q.2015.Pte:Predic$vetextem-beddingthroughlarge-scaleheterogeneoustextnetworks.SIGKDD.
![Page 17: Joint Representation Learning for Multi-Modal …Joint Representation Learning for Multi-Modal Transportation Recommendation Hao Liu1, Ting Li2, Renjun Hu3, Yanjie Fu4, Jingjing Gu5,](https://reader033.fdocuments.us/reader033/viewer/2022042410/5f281e619323d963c078c66d/html5/thumbnails/17.jpg)
Thanks ! Q & A