all categories internal usage - ClopiNet€¦ · PPT file · Web view · 2009-06-26Sampath...

35
KDD Cup 2009 Fast Scoring on a Large Database Presentation of the Results at the KDD Cup Workshop June 28, 2008 The Organizing Team

Transcript of all categories internal usage - ClopiNet€¦ · PPT file · Web view · 2009-06-26Sampath...

  • KDD Cup2009

    Fast Scoring on a Large Database

    Presentation of the Results at the KDD Cup Workshop

    June 28, 2008

    The Organizing Team

  • KDD Cup 2009 Organizing Team

    Project team at Orange Labs R&D:

    Vincent Lemaire Marc Boull Fabrice Clrot Raphal Fraud Aurlie Le Cam Pascal Gouzien

    Beta testing and proceedings editor:

    Gideon Dror

    Web site design:

    Olivier Guyon (MisterP.net, France)

    Coordination (KDD cup co-chairs):

    Isabelle Guyon David Vogel

    The main objective of the challenge is to make good predictions of the target variables. The prediction of each target variable is thought of as a separate classification problem. The results of classification, obtained by thresholding the prediction score, may be represented in a confusion matrix, where tp (true positive), fn (false negative), tn (true negative) and fp (false positive) represent the number of examples falling into each possible outcome:

    PredictionClass +1Class -1TruthClass +1tpfnClass -1fptn

    We define the sensitivity (also called true positive rate or hit rate) and the specificity (true negative rate) as:Sensitivity = tp/posSpecificity = tn/negwhere pos=tp+fn is the total number of positive examples and neg=tn+fp the total number of negative examples.

  • Thanks to our sponsors

    OrangeACM SIGKDDPascalUnipenGoogleHealth Discovery CorpClopinetData Mining SolutionsMPS

  • Record KDD Cup Participation

    Year# Teams1997451998571999242000312001136200218200357200410220053720066820079520081282009453

    Chart1

    45

    57

    24

    31

    136

    18

    57

    102

    37

    68

    95

    128

    453

    # Teams

    Year

    KDD Cup Participation By Year

    Sheet4

    Domain ExtensionCountry

    DomainTotalArgentina3a(unknown)

    Count of Domain.arar3Argentina3Australia5bitnet(unknown)

    DomainTotal.atat3Austria3Austria3acUnited Kingdom academic institutions

    .ar3.auau11Australia11Belgium11adAndorra

    .at3.bebe11Belgium11Brazil1aeUnited Arab Emirates

    .au11.bgbg1Bulgaria1Bulgaria1afAfghanistan

    .be11.bizbiz202Canada12agAntigua and Barbuda

    .bg1.brbr1Brazil1Chile1aiAnguilla

    .biz2.c.zac.za101China25alAlbania

    .br1.caca12Canada12Fiji25amArmenia

    .c.za1.chch4Switzerland4Finland25anNetherlands Antilles

    .ca12.clcl1Chile1France25aoAngola

    .ch4.cncn25China25Germany37aqAntarctica

    .cl1.comcom780US Commercial780Greece2arArgentina

    .cn25.com1com1101Hong Kong20asAmerican Samoa

    .com780.com2com2101Hungary14atAustria

    .com11.coomcoom101India14auAustralia

    .com21.dede37Germany37Iran14awAruba

    .coom1.eduedu72US Educational72Ireland2azAzerbaijan

    .de37.eses12Spain12Israel1baBosnia and Herzegovina

    .edu72.eueu101Italy6bbBarbados

    .es12.fifi3Finland3Japan6bdBangladesh

    .eu1.fjfj1Fiji1Jordan2beBelgium

    .fi3.frfr31France31Latvia1bfBurkina Faso

    .fj1.grgr2Greece2Malaysia1bgBulgaria

    .fr31.hkhk20Hong Kong20Mexico1bhBahrain

    .gr2.huhu14Hungary14Netherlands1biBurundi

    .hk20.ieie2Ireland2New Zealand3bjBenin

    .hu14.ilil3Israel3Pakistan1bmBermuda

    .ie2.inin11India11Portugal1bnBrunei Darussalam

    .il3.irir1Iran1Romania1boBolivia

    .in11.itit6Italy6Russian Federation25brBrazil

    .ir1.jojo2Jordan2Singapore10bsBahamas

    .it6.jpjp21Japan21Slovak Republic1btBhutan

    .jo2.krkr8Korea (South)8Slovenia1bvBouvet Island

    .jp21.lvlv1Latvia1South Africa1bwBotswana

    .kr8.mxmx1Mexico1South Korea1byBelarus

    .lv1.mymy1Malaysia1Spain1bzBelize

    .mx1.netnet17US network17Sweden1caCanada

    .my1.nlnl6Netherlands6Switzerland1ccCocos (Keeling) Islands

    .net17.nznz2New Zealand (Aotearoa)2Taiwan52cfCentral African Republic

    .nl6.oeoe101Turkey52cgCongo

    .nz2.orgorg14US Non-Profit Organization14Uganda1chSwitzerland

    .oe1.pkpk1Pakistan1United Kingdom19ciCote d'Ivoire (Ivory Coast)

    .org14.plpl21021Uruguay1ckCook Islands

    .pk1.ptpt5Portugal5United States887clChile

    .pl21.roro1Romania1cmCameroon

    .pt5.rsrs202cnChina

    .ro1.ruru25Russian Federation25coColombia

    .rs2.sese1Sweden1comUS Commercial

    .ru25.sgsg10Singapore10crCosta Rica

    .se1.sisi4Slovenia4csCzechoslovakia (former)

    .sg10.sksk1Slovak Republic1cuCuba

    .si4.trtr2Turkey2cvCape Verde

    .sk1.twtw52Taiwan52cxChristmas Island

    .tr2.ugug1Uganda1cyCyprus

    .tw52.ukuk19United Kingdom19czCzech Republic

    .ug1.uyuy1Uruguay1deGermany

    .uk19.vivi1Virgin Islands (US)1djDjibouti

    .uy1.xxxx101dkDenmark

    .vi1.yyyy101dmDominica

    .xx1.zaza2South Africa2doDominican Republic

    .yy1Grand Totalrand Total1299dzAlgeria

    .za2ecEcuador

    Grand Total1299eduUS Educational

    eeEstonia

    egEgypt

    ehWestern Sahara

    erEritrea

    esSpain

    etEthiopia

    fiFinland

    fjFiji

    fkFalkland Islands (Malvinas)

    fmMicronesia

    foFaroe Islands

    frFrance

    fxFrance (Metropolitan)

    gaGabon

    gbGreat Britain (UK)

    gdGrenada

    geGeorgia

    gfFrench Guiana

    ghGhana

    giGibraltar

    glGreenland

    gmGambia

    gnGuinea

    govUS Government

    gpGuadaloupe

    gqEquatorial Guinea

    grGreece

    gsSouth Georgia and South Sandwich Islands

    gtGuatemala

    guGuam

    gwGuinea-Bissau

    gyGuyana

    hkHong Kong

    hmHeard and McDonald Islands

    hnHonduras

    hrCroatia (Hrvatska)

    htHaiti

    huHungary

    idIndonesia

    ieIreland

    ilIsrael

    inIndia

    ioBritish Indian Ocean Territory

    iqIraq

    irIran

    isIceland

    itItaly

    jmJamaica

    joJordan

    jpJapan

    keKenya

    kgKyrgyzstan

    khCambodia

    kiKiribati

    kmComoros

    knSaint Kitts and Nevis

    kpKorea (North)

    krKorea (South)

    kuKuwait

    kyCayman Islands

    kzKazakhstan

    laLaos

    lbLebanon

    lcSaint Lucia

    liLiechtenstein

    lkSri Lanka

    lrLiberia

    lsLesotho

    ltLithuania

    luLuxembourg

    lvLatvia

    lyLibya

    maMorocco

    mcMonaco

    mdMoldova

    mgMadagascar

    mhMarshall Islands

    milUS Military

    mkMacedonia

    mlMali

    mmMynamar

    mnMongolia

    moMacau

    mpNorthern Mariana Islands

    mqMartinique

    mrMauritania

    msMontserrat

    mtMalta

    muMauritius

    mvMaldives

    mwMalawi

    mxMexico

    myMalaysia

    mzMozambique

    naNamibia

    ncNew Caledonia

    neNiger

    netUS network

    nfNorfolk Island

    ngNigeria

    niNicaragua

    nlNetherlands

    noNorway

    npNepal

    nrNauru

    ntNeutral Zone

    nuNiue

    nzNew Zealand (Aotearoa)

    omOman

    orgUS Non-Profit Organization

    paPanama

    pePeru

    pfFrench Polynesia

    pgPapua New Guinea

    phPhilippines

    pkPakistan

    pl>Poland

    pmSaint Pierre and Miquelon

    pnPitcairn

    prPuerto Rico

    ptPortugal

    pwPalau

    pyParaguay

    qaQatar

    reReunion

    roRomania

    ruRussian Federation

    rwRwanda

    saSaudi Arabia

    sbSolomon Islands

    scSeychelles

    sdSudan

    seSweden

    sgSingapore

    shSaint Helena

    siSlovenia

    sjSvalbard and Jan Mayen Islands

    skSlovak Republic

    slSierra Leone

    smSan Marino

    snSenegal

    soSomalia

    srSuriname

    stSao Tome and Principe

    suUSSR (former)

    svEl Salvador

    sySyria

    szSwaziland

    tcTurks and Caicos Islands

    tdChad

    tfFrench Southern Territories

    tgTogo

    thThailand

    tjTajikistan

    tkTokelau

    tmTurkmenistan

    tnTunisia

    toTonga

    tpEast Timor

    trTurkey

    ttTrinidad and Tobago

    tvTuvalu

    twTaiwan

    tzTanzania

    uaUkraine

    ugUganda

    ukUnited Kingdom

    umUS Minor Outlying Islands

    usUnited States

    uyUruguay

    uzUzbekistan

    vaVatican City State

    vcSaint Vincent and the Grenadines

    veVenezuela

    vgVirgin Islands (British)

    viVirgin Islands (US)

    vnViet Nam

    vuVanuatu

    wfWallis and Futuna Islands

    wsSamoa

    yeYemen

    ytMayotte

    yuYugoslavia

    zaSouth Africa

    zmZambia

    zrZaire

    zwZimbabwe

    Bulgaria

    Bhutan

    Belize

    Denmark

    Falkland Islands (Malvinas)

    Georgia

    Cambodia

    Cayman Islands

    French Polynesia

    Slovenia

    Uganda

    Uzbekistan

    Zambia

    Sheet1

    Domain

    Pablo Mouzo 4138.ar

    Marcelo Ferreyra 4643.ar

    Andrs Parra 4138.ar

    Michael Jahrer 4340.at

    Markus Muhr 3431.at

    Michael Pfeiffer 4138.at

    Vladimir Nikulin 4340.au

    Hugh Miller 4037.au

    Gad Abraham 4239.au

    Stephen Mc Ateer 4643.au

    Tung Pham 3431.au

    Luo Wei 2825.au

    Tim Manns 3431.au

    Mai Dang 3229.au

    Jojo the Monkey 3431.au

    vateesh chand 3734.au

    Mai Dang 3936.au

    Kevin Franoisse 4946.be

    Pierre Geurts 3431.be

    Peter Clerinx 4643.be

    Kris De Brabanter 5350.be

    Peter Karsmakers 5249.be

    saaid hasbi 3633.be

    cedric simon 4037.be

    Catharina Olsen 3431.be

    Damien Franois 4138.be

    Kristof Coussement 5249.be

    koen de bock 3532.be

    vencisalv trifonov 3633.bg

    Phil Brierley 3430.biz

    Sharon Lewis 3430.biz

    Robson Cordeiro 3633.br

    Musa Gabere 3025.c.za

    Gordon Cormack 3835.ca

    Abdellali Kelil 4845.ca

    zhengzheng xing 3330.ca

    Radford Neal 3936.ca

    Alexei Nordell 4037.ca

    Ruiwen Chen 3330.ca

    Da Kuang 2421.ca

    Tengke Xiong 4239.ca

    Salar Sayyad 3936.ca

    Lyon Chang 3128.ca

    Vlado Keselj 3027.ca

    Marcin Mizianty 3835.ca

    Adam Woznica 3633.ch

    Ali Ajdari Rad 3835.ch

    Christian Sommer 3027.ch

    Thomas Fuchs 3936.ch

    Cristian Bravo 3936.cl

    Ray Sun 2522.cn

    Luheng He 3734.cn

    bin wu 2623.cn

    Minlong Lin 3936.cn

    Wang Pu 3532.cn

    Xiaoxing Yang 4037.cn

    rui wang 3633.cn

    guoxian yu 4037.cn

    you zhou 2926.cn

    Weng Xiaoqing 3532.cn

    Yuchen Zhang 4744.cn

    Fang Wang 3532.cn

    Jianmin Li 4340.cn

    Wenbin Tang 4441.cn

    Dong Wang 4138.cn

    Chen Hongyan 3431.cn

    Xi Chen 3128.cn

    Han Zhi 2724.cn

    Shan Long 2623.cn

    Yue Yi 2623.cn

    zhang liming 3431.cn

    Xinshun Liu 4340.cn

    Yang Yu 3027.cn

    Min Xiao 3128.cn

    Maolin Yang 3128.cn

    Isabelle Guyon 3531.com

    Vincent Lemaire 5248.com

    Chiwoo Park 3531.com

    Patrick Abbas 4137.com

    Nena Marin 3733.com

    srivatsava daruru 3733.com

    Abhishek Shrivastava 4036.com

    rohit paravastu 4238.com

    John Moe 2723.com

    Krishna Mehta 3531.com

    Hossein Rahimy 4238.com

    Tal Galili 3329.com

    Leonardo Kury 3329.com

    Vempati Sreekanth 4743.com

    jack daniels 3531.com

    Naser Peiravian 3935.com

    Isabelle Boser 3834.com

    Al Rashid 2925.com

    Umair ul Hassan 4339.com

    regis vert 3430.com

    Sudarshan Lamkhede 4945.com

    Ming Li 3026.com

    pop tt 2723.com

    Rakesh Ramakrishnan 4137.com

    Isidor Passy 3228.com

    Vahan Grigoryan 3632.com

    Jaya Kawale 3531.com

    goon goon 3127.com

    Rong Pan 2925.com

    Haisu Zhang 2925.com

    Baochen Sun 3228.com

    SANGUN YUN 3329.com

    Huiwen Cai 3430.com

    Tae-hyun Shin 3733.com

    Soujanya Vadapalli 4036.com

    feng wang 3733.com

    Cathy Wat 3127.com

    Wei Xiong Shang 3632.com

    Kim Jae Wang 3733.com

    Yan Zhu 2824.com

    Shuangyong Song 3531.com

    4743.com

    Gouchol Pok 3329.com

    shobha prabhakar 3834.com

    Junjie Wu 2824.com

    Hung-Yi Lo 2925.com

    padmasini rengabashyam 6258.com

    sar ch 2824.com

    Chuancong Gao 3531.com

    Srividhya Kannan 4137.com

    Varun Mithal 3733.com

    Chun Li 3228.com

    Didier Baclin 3329.com

    Evan Xiang 2824.com

    Jaco Cronje 3329.com

    Nghe Nguyen 3834.com

    Varun Aggarwal 3430.com

    Pei He 2824.com

    suicheng gu 3026.com

    Peng-Jen Chen 3329.com

    Daniel Sparing 4137.com

    Mohammad Rouzbeh 4137.com

    Jozo Kovac 3329.com

    Mahmood Deypir 3430.com

    B T 2925.com

    Tu Wei-Che 3430.com

    Christian Colot 3935.com

    Dymitr Ruta 3228.com

    Alexander Larko 4137.com

    James Fink 3329.com

    Shin Ando 3329.com

    Jean Adams 3127.com

    li qingyang 3733.com

    Yue-Shiang Liu 3935.com

    Xiangliang ZHANG 4440.com

    David Knowles 3531.com

    I F 2319.com

    Grzegorz Swirszcz 3935.com

    kuanliang liu 3531.com

    Amin DAOUD 3026.com

    Celine Theeuws 3430.com

    tus1337 tus1337 4036.com

    Mathias Adankon 4339.com

    Jean-Philippe Vert 3733.com

    baptiste pigaux 3632.Com

    Roger Zhan 2925.COM

    Kyong Shim 3127.com

    Jian Luo 2622.com

    saeed dehghan 3531.com

    Yuyin Sun 3834.com

    Guang-Tong Zhou 4137.com

    Xinyu Wu 2824.com

    monica wei 3834.com

    chenliang xu 3430.com

    Kelvin Sim 3127.com

    pirouz navidi 3632.com

    Guli Lin 2824.com

    Chao-An Wang 3228.com

    Awais Athar 3531.com

    Teng-Kai Fan 3531.com

    Jason McFall 3733.com

    Jiesheng Zheng 3228.com

    Alexander Borisov 4743.com

    Marcin Kowalski 4440.com

    Tae Sik Yoon 3329.com

    Jorge Sueiras 4541.com

    Jun Gong 2824.com

    Yang Qu 3127.com

    Mehmet Ali Yatbaz 4036.com

    Vincenzo D\'Elia 4036.com

    babak Yadegari 4137.com

    Sangwoo Kim 3026.com

    Yongseok Lee 3632.com

    Pascal Gouzien 5046.com

    ali taheri 3026.com

    Thierry VAN DE MERCKT 3733.com

    Dave Tahmoush 3531.com

    John Ries 3733.com

    Michal Tafil 3531.com

    Nitesh Chawla 3935.com

    gang wang 2925.com

    Carlos Leandro 3228.com

    guiduo yan 3329.com

    Sreedal Menon 3329.com

    shobha prabhakar 5046.com

    Akash Narayana 3430.com

    Ferenc Szidarovszky 5854.com

    Changliang Wang 4339.com

    Lin Zhu 3430.com

    Boris Chidlovskii 4642.com

    Yi Zhen 2824.com

    bhupinder juneja 4541.com

    Tariq Tiger 3733.com

    sclu sclu 3127.com

    Jessica Enneking 3935.com

    chun han 3026.com

    Supreet Mandala 3632.com

    Ya Luo 2622.com

    Hugo Gvert 3531.com

    Andrzej Janusz 3935.com

    Wei Liu 2925.com

    Essraa Abbas 3733.com

    Chenhao Tan 3329.com

    Haris Anwer 3935.com

    juli prn 2824.com

    Wenyin Gong 3531.com

    Jianfei Wu 3632.com

    Marcin Gruszka 3733.com

    Ji You 2622.com

    Hao Wu 2622.com

    Yuzhou Zhang 3430.com

    Bin Cao 2723.com

    Shrikant Kashyap 4541.com

    Feng Shi 3026.com

    Lixing WANG 3026.com

    Tang Zhigao 2723.com

    Roy Lee 2622.com

    Vijeth Dinesha 3531.com

    Cheng-Han Chen 3632.com

    Pierre Gramme 3026.com

    Claudia Perlich 3632.com

    Fan Long 3026.com

    Venu Satuluri 3834.com

    Tony Montana 4137.com

    Catia Cunha 3733.com

    Carlos Joa 3228.com

    Steve Jin 3127.com

    Yin Zhu 3026.com

    Kathryn Buttler 3430.com

    James Bezetek 3329.com

    Anuradha Bhamidipaty 4238.com

    Pei Yang 2925.com

    kiran G V R 3329.com

    Francesc Martnez i Fuster 5248.com

    Camelia Vidrighin 3632.com

    Jos Luis Ferreiro 5046.com

    Dan Zisu 2723.com

    Yuri Botnikov 3228.com

    q jw 2319.com

    Pablo Hadjinian 4440.com

    Peter Chu 3026.com

    diego ma 2723.com

    Ravi Shankar 4036.com

    Kenneth Lawhorne 3935.com

    quan yuan 3531.com

    John Spooner 3935.com

    Prakhar Jain 3329.com

    Neil Eklund 3632.com

    dodo twokay 3127.com

    Rong Duan 2824.com

    Dihua Guo 2925.com

    george hayduke 3935.com

    Sara Mostafavi 4137.com

    rajan lukose 3733.com

    Eric Lucas 2925.com

    Jie Cheng 3127.com

    Matthew Perone 3531.com

    Wieslaw Pietruszkiewicz 5147.com

    Jianjun Xie 3026.com

    Guy Shani 3430.com

    Vikas Raykar 3632.com

    Chuck Lam 3430.com

    DFT Boy 3127.com

    S. Joshua Swamidass 4137.com

    Wei Jiang 3228.com

    Vineet Agrwal 3834.com

    Ya Hu 2824.com

    na na 2016.com

    rovenna chu 3127.com

    Lokesh Kulkarni 4339.com

    William-Chandra Tjhi 4440.com

    xia wei 2521.com

    Kazuki Tsubo 3531.com

    Jana Bilesova 4137.com

    Ricardo Martinez 3733.com

    Cassio Melo 3531.com

    David Lazaridis 3632.com

    Jiexun Li 2824.com

    Harold Christopher Burger 5652.com

    romakanta irungbam 4036.com

    Yifan Hu 3531.com

    DeDe Paul 2824.com

    Weizhong Yan 2925.com

    Joao Botelho 3935.com

    Sumit Chopra 3329.com

    mohamed amir esseghir 4541.com

    Cailing Dong 3632.com

    Rui Zhang 3228.com

    Siddharth Pal 3329.com

    Abdel Halim Hiassat 4339.com

    67890 12345 3127.com

    Ning Situ 3026.com

    fabon dzogang 3935.com

    Carlos Roberto Arias 4743.com

    Diego Funes 3127.com

    Ravi Sikhakolanu 4642.com

    Yongming Shen 3531.com

    Mohammad Taheri 3935.com

    ASHISH TENDULKAR 3733.com

    Mahesh Panchal 3733.com

    anil rohilla 3632.com

    Raghvendra Mall 4743.com

    Tom Lewicke 3834.com

    Polly Wong 3329.com

    David Cieslak 3935.com

    Frdric Clerc 3430.com

    Jianming Jin 3531.com

    Moninder Singh 3632.com

    Ovidiu Negrila 3632.com

    Shengwen Yang 3834.com

    Yu GONG 2723.com

    Lian Liu 3430.com

    Mai Dang 2925.com

    Sangameshwar Patil 4743.com

    Timothy Clendaniel 5046.com

    Shuying Wang 3228.com

    hsu sheng 3228.com

    Stefan Skogwik 3531.com

    long top 3228.com

    Bartosz Witorzenc 3632.com

    Ye Wang 3228.com

    Srivani Narra 3329.com

    Wei Mingjun 2925.com

    Marcin Pionnier 4339.com

    Amit Thombre 3834.com

    Henree Domen 3632.com

    Arek Beer 2925.com

    Hao Liu 3228.com

    Meejung Cheigh 3733.com

    wenhao jiang 3127.com

    Mohammad ali Nikouei 3935.com

    Qiong Chen 3026.com

    Xu Xu 3329.com

    Mohammad Aziz 3531.com

    arif akbulut 3228.com

    Hanumesh Nagur 4238.com

    jiang jie 3026.com

    X X 2319.com

    Matthew Walker 4541.com

    Louis Duclos-Gosselin 5046.com

    Braulio Medina Dias 4642.com

    Zhibo Xiao 3228.com

    Omar Cardona 3632.com

    Bill Chen 3329.com

    yasmeen mohammed zafer 5248.com

    Narayan Bhamidipati 4440.com

    ayad handalianpour 4036.com

    zahra hamidian 3733.com

    Ravi Kumar Sikhakolanu 5652.com

    XUEYAO LIANG 4339.com

    uday kiran 3228.com

    Joe Urban 3026.com

    Junfeng Pan 3430.com

    Jae Poong Jeong 3430.com

    tae hyung Kim 3430.com

    hyunkuyn yoo 3228.com

    Heesu park 3329.com

    Yong-Mook Lee 3329.com

    Jun Soo Oh 3430.com

    Hwang Heejin 3733.com

    xu yanzi 2521.com

    Wei Fan 2723.com

    Hao Zhang 3127.com

    huang chih-kai 3935.com

    Tiantian Xu 3531.com

    Swetha Krish 3632.com

    Xiangjun Zhou 3733.com

    Martin Hong 3935.com

    Heine Bodekaer 4036.com

    Alain Pernot 3329.com

    Attila Kiss 3632.com

    HUANG TERRY 3228.com

    Suraj MS 3127.com

    wt zh 2521.com

    Toshihiro Nakae 3834.com

    Shih-Wei Tu 3026.com

    Kenn Slagter 3834.com

    Chia-Yu Tsai 3632.com

    Yen Paniagua 3127.com

    Davis Muhajereen Dimalen 4541.com

    John Trenkle 4541.com

    Sheng-Kai Wang 3430.com

    aravind gowrisankar 4339.com

    Zainab Jamal 3430.com

    Erheng Zhong 3329.com

    up chen 2622.com

    Tams Schradi 3935.com

    Anung Ariwibowo 3632.com

    Satya Sandeep Lakkaraju 4541.com

    Arun Kumar Mani 4339.com

    Bo Han 2622.com

    Claudio Favre 3632.com

    Yuanming Yu 3026.com

    PU Jiansu 3430.com

    Lacr Wang 2925.com

    Mariusz Gdarowski 4238.com

    Marcin Piechota 4339.com

    raghav ghaiee 3834.com

    Marcos Souza 4238.com

    Giuseppe Paleologo 3733.com

    Nathaniel Ramm 4137.com

    Brendan O\'Connor 3834.com

    Willy Duville 4036.com

    Li-Chin Lee 3430.com

    heart Wang 3127.com

    Evgenija Odinokova 3834.com

    Ajay Ohri 3733.com

    Botao Hu 2824.com

    Xiatian Zhang 3935.com

    Ihor Menshykov 3531.com

    William Lao 3228.com

    M Hofmann 3733.com

    Alexandru Niculescu-Mizil 4642.com

    Raghava Kothapalli 4137.com

    Bahman Sodeir Abedi 3935.com

    Dubravko Culibrk 4541.com

    Marek Slonski 3935.com

    Chih-heng Chung 4036.com

    dima dow 2824.com

    Yanjuan Bian 3632.com

    Aisha Zaidan 3531.com

    Wayne Thompson 3935.com

    omar darwish 3632.com

    Kurt Schwenke 3329.com

    Ritendra Datta 4137.com

    Mohammad Al-Masri 3632.com

    ujwala kotti 3329.com

    Alex Lin 3127.com

    Ex Maple 2824.com

    Mauricio Mercado 4137.com

    Balazs Kegl 3531.com

    Stefka Dabijeva 3632.com

    Hsiao-Shan Hsieh 4137.com

    ma shengxiang 3834.com

    cloud computing 4945.com

    Moon Shotter 3632.com

    Ivan Karlovic 3430.com

    Elias Blomqvist 4339.com

    Sachin Gangaputra 3834.com

    Test Account 3531.com

    Li Sheng 3329.com

    dario nappa 2925.com

    wang yue 3026.com

    zephyr martin 3026.com

    Andy Saputra 3632.com

    Anshuman Verma 3733.com

    Ping Hua Yang 4541.com

    Anastasia Novokreshenova 4642.com

    Taiyeong Lee 3531.com

    Gastn L\'Huillier 3935.com

    Dean Barker 3531.com

    David Poole 3632.com

    Jing Xiao 3228.com

    SARAFIDIS MICHAEL 4339.com

    Allan Engelhardt 3632.com

    Dryomov Dmitry 3430.com

    Prem Swaroop 3632.com

    Tariq Islam 3329.com

    Lester Mackey 4137.com

    Dong Wang 2925.com

    qdsunstar qdsunstar 3935.com

    Shixi Chen 3228.com

    Putri Novianti 3430.com

    Ryan Skar 2925.com

    Haoru Chen 3026.com

    Sarika T 3834.com

    Mohammad Ali Ghaderi 4339.com

    chun-yao Huang 3935.com

    Irina Dolgopolova 4440.com

    Yu Cheng 3026.com

    Xiaoyuan Hu 3430.com

    ekker ko 2723.com

    chunghung lin 3935.com

    Yogesh Badhe 3127.com

    Nikita Romashkin 4541.com

    Yu Jin 2824.com

    Patrick Haffner 4238.com

    Raghav Ghaiee 3329.com

    aviah laor 2925.com

    Yetian Chen 3430.com

    Shenghuo Zhu 3430.com

    D. Sculley 3228.com

    Elham Chitsaz 3935.com

    Tom Au 2016.com

    Larry Schumacher 4945.com

    Gekko Gordon 3935.com

    Ioannis Antonellis 4945.com

    upload test 3430.com

    Michael Fox 2723.com

    April Tse 3329.com

    George Ma 2319.com

    tingwei cha 3329.com

    Oleg Danilchenko 4541.com

    Shiwan Zhao 3026.com

    Shiwan Zhao 3026.com

    Bomg Sundy 3228.com

    Chine Lown 3329.com

    Quan Yuan 3531.com

    dinopa dinopa 3228.com

    Ronak Tamra 3632.com

    Chang Chun-Min 3329.com

    sai vadrevu 3531.com

    NARAYANA SUBBARAO, NARAYANA 5652.com

    Y.J. Yang 2824.com

    Libei Chen 3329.com

    Puja Potdar 3733.com

    Mingda Zhang 3127.com

    dummy sport 3531.com

    rr ff 3228.com

    Don Vito 3329.com

    Nik Ki 2925.com

    Scar Face 4036.com

    Son Ny 2925.com

    Greed Is Good 4137.com

    Aaron Lin 3127.com

    nice shot 3127.com

    Data Mining 3935.com

    tin apon 2218.com

    Sonja Lim 2925.com

    sunny killer 3430.com

    t109 t109 2723.com

    subhakant nayak 3733.com

    Deepa Karthik 3430.com

    Ranjani Subramaniam 4440.com

    hari kumar 3834.com

    Tang zhigao 3430.com

    Anch swamy 3632.com

    Shanky Dude 3531.com

    syluvai anthony 3531.com

    sumo sumo 2723.com

    Kyle Broflosky 3531.COM

    O G 2521.com

    lixiong zheng 3228.com

    Priya Balakrishnan 5450.com

    Krishnan Venkata 5046.com

    sup sup 2420.com

    Jahui Chang 3531.com

    f f 1713.com

    wei wei 2723.com

    Avi Sharma 3632.com

    naive naive 3026.com

    core2 core2 3430.com

    KDD Cup 3026.com

    David Vogel 3228.com

    Dima Chetverikov 4238.com

    K C 2622.com

    S B 2925.com

    Tanya Silva 3531.com

    Ray Poo 2824.com

    ccv ccv 2824.com

    ppt llm 2117.com

    lao shi 2723.com

    ni hao 2420.com

    Chi Chen 2622.com

    Hello World 3026.com

    Bill Cassill 4844.com

    Sassoon Kosian 3430.com

    lixiong Zheng 3632.com

    Dallas Zhu 3026.com

    CJ Kuo 2622.com

    Cong Guan 3531.com

    Tingting Wu 3733.com

    Sihong Xie 3026.com

    Junfeng Pan 3531.com

    Jami Xu 2521.com

    Ning ZHANG 3733.com

    Bajpai Rahul 4036.com

    Gaurav Kumar Mehta 4238.com

    nupur garg 3026.com

    Puneet Lakhina 4137.com

    nupur garg 3329.com

    Vassilis Syrris 3430.com

    Mon Sonu 2824.com

    amol deshmukh 4036.com

    Qu Gong 2925.com

    axilay . 2723.com

    Y-S L 3228.com

    Manish Jain 3026.com

    minhui zhou 3026.com

    Abhinav Mishra 3632.com

    oo pp 2117.com

    Dominik Perykasza 3935.com

    lixiong zheng 3733.com

    Xi Hu 2016.com

    Anshuk Aggarwal 3834.com

    Abhishek Dubey 3733.com

    Ramesh Hariharan 4036.com

    Tao Xia 2622.com

    Erik Erlandson 4036.com

    rohit gupta 3935.com

    weirdo weirdo 3127.com

    February February 3632.com

    Gaurav Sir 3228.com

    Robert Bell 3632.com

    Joe Yang 3228.com

    qui Bec 2420.com

    mar mar 2319.com

    april april 2723.com

    Raja A 3026.com

    Any Anon 2824.com

    lixiong zheng 3935.com

    C B 3430.com

    Grace PENG 2925.com

    John Paul 3329.com

    Hsiu-Khuern Tang 4238.com

    Monkey D 3228.com

    Bin Lu 2925.com

    ST Chen 2622.com

    jfpoq [q 2218.com

    shoude lin 2925.com

    Chia-yu Tsai 3430.com

    ROHIT GUPTA 2723.com

    Grace PENG 2925.com

    June June 2521.com

    hh dd 1915.com

    N F 2319.com

    Adrienn Szabo 4036.com

    July July 2521.com

    Grace PENG 3632.com

    tt mm 2218.com

    Arman Kavian 4137.com

    Shankar Yaddanapudi 4440.com

    Taiping He 3127.com

    abc 123 2622.com

    David Duling 3531.com

    Tatsunori Hashimoto 4238.com

    feng chen 3026.com

    Yogesh Pandit 3531.com

    Kiran Palla 3228.com

    upendra chitnis 3632.com

    sanjog ray 3228.com

    jean kiks 3329.com

    Nami Mish 3026.com

    Xiaoyuan Su 3430.com

    li yingan 3127.com

    Jose Padros-Escalante 4440.com

    fateme faraji daneshgar 4541.com

    Light Liang 3632.com

    Andrs Parra 3834.com

    bharath raj 3834.com

    Bahar Salehi 3632.com

    Donglin Niu 3531.com

    Rory Winston 4238.com

    darren chai 3329.com

    Yirui Yuan 3026.com

    Dibyanshu Dwivedi 4743.com

    zhuo zhang 3329.com

    meraa mm 3632.com

    nazm ulukaya 4137.com

    Zeina Nehme 3228.com

    Bulent Egrilmez 3632.com

    Riya J 2925.com

    Grozavu Nistor 4137.com

    Don Cozine 3127.com

    Trung-Nghia Vu 3531.com

    Bharatheesh Jaysimha 5147.com

    xilan tian 3228.com

    Dibyanshu Dwivedi 4440.com

    Christophe BURGAUD 5046.com

    Koon-Kiu Yan 3228.com

    Jaehoon Ha 3127.com

    Tai He 3430.com

    Sheng Guo 3329.com

    Piyasak Jeatrakul 4036.com

    Jay Mount 3026.com

    EnglishWarman EnglishWarman 5854.com

    Dibyanshu Dwivedi 4945.com

    Anshuman Verma 3834.com

    Shankar Yaddanapudi 4440.com

    narjes mirshekari 4844.com

    Michael Angstadt 4238.com

    cunruiwang cunruiwang 4036.com

    katya cherniak 3430.com

    Micheline Elias 4036.com

    Suchitra Chikhalkar 5046.com

    mohak narkhede 3430.com

    [email protected] [email protected] 4945.com

    Raymond Falk 3329.com

    Pavlo Golik 3127.com

    Russell Spangler 4642.com

    Charles Curry 2925.com

    olivier van overstraeten 5147.com

    Toni Cebrin 3329.com

    ... ... 2319.com

    Louis Duclos-Gosselin 5349.com

    Miguel Otero 3329.com

    Evan Kirshenbaum 4137.com

    Paul Deignan 4036.com

    Tzongze Pan 3127.com

    nnd dnn 2420.com

    saeid goodarzian 4238.com

    Jan Hosang 3329.com

    pirooz shamsinejad 3935.com

    Shlomi Lifshits 3935.com

    arm gong 3329.com

    ting liu 3127.com

    Yasser Jaffal 3430.com

    Mohammad Rouzbeh 3632.com

    wade Lin 3430.com

    Ali Tabandeh 3127.com

    Petri Fast 3329.com

    ehsan ahmadi 3834.com

    [email protected] [email protected] 7066.com

    Errol Caby 2622.com

    leila seyedhossein 4036.com

    sanjay kumar 3127.com

    Amy Nicklson 3430.com

    vivekanand samant 4137.com

    dec dec 2117.com

    nasrin asadi 3733.com

    Ian Hansel 3329.com

    Leonardo D'Ambrosi 4844.com

    Grzegorz Goldian 4137.com

    Minsang Kim 3228.com

    Bob Samra 3026.com

    Jozef Vilcek 3430.com

    Carlos Guerrero 3935.com

    Ching-Hua Yu 3632.com

    Cai Tomara 3430.com

    Alejandro Tobon 4339.com

    ELMEZIANE Rachid 4440.com

    Mumbai Indians 4339.com

    Nekhil Agrawal 3329.com

    Tom Lewicke 3127.com

    Anton Treialt 4036.com

    Nicholas Lim 3228.com

    yg luo 2218.com

    manivelu r 3430.com

    qin tian 3026.com

    shankey jain 4036.com

    Cristina Soviany 4541.com

    Jin Li 2622.com

    Renato Vimieiro 3733.com

    Jason Mackay 3632.com

    vaibhav p 2521.com

    Ilya Narsky 3531.com

    Erez Silberschein 3733.com

    Majid Kalkatechi 4036.com

    Marcin Nawrocki 4036.com

    Cijo Abraham Mani 4036.com

    ARUN AGGARWAL 3430.com

    Zhenglu Yang 3430.com

    Suzan Amareen 3531.com

    Nan Zhang 3430.com

    bahar salehi 3430.com

    ho shou-wei 3228.com

    Poska Spi 3127.com

    Jason Jung 2925.com

    ajay chandradurai 4642.com

    ee e 2016.com

    Matthias Meul 3935.com

    Hk Ji 2218.com

    Yifei Yuan 3127.com

    sri vatsava 3228.com

    khattab ali 3430.com

    Tony Diaz 3026.com

    Bo Han 2622.com

    Marcin Choinski 3834.com

    Damian oziski 3430.com

    Bo Han 3026.com

    matan matan 3531.com

    Minh Doan 3127.com

    Alex Zhang 3430.com

    Andrey Bernstein 4036.com

    Krishnaiah J 3733.com

    zwj zwj 2723.com

    Jacob Smithe 3329.com

    Indir Jaganjac 3632.com

    Xinjian Guo 3834.com

    Lev Itskovich 3127.com

    Tian Zhu 3127.com

    Semanur Konezoglu 3834.com

    Srdjan Sladojevic 4945.com

    Taotao Wang 3531.com

    wade Lin 3228.com

    Pei-Ling Hsu 3228.com

    Alex Zhang 3026.com

    Peter Stengard 4238.com

    Nuno Antonio 4036.com

    ota masaki 3228.com

    yusuke yokota 4238.com

    willer qu 3430.com

    Lian Liu 2420.com

    Musa Nur 2824.com

    li wan 2420.com

    Alvin Chua 3834.com

    Edinaldo Garcia (Ed Garcia) 4945.com

    TAnaMiksa TAnaMiksa 4339.com

    sahar soussa 3632.com

    Anton Sarychev 3329.com

    saravanaraj duraisamy 6561.com

    Liberat Houenou 3531.com

    Marzena Dasko 3935.com

    Victor azov-info.com 4642.com

    x yd 2723.com

    Mpho Buthelezi 3632.com

    Marko Jevremovic 3531.com

    Vineet Jain 3430.com

    anteAngel anteAngel 4137.com

    Ya Hu 2824.com

    Maciej Kawecki 4137.com

    Dimitrios Athanasakis 4642.com

    bob obob 2824.com

    Jeff Zanooda 3430.com

    daniel wang 3026.com

    rajendran veerasamy 4642.com

    Masahito Yamaguchi 5147.com

    June Kim 2925.com

    devdatta akhawe 4036.com

    Washington Bastidas 4339.com

    Cezary Glowinski 3733.com

    Christian Langreiter 4339.com

    Liang Xie 3430.com

    Sun Yongxin 3430.com

    Alok Dobriyal 4036.com

    meng shi 2925.com

    BigBoyNY BigBoyNY 4036.com

    seyed hossein kalantar 4541.com

    Xu-Ying Liu 3329.com

    Uwe Mayer 3329.com

    Vahid Taslimi Tehrani 4238.com

    Takahiko HORIUCHI 3632.com

    Esin Yilmaz 3329.com

    Maidul Islam 3430.com

    Jim Herberts 3228.com

    charlie berger 4238.com

    S Sarkar 3329.com

    dakshya pati 3935.com

    Ahmad Julashokri 4541.com

    aaaaa aaaaa 4036.com

    svatera svatera 4339.com

    Alastair Muir 3632.com

    meitner cadena 4137.com

    Rita Agarwal 3733.com

    Sithar Dorji 3733.com

    Achofanto Drukpa 3834.com

    Wenxuan Gao 3531.com

    Eftim Zdravevski 3531.com

    Jose Luis Jimnez Andrade 4541.com

    qiyang duan 3632.com

    gg ff 2318.com1

    r r 2116.com2

    meraa moon 3934.coom

    Cristian Grozea 5350.de

    Ernesto Diaz-Aviles 3330.de

    Zeno Gantner 3128.de

    Wiebke Sieben 4441.de

    Felix Jungermann 5350.de

    Christian Buck 3229.de

    Hannes Nickisch 3128.de

    Arne Mauser 3835.de

    Frank Lemke 2926.de

    Lars Schmidt-Thieme 4542.de

    Krisztian Buza 3027.de

    Domonkos Tikk 4441.de

    Mike Nike 2825.de

    Joerg Wichard 3734.de

    Holger Arndt 3734.de

    Jrme Kunegis 3532.de

    Stefan Hoevel 4037.de

    Yasser Jaffal 3835.de

    Karen Schettlinger 5855.de

    Artus Krohn-Grimberghe 3936.de

    Patrick Doetsch 4845.de

    Leandro Balby Marinho 4037.de

    Christine Preisach 5350.de

    Christoph Freudenthaler 4845.de

    Mr X 3128.de

    Arne Mauser 4946.de

    Michael Kramp 4441.de

    Johannes Laudenberg 5653.de

    Christian Oberdrfer 6057.de

    Niklas Hoppe 4239.de

    Wiebke Sieben 4744.de

    Jens Forster 4037.de

    Stefan Kaiser 2926.de

    Michael Kramp 3633.de

    Pascal Steingrube 5249.de

    Sebastian Schueller 6764.de

    Timo Rseler 4037.de

    david delgado 3733.edu

    Myles Turner 3329.edu

    Christos Boutsidis 3834.edu

    Wei-Yin Loh 3127.edu

    Andy Perkins 3228.edu

    Aynur Dayanik 3632.edu

    Yukun Chen 3834.edu

    Bill Bame 2521.edu

    Jianfei Wu 3228.edu

    Ramanathan Narayanan 5147.edu

    xue-wen Chen 2824.edu

    Aayush Sharma 3834.edu

    Chengwei Yuan 3127.edu

    Huzefa Rangwala 3733.edu

    guangzhi qu 2925.edu

    Hui Wu 2420.edu

    Venu Satuluri 4339.edu

    George Karypis 3531.edu

    Perfecto Herrera 4339.edu

    Susan Brudvig 3228.edu

    Alexander Statnikov 5652.edu

    Saurabh Kataria 3834.edu

    Matthew Whitehead 4036.edu

    Mark Embrechts 3127.edu

    Michael Krein 3127.edu

    Patrick Kelly 3733.edu

    Jong Cheol Jeong 3329.edu

    Anup Sawant 3329.edu

    Enric Guaus 3329.edu

    Adrian Barb 3228.edu

    Joan Garriga 3531.edu

    Wenjun Zhou 4036.edu

    Laura Brown 4238.edu

    Saurabh Kataria 3329.edu

    Albert Shieh 3632.edu

    Daria Sorokina 3329.edu

    Xifeng Yan 2925.edu

    Terry Letsche 4238.edu

    xuning tang 2925.edu

    Ariel Kleiner 4238.edu

    Hossam Sharara 3430.edu

    Xu He 2622.edu

    Ioannis Antonellis 4238.edu

    Dimitrios Tsamis 3935.edu

    Panagiotis Papadimitriou 5349.edu

    Dimitrios Tsamis 3834.edu

    Seungil Huh 3329.edu

    P P 2723.edu

    DIAL DIAL 2622.edu

    KyoungTak Cho 3632.edu

    Prem Swaroop 3935.edu

    Chung Ching Wang 3733.edu

    Yang Liu 2824.edu

    Karla Caballero 3632.edu

    Pedro Alves 3430.edu

    boya xie 2521.edu

    Tingda Lu 3026.edu

    Derek Anderson 5248.edu

    Artun Artun 3935.edu

    Henry Tzeng 2723.edu

    Debasish Das 3632.edu

    X YZ 2420.edu

    as as 3228.edu

    Berkin Ozisikyilmaz 4945.edu

    Chetan Kumar 4036.edu

    Daria Sorokina 3733.edu

    ping Li 2824.edu

    Kavitha Kumar R 3632.edu

    Robert Grossman 3430.edu

    Cooper Francis 3430.edu

    meng shi 3228.edu

    Juan Huerta 3531.edu

    Sergio Jimenez 3532.es

    Angel Navia-Vazquez 3936.es

    Heath Hohwald 2825.es

    Roberto Paredes 3835.es

    Carlos Molina 3330.es

    Pablo Bermejo 3633.es

    Xavier Amatriain 2926.es

    Manuel Cebrian 3128.es

    Esteban Garca 4037.es

    juan j. rodriguez 3835.es

    JESSICA VIQUEIRA 4340.es

    Fernando Tusell 4037.es

    Thanh-Nghi Do 4138.eu

    ger bill 3532.fi

    Tuomas Lepola 3633.fi

    Teemu Mutanen 3633.fi

    vignesh krishna 3734.fj

    Martine Cadot 3835.fr

    ali qamar 3734.fr

    Xiangliang Zhang 3330.fr

    Ludovic DENOYER 4138.fr

    Trinh-Minh-Tri DO 4542.fr

    Sergio Rodrigues de Morais 6865.fr

    Sbastien Derivaux 4845.fr

    Germain Forestier 4845.fr

    Hartland Cdric 4037.fr

    Berti Laure 2825.fr

    Nicolas Lachiche 4643.fr

    Nicolas Saunier 3835.fr

    Francois-Xavier Jollois 6764.fr

    Franois Rosset 3431.fr

    rushed kanawati 5451.fr

    john martin 2825.fr

    Isabelle Rivals 4239.fr

    Haytham Elghazel 4845.fr

    Stphane CANU 3532.fr

    larbi mounir 3633.fr

    Grgory Thibault 4340.fr

    Paul MATHIEU 3229.fr

    Gilles Gasso 4138.fr

    Romaric Gaudel 3835.fr

    Alex Aussem 3532.fr

    Lelu Alain 2926.fr

    orfane orfane 3431.fr

    Erick Alphonse 4643.fr

    med bou 2724.fr

    Nicolas Quilles 3633.fr

    Haoyuan Li 2421.fr

    Anestis Gkanogiannis 3734.gr

    V S 2118.gr

    Weifeng Su 2825.hk

    Haiqin Yang 3633.hk

    Raymond Chi-Wing Wong 4239.hk

    Yang Ruan 3128.hk

    Yueqi Li 2926.hk

    XIN XIN 3027.hk

    Jeff HUANG 3128.hk

    Janice Leung 3835.hk

    Derek Hao Hu 3330.hk

    Jackie Lo 3128.hk

    shiming zhang 3330.hk

    zhili wu 3532.hk

    qian wan 2623.hk

    lisa luo 2522.hk

    Grace PENG 2724.hk

    Wei Jiang 2724.hk

    Grace PENG 3128.hk

    Lei Shi 3027.hk

    Wa Lun Hong 3734.hk

    Ho Wai Pang 2724.hk

    Miklos Kurucz 4340.hu

    Zsolt Kardkovcs 4340.hu

    Gabor Szucs 3128.hu

    Bob Hope 2825.hu

    Gbor Veress 3936.hu

    Gabor Takacs 2926.hu

    Istvan Nagy 3734.hu

    Sndor Szaszk 3633.hu

    Robert Busa-Fekete 4542.hu

    Zoltn Prekopcsk 4441.hu

    Mikls Molnr 3835.hu

    Csaba Gspr-Papanek 4340.hu

    Robert Iwatt 3633.hu

    Mikls Erdlyi 4037.hu

    Santiago Villalba 4441.ie

    Guillem LEFAIT 3835.ie

    gideon dror 3027.il

    Alon Kaufman 3532.il

    Gandalf The Grey 4037.il

    Sudheer Tumu 3835.in

    Upasna Singh 3835.in

    Ambuj Mahanti 3128.in

    Luvly geet 3835.in

    Dibyanshu Dwivedi 5451.in

    Abhinav Mishra 4845.in

    Shuchi Pandya 3936.in

    samir kelkar 3229.in

    pp34 pp34 4138.in

    Dirisala J Nagendra Kumar 5552.in

    sam pai 2926.in

    Hamid Farvaresh 4138.ir

    Alessandro Fiori 4542.it

    Claudio Carpineto 3532.it

    Paolo Garza 3532.it

    Alessandro Ghio 4239.it

    Dipankar Bachar 3633.it

    Haoyuan Li 2724.it

    ahmad awad 4037.jo

    Hassan Najadat 3633.jo

    Hirotaka Moriguchi 3633.jp

    kazuaki komoto 3330.jp

    Homete Yaruzo 3633.jp

    Naoko Matsushita 3936.jp

    Hikari Aikawa 3229.jp

    Noriaki KAWAMAE 5451.jp

    yuji tagaya 4138.jp

    Junichi Kobayashi 3936.jp

    lu yongyan 2724.jp

    TOMO FUJIE 2825.jp

    Naoyuki Harada 4441.jp

    suganuma hajime 3633.jp

    Shinya Yazawa 3229.jp

    FEG B-TEAM 4037.jp

    h oge 3330.jp

    feg feg 2623.jp

    Sampei Nakabayashi 3532.jp

    Kota Hiranuma 3734.jp

    FEGteamC mycharagne 3734.jp

    Kotan numa 3431.jp

    Yasuhiro Matsushita 4643.jp

    Gouchol Pok 3128.kr

    Kyunglag KWON 3633.kr

    Hyejin Park 3330.kr

    SeungHo Yang 3734.kr

    Namhyoung Kim 3633.kr

    sungwoo cho 3330.kr

    Cheng Hao Jin 4643.kr

    Mikael Fernandus Simalango 4643.kr

    Serge Parshutin 4340.lv

    Hugo Jair Escalante 4441.mx

    Rasimah Aripin 4340.my

    Olivier Guyon 3531.net

    Gang Wang 3733.net

    Frank Vanden Berghen 5248.net

    mars wang 3531.net

    Kamil Kujawinski 3935.net

    you gillsang 3834.net

    Son Dameun 3531.net

    Kim Mun Uk 3531.net

    David Slate 3430.net

    Steffen Springer 3531.net

    atll llace 2723.net

    Yuming Chang 4036.net

    me me 2521.net

    Guang qin 2824.net

    Phil Sherrod 3834.net

    Volodymyr Lytvynenko 3834.net

    Jaehoon Koo 3430.net

    Jilles Vreeken 4037.nl

    Wojtek Kowalczyk 3431.nl

    Peter Rijnbeek 4037.nl

    Hans de Wit 3532.nl

    Wojtek Kowalczyk 3532.nl

    Eric Melse 3229.nl

    Andrew Cathie 3835.nz

    Bernhard Pfahringer 4744.nz

    xni xni 2118.oe

    clay woolam 2925.org

    Ashkan Sami 2824.org

    Mehboob Alam 2824.org

    rofu rofu 2824.org

    Julius Campbell 4238.org

    Amir Saffari 2824.org

    Fernando Berzal 3228.org

    Liang Chen 2723.org

    Chris Brew 2622.org

    Donald Wedding 3329.org

    Shyh-hao Kuo 2824.org

    Abhinav Mishra 3935.org

    Abhinav Mishra 3834.org

    Edinaldo Garcia (Ed Garcia) 4642.org

    Mehreen Saeed 3936.pk

    Radomir Mastalerz 3633.pl

    Sebastian Stawicki 4239.pl

    Michal Lyczek 3633.pl

    - - 1512.pl

    Marcin Kuzawiski 5047.pl

    Wojtek Swieboda 3633.pl

    Marcin Woch 3431.pl

    Andrzej Gdula 3128.pl

    Marek Sikora 3633.pl

    K C 3229.pl

    Marcin Blachnik 4239.pl

    Michal Sapinski 2926.pl

    Piotr Rzepakowski 3936.pl

    Krzysztof Ciebiera 4037.pl

    Calculatius Calc 4340.pl

    Marcin Kowalski 2926.pl

    Wiktor Adamski 4441.pl

    Michal Draminski 3229.pl

    anna stachowiak 3431.pl

    Damian oziski 4643.pl

    marcin ow 3229.pl

    Ftima Rodrigues 3431.pt

    Teresa Goncalves 3229.pt

    Miguel Pironet 4239.pt

    Vtor Santos Costa 3734.pt

    Luis Alexandre 3229.pt

    Cristian Grozea 3431.ro

    Damjan Krstajic 4441.rs

    Milan Milosavljevic 4138.rs

    Alex Larko 2926.ru

    Mikhail Klimushkin 3936.ru

    Mariya Povorova 3734.ru

    Vadim Khrustalev 3936.ru

    zelma zelma 3128.ru

    Plot Plot 3128.ru

    tester tester 3229.ru

    Group PMI 3330.ru

    VeroNika Alb 3128.ru

    Volodymyr Lytvynenko 4239.ru

    Alexey Volodskih 3431.ru

    Sergey Kolotienko 3734.ru

    Zhenya Markelov 3835.ru

    Andrew Kramarenko 3835.ru

    Polina Bondareva 3835.ru

    Alexey Didyk 2926.ru

    Vladimir Dremov 3936.ru

    Lev Itskovich 3431.ru

    Ca5h Raver 3229.ru

    epheryimpessy epheryimpessy 4845.ru

    SufleOtRihanni SufleOtRihanni 5451.ru

    pacanchchik pacanchchik 4643.ru

    WiiGamesNew WiiGamesNew 4542.ru

    SexOfPhone SexOfPhone 4138.ru

    Warcraft Gold Warcraft Gold 5148.ru

    Sampath Deegalla 3532.se

    Kar Leong Tew 3936.sg

    Zeyar Aung 3734.sg

    Dhaval Patel 3734.sg

    Tuan Zea Tan 3936.sg

    Chuan Sheng Foo 4138.sg

    Kenny Chua 3734.sg

    Guimei Liu 3431.sg

    Yang Jian 3330.sg

    Zhenzhen Fan 3431.sg

    Yew Haur Lee 3431.sg

    Jan Rupnik 3027.si

    Mitja Trampus 3633.si

    Lan Zagar 3532.si

    Ruben Sipos 3229.si

    Viktor Beles 4239.sk

    Deniz Yuret 3027.tr

    aysegul ozturk 3532.tr

    shang Lin 3633.tw

    shou-de lin 3532.tw

    tai-wei Cheng 3835.tw

    Hung-Che Lai 3835.tw

    Howard Chang 3734.tw

    Tsung-Ting Kuo 4239.tw

    ll Lin 3128.tw

    CC Tung 3633.tw

    Sheng-Jie Weng 3835.tw

    Pei-Hsun Hsu 3532.tw

    Chun-Sung Ferng 4037.tw

    gs wang 3027.tw

    chih yu Lin 4037.tw

    Sheng-Da Huang 4340.tw

    Chia-Chun Shih 3633.tw

    Yuming Chang 3936.tw

    Kissinger SHU 4239.tw

    chen ting 3835.tw

    chung Kevin 3431.tw

    shihcheng chang 4441.tw

    Liao Ying Chiao 3936.tw

    rf h 2926.tw

    Katrina Chuang 3734.tw

    Saiko Ashiyasayo 4138.tw

    Chia-Hui Chang 3734.tw

    sz Chen 3229.tw

    Kevin Hsu 3633.tw

    Song Xiang-Yu 4239.tw

    Yi-Shan Lin 3734.tw

    Chih-Ta Lin 3229.tw

    Shou-Wei Ho 4037.tw

    Miles Chiang 3734.tw

    Cheng-Ju Kuo 4138.tw

    YiHung Huang 3835.tw

    Yu-Shi Lin 3936.tw

    Bill Ko 3229.tw

    Hungyi Lo 3633.tw

    Y.S Lin 3027.tw

    Hung-Yi Lo 3330.tw

    A Lin 3431.tw

    Li-Jen Chien 3128.tw

    Hungyi Henry Lo 3734.tw

    dog oauk 3431.tw

    ekker ko 3128.tw

    Eric Chen 3734.tw

    ee rrrr 2623.tw

    Shang-Tse Chen 3734.tw

    Jan Nien-yi 3027.tw

    Chao Chao Yen 4138.tw

    Yu HsingChuan 3633.tw

    Chih-ta Lin 4037.tw

    Li-Jen Chien 3330.tw

    Ernest Mwebaze 3936.ug

    Tom Diethe 3431.uk

    Gavin Cawley 3229.uk

    Bassam Farran 3734.uk

    Jurgen Van Gael 3330.uk

    Markovsky Ivan 3532.uk

    Andrzej Ruta 4037.uk

    Paul Coleman 3734.uk

    l han 2320.uk

    Ben Galvin 3734.uk

    Pam Green 3330.uk

    Daniel Berrar 3835.uk

    Leto Peel 3936.uk

    Geoffrey Guile 3431.uk

    Ashish Ganda 4037.uk

    Enrico Glaab 3229.uk

    Padmasini rengabashyam 4845.uk

    Fedor Zhdanov 3532.uk

    Ching-Wei Wang 3734.uk

    Ian Hurst 3330.uk

    Esther Hochsztain 3835.uy

    Durban Frazer 3229.vi

    lol lol 2421.xx

    test test 2522.yy

    Adriaan Pottas 4138.za

    Dr. Sundararajan Vijayaraghava Seshadri 6057.za

    Sheet2

    Domain ExtensionCountry

    a(unknown)

    bitnet(unknown)

    acUnited Kingdom academic institutions

    adAndorra

    aeUnited Arab Emirates

    afAfghanistan

    agAntigua and Barbuda

    aiAnguilla

    alAlbania

    amArmenia

    anNetherlands Antilles

    aoAngola

    aqAntarctica

    arArgentina

    asAmerican Samoa

    atAustria

    auAustralia

    awAruba

    azAzerbaijan

    baBosnia and Herzegovina

    bbBarbados

    bdBangladesh

    beBelgium

    bfBurkina Faso

    bgBulgaria

    bhBahrain

    biBurundi

    bjBenin

    bmBermuda

    bnBrunei Darussalam

    boBolivia

    brBrazil

    bsBahamas

    btBhutan

    bvBouvet Island

    bwBotswana

    byBelarus

    bzBelize

    caCanada

    ccCocos (Keeling) Islands

    cfCentral African Republic

    cgCongo

    chSwitzerland

    ciCote d'Ivoire (Ivory Coast)

    ckCook Islands

    clChile

    cmCameroon

    cnChina

    coColombia

    comUS Commercial

    crCosta Rica

    csCzechoslovakia (former)

    cuCuba

    cvCape Verde

    cxChristmas Island

    cyCyprus

    czCzech Republic

    deGermany

    djDjibouti

    dkDenmark

    dmDominica

    doDominican Republic

    dzAlgeria

    ecEcuador

    eduUS Educational

    eeEstonia

    egEgypt

    ehWestern Sahara

    erEritrea

    esSpain

    etEthiopia

    fiFinland

    fjFiji

    fkFalkland Islands (Malvinas)

    fmMicronesia

    foFaroe Islands

    frFrance

    fxFrance (Metropolitan)

    gaGabon

    gbGreat Britain (UK)

    gdGrenada

    geGeorgia

    gfFrench Guiana

    ghGhana

    giGibraltar

    glGreenland

    gmGambia

    gnGuinea

    govUS Government

    gpGuadaloupe

    gqEquatorial Guinea

    grGreece

    gsSouth Georgia and South Sandwich Islands

    gtGuatemala

    guGuam

    gwGuinea-Bissau

    gyGuyana

    hkHong Kong

    hmHeard and McDonald Islands

    hnHonduras

    hrCroatia (Hrvatska)

    htHaiti

    huHungary

    idIndonesia

    ieIreland

    ilIsrael

    inIndia

    ioBritish Indian Ocean Territory

    iqIraq

    irIran

    isIceland

    itItaly

    jmJamaica

    joJordan

    jpJapan

    keKenya

    kgKyrgyzstan

    khCambodia

    kiKiribati

    kmComoros

    knSaint Kitts and Nevis

    kpKorea (North)

    krKorea (South)

    kuKuwait

    kyCayman Islands

    kzKazakhstan

    laLaos

    lbLebanon

    lcSaint Lucia

    liLiechtenstein

    lkSri Lanka

    lrLiberia

    lsLesotho

    ltLithuania

    luLuxembourg

    lvLatvia

    lyLibya

    maMorocco

    mcMonaco

    mdMoldova

    mgMadagascar

    mhMarshall Islands

    milUS Military

    mkMacedonia

    mlMali

    mmMynamar

    mnMongolia

    moMacau

    mpNorthern Mariana Islands

    mqMartinique

    mrMauritania

    msMontserrat

    mtMalta

    muMauritius

    mvMaldives

    mwMalawi

    mxMexico

    myMalaysia

    mzMozambique

    naNamibia

    ncNew Caledonia

    neNiger

    netUS network

    nfNorfolk Island

    ngNigeria

    niNicaragua

    nlNetherlands

    noNorway

    npNepal

    nrNauru

    ntNeutral Zone

    nuNiue

    nzNew Zealand (Aotearoa)

    omOman

    orgUS Non-Profit Organization

    paPanama

    pePeru

    pfFrench Polynesia

    pgPapua New Guinea

    phPhilippines

    pkPakistan

    pl>Poland

    pmSaint Pierre and Miquelon

    pnPitcairn

    prPuerto Rico

    ptPortugal

    pwPalau

    pyParaguay

    qaQatar

    reReunion

    roRomania

    ruRussian Federation

    rwRwanda

    saSaudi Arabia

    sbSolomon Islands

    scSeychelles

    sdSudan

    seSweden

    sgSingapore

    shSaint Helena

    siSlovenia

    sjSvalbard and Jan Mayen Islands

    skSlovak Republic

    slSierra Leone

    smSan Marino

    snSenegal

    soSomalia

    srSuriname

    stSao Tome and Principe

    suUSSR (former)

    svEl Salvador

    sySyria

    szSwaziland

    tcTurks and Caicos Islands

    tdChad

    tfFrench Southern Territories

    tgTogo

    thThailand

    tjTajikistan

    tkTokelau

    tmTurkmenistan

    tnTunisia

    toTonga

    tpEast Timor

    trTurkey

    ttTrinidad and Tobago

    tvTuvalu

    twTaiwan

    tzTanzania

    uaUkraine

    ugUganda

    ukUnited Kingdom

    umUS Minor Outlying Islands

    usUnited States

    uyUruguay

    uzUzbekistan

    vaVatican City State

    vcSaint Vincent and the Grenadines

    veVenezuela

    vgVirgin Islands (British)

    viVirgin Islands (US)

    vnViet Nam

    vuVanuatu

    wfWallis and Futuna Islands

    wsSamoa

    yeYemen

    ytMayotte

    yuYugoslavia

    zaSouth Africa

    zmZambia

    zrZaire

    zwZimbabwe

    Bulgaria

    Bhutan

    Belize

    Denmark

    Falkland Islands (Malvinas)

    Georgia

    Cambodia

    Cayman Islands

    French Polynesia

    Slovenia

    Uganda

    Uzbekistan

    Zambia

    Sheet3

    ArgentinaGermanyMalaysiaSouth KoreaYear# TeamsregRecordsPredictors

    AustraliaGreeceMexicoSpain199745

    AustriaHong KongNetherlandsSweden199857

    BelgiumHungaryNew ZealandSwitzerland199924

    BrazilIndiaPakistanTaiwan200031170

    BulgariaIranPortugalTurkey20011361909139351

    CanadaIrelandRomaniaUganda200218

    ChileIsraelRussian FederationUnited Kingdom200357

    ChinaItalySingaporeUruguay2004102

    FijiJapanSlovak RepublicUnited States200537

    FinlandJordanSlovenia200668

    FranceLatviaSouth Africa200795netflix

    2008128

    2009453

    Sheet3

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    # Teams

    Year

    KDD Cup Participation By Year

  • Participation Statistics

    1299 registered teams7865 entries46 countries :

    ArgentinaGermanyMalaysiaSouth KoreaAustraliaGreeceMexicoSpainAustriaHong KongNetherlandsSwedenBelgiumHungaryNew ZealandSwitzerlandBrazilIndiaPakistanTaiwanBulgariaIranPortugalTurkeyCanadaIrelandRomaniaUgandaChileIsraelRussian FederationUnited KingdomChinaItalySingaporeUruguayFijiJapanSlovak RepublicUnited StatesFinlandJordanSloveniaFranceLatviaSouth Africa

  • A worlwide operator

    One of the main telecommunication operators in the world

    Providing services to more than 170 millions customers over five continents

    Including 120 millions under the Orange Brand

  • KDD Cup 2009 organized by OrangeCustomer Relationship Management (CRM)

    Three marketing tasks: predict the propensity of customers

    to switch provider: Churn

    to buy new products or services: Appentency

    to buy upgrades or new options proposed to them: Up-selling

    Objective: improve the return of investments (ROI) of marketing campaigns

    Increase the efficiency of the campaign given a campaign cost

    Decrease the campaign cost for a given marketing objective

    Better prediction leads to better ROI

  • Data, constraints and requirements

    Input data

    Relational databases

    Numerical or categorical

    Noisy

    Missing values

    Heavily unbalanced distribution

    Train data

    Hundreds of thousands of instances

    Tens of thousand of variables

    Deployment

    Tens of millions of instances

    Train and deploy requirements

    About one hundred models per month

    Fast data preparation and modeling

    Fast deployment

    Model requirements

    Robust

    Accurate

    Understandable

    Business requirement

    Return of investment for the whole process

  • In-house systemFrom raw data to scoring models

    Data warehouse

    Relational data base

    Data mart

    Star schema

    Feature construction

    PAC technology

    Generates tens of thousands of variables

    Data preparation and modeling

    Khiops technology

    Data feeding

    PAC

    Khiops

  • Design of the challenge

    Orange business objective

    Benchmark the in-house system against state of the art techniques

    Data

    Data store

    Not an option

    Data warehouse

    Confidentiality and scalability issues

    Relational data requires domain knowledge and specialized skills

    Tabular format

    Standard format for the data mining community

    Domain knowledge incorporated using feature construction (PAC)

    Easy anonymization

    Tasks

    Three representative marketing tasks

    Requirements

    Fast data preparation and modeling (fully automatic)

    Accurate

    Fast deployment

    Robust

    Understandable

  • Data sets extraction and preparation

    Input data

    10 relational table

    A few hundreds of fields

    One million customers

    Instance selection

    Resampling given the three marketing tasks

    Keep 100 000 instances, with less unbalanced target distributions

    Variable construction

    Using PAC technology

    20000 constructed variables to get a tabular representation

    Keep 15 000 variables (discard constant variables)

    Small track: subset of 230 variables related to classical domain knowledge

    Anonymization

    Discard variable names, discard identifiers

    Randomize order of variables

    Rescale each numerical variable by a random factor

    Recode each categorical variable using random category names

    Data samples

    50 000 train and test instances sampled randomly

    5000 validation instances sampled randomly from the test set

  • Scientific and technical challenge

    Scientific objective

    Fast data preparation and modeling: within five days

    Large scale: 50 000 train and test data, 15 000 variables

    Hetegeneous data

    Numerical with missing values

    Categorical with hundreds of values

    Heavily unbalanced distribution

    KDD social meeting objective

    Attract as many participants as possible

    Additional small track and slow track

    Online feedback on validation dataset

    Toy problem (only one informative input variable)

    Leverage challenge protocol overhead

    One month to explore descriptive data and test submission protocol

    Attractive conditions

    No intellectual property conditions

    Money prizes

  • Business impact of the challenge

    Bring Orange datasets to the data mining community

    Benefit for community

    Access to challenging data

    Benefit for Orange

    Benchmark of numerous competing techniques

    Drive the research efforts towards Orange needs

    Evaluate the Orange in-house system

    High number of participants and high quality of the results

    Orange in-house results:

    Improved by a significant margin when leveraging all business requirements

    Almost Parretto optimal when other criterions are considered

    (automation, very fast train and deploy, robustness and understandability)

    Need to study the best challenge methods to get more insights

  • KDD Cup 2009: Result Analysis

    Best Result (period considered in the figure)

    In House System (downloadable : www.khiops.com)

    Baseline (Nave Bayes)

  • Overall Test AUC Fast

    Good Result Very Quickly

    Best Results (on each dataset) Submissions

  • Overall Test AUC Fast

    Good Result Very Quickly

    Best Results (on each dataset) Submissions

    In House (Orange) System:

    No parametersOn 1 standard laptop (mono proc)If deal as 3 different problems

  • Overall Test AUC Fast

    Very Fast Good Result

    Small improvement after the first day

    (83.85 84.93)

  • Overall Test AUC Slow

    Very Small improvement after the 5th day

    (84.93 85.2)

    Improvement due to unscrambling?

  • Overall Test AUC Submissions

    23.24% of the submissions (>0.5)< Baseline

    15.25% of the submissions (>0.5)> In House

    84.75% of the submissions (>0.5)< In House

  • Overall Test AUC 'Correlation' Test / Valid

  • Overall Test AUC'Correlation' Test / Train

    Random Values Submitted

    Boosting Method orTrain Target Submitted

    Over fitting

    ?

  • Overall Test AUC

    Test AUC - 12 hours

    Test AUC - 24 hours

    Test AUC 36 days

    Test AUC 5 days

  • Overall Test AUC

    Test AUC - 12 hours

    Test AUC 36 days

    time to adjust model parameters ? time to train ensemble method ? time to find more processors ? time to test more methods time to unscramble ?

    Difference between :

    best result at the end of the first day and best result at the end of the 36 days

    =1.35%

  • Test AUC = f (time)

    Easier ?

    Harder ?

    Churn Test AUC day [0:36]

    Appetency Test AUC day [0:36]

    Up-selling Test AUC day [0:36]

  • Test AUC = f (time)

    Easier ?

    Harder ?

    Difference between :

    best result at the end of the first day and best result at the end of the 36 days

    =1.84%

    =1.38%

    =0.11%

    Churn Test AUC day [0:36]

    Appetency Test AUC day [0:36]

    Up-selling Test AUC day [0:36]

  • CorrelationTest AUC / Valid AUC (5 days)

    Easier ?

    Harder ?

    Churn Test/Valid day [0:5]

    Appetency Test/Valid day [0:5]

    Up-selling Test/Valid day [0:5]

  • CorrelationTrain AUC / Valid AUC (36 days)

    Difficulty to conclude something

    Churn Test/Train day [0:36]

    Appetency Test/Train day [0:36]

    Up-selling Test/Train day [0:36]

  • HistogramTest AUC / Valid AUC ([0:5] or ]5-36] days)

    Knowledge (parameters?) found during 5 days helps after ?

    Churn Test AUC day [0:36]

    Appetency Test AUC day [0:36]

    Up-selling Test AUC day [0:36]

  • HistogramTest AUC / Valid AUC ([0:5] or ]5-36] days)

    Knowledge (parameters?) found during 5 days helps after ?

    YES !

    Churn Test AUC day [0:36]

    Appetency Test AUC day [0:36]

    Up-selling Test AUC day [0:36]

    Churn Test AUC day ]5:36]

    Appetency Test AUC day ]5:36]

    Up-selling Test AUC day ]5:36]

  • Fact Sheets:Preprocessing & Feature Selection

    0

    20

    40

    60

    80

    Principal Component Analysis

    Other prepro

    Grouping modalities

    Normalizations

    Discretization

    Replacement of the missing values

    PREPROCESSING (overall usage=95%)

    Percent of participants

    0

    10

    20

    30

    40

    50

    60

    Wrapper with search

    Embedded method

    Other FS

    Filter method

    Feature ranking

    FEATURE SELECTION (overall usage=85%)

    Percent of participants

    Forward / backward wrapper

  • Fact Sheets:Classifier

    0

    10

    20

    30

    40

    50

    60

    Bayesian Neural Network

    Bayesian Network

    Nearest neighbors

    Nave Bayes

    Neural Network

    Other Classif

    Non-linear kernel

    Linear classifier

    Decision tree...

    CLASSIFIER (overall usage=93%)

    Percent of participants

    About 30% logistic loss, >15% exp loss, >15% sq loss, ~10% hinge loss. Less than 50% regularization (20% 2-norm, 10% 1-norm). Only 13% unlabeled data.

  • Fact Sheets: Model Selection

    0

    10

    20

    30

    40

    50

    60

    Bayesian

    Bi-level

    Penalty-based

    Virtual leave-one-out

    Other cross-valid

    Other-MS

    Bootstrap est

    Out-of-bag est

    K-fold or leave-one-out

    10% test

    MODEL SELECTION (overall usage=90%)

    Percent of participants

    About 75% ensemble methods (1/3 boosting, 1/3 bagging, 1/3 other). About 10% used unscrambling.

  • Fact Sheets: Implementation

    Run in parallel

    Multi-processor

    None

    Memory

    Operating System

    Parallelism

    Software Platform

  • Winning methods

    Fast track:

    IBM research, USA +: Ensemble of a wide variety of classifiers. Effort put into coding (most frequent values coded with binary features, missing values replaced by mean, extra features constructed, etc.) ID Analytics, Inc., USA +: Filter+wrapper FS. TreeNet by Salford Systems an additive boosting decision tree technology, bagging also used. David Slate & Peter Frey, USA: Grouping of modalities/discretization, filter FS, ensemble of decision trees.

    Slow track:

    University of Melbourne: CV-based FS targeting AUC. Boosting with classification trees and shrinkage, using Bernoulli loss. Financial Engineering Group, Inc., Japan: Grouping of modalities, filter FS using AIC, gradient tree-classifier boosting. National Taiwan University +: Average 3 classifiers: (1) Solve joint multiclass problem with l1-regularized maximum entropy model. (2) AdaBoost with tree-based weak leaner. (3) Selective Nave Bayes.

    (+: small dataset unscrambling)

  • Conclusion

    Participation exceeded our expectations. We thank the participants for their hard work, our sponsors, and Orange who offered:

    A problem of real industrial interest with challenging scientific and technical aspects

    Prizes.

    Lessons learned:

    Do not under-estimate the participants: five days were given for the fast challenge, only a few hours sufficed to some participants.

    Ensemble methods are effective.

    Ensemble of decision trees offer off-the-shelf solutions to problems with large numbers of samples and attributes, mixed types of variables, and lots of missing values.

    presentation title

    The main objective of the challenge is to make good predictions of the target variables. The prediction of each target variable is thought of as a separate classification problem. The results of classification, obtained by thresholding the prediction score, may be represented in a confusion matrix, where tp (true positive), fn (false negative), tn (true negative) and fp (false positive) represent the number of examples falling into each possible outcome:

    PredictionClass +1Class -1TruthClass +1tpfnClass -1fptn

    We define the sensitivity (also called true positive rate or hit rate) and the specificity (true negative rate) as:Sensitivity = tp/posSpecificity = tn/negwhere pos=tp+fn is the total number of positive examples and neg=tn+fp the total number of negative examples.

    presentation title

    presentation title

    presentation title

    0,n

    1,n

    0,n

    1,n

    0,1

    0,n

    0,n

    0,n

    1,1

    1,1

    1,n

    0,n

    0,n

    1,1

    1,n

    1,n

    1,n

    1,n

    0,n

    1,n

    1,1

    0,n

    1,n

    1,1

    0,n

    1,1

    0,n

    1,1

    0,n

    1,n

    0,n

    1,1

    1,1

    0,n

    1,n

    1,1

    1,n

    (1,1)

    1,1

    0,n

    1,n

    0,n

    1,1

    0,n

    0,n

    0,1

    1,1

    1,n

    1,n

    0,1

    0,n

    1,1

    1,1

    0,1

    0,n

    Heritage tiers

    Heritage offre commerciale

    0,n

    1,n

    1,1

    0,n

    (1,1)

    0,n

    0,n

    0,n

    0,1

    0,1

    0,n

    1,n

    1,n

    0,n

    1,n

    0,n

    (1,1)

    Fu appartient type FU

    1,1

    1,n

    Offre

    Id offre

    Libell offre

    Produit & Service

    Id PS

    Date fin validit du P&S

    Date dbut validit du P&S

    Date cration du P&S

    Libell P&S

    Identit Tiers

    Id identit tiers

    Login

    Type identit tiers

    O compose de PS

    Elment De Parc

    Id EDP

    Date dernire utilisation EDP

    Date premire utilisation EDP

    Modle Conceptuel de Donnes

    Modle : MCD PAC_v4

    Package :

    Diagramme : Tiers Services

    Auteur : claudebe

    Date : 14/06/2005

    Version :

    PS a pour FU

    T utilise IT

    EDP souscrit ds O

    Date dbut souscription offre

    Date fin souscription offre

    D

    D

    CRU concerne FU

    Gamme

    Id gamme

    Libell gamme

    Date cration gamme

    Date fin de gamme

    G compose de PS

    Fonction Usage

    Id fonction d'usage

    Libll fonction usage

    T dtient EDP

    Date dbut dtention EDP

    Date fin dtention EDP

    D

    D

    Compte Facturation

    Id compte facturation

    Date dbut validit compte facturation

    Date fin validit compte facturation

    F mise pour CF

    Compte Rendu Usage

    Id compte rendu usage

    Date dbut CRU

    Date fin CRU

    Volume descendant CRU

    Volume montant CRU

    Type transmission

    IT gnre CRU

    CRU gnr par EDP

    Ligne Facture

    Id ligne de facture

    Ligne affiche sur facture

    Montant HT

    Montant TTC

    Type Ligne Facture

    Id type ligne facture

    Libell type ligne facture

    Facture

    Id facture

    Date chance facture

    LF correspond EDP

    LF compose F

    EDP factur sur CF

    Tiers

    Id tiers

    Prnom tiers PP

    Nom tiers PP

    Nom marital PP

    Genre PP

    Date naissance tiers PP

    Date cration tiers

    Date clture tiers

    Date modification tiers

    Type Tiers

    T a pour relation avec T

    Foyer

    Id foyer

    Date cration foyer

    Date fin foyer

    Nb personnes foyer

    Adresse

    Id adresse

    Code postal distribution

    Commune

    Nb habitants commune

    Dpartement

    F a pour A

    Date dbut adresse

    Date fin adresse

    D

    D

    T a pour F

    Date dbut appartenance foyer

    Date fin appartenance foyer

    Role tiers ds foyer

    D

    D

    VA1

    Type Relation Tiers

    Id type relation

    Date cration type de relation tiers

    Libell type relation tiers

    Statut Oprateur

    Id statut oprateur

    Libell statut oprateur

    Operateur

    Id oprateur

    Libell oprateur

    T a pour S

    Date dbut statut tiers

    Date fin statut tiers

    D

    D

    CSP

    Id CSP 350

    Libell CSP 350

    Id CSP 23

    Libell CSP 23

    Id CSP 5

    Libell CSP 5

    T a pour CSP

    LF a pour TLF

    Classification Offre

    Id classification offre

    Libell classification offre

    O positionne ds C

    CO hirarchie

    Groupe de CRU

    Id groupe de CRU

    CRU appartient la CCRU

    Cercle Relationnel

    Id CR

    Libll cercle relationnel

    CRU a pour OCR

    CRU a pour DCR

    Coordonnes Tiers

    Id coordonne tiers

    Date cration coordonne

    Libell coordonne tiers

    T titulaire CT

    C correspond M

    Donnes payeur

    Inscription fichier contentieux

    Nb dossiers recouvrement actifs

    Nb dossiers rclamation actifs

    Nb dossiers recouvrement

    Nb dossiers rclamation

    Niveau risque courant

    Niveau risque prcdent

    Classe de risque

    Id classe risque

    Libell classe risque

    Libell court classe risque

    Niveau risque minimum

    Niveau risque maximum

    T a pour CR

    Date dbut tiers ds classe risque

    Date fin tiers ds classe risque

    D

    D

    Offre compose

    Id offre compose

    Libell offre compose

    Offre commerciale

    Id offre commerciale

    Libell offre commerciale

    Date cration offre

    Date clture offre

    O fait partie OC

    Date dbut rattachement offre

    Date fin rattachement offre

    D

    D

    EDP correspond PS

    Positionnement classification

    Id positionnement

    Libell positionnement

    P dans O

    P hirarchie

    CRU Enchainement

    Mdia

    Id mdia

    Libell mdia

    EDP a EU

    mois

    valeur

    VA6

    N10

    T payeur du CF

    DP pour O

    Etat Usage

    Id EU

    libell tat usage

    Type de fonction d'usage

    id type FU

    lib typ FU

    Id customerzip codeNb call/monthNb calls/hourNb calls/month,weekday,hours,service

    KDD Cup Participation By Year

    45

    57

    24

    31

    136

    18

    57

    102

    37

    68

    95

    128

    453

    0

    50

    100

    150

    200

    250

    300

    350

    400

    450

    500

    1997199819992000200120022003200420052006200720082009

    Year