Genn.ai introduction for Buzzwords

Click here to load reader

  • date post

    03-Jul-2015
  • Category

    Technology

  • view

    243
  • download

    0

Embed Size (px)

description

We are making a realtime marketing system for general marketplace type web sites with Kafka + Strom. http://genn.ai/

Transcript of Genn.ai introduction for Buzzwords

  • 1. Genn.ai ()Attached realtime marketing platform1

2. http://genn.ai/Who am I?Takeshi NAKANOSenior Researcher / Architect.Co-authered Getting started with Solr in Japanese.Co-authered Hadoop Hacks in Japanese.2Writing Getting started with Kafka in Japanese. 3. http://genn.ai/3Company Name Recruit Holdings Co., LtdFounded March 31, 1960 (incorporated: August 26, 1963governance changed from Recruit Co., Ltd since October 1st, 2012Headquarters Tokyo, JapanNumber of Group Employees 22,368as of the end of March,2012Number of Group Companies 102Consolidated Subsidiaries/as of the end of June,2012Capital 3,002.64 million yensince March 1st, 1995Consolidated Sales 806,661 million yenfrom April 1st 2011 to March 31st 2012Consolidated OrdinaryIncome117,617 million yen from April 1st 2011 to March 31st 2012) 4. http://genn.ai/4RecruitHoldingsHeadquartersFunctionGlobal BusinessBusiness R&DRecruit Career Co., LtdRecruit Jobs Co., LtdRecruit Sumai Company LtdRecruit Marketing Partners Co., LtdRecruit Lifestyle Co., LtdRecruit Administration Co., LtdRecruit Technologies Co., LtdRecruit Communications Co., LtdOperating Companies DomainRecruit Stafng Co., LtdSTAFF SERVICE HOLDINGS CO., LTD 5. http://genn.ai/What is our challenge?Clients can get the information about their customerschecking their items on our web sites.Providing a printed report monthly.Send them more detailed report in real time.In real time fashion, the value of the information will bedramatically increased!5 6. http://genn.ai/Agenda What is genn.ai? Overview How to gather clicks 0) Injecting our javascripts6 Genn.ai core details Structure 1) Setting up lters 2) Filtering customers 3) Visualizing them Future Dealing with historical streams Genn.ai wants friends. 7. http://genn.ai/Overview of using Genn.ai7CustomerACustomerBCustomerCRealtime AnalysisPlatformGenn.aiWeb SitesClient(Hotels)MarketerAnalyst 8. http://genn.ai/Demo videos!They are based on a hotel reservation site likeExpedia.com.An animation and a screencapture what themarketting sta see using Genn.ai.At this moment ..What prefectures are all my users searching for? (0:26)Comparing the number of searches in today s andyesterday s regarding places. (2:50)8 9. http://genn.ai/Overview of using Genn.ai9CustomerACustomerBCustomerCRealtime AnalysisPlatformGenn.aiWeb SitesClient(Hotels)MarketerAnalyst 10. http://genn.ai/Overview of using Genn.ai10CustomerACustomerBCustomerCWeb SitesRealtime AnalysisPlatformGenn.aiClient(Hotels)MarketerAnalyst 11. http://genn.ai/How to gain access to Genn.ai11CustomerACustomerBCustomerCApp(ex.Tomcat)ApacheGenn.ai will be able to be attached! 12. http://genn.ai/How to gain access to Genn.aiA custom apache module for embeding ourcode on returning html sources from App.12CustomerBApacheGenn.ai will be able to be attached!modGenn.aiApp(ex.Tomcat) 13. http://genn.ai/How to gain access to Genn.ai13The embedded values and js on a html. 14. http://genn.ai/Structure14 15. http://genn.ai/Steps for getting resultsThere are 3 steps! (after getting clicks)151 2 3Pouring clicksinto KafkaRecording themas behaviorsVisualizing them0gather clicks 16. Backend for Genn.aihttp://genn.ai/Structure (Pouring into Kafka)16StormTopology(Filter)Kafkakakan.puTomcatiPad(APP)MongoDB321Genn.ai Front(Based on Finagle)Querygenn.ai/web)Click StreamsFrontend for Genn.aiVisualizer for Genn.aiZoomdata 17. Backend for Genn.aihttp://genn.ai/Structure (Recording as behaviors)17StormTopology(Filter)Kafkakakan.puTomcatiPad(APP)MongoDBGenn.ai Front(Based on Finagle)Querygenn.ai/web)Click Streams321Visualizer for Genn.aiFrontend for Genn.aiZoomdata 18. Backend for Genn.aiVisualizer for Genn.aihttp://genn.ai/Structure (Visualizing customers)18StormTopology(Filter)Kafkakakan.puTomcatiPad(APP)MongoDBGenn.ai Front(Based on Finagle)Querygenn.ai/web)Click Streams321Frontend for Genn.aiZoomdata 19. http://genn.ai/How is Genn.ai composed?Genn.ai Frontend! : Gathering clicks.Genn.ai Backend! : Recording behaviors.Genn.ai Visualizer! : Visualizing customers attrs.Kafka : Used for message busMongoDB : Storing Additional tablesStorm : Processing click streams19 20. http://genn.ai/Details of recording behaviors.20 21. Backend for Genn.aihttp://genn.ai/Structure (Recording behaviors)21StormTopology(Filter)Kafkakakan.puTomcatiPad(APP)MongoDBGenn.ai Front(Based on Finagle)Querygenn.ai/web)Click Streams321Visualizer for Genn.aiZoomdataFrontend for Genn.ai 22. http://genn.ai/Calculated valuesRecording behaviorsUseful values for each sites Ex.)Hotel reservation site,22{"_id": 154,"largeAreaCntAcl": {"Tokyo": 3,"Berlin": 1},"largeAreaCntPre": {"Tokyo": 4,"Berlin": 3},"rateAvgAcl": 4200,"rateAvgPre": 6200,"rateCntAcl": [4, 1, 0, 0, 0],"rateCntPre": [4, 1, 2, 0, 0],"lastUpdate": "2013-04-16T04:04:37.432Z"} 23. http://genn.ai/Calculated valuesRecording behaviorsUseful values for each sites Ex.)Hotel reservation site, Where is the favorite place?23{"_id": 154,"largeAreaCntAcl": {"Tokyo": 3,"Berlin": 1},"largeAreaCntPre": {"Tokyo": 4,"Berlin": 3},"rateAvgAcl": 4200,"rateAvgPre": 6200,"rateCntAcl": [4, 1, 0, 0, 0],"rateCntPre": [4, 1, 2, 0, 0],"lastUpdate": "2013-04-16T04:04:37.432Z"} 24. http://genn.ai/Calculated valuesRecording behaviorsUseful values for each sites Ex.)Hotel reservation site, Where is the favorite place? Where is the this searching place?24{"_id": 154,"largeAreaCntAcl": {"Tokyo": 3,"Berlin": 1},"largeAreaCntPre": {"Tokyo": 4,"Berlin": 3},"rateAvgAcl": 4200,"rateAvgPre": 6200,"rateCntAcl": [4, 1, 0, 0, 0],"rateCntPre": [4, 1, 2, 0, 0],"lastUpdate": "2013-04-16T04:04:37.432Z"} 25. http://genn.ai/Calculated valuesRecording behaviorsUseful values for each sites Ex.)Hotel reservation site, Where is the favorite place? Where is the this searching place? What price range has he booked?25{"_id": 154,"largeAreaCntAcl": {"Tokyo": 3,"Berlin": 1},"largeAreaCntPre": {"Tokyo": 4,"Berlin": 3},"rateAvgAcl": 4200,"rateAvgPre": 6200,"rateCntAcl": [4, 1, 0, 0, 0],"rateCntPre": [4, 1, 2, 0, 0],"lastUpdate": "2013-04-16T04:04:37.432Z"} etc.. 26. http://genn.ai/Functions for recording behaviorsTime shift functionTime sensitive counter (old item will be expired and deleted)Facet ( like Solr )Categorized counterRange Facet ( like Solr )Dynamic categorized counter26321 27. http://genn.ai/Functions for recording behaviorsTime shift functionTime sensitive counter (old item will be expired and deleted)Facet ( like Solr )Categorized counterRange Facet ( like Solr )Dynamic categorized counter27321 28. http://genn.ai/Time Shift function?We make 3 min time window and now start.( no clicks here )28tNow 29. http://genn.ai/Time Shift function?In our 3 min time window, 1.5 min are captured.2 clicks have been recorded.291.5 minsNowTime windowClickClick2 clicks are captured.t 30. http://genn.ai/Time Shift function?There are 3 clicks in the past.The length of the capturing window becomesmax.303 minsNowTime windowClickClickClickt 31. http://genn.ai/Time Shift function?There are 4 clicks in the past.Ignore the rst 2 clicks31ClickClickNowTime windowClickClick1.5 min 3 minst 32. http://genn.ai/Time Shift function?There are 4 clicks in the past.Ignore the rst 3 clicks32ClickClickNowTime windowClickClick3 mins3 minst 33. http://genn.ai/Functions for recording behaviorsTime shift functionTime sensitive counter (old item will be expired and deleted)Facet ( like Solr )Categorized counterRange Facet ( like Solr )Dynamic categorized counter33321 34. http://genn.ai/Functions for recording behaviorsTime shift functionTime sensitive counter (old item will be expired and deleted)Facet ( like Solr )Categorized counterRange Facet ( like Solr )Dynamic categorized counter34321 35. http://genn.ai/Counting the frequencyThis function is under the eect of the Timeshift function.35{"_id": 154,"largeAreaCntAcl": {"Tokyo": 3,"Berlin": 1},"largeAreaCntPre": {"Tokyo": 4,"Berlin": 3},"rateAvgAcl": 4200,"rateAvgPre": 6200,"rateCntAcl": [4, 1, 0, 0, 0],"rateCntPre": [4, 1, 2, 0, 0],"lastUpdate": "2013-04-16T04:04:37.432Z"} 36. http://genn.ai/Counting the frequencyThis function is under the eect of the Timeshift function.You have booked 1 stay in Berlin and 3 staysin Tokyo in this 3 weeks.36L.A.S.F.tNowTime windowBerlinTokyo1.5 week 3 weeksTokyo"largeAreaCntAcl": {"Tokyo": 3,"Berlin": 1}Tokyo 37. http://genn.ai/Details of visualization.37 38. Backend for Genn.aihttp://genn.ai/Visualizing the customers38StormTopology(Filter)Kafkakakan.puTomcatiPad(APP)MongoDBGenn.ai Front(Based on Finagle)Querygenn.ai/web)Click Streams321Visualizer for Genn.ai 39. Backend for Genn.aihttp://genn.ai/Visualizing the customers39StormTopology(Filter)Kafkakakan.puTomcatiPad(APP)MongoDBGenn.ai Front(Based on Finagle)Querygenn.ai/web)Click Streams321Visualizer for Genn.ai 40. http://genn.ai/Kafka consumerKakan.puThe program for relaying clicks from Kafka to theother systems.The enriching function (joining the MongoDB data)The converting function (for following systems)40 41. Backend for Genn.aihttp://genn.ai/Visualizing the customers41StormTopology(Filter)Kafkakakan.puTomcatiPad(APP)MongoDBGenn.ai Front(Based on Finagle)Querygenn.ai/web)Click Streams321Visualizer for Genn.aiZoomdata 42. http://genn.ai/Visualizing toolZoomdataA product containing a set of server app and ipadapp for visualizing event data.42321 43. http://genn.ai/ZoomdataZoomdata has the functions like..WebAPI for streaming data.Own event processor for vizualizing.43321The bar and bubble graph on default setting.Customizable visualizing templates. 44. http://genn.ai/Feature plan.44 45. http://genn.ai/Genn.ai EvolutionAdding functions on Genn.ai core.Genn.ai should have ltering / aggrigation functionsfor federating with other systems.Historical rewinding function will be implemented withthe Hadoop cluster.45 46. http://genn.ai/Future planGenn.ai wants friends! We are looking for co-challenging partners for tuningand improving Genn.ai. You can use Genn.ai on your web sites for totally