Java Tech & Tools | Social Media in Programming in Java | Khanderao Kand

Social Media Programming Khanderao Kand

CTO GloMantra Inc

mybantu virtual personalized Assistance

@khanderao Nov 2, 2011 at JAX Conference, London

*  CTO of Glomantra and myBantu in Silicon Valley, CA, USA *  AI based Relevance Engine / recommendation technology with Social Intelligence *  Mybantu is for your daily needs available on Android, iPhone, Facebook and

mybantu.com *  Projects on Social Commerce and Social CRM

*  BIO: *  20 years in Industry (currently AI, SOA, Social, Mobile and Cloud Technologies) *  Lead Oracle’s SOA and BPM Architecture *  Chief Consultanting Architect for SOA and BPM to 50+ architects & hundreds of

developers of Oracle’s next Generation Fusion Apps *  Lead Architect of Peopletools *  Architect in CRM, Online Dialog Marketing, OLAP Hyperion *  Participated in Standards of BPEL and SCA at OASIS *  9 Patents filled *  Frequent Speaker in conferences *  Authored articles in JDJ, WSJ, Cloud Computing *  Upcoming books: SOA with Java, Social Media Programming on Java

Khanderao Kand

*  Business Case: Social Media for Businesses *  Social Media Apps: Social CRM and Social Commerce *  Higher Level Architectures of S-‐CRM and S-‐Commerce *  APIs, Frameworks and Tools: *  Key Technologies, APIs Interfacing with Popular Social

media *  Storage *  Social Media Data Processing

Agenda

Data Point 1

Data Point 2: Facebook Crossed Google

*  Social Media Valuations *  Facebook 65Billion, *  Twitter $7billion (April $4.4b in Jan) , *  Groupon $10 ?billion Zynga $9billion *  LinkedIN $8.5B

Explosion of Social Media Valuation

Is it a Hype?

*  Revenues *  Facebook 2010 (6th year) $2 billion *  Zynga 2010(3rd year) : $850 million *  Groupon 2010 (2nd year) 750million *  Twitter (7th year) $100 million

Revenues and Valuations

*  Compare with *  Novell, took three years for $50million. *  Microsoft crossed the $50 million barrier in eight years; *  Oracle, 10 years.

*  Market Caps: Compare with: *  Amazon 82billion, Google 186Billion, eBay $40billion,

Dell 28B, IBM 200 billion, *  Ford 56B, GM 47B,

*  Gartner predicts 40% of fortune 1000 would use Social Media Apps in 2012-‐13

Future

*  JC Penny *  We go where customers are

*  GNC *  You do not need to leave your favorite site for doing shopping *  Gap *  1-‐800-‐flowers *  Delta airline *  Nike *  Dexter *  Coca-‐cola *  Starbucks *  Apple Apps Stores

Big Brands on Facebook

*  Facebook has brought your social world online with real identities *  Social Media enables how you interact with leverage your social world *  For example, you share … , you ask … before deciding, you

provide feedback, you express approvals/disapprovals and you learn about others, learn from others *  See how it correlates to your Social Media Activities: *  Share : Photos, Video, News, Thoughts, Personal Updates *  Ask: Get opinions / get recommendations *  Feedback: Comment FB/LinkedIn, tweet, Shoutouts *  (Dis) Approval: Like / Dislike on anything *  Observe get influenced : Follow popular tweets, become Fan

How does your Social relationship work on SMNs?

*  Consumers get information about products and services from Social Media Network

Marketing *  Consumer decide based to buy or use service on friends and social

buzz Peer influence generating sales / social commerce *  Consumers buy/usage experience action is shared and kicks in virality Viral growth / cross sale *  Consumers rate, review, shoutout/like/dislike influences decisions of

others as well as image / brand of product / company / service Positive / negative branding affects future sales / revenue

Usages of consumers social behavior on businesses….

Recent Activities …… *  Radian6 got acquired by Salesforce at $350 million *  Overtone got acquired by eGain (not disclosed) *  Google Acquired Y-‐Combinator startup TalkBean at$35m *  BEA founder started Magnet for Social Media with 15m infusion *  Many startups getting funded in the space

Social CRM frenzy…

Social Media Business Apps

*  Business Apps on Social Media

*  Enabling Social Media Widgets on Business Apps

*  Processing Social Media Data for Businesses

Types of Social Media Business Apps

*  Ability to quickly spread awareness about product, events, services, news at a low cost (Social Media Marketing)

*  Ability to get insight of customers reactions, and views (Social Media Market Research / Media Monitoring)

*  Monitor Trend (Social Media Market Research / Media Monitoring) *  Monitor Brand (Social Media Market Research / Media Monitoring) *  Ability to engage customers to develop loyalty or minimize damage (Social

CRM / Social Media Engagement) *  Sales: Increase revenue by influencing decisions and taking transactions

where customers are (Social Commerce ) *  Social CRM / Marketing : Social Analytics: *  Better Understanding of customer and market *  Competitive analysis

*  Actionable Social Intelligence: Crowd Intelligence, New ideas, feature enhancement request

CRM / Commerce Use Cases for businesses for using Social Media …

*  IBM’s “Smart Commerce” initiative started with $2.5 billion acquisitions: *  Sterling Commerce (connect online and offline networks of suppliers and

customers) *  Unica (customer preferences in marketing campaigns) and *  Coremetrics (real-‐time analysis of customer comments and behavior)

*  Goal: *  New approach to buying, selling, and marketing. *  Insight into how customers feel about and use products, and influence

their & their social circles’ buy decisions *  Addressable market $70billion (Software, & services) *  Deploying 1200 salesmen and about 1000 services people *  Launched “Smarter Commerce University”

IBM’s Social Commerce Initiative

Social CRM Architecture

CRM Agent

Community Manager

Marketing PR Analyst

CRM Case

Management

Marketing Campaigns

Contact Management

Sales Case Mgmt

Social Monitoring

REST / RSS JSON / XML

Social Analytics

Unstructured Text Process

ETL BI/ DW NLP/ AI Sentiment,

Brand,

Relevance / Interest Graph

Community Content

Marketing Content Reviews

Content

OAuth OID

Infograph of F-‐Commerce

Facebook Widgets

myBantu

Social Commerce Architecture

Social Commerce Widgets: myBantu (GloMantra) getOpinion, Like/Thrash, Rate, Shoutout, Recommendation, socialBuzz, hotmeter BazarVoice Reviews, Ratings, BrandVoice, BrandAnswers, Ask&Answer Pluck: Popular Contents, Comments, Active Users, Recommendation,

IBM Smart Commerce WebSphere Platform

Find Similar

gloMantra MyBantu Social Commerce Widgets on An Apparel Site

*  Foundation Protocols / Interfaces *  HTTP REST *  JSON XML *  RSS *  Oauth *  OpenId

*  Social Media APIs and Interfaces *  Open Social *  Facebook API: Graph API and Open Graph 1 and 2 *  Twitter API *  Google+ *  Social Graph from Google

*  Processing Technologies *  Hadoop, NOSQL, GraphDB *  Unstructured Processing / Text Processing / Text Analytics *  NLP, Machine Learning

Interfacing with Key Social Media

Foundational Protocols and Standards for Social Media

*  OAuth is open standard for Authorizing Access (typically data access / actions)

*  Oauth hands over access token and not user credentials

*  Oauth is essential to developing social-‐ready applications

*  OAuth protocol (Dance) involves series of 3 party (consumer browser, Service provider – Social media Site and App).

*  OAuth, including Twitter, Facebook, LinkedIn, TripIt, and Foursquare, as well as the Google and Yahoo APIs.

*  Different SM supports either Oauth 1.0 and Oauth 2.0

*  Visit : http://oauth.net

*  APIs: Jersery (JAX-‐RS), Spring Security API, Googlecode,

OAuth

*  Open ID is open source standard for multi-‐site authentication *  OpenId is for authentication as against Oauth *  No need of adhoc and site specific authentication *  Oauth and OpenID can coexist *  Open ID providers : think of Google login *  Providers pass ID attributes to the relying party (app) *  However, Oauth is more popular and common in Social Media

interfaces *  Visit http://www.openid.org *  Java API: openid4java, GAE supports,

Open ID

*  Representational State Transfer (Roy Fielding’s paper for Architectural Style over HTTP)

*  Basic GET, PUT, POST, DELETE operations on a resource

*  Internet world REST is currently winning

*  Google, Yahoo, Facebook, etc etc.

*  Java: Jersey (JAX-‐RS), Restlet, RestFul Services, JAX-‐WS

REST

*  Javascript Simple Object Notation as lightweight data interchange format *  Lighter than XML *  Can be manipulated on browsers *  No need for parsing into DOM *  http://www.json.org *  Java API: GSON, Jackson, JSONLib, FlexJSON *  XML to/from JSON conversion: JSONLib

JSON

*  Really Simple Syndication (originally RDF Site Summary) *  Started with news, site, blogs update *  Simple and lightweight *  Popular Readers: Google Reader, Feedly etc *  Java API: ROME, Simple XML parsing

RSS

APIs of Popular Social Media

*  Representing objects and relationships in Facebook’s social network *  Simple, consistent and REST style representation *  Objects: People, events, pages, and photos *  People: http://graph.facebook.com/659690992/picture *  Events: http://graph.facebook.com/331218348435/picture *  Groups: http://graph.facebook.com/69048030774/picture *  Pages: http://graph.facebook.com/DoloresPark/picture *  Applications: http://graph.facebook.com/2318966938/picture *  Photo Albums: http://graph.facebook.com/platform/picture *  Access API: https://graph.facebook.com/ID/CONNECTION_TYPE

Facebook Graph API

*  https://graph.facebook.com/ID/CONNECTION_TYPE

*  Friends: https://graph.facebook.com/me/friends?access_token=... *  News feed: https://graph.facebook.com/me/home?access_token=... *  Profile feed (Wall): https://graph.facebook.com/me/feed?access_token=... *  Likes: https://graph.facebook.com/me/likes?access_token=... *  Movies: https://graph.facebook.com/me/movies?access_token=... *  Music: https://graph.facebook.com/me/music?access_token=... *  Books: https://graph.facebook.com/me/books?access_token=... *  Notes: https://graph.facebook.com/me/notes?access_token=... *  Permissions: https://graph.facebook.com/me/permissions?access_token=... *  Photo Tags: https://graph.facebook.com/me/photos?access_token=... *  Photo Albums: https://graph.facebook.com/me/albums?access_token=... *  Video Tags: https://graph.facebook.com/me/videos?access_token=... *  Video Uploads: https://graph.facebook.com/me/videos/uploaded?access_token=... *  Events: https://graph.facebook.com/me/events?access_token=... *  Groups: https://graph.facebook.com/me/groups?access_token=... *  Checkins: https://graph.facebook.com/me/checkins?access_token=…

Facebook Graph API: Access

*  All public objects *  https://graph.facebook.com/search?q=QUERY&type=OBJECT_TYPE

*  All public posts: https://graph.facebook.com/search?q=socialcommerce&type=post

*  People: https://graph.facebook.com/search?q=Khanderao&type=user *  Pages: https://graph.facebook.com/search?q=glomantra&type=page *  Events: https://graph.facebook.com/search?q=conference&type=event *  Groups: https://graph.facebook.com/search?q=programming&type=group *  Places:

https://graph.facebook.com/search?q=coffee&type=place&center=37.76,122.427&distance=1000

*  Checkins: https://graph.facebook.com/search?type=checkin

Facebook Search using Graph API

*  HTTP Post *  Access token *  curl -‐F 'access_token=...' \ https://graph.facebook.com/313449204401/likes

Facebook: Publishing on Wall

Example String url = "h3ps://graph.facebook.com/"+facebook ID+"/feed"; String data = URLEncoder.encode("access_token", "UTF-‐8") + "=" + URLEncoder.encode(“ACCESS TOKEN STRING”, "UTF-‐8"); data += "&" + URLEncoder.encode("message", "UTF-‐8") + "=" + URLEncoder.encode(“wallMessageText”, "UTF-‐8"); data += "&" + URLEncoder.encode("picture", "UTF-‐8") + "=" + URLEncoder.encode(”IMAGE URL”, "UTF-‐8"); data += "&" + URLEncoder.encode("name", "UTF-‐8") + "=" + URLEncoder.encode(“POST TITLE STRING”, "UTF-‐8"); data += "&" + URLEncoder.encode("capOon", "UTF-‐8") + "=" + URLEncoder.encode(“CAPTION STRING”, "UTF-‐8"); data += "&" + URLEncoder.encode("descripOon", "UTF-‐8") + "=" + URLEncoder.encode(“DESCRIPTION”, "UTF-‐8"); data += "&" + URLEncoder.encode("acOons", "UTF-‐8") + "=" + URLEncoder.encode("{\"name\": \”acOon_name\", \"link\": \” h3p://192.169.0.46/app/valid url1\"}", "UTF-‐8"); data += "&" + URLEncoder.encode("link", "UTF-‐8") + "=" + URLEncoder.encode(“LINK URL”, "UTF-‐8"); // ConstrucNng acNon links JSONObject obj = new JSONObject(); LinkedHashMap<String, Object> m1 = new LinkedHashMap<String, Object>(); m1.put("text“,”LINK1 TEXT”); m1.put("href", “h3p://192.169.0.46/app/valid acNon url1”); obj.put(”LINK1 LABEL”, m1);

SAMPLE POST ON YOUR WALL

*  Twitter is ranked as one of the ten-‐most-‐visited websites worldwide by Alexa's web traffic analysis.

*  March 2010 10 billionth tweet

*  Twitter has a user retention rate of forty percent. *  Value: Conversational 30%, Pointless Babble 30%, Promotion 8%, News 8%, news 7-‐8% and spam

*  Good source: news validation, trend, topic updates

Twitter

*  "Twitter Revolutions" and which include the 2011 Egyptian revolution, 2010–2011 Tunisian protests, 2009–2010 Iranian election protests, and 2009 Moldova civil unrest *  Twitter is also increasingly used for making TV more interactive and

social *  Most of the celebrities are on Twitter *  President Obama and many others effectively used Twitter for

campaigning *  Twitter has become a great source for latest news *  Twitter is an indicator for buzz around latest topics and trends *  Brands are using twitters for promotions of products and spreading

deals

Twitter Usage

*  Twitter4J : Open Source BSD

*  Java-‐Twitter

*  Jtwitter : Open Source *  Twitter Client Streaming API

3rd party Twitter Java APIs

*  Twitter API allows to *  Tweet *  Direct message *  Search *  Get tweets

Twitter

*  Twitter4J (third party – open source) *  Access token / consumer key in twitter.properties

Twitter twitter = new TwitterFactory().getInstance(); Query query = new Query(”mybantu"); QueryResult result = twitter.search(query); // uodate status Status status = twitter.updateStatus(latestStatus);

Twitter4J API

*  GNIP 50% at approximately 360K per year *  For analysis but not for resale or display tweets

*  Spritzer 2% random for free

Twitter Fire hose

*  "For the question about Google plans to allow developers to create Google+ apps deploying the OpenSocial APIs, a spokeswoman of Google said that by means of email nowadays the Google+ platform can’t support the OpenSocial APIs.

*  She also added that nevertheless, they are utilizing many of the technology which was designed as part of OpenSocial, along with the gadget application packaging model, and power Google+ games and the Portable Contacts JSON schema. Because they define the +Platform APIs, they are drawing close attention to the future tendency of the OpenSocial APIs, as well as converging everything possible."

Open Social Project

*  Apache Opensource project in incubation *  Shindig

Developing Social Network

Great initiative with huge promise *  A common “open source” API for social media launched in 2007 *  Early movers as open social containers: Google Orkut, hi5, LinkedIn,

MySpace, Netlog, Ning, orkut, and Yahoo! *  Opensource OpenSocial Based container : Apache Shindig

Bummer… *  Except Facebook *  Google+ is not yet committed to Open Social

Open Social

*  By supporting open Web standards for describing connections between people, web sites can add to the social infrastructure of the web

*  The API returns web addresses of public pages and publicly declared connections between them.

*  Allows developers to create a button for a Web site that would allow a registered user to easily "add friends.”

*  index the public Web for Friends of A Friend (FOAF) and Friends Network

* 

Google Social Graph API

*  The Social Graph API looks for two types of publicly declared connections:

1.  It looks for all public URLs that belong to you and are interconnected. This could be your blog (a1), your LiveJournal page (a2), and your Twitter account (a3).

2.  It looks for publicly declared connections between people. For example, a1 may link to b's blog while a1 and c link to each other.

Google’s Public Social Graph

Searching Public posts GET https://www.googleapis.com/plus/v1/activities?query=cookie%20recipes&orderBy=best&key=[yourAPIKey] People Search GET https://www.googleapis.com/plus/v1/people?query=Khanderao%20kand&key=[yourAPIKey] View People Interacting with Posts GET https://www.googleapis.com/plus/v1/activities/{activityId}/people/resharers?key=[yourAPIKey] GET https://www.googleapis.com/plus/v1/activities/{activityId}/people/plusoners?key=[yourAPIKey]

Google+ API https://developers.google.com/+/api/

*  An extensible service provider framework that greatly simplifies the process of connecting local user accounts to hosted provider accounts. *  A connect controller that handles the authorization flow between

your Java/Spring web application, a service provider, and your users. *  Java bindings to popular service provider APIs such as Facebook,

Twitter, LinkedIn, TripIt, GitHub, and Gowalla. *  A sign-‐in controller that enable users to authenticate with your

application by signing through a service provider.

Spring Social

Spring Social provides templates for REST based API handling OAUth and providing operations specific to Social Media

*  TwitterTemplate *  FacebookTemplate *  LinkedInTemplate *  TripItTemplate

*  http://blog.springsource.com/2010/11/03/socializing-‐spring-‐applications/ *  http://greenhouse.springsource.org

Spring Social

Technologies for Building Applications on Social Media

*  Social Media Contains: Huge Data, Often available via JSON, XML , RSS

Requires different type of DBs…. *  Couch DB *  MongoDB *  Cassandra *  Hbase *  Neo4J *  Hadoop for Big Data Processing

Social Media Data Often Needs Different Type of Database

CouchDB MongoDB

Data Model Document Oriented Document Oriented

Data Format JSON BSON

Interface REST TCP/IP

Query MapReduce Java Script Jmap Reduce ava Script + Query

Fault Tolerance Master Master Master Slave

Concurrency MVCC (Versioning) Updates in place

Performance Comparatively lower Higher

Cassandra HBase

Type Column DB Colletcion-‐ column DB

Type Big Table Big Table

Protoco Binary (Thrift) Binary (Thrift) HTTP/REST

Map Reduce Via Hadoop Via Hadoop

Dealing with Big Data

*  Text Processing Frameworks: *  GATE: General Architecture for Text Engineering -‐ Java toolkit *  OpenPipeline: Open Source Java software for crawling, parsing, analyzing

and routing documents. *  UIMA: Unstructured Information Management Architecture -‐ Java

framework *  Mallet(Machine Learning for Language Toolkit) *  Doc classification, clustering

*  Apache Mahout: Scalable Machine Learning and Data mining *  ML: Mahout, LingPipe, UIMA, Mallet *  NLP: Apache OpenNLP, Gate, LingPipe, Stanford POS, NLTK *  Dictionary and entity databases: *  Wordnet, SentiWordNet

*  Entity Recoginition: Alchemy, Stanford NER

Text Processing and ML

*  Collaborative Filtering *  User and Item based recommenders *  K-‐Means, Fuzzy K-‐Means clustering *  Mean Shift clustering *  Dirichlet process clustering *  Latent Dirichlet Allocation *  Singular value decomposition *  Parallel Frequent Pattern mining *  Complementary Naive Bayes classifier *  Random forest decision tree based classifier

Algorithms in Apache Mahout

*  Synonymous *  Most of the Social Media Apps are built for Cloud usage *  Frameworks: *  Amazon AMI (many images for Java stack) *  VMWare SpringSource CloudFoundry *  JBOSS : Openshift *  Salesforce: Heroku for Java *  Oracle: Java MW on Cloud *  Google App Engine

Social Media Apps and Cloud

*  1. The presentation though quotes examples of my Company and products, the presentation is based on my knowledge and experience and my company is not liable.

*  2. Attributed to most of the known source, if any attribution is missing, it is not due to intention but might not have got the information of the original source or might be be negligence in my due diligence. It can be corrected if you contact

*  3. Feel free to share some / all content however please attribute to Khanderao Kand

*  4. Social Media Networks mentioned in the presentation are more foe example but the approach is applicable to most of all other social media and there is no intention to promote / discourage any media

*  5. User usage data and financial numbers are dated and would be stale / outdated over a period of time. You may need to look for the latest data.

*  6. APIs , tools and libraries are third party and not from me / my company. Though they are listed based on my experience and knowledge, use them at your risk.

*  7. We have products in Social Commerce, Social CRM, and recommendation. We also take projects around the space and can be done outsource / jointly if interested in.

*  Contact : [email protected] or [email protected] Twitter @khanderao

*  For original Slide deck (ppt) feel free to contact.

Disclaimer and Request

Java Tech & Tools | Social Media in Programming in Java | Khanderao Kand

Technology

Transcript of Java Tech & Tools | Social Media in Programming in Java | Khanderao Kand