Powered by Asciidoctor, Hyla & RevealJS

DEVELOPMENT OF SOCIAL MEDIA PROJECTSWITH CAMELDevNation / 14th of April - 2014

Architect, Engineer & Committer

Charles Moulliard

SPEAKERAgricultural Engineer & Zoologist

19 years of experience in IT world development

Project manager in Bank, Financial, Telco world

Specialized in new technologies Web & Integration

Architect/Engineer @Red Hat

Committer : Apache ServiceMix, Karaf (PMC), Camel (PMC),Fabric8, Drools, Hawtio, Asciidoctor

SPEAKER - CON’TTwitter :

LinkedIn :

Blog :

Slideshare :

http://twitter.com/cmoulliard

http://www.linkedin.com/in/charlesmoulliard

http://cmoulliard.github.io

http://www.slideshare.net/cmoulliard

COUNTRY WHERE I LIVE

WHAT YOU SHOULD KNOW ABOUT BELGIUM

MY PASSION(S)

AGENDAIntroduction

Social Media Hype

Use cases covered

Integration & Camel

Technology

What JBoss Fuse can offer

Conclusion

INTRODUCTIONWeb 2.0 “Revolution”,User is an actorcollaborating

Consequences :

HTTPs requests

Volume of dataexchanged

INTRODUCTION - CON’TTechnology Transformation

HTML5

JavaScript / JSon / NoSQL

Push Ajax - WebSocket

AGENDAIntroduction

Social Media Hype

Use cases covered

Integration & Camel

Technology

What JBoss Fuse can offer

Conclusion

SOCIAL MEDIA HYPEBusiness - Companies

Marketing Strategy,

Product Promotion,

Launch or Campaignannoucement

Seeking candidates

…

Use Web2.0 but Social Medias

SOCIAL MEDIA HYPE - CON’TWhy To create content that attracts attention & encouragesreaders to share it with their social networks

AGENDAIntroduction

Social Media Hype

Use cases covered

Integration & Camel

Technology

What JBoss Fuse can offer

Conclusion

USE CASES COVERED - DEMO1Pattern : Real Time & Broadcasting

Definition : Collect data in real time from different providers(twitter, facebook, …) and broadcast/multicast data tosubscribed channels

Business : Event, conference, meeting room

USE CASES COVERED - DEMO2Pattern : Collect Data Metrics/Statistics

Definition : Retrieve data from Social Media and store them inorder to query the data to analyze the results

Business : Measure performances of a campaign, productlaunch, analyze data to design marketing strategies

USE CASES COVERED - DEMO3Pattern : Business Activity Monitoring

Definition : Monitor activities, operational days usingreporting, dashboard tools.

Business : Measure performances, rentability, return ofinvestment

AGENDAIntroduction

Social Media Hype

Use cases covered

Integration & Camel

Technology

What JBoss Fuse can offer

Conclusion

INTEGRATIONIntegration is really hard

protocols, standards,data formats, systems

Long story since CommonObject Request BrokerArchitecture

INTEGRATION - CON’TConcept

Decouple Producerfrom Consumer

Message transportinformation

INTEGRATION - CON’T Layer to transportmessages : BUS

ESB, JBI, SCA

INTEGRATION - CON’TOpenSource Java Integration Framework

Designed around : Domain Specific Language

Implement Enterprise Integration Patterns

CAMEL implemented

and more : Loadbalancer, Throttler, Delayer, …

> 50 patterns

CAMEL - CON’TKey features

Component

Endpoint

Consumer

Producer

CAMEL - CON’TKey features : route, processor

CAMEL - CON’TInterceptor : trace, log, capture business events

CAMEL - CON’TContainer for theroutes CamelContext

CAMEL - CON’TCross communicationnot allowed usingdirect, seda

CAMEL - CON’TBut possible UsingBUS like NMR, Broker,Shared Componentwith direct-vm, vm

CAMEL - CON’TType Converter Strategy

Allow to convert the body payloadsfrom one type to another

To and From these types

File

String

byte[] and ByteBuffer

InputStream and OutputStream

Reader and Writer

CAMEL - CON’TData Transformation for complex use case

package org.devnation.camel;

import java.io.InputStream;import java.io.OutputStream;import org.apache.camel.Exchange;

public interface DataFormat {

void marshal(Exchange exchange, Object graph, OutputStream stream) throws Exception;

Object unmarshal(Exchange exchange, InputStream stream) throws Exception;}

Marshalling : Object XML (JAXB)

Unmarshalling : XML Object (JAXB)

CAMEL - CON’T> 20 Data Format

CAMEL - CON’T

CAMEL - CON’T

CAMEL - CON’TFluent API

package org.devnation.camel;

import org.apache.camel.builder.RouteBuilder;

public class ExampleRouteBuilder extends RouteBuilder { @Override public void configure() throws Exception {

from("amq:queue:quotes") .filter().xpath("/quote/product/ = 'widget") .bean("QuotesService", "widget") .filter().xpath("/quote/product/ = 'gadget") .bean("QuotesService","gadget");

}}

CAMEL - CON’TAlternative : Spring, Blueprint DSL

<?xml version="1.0" encoding="UTF-8"?><beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd "> <bean id="quotesService" class="org.devnation.camel.QuotesService"/>"

<camelContext xmlns="http://camel.apache.org/schema/spring"> <route> <from uri="amq:queue:quotes"/> <filter> <xpath>"/quote/product/ = 'widget"</xpath> </filter> <bean id="quotesService" method="widget"/> <filter> <xpath>"/quote/product/ = 'gadget"</xpath> </filter> <bean id="quotesService" method="gadget"/> </route> </camelContext>

</beans>

CAMEL - CON’TIn memory bus / alternative to JBI, SCA, NMR

Transport objects : XML, File, Stream, Bytes

Predicate & Expression language (xslt, xpath, …)

Sync/Async exchanges

Threads Management,

Tx Architecture

Error and exception handling

Policy driven

Container agnostic

CAMEL - CON’T

AGENDAIntroduction

Social Media Hype

Use cases covered

Integration & Camel

Technology

What JBoss Fuse can offer

Conclusion

TECHNOLOGY - CAMEL

combined with

camel-gmail

camel-dropbox

camel-ajax, camel-websocket,

camel-sql, camel-jdbc, …

camel-elasticsearch

TECHNOLOGY - COMMONOpen standard for Authorization

OAuth

TECHNOLOGY - CAMELComponent : FaceBook

Info :

Grant Access :

API :

http://camel.apache.org/facebook.html

https://developers.facebook.com/apps

http://facebook4j.org/en/index.html

// start with a 30 day window for the first delayed poll String since = "RAW(" + new SimpleDateFormat(FACEBOOK_DATE_FORMAT).format( new Date(System.currentTimeMillis() - TimeUnit.MILLISECONDS.convert(30, TimeUnit.DAYS))) + ")";

from("facebook://searchPosts?reading.limit=10&" + "reading.locale=en.US&reading.since=" + since + "&" + "consumer.initialDelay=1000&" + "consumer.sendEmptyMessageWhenIdle=true&" + getOauthParams());

TECHNOLOGY - CAMELOAuth

protected String getOauthParams() { return "oAuthAppId=" + properties.get("oAuthAppId") + "&oAuthAppSecret=" + properties.get("oAuthAppSecret") + (properties.get("oAuthAccessToken") != null ? ("&oAuthAccessToken=" + properties.get("oAuthAccessToken")) : ""); }

Externalize tokens with Property PlaceHolderparameters

consumer.key=rtPyyyyyyyyyyyyyyyygconsumer.secret=HyoWW3xxxxxxxxxxxxxxxxTuyEz8qrkaccess.token=7976eeeeeeeeeeeeeeeeeeeuRmG0IPaccess.token.secret=VNtttttttttttttttttg0B8jWGs

TECHNOLOGY - CAMELExample :

To create a post within your Facebook profile

Send a facebook4j.PostUpdate body to a camel facebookproducer

PostUpdate post = new PostUpdate(new URL("http://facebook4j.org")) .picture(new URL("http://facebook4j.org/images/hero.png")) .name("Facebook4J - A Java library for " + "the Facebook Graph API") .caption("facebook4j.org") .description("Facebook4J is a Java library" + " for the Facebook Graph API.");

from("direct:post") .setBody().constant(post) .to("facebook://postFeed/inBody=postUpdate");

TECHNOLOGY - CAMELComponent Twitter

Info :

Grant Access :

API :

http://camel.apache.org/twitter.html

https://dev.twitter.com/apps/new

http://twitter4j.org/en/index.html

Example from("twitter://search?type=polling&delay=" + delay + "&useSSL=true&keywords=" + keywords + "&" + getUriTokens()) .choice() .when().simple("${body.tweet.text} > 'java'") .bean("Service", "push") .otherwise() .to(">> Tweets received");

TECHNOLOGY - CAMELOAuth

protected String getUriTokens() { return "consumerKey=" + consumerKey + "&consumerSecret=" + consumerSecret + "&accessToken=" + accessToken + "&accessTokenSecret=" + accessTokenSecret; }

Advanced Query

https://support.twitter.com/articles/71577-using-advanced-search

TECHNOLOGY - CAMELSimple Example to post a tweet

Date now = new Date(); String tweet = "Demo: this is a tweet posted on " + now.toString();

from("direct:post") .setBody().constant(tweet) .to("twitter://timeline/user?" + getUriTokens());

TECHNOLOGY - CAMELObject / JSON

Use utility like Google JSON API

Marshall/Unmmarshall JSON

TWITTER - CON’TBAD -

String data = "{ " + " \"" + "timestamp" + "\" : \"" + formatDate(generateTimeStamp()) + "\"," + " \"" + "createdAt" + "\" : \"" + status.getCreatedAt().toString() + "\"," + " \"" + "id" + "\" : \"" + status.getId() + "\"," + " \"" + "text" + "\" : \"" + status.getText() + '\'' + "\"," + " \"" + "isFavorited" + "\" : \"" + status.isFavorited() + "\"," + " \"" + "isRetweeted" + "\" : \"" + status.isRetweeted() + "\"," + " \"" + "favoriteCount" + "\" : \"" + status.getFavoriteCount() + "\"," + " \"" + "inReplyToScreenName" + "\" : \"" + status.getInReplyToScreenName() + '\'' + " \"" + "geoLocation" + "\" : \"" + status.getGeoLocation() + "\"," + " \"" + "place" + "\" : \"" + status.getPlace() + "\"," + " \"" + "retweetCount" + "\" : \"" + status.getRetweetCount() + "\"," + " \"" + "isoLanguageCode" + "\" : \"" + status.getIsoLanguageCode() + "\"," + " \"" + "user" + "\" : \"" + status.getUser().getName() + "\"," + " \"" + "country" + "\" : \"" + country + "\"" + " }";

TECHNOLOGY - CON’TBETTER -

private void init(JSONObject json) throws TwitterException { id = getLong("id", json); source = getUnescapedString("source", json); createdAt = getDate("created_at", json); isTruncated = getBoolean("truncated", json); inReplyToStatusId = getLong("in_reply_to_status_id", json); inReplyToUserId = getLong("in_reply_to_user_id", json); isFavorited = getBoolean("favorited", json);

TECHNOLOGY - CAMELRouting engineCamel & EIPPatterns like

Aggregator

Splitter

Transform

Multicast

TECHNOLOGY - REAL TIMEWebsocket

Full-duplex singlesocket connection

HTTP request followedby WebSocket dataPackets exchange

ws:// and wss:// protocol

Part of HTML5 initiative

Specification rfc-6455(Dec-2011) managed byIETF

TECHNOLOGY - REAL TIME - DEMO1Integrated within ActiveMQ & Camel

More info : Develop Real Time HTML5 applications using WebSocketwith Apache Camel & ActiveMQ - CamelOne 2012

TECHNOLOGY - DATA METRICS

Kibana & ElasticSearch Insight Technology

Collect Logs, Camel metrics …

TECHNOLOGY - DATA METRICS

USE CASES COVERED - DATA METRICS

TECHNOLOGY - DATA METRICSCreate a bean recuperating Message/Exchange using@Header, @Body

Store it using org.fusesource.insight.storage.StorageService

public class StoreService {

private static String ES_TYPE = "insight-tweet"; private static StorageService storageService;

public static void store(@Header("tweet-full") String data) { storageService.store(ES_TYPE, generateTimeStamp(), data); }

public void setStorageService(StorageService storageService) { StoreService.storageService = storageService; }

TECHNOLOGY - DATA METRICSInject Storage service

<?xml version="1.0" encoding="UTF-8" standalone="no"?><blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd">

<!-- Service Service used to send tweets to ES Storage, parse JSON --> <bean id="helper" class="org.devnation.demo.camel.Service"> <property name="storageService"> <reference interface="org.fusesource.insight.storage.StorageService" /> </property> </bean>

</blueprint>

TECHNOLOGY - DATA METRICSCall it from your Camel route

from("twitter://search?type=polling&delay=" + delay + "&useSSL=true&keywords=" + keywords + "&" + getUriTokens()) .routeId("Tweet-Store-WS") .delay(5000)

// We receive the Twitter4J Status that we will use // to extract info text, isfavorited, is Retweeted, geolocation, isoLanguageCode, contributors // and create A JSON message used later on to store it in ES .setHeader("tweet-full").method(Service.class, "getJSONTweet")

// Message is stored using insight in ElasticSearch .bean(Service.class, "store")

TECHNOLOGY - DATA METRICS - DEMO2

TECHNOLOGY - DASHBOARD

Hawtio

Lightweight & modular HTML5 web console with plugins formanaging Java MBeans

Javascript / REST front end jolokia JMX translator

Heart of the new Fuse Management Console

TECHNOLOGY - DASHBOARDExtend Hawtio Dashboard

TECHNOLOGY - DASHBOARDDeclare MBean Interface

public interface SocialMediaMBean {

/* Attributes */ void setTweetsCounter(Integer val); public Integer getTweetsCounter();

/* Operations */ List<String> searchTweets(String keywords) throws TwitterException; String userInfo(String id) throws TwitterException;

TECHNOLOGY - DASHBOARDService Implementation & MBean registration

public List<String> searchTweets(String keywords) throws TwitterException { return twitterService.searchTweets(keywords); }

public void init() { try { if (objectName == null) { objectName = new ObjectName("hawtio:type=SocialMedia"); } if (mBeanServer == null) { mBeanServer = ManagementFactory.getPlatformMBeanServer(); } try { mBeanServer.registerMBean(this, objectName); } catch (InstanceAlreadyExistsException iaee) { // Try to remove and re-register LOG.info("Re-registering SchemaLookup MBean"); mBeanServer.unregisterMBean(objectName); mBeanServer.registerMBean(this, objectName); } } catch (Exception e) { LOG.warn("Exception during initialization: ", e); throw new RuntimeException(e); } }

TECHNOLOGY - DASHBOARD - DEMO3Inject Service & init beans

<?xml version="1.0" encoding="UTF-8" standalone="no"?><blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0">

<cm:property-placeholder id="twitterConfig" persistent-id="twitter" update-strategy="reload"/>

<bean id="twitterFactory" class="org.devnation.demo.service.TwitterFactory"> <property name="consumerKey" value="${consumer.key}"/> <property name="consumerSecret" value="${consumer.secret}"/> <property name="accessToken" value="${access.token}"/> <property name="accessTokenSecret" value="${access.token.Secret}"/> </bean>

<bean id="twitterService" class="org.devnation.demo.service.TwitterService"> <property name="twitterFactory" ref="twitterFactory"/> </bean>

<bean id="socialDataMBean" class="org.devnation.demo.service.SocialMedia" init-method="init" destroy-method="destroy" scope="singleton"> <property name="twitterService" ref="twitterService"/> </bean></blueprint>

TECHNOLOGY - PLUGINCreate your own hawtio plugin

Web project (war, bundle)

Require to register io.hawt.web.plugin.HawtioPlugin withparameters

Context

JS Scripts location

Name and domain (used by controller)

TECHNOLOGY - PLUGIN<?xml version="1.0" encoding="UTF-8" standalone="no"?><blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0">

<bean id="plugin" class="io.hawt.web.plugin.HawtioPlugin" init-method="init" destroy-method <property name="name" value="${plugin-name}"/> <property name="context" value="${plugin-context}"/> <property name="domain" value="${plugin-domain}"/> <property name="scripts" value="${plugin-scripts}"/> </bean>

</blueprint>

TECHNOLOGY - PLUGINDesign Front end (AngularJS)

Send REST request to call Service

$scope.searchTweets = function () { if (Core.isBlank($scope.keywords)) { return; } jolokia.request({ type: 'exec', mbean: SOCIAL.mbean, operation: 'searchTweets', arguments: [$scope.keywords] }, { method: 'POST', success: function (response) { /* Simple Table */ $scope.tweets = response.value.map(function (val) { return { tweet: val }; }); Core.$apply($scope); }, error: function (response) { SOCIAL.log.warn("Failed to search for Tweets: ", response.error); SOCIAL.log.info("Stack trace: ", response.stacktrace); Core.$apply($scope); }

TECHNOLOGY - PLUGIN<div class="row-fluid gridStyle" ng-controller="SOCIAL.FormController"> <h3>Twitter</h3>

<div class="span12"> <div> <form name="inputForm" ng-submit="searchTweets()"> <div> Keyword(s) : <input type="text" class="entry-widget" ng-model="keywords" </div> <p class="span4 centered"> <input type="submit" class="btn btn-success" ng-disabled="!keywords" </p> </form> </div> </div>

<div class="span8"> <div class="gridStyle" ng-grid="tweetsGrid"/> </div>

</div>

TECHNOLOGY - PLUGIN - DEMO3

AGENDAIntroduction

Social Media Hype

Use cases covered

Integration & Camel

Technology

What JBoss Fuse can offer

Conclusion

WHAT JBOSS FUSE CAN OFFER

WHAT JBOSS FUSE CAN OFFER - CON’T

WHAT JBOSS FUSE CAN OFFER - CON’T

TECHNOLOGY - CON’T

AGENDAIntroduction

Social Media Hype

Integration & Camel

Use cases covered

Technology

What JBoss Fuse can offer

Conclusion

CONCLUSIONFuse Technology ready design Social Media Projects

To cover use cases

Real Time,

Data collection,

Fuse Analytics,

BAM

…

QUESTIONSTwitter :

More info : , ,

@cmoulliard

Apache Camel Hawtiofabric8

Code of demos : https://github.com/cmoulliard/devnation-2014-camel