The Art of Picking Your Poison - Amazon Web...

Open Source Across the Enterprise

The Art of Picking Your PoisonEugene Ciurana,

Director, Systems Infrastructure

Leap Frog Enterprises

May 23, 2007



Who is this Eugene character? 15+ years of experience building mission-critical, high-

availability systems infrastructure• Not an end-user applications kinda guy

10+ years of Java work

Engaged by the largest companies in the world• Retail

• Finance

• Oil industry

Background ranges from industrial robotics to on-line retail systems


And about open-source...? Eugene’s open-source professional experience goes

back to 1997 Contributed code to various projects

Operating systems: Linux Java: Jetty servlet container Cross-platform tools: awke

Advocacy OpenOffice.org - Computerworld anti-FUD from Microsoft article 2004 Mule ESB Linux, Apache, CakePHP, RoR, others

Enterprise Introduced Linux to Wal-Mart Stores in 2006 - first official open-

source product for production! Seeded first open-source education program there too - SourceLabs

Geek!

Suit!

Troll!


In the beginning...A surgeon, a civil engineer, and an enterprise architect argued about whose profession came up first:

“Mine,” said the surgeon. “God created Eve from one of Adam’s ribs.”

“Wrong! The first profession was civil engineering,” the next guy interrupted. “Remember that God first created the universe from the chaos.”

“You’re both wrong,” the enterprise architect said. “Who do you think created the chaos?”


What is the Architect’s Job?• Assuming that you are a system architect...

• ...what’s the job description?

• Architecting is not only about programming

• Excellent coders are not, by default, excellent architects

•This is something that you grow into

•Architects must have a multi-disciplinary background

•Being a Java architect is a bit of an oxymoron

•Architects must articulate their works...

•...in technological terms

•...in business terms

•Many tech folks want to become architects... why?


What is the Architect’s Job?•Commercial software is not the right answer

•Open-source software is not the right answer

•Mixed environments are the right answer!

• The architect must have a mix of skills

•In the past, tech kung f00 + problem domain knowledge was enough

•Today, the architect needs that, plus understanding of legal implications of open-source

The best enterprise environments are a combination of commercial and open-source software. The right balance is up to you.


What is the Architect’s Job?•Commercial software is not the right answer

•Open-source software is not the right answer

•Mixed environments are the right answer!

• The architect must have a mix of skills

•In the past, tech kung f00 + problem domain knowledge was enough

•Today, the architect needs that, plus understanding of legal implications of open-source

The best enterprise environments are a combination of commercial and open-source software. The right balance is up to you.

In an open-source world, the architect must not only help in shaping the enterprise infrastructure

and applications. The architect must lead in shaping the company’s policies.


A Warning...

Avoid Marketectures

Don’t let the vendors, commercial or open-source,

bamboozle you.


Architectures...

•...are often confused with frameworks• In some cases, framework and architecture map 1:1

• In other cases, vendors like to fan the confusion

•Is your architecture tied to a specific vendor or technology? Is this a good idea?

•Architectures are vendor- and technology-agnostic• Lay out the architecture, then evaluate technology, and then figure out

which products will become part of it

• J2EE?

• .Net?

• Open-source products?

• Why?


Open-source/Cutting Edge Software•Larger enterprises sometimes don’t benefit

from using these tools•Corporate policy gets in the way

•Fear of litigation

•Aversion to being first

•Inertia

•Vendor lock-in

•The smart companies, though, figure out ways of implementing these•Pilot programs

•Partnerships


Open-source/Cutting Edge Software• How do you go about changing corporate policy?

• Through education!• Most large IT organizations are too busy with day-to-day business to

pay attention

• Many lack, or forego, ongoing formal training for their engineering and engineering management groups

• Remember: vendors have their own best interest at heart!

• Seek education from outside sources/third-parties who are in that business, not in the business of selling software• Open Source Initiative

• Trade magazines

• WWW: TheServerSide, Slashdot, Reddit, java.net, Digg, InfoWeek, etc.

Are you spending at least 30

minutes researching new technologies and trends every day?

Why?


Open-source/Cutting Edge Software•Multiple tiers

•Each tier is implement on different technology

•Coexistence of legacy and new systems

•Different skill sets for each tier

•Multiple physical locations•Ad hoc interfaces between locations

•Multiple protocols

•Data and process consolidation•Mainframes are at the core

•Cross-coupling between systems makes it difficult to add new functionality

•New integration projects == lengthy development cycle


Typical Application Architecture

• Systems are built “as needed”• Point-only integration

• New systems require new interface definition• Interface duplication or incompatibility

• Queuing (MQ Series, JMS, etc.) seen as silver bullets• It works as long as all the participants support queuing

DatabaseMid- or front-tier

Server

Provider

poll, query

push, update

XML

EDI

CSV

Data dispatch in-line

Architecture evolved toward maximum flexibility

and quick development turn-around

Perl, Java, PHP, etc.many discrete modules


Typical Application Architecture

• Systems are built “as needed”• Point-only integration

• New systems require new interface definition• Interface duplication or incompatibility

• Queuing (MQ Series, JMS, etc.) seen as silver bullets• It works as long as all the participants support queuing

DatabaseMid- or front-tier

Server

Provider

poll, query

push, update

XML

EDI

CSV

Data dispatch in-line

Architecture evolved toward maximum flexibility

and quick development turn-around

Perl, Java, PHP, etc.many discrete modules

What’s the problemwith

Point-to-Point?


Point-to-Point Problem: Case Study Top-3 in the US financial operation/Integrion

On-line banking system has a three-tier architecture• Web browser front-end

• AIX/app server middle-tier

• Mainframe middle-tier

• Mainframe back-end

MQ Series was chosen as the common conduit...• Brittle data format

COBOL-like data structures, positional

• Not scaleable

• Lots of processing overhead because of native-to-uniform data formats

It worked, but it became very expensive to operate and enhance


Internet

AIX app server

JavascriptJavaOther

WebBrowser

Javascript

Bank Mainframe

zOSCOBOL

PL/IYUCK!

Service Provider

Mainframe

zOSCOBOL

PL/I

Other BankApplications

Windows ServerSQLServer

etc.

MQ Series Common Conduit

MQ everywhere!

MQ MQ

MQ MQ

MQMQ

Point-to-Point Problem: Case Study


Every problem becomes a new project

Different systems may support different interoperability protocols• JMS

• SOAP

• REST

• BPEL

• etc.

Point-to-point integration is more expensive• Consumes more resources

• Requires more regression testing

• Creates lock-in/artificial dependencies between systems

Point-to-Point Problem


Point-to-Point Problem

Applications run on different platforms

• Mainframes

• Windows

• *NIX

Adding a new application and protocol (i.e. BPEL) requires extensive changes

Some applications become hubs... and potential bottlenecks.

ApplicationA

ApplicationF

ApplicationE

ApplicationC

ApplicationB

ApplicationD

JMS

JMS

JMS

SOAP

REST

SOAP

FTP


You picked your applications

You leveraged open-source options

Next comes the fun part: Integrating all application and subsystems to play together

Assumption: each of the systems in the mix supports at least one way of communicating with the outside world• Database

• SOAP

• BPEL

• REST

• FTP

• JMS/queueing

• JavaSpaces

Et cetera!

Solving the Integration Problem


Solving the Integration Problem It’s buzzword time!

SOA = service-oriented architectures

Web services are *part* of this

It means that functionality is deployed in dedicated systems that interact with one another asynchronously over some transport

ROC = resource-oriented computing

Think of it as SOA 2.0

Services provide not only data but computational capability


Solving the Integration Problem Shift effort from software development and data replication and

consolidation toward integration

Don’t throw away your enterprise applications

They are domain-specific

Adopt new technology quickly

The fastest pace of innovation now is in the open-source ecosystem

Open-source software enables SOA and ROC

Good open-source products come about to solve a problem, not to sell frameworks, products, or services


Solving the Integration Problem Shift effort from software development and data replication and

consolidation toward integration

Don’t throw away your enterprise applications

They are domain-specific

Adopt new technology quickly

The fastest pace of innovation now is in the open-source ecosystem

Open-source software enables SOA and ROC

Good open-source products come about to solve a problem, not to sell frameworks, products, or services

Hibernate vs. EJBsMule ESB vs. IBM Websphere ESB


Solving the Integration Problem Would you use open-source software for everything?

Why?

In-house applications and legacy - leave them where they are and continue their development/maintenance cycle

Commercial software addresses domain-specific problems when is cheaper to acquire than to build in-house

Open-source software works best for satisfying infrastructure requirements

Operating systems Network transports Programming languages

Development frameworks Caching systems etc.

Why?


Why is Open-source Better for Infrastructure?

Successful open-source projects attract many developers from all walks of life

Most ARE NOT domain-specific Specialization is expensive

The ecosystem evolves to become commercially viable as it thrives, giving birth to support subscription models

Examples: Mule ESB Spring framework Apache projects Hibernate Linux


Hardware and operating system(Solaris)

Java Virtual Machine

(App server, caching systems, filtering, search, single sign-on technology)(Spring, Glassfish, JBoss, other)

Product CMS

Database interfaces

Services, iBatis, Hibernate, etc.

3rd party interfaces as

services(ESB, SOA)

Vendor application

(CRM, CMS, PIM)Web site

(Wicket, tagging, templates,etc.)

In-house customization

to vendor tools

(Ruby on Rails, PHP)

Off-the-shelf application infrastructure(Oracle apps, SAP, other third-parties)

Application-specific components and services

Open-source Architecture


Open-source Architecture

Mule ESB

SOAP, REST, JMS, in-memory messaging

MySQL

Database

Workflow, BPEL

applicationS

OpenSSO, OpenID, other authentication systems

MySQLCustomOpen

LDAP

Internet

HTTPHTML

Webservices

SOAP

ERP systems,

other

databases

Content

Management

System

Spring and server application

mycompany.com

ORMServlet container

Packages

HTML

etc.

binary

data

REST

WebServices

auth

Database

Business

Intelligence

Third-party.com

Web client (Firefox) or dedicatedapplication

Other

HTTP / HTML


Conclusions

Architecture is about figuring out what and why first Avoid marketectures

Open-source is one of the answers to “how” Commercial In-house

Open-source is as much as policy and governance issue as a technological one

Shift effort toward integration rather than acquisition or development

Open-srouce works best for infrastructure Domain-specific, specialized apps? In-house or commercial work better


Q&AThanks for coming!

Eugene CiuranaDirector, Systems InfrastructureLeapFrog Enterprises, Inc.

Download this presentation from:http://eugeneciurana.com/OSBC2007

[email protected] @ irc://irc.freenode.net ##java, #esb, #awk, #security

The Art of Picking Your Poison - Amazon Web...

Documents

Transcript of The Art of Picking Your Poison - Amazon Web...