Ambari Meetup: 2nd April 2013: Teradata Viewpoint Hadoop Integration with Ambari
-
Upload
hortonworks -
Category
Documents
-
view
2.613 -
download
4
description
Transcript of Ambari Meetup: 2nd April 2013: Teradata Viewpoint Hadoop Integration with Ambari
Steve RatayChief Architect, Teradata ViewpointApril 2, 2013
TERADATA VIEWPOINTHADOOP INTEGRATION WITH AMBARI
2 Copyright Teradata 2012
• Combination of Teradata Database, Aster, and Hadoop in a single appliance• Connectors between the different systems• Monitor the entire appliance with Teradata Viewpoint
Teradata Unified Data Architecture
DISCOVERY PLATFORM
CAPTURE | STORE | REFINE
INTEGRATED DATA WAREHOUSE
Aster Connector for Hadoop Teradata Connector for Hadoop
Aster Teradata Connector
SQL-H SQL-H
3 Copyright Teradata 2012
Viewpoint Architecture
TD 1
DataCollectionService
PostgresDatabase
ViewpointPortal
Viewpoint Managed Server
ActiveMQ
AlertService
TD n
Aster 1
Aster n
HDP 1
HDP nEmail SNMP Portlet
4 Copyright Teradata 2012
• Different technologies and formats:> Ganglia> Nagios> JMX> Screen scraping HTML
• Connection to multiple nodes:> Connectivity to all nodes> Access to several ports> Location of each service, especially when using high availability
Monitoring Hadoop without Ambari
5 Copyright Teradata 2012
• RESTful APIs• JSON response format• Single server and port
Ambari to the Rescue!
6 Copyright Teradata 2012
• Viewpoint uses Java and Spring• Used Spring’s RestTemplate class:
> Makes HTTP call to specified URL> Binds JSON results to Java model objects
• Data is collected every minute and stored in Viewpoint’s Postgres database> Use of Rewind to view previous state of system> Easy generation of trending charts and graphs
Accessing Ambari APIs
7 Copyright Teradata 2012
System Health
8 Copyright Teradata 2012
Metric Analysis
9 Copyright Teradata 2012
Node Monitor
10 Copyright Teradata 2012
Hadoop Services
11 Copyright Teradata 2012
Hadoop Alerts
12 Copyright Teradata 2012
• Viewpoint delivered comprehensive Hadoop monitoring solution in a couple of months• Java and web developers focused on collecting data and
building UI• Didn’t waste time dealing with intricacies of Hadoop and
Hadoop monitoring
Benefits of Using Ambari