Steve RatayChief Architect, Teradata ViewpointApril 2, 2013
TERADATA VIEWPOINTHADOOP INTEGRATION WITH AMBARI
2 Copyright Teradata 2012
• Combination of Teradata Database, Aster, and Hadoop in a single appliance• Connectors between the different systems• Monitor the entire appliance with Teradata Viewpoint
Teradata Unified Data Architecture
DISCOVERY PLATFORM
CAPTURE | STORE | REFINE
INTEGRATED DATA WAREHOUSE
Aster Connector for Hadoop Teradata Connector for Hadoop
Aster Teradata Connector
SQL-H SQL-H
3 Copyright Teradata 2012
Viewpoint Architecture
TD 1
DataCollectionService
PostgresDatabase
ViewpointPortal
Viewpoint Managed Server
ActiveMQ
AlertService
TD n
Aster 1
Aster n
HDP 1
HDP nEmail SNMP Portlet
4 Copyright Teradata 2012
• Different technologies and formats:> Ganglia> Nagios> JMX> Screen scraping HTML
• Connection to multiple nodes:> Connectivity to all nodes> Access to several ports> Location of each service, especially when using high availability
Monitoring Hadoop without Ambari
5 Copyright Teradata 2012
• RESTful APIs• JSON response format• Single server and port
Ambari to the Rescue!
6 Copyright Teradata 2012
• Viewpoint uses Java and Spring• Used Spring’s RestTemplate class:
> Makes HTTP call to specified URL> Binds JSON results to Java model objects
• Data is collected every minute and stored in Viewpoint’s Postgres database> Use of Rewind to view previous state of system> Easy generation of trending charts and graphs
Accessing Ambari APIs
7 Copyright Teradata 2012
System Health
8 Copyright Teradata 2012
Metric Analysis
9 Copyright Teradata 2012
Node Monitor
10 Copyright Teradata 2012
Hadoop Services
11 Copyright Teradata 2012
Hadoop Alerts
12 Copyright Teradata 2012
• Viewpoint delivered comprehensive Hadoop monitoring solution in a couple of months• Java and web developers focused on collecting data and
building UI• Didn’t waste time dealing with intricacies of Hadoop and
Hadoop monitoring
Benefits of Using Ambari
Top Related