Build-Deploy-Run large scale logging infrastructure for ...
Transcript of Build-Deploy-Run large scale logging infrastructure for ...
![Page 1: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/1.jpg)
Istvan Zoltan Ballok
Hariharan Gandhi
Build-Deploy-Run large scale logging infrastructure for
SAP Cloud Platform and Cloud Applications
GridKa School 2018August 30, 2018, SCC at KIT Karlsruhe, Germany
![Page 2: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/2.jpg)
Istvan Zoltan BallokDeveloper
Jörg BischofDevelopment Manager
Hariharan GandhiDeveloper
![Page 3: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/3.jpg)
SAP is… a long time trusted leader
in transforming businesses
✓ 90,000 employees, 120 nationalities worldwide, in 130 countries,
45% Millennials, 32% Women
✓ 97 of the 100 top universities in the world run SAP solutions
Headquartered
in Walldorf,
Germany
>90,000
employees in
130 countries
Customers in
190 countries
97 of the 100
top universities
in the world run
SAP solutions
76% of the world’s
transaction revenue
touches an SAP
system.
75% of the beer
drunk in the world
is powered by
SAP 23,300+
employees in
research and
development
![Page 4: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/4.jpg)
Our Customers
Produce more than 79% of
chocolates produced in the
world
Our Customers
Brew more than 77% of
beer produced in the world
Our Customers
Produce more than 86% of
sport shoes produced in
the world
![Page 5: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/5.jpg)
What SAP does
![Page 6: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/6.jpg)
Outline
What SAP does
• Business applications
• Technology Platform
What we do
• From Open Source to an Enterprise-ready Cloud Platform
• Cloud Services at Karlsruhe
Technical focus
• Cloud Foundry
• Application Logging pipeline
![Page 7: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/7.jpg)
Infrastructure(as a Service)
Applications
Runtimes
Security
Databases
Server
Virtualization
Server HW
Storage
Networking
Applications
Runtimes
Security
Databases
Server
Virtualization
Server HW
Storage
Networking
Applications
Runtimes
Security
Databases
Server
Virtualization
Server HW
Storage
Networking
Platform(as a Service)
Software(as a Service)
Source: http://cloudacademy.com/blog/cloud-foundry-benefits/
Managed by V
endor
Managed by V
endor
Managed by V
endor
Man
aged
by
you
Man
aged
by
you
![Page 8: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/8.jpg)
“Cloud is about how you do computing,
not where you do computing.”
Cloud Platform
Data Platform Big Data
Business Services
Analytics
Elastic Multi-Tenant
- Paul Maritz
IoT- Platform
IoT-Services
![Page 9: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/9.jpg)
Cloud Foundry
is an Open Source PaaS driven by 60+ strong partners providing choice of
clouds, frameworks, and application services
![Page 10: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/10.jpg)
Cloud Foundry: PaaS
• “Here is my source code,
run it on the cloud for me,
I do not care how ”
`cf push` experience
• Security updates for operating
system, runtimes
and “app servers” (e.g. Tomcat) are
handled by CF
Bare Metal
Apps
uses
Services
Services
![Page 11: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/11.jpg)
Cloud Foundry – from the developer’s perspective
• Create the first version and add a manifest
• Deploy new versions rapidly
cf push myApp
• Assemble services to create integrated applications
cf create service dbService medium myDB
cf bind-service myApp myDB
• Scale applications
cf scale myApp -i 100
Don’t worry about underlying operating systems. Easier backup and recovery
strategies using platform functionality.
http://bit.ly/gridka-chat
![Page 12: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/12.jpg)
Cloud Foundry Architecture
https://docs.cloudfoundry.org/concepts/architecture/
![Page 13: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/13.jpg)
Cloud Foundry, Diego Architecture
https://docs.cloudfoundry.org/concepts/diego/diego-architecture.html
![Page 14: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/14.jpg)
CF Loggregator Architecture
https://docs.cloudfoundry.org/loggregator/architecture.html
![Page 15: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/15.jpg)
What we do
![Page 16: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/16.jpg)
Application Loggingmaking lives of developers easier
![Page 17: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/17.jpg)
ES ClusterLoggregator QueueES
ClusterLog
ForwarderParser Web
App KE L
Quota andBurst management
CF applicationlogging subsystem
Buffer / Peak load handling
Log Parsing
Distributed Search andAnalytics
UI and Security
Application LoggingPipeline
![Page 18: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/18.jpg)
ES ClusterLoggregator QueueES
ClusterLog
ForwarderParser Web
Demo Chat App
Let’s check if the demo works?
http://bit.ly/gridka-chat
Try it! https://cloudplatform.sap.com
![Page 19: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/19.jpg)
• We can not use Cloud Foundry Application Runtime to deploy stateful services
• Bosh
• The same tool that is used to deploy and manage the lifecycle of Cloud Foundry itself
BOSH is a project that unifies release engineering, deployment, and lifecycle management of small
and large-scale cloud software.
• Immutable infrastructure (pet vs. cattle)
• Infrastructure agnostic (CPI)
• Bosh release, jobs, packages
• Deployment manifest
• Rolling upgrade
• Health checks
How do we deploy the logging stack?
![Page 20: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/20.jpg)
Why is parsing needed?
cf-logging-monitor.cf.sap.hana.ondemand.com - [2018-08-29T12:58:52.421+0000] "GET /health HTTP/1.1" 200 0
129 "-" "Apache-HttpClient/4.5.3 (Java/1.8.0_172)" "-" "10.0.137.45:62962" x_forwarded_for:"-"
x_forwarded_proto:"https" vcap_request_id:"72fdd51d-eb6d-48cb-4001-69681d02573e" response_time:0.00129951
app_id:"26f8ad18-eddf-41df-9f38-6f7d44036867" app_index:"0" x_b3_traceid:"9ca7e3e7b875fca7"
x_b3_spanid:"9ca7e3e7b875fca7" x_b3_parentspanid:"-"
{
"msg": "cf-logging-monitor.cf.sap.hana.ondemand.com - [2018-08-29T12:58:52.421+0000] \"GET /health HTTP/1.1\" 200 0 129 \"-\"
(Java/1.8.0_172)\" \"-\" \"10.0.137.45:62962\" x_forwarded_for:\"-\" x_forwarded_proto:\"https\" vcap_request_id:\"72fdd51d-eb6d
response_time:0.00129951 app_id:\"26f8ad18-eddf-41df-9f38-6f7d44036867\" app_index:\"0\" x_b3_traceid:\"9ca7e3e7b875fca7\"
x_b3_spanid:\"9ca7e3e7b875fca7\" x_b3_parentspanid:\"-\"",
"referer": "-",
"request_size_b": 0,
"channel": "OUT",
"type": "request",
"remote_user": "-",
"protocol": "HTTP/1.1",
"remote_ip": "\"-\"",
"request_received_at": "2018-08-29T12:58:52.421Z",
"component_id": "26f8ad18-eddf-41df-9f38-6f7d44036867",
"method": "GET",
"level": "INFO",
...
![Page 21: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/21.jpg)
Emit structured logshttps://cloudplatform.sap.com/enterprise-paas/cloudfoundry/cf-contributions.html
![Page 22: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/22.jpg)
Inside Elasticsearch
![Page 23: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/23.jpg)
Pain Points, Challenges
• Regular Expr performance during parsing
• Rolling update of Elasticsearch and all components staying ZDM
• Resizing persistent disks used by Elasticsearch - attached volumes cannot be expanded by Bosh
• Tuning for indexing performance in Elasticsearch (async flush to disk)
• Quota to protect the stack
ES ClusterLoggregator QueueES
ClusterLog
ForwarderParser Web
App
![Page 24: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/24.jpg)
How do we ship it?
![Page 25: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/25.jpg)
8
A team of
DevOps
working in a
to multiple live landscapes on
SAP DC, AWS, GCP, Azure
people mode
2 weeks
shipping every
![Page 26: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/26.jpg)
DevOps
development
and testingdelivery and
deploymentOperations
Service Ownership, Full
responsibility
Release Operate
Test Deploy Monitor
Build
![Page 27: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/27.jpg)
SAP Cloud Platform:
Centerpiece of the Intelligent Enterprise
![Page 28: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/28.jpg)
Integration &
Orchestration
Data Privacy &
SecurityUX & Mobile
Dev XP &
DevOpsCollaboration
Data
Management
*planned innovations / future direction
UI5 / Fiori Mobile Co-PilotWebIDE/
Cockpit
App Center/
Marketplace
APIs & Events
Commerce Globalization Master DataRates &
Measures
LoBs &
Industries…
AnalyticsMachine
LearningBlockchain IoT
Data
IntelligenceBig Data
Knative *
SAP Cloud Platform: An ongoing journey,especially around Open Source technologies
![Page 29: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/29.jpg)
Questions?
![Page 30: Build-Deploy-Run large scale logging infrastructure for ...](https://reader030.fdocuments.us/reader030/viewer/2022012700/61a35a622bd41403f3643db6/html5/thumbnails/30.jpg)
DEVELOPERS WANTED
Kubernetes, Knative
Cloud Foundry
Security
Performance
Cloud Cockpit
Logging
Big Data
PersistenceLINK TO JOB LISTING