Post on 07-Jan-2020
Keep an eye on yourPostgreSQL clusters
Open PostgreSQL Monitoring & PostgreSQLWorkload Analyzer
Julien Rouhaud
Dalibo - www.dalibo.org
pgconf.ru 2015 - February
[ 1 / 39 ]
Monitoring?
I Service availabilityI Service, host, network...I Alerting
I Service performanceI GraphingI TrendingI Analysis
[ 2 / 39 ]
At what cost?
I Open source projects
[ 3 / 39 ]
What already exists
I Different kind of toolsI Command line tools / datasourcesI Generic solutions with probesI Dedicated solutions
[ 4 / 39 ]
Command lineAnd data sources
I Nice featuresI But more useful for emergency situationsI Or need some external tool for best usage
[ 5 / 39 ]
Command lineSome examples
I command lineI pg_viewI pg_activityI pgstats
I Data sourceI all pg_stat* catalogI pg_stat_statements, pg_stat_plansI pg_proctab
[ 6 / 39 ]
Probe
I Gather informations for another toolI Can usually be used in standalone
[ 7 / 39 ]
ProbeExample
I check_postgresI pg_monz
[ 8 / 39 ]
Dedicated solution
I Complete for its purposeI More pertinentI But not numerous
[ 9 / 39 ]
OPM and PoWA?
I OPMI aim to be a dedicated solutionI relies on existing or new probes and components
I PoWAI Dedicated performance solution
[ 10 / 39 ]
EcosystemGeneric solution based on probes
Name Native graphing AlertingNagios No YesZabbix Yes YesMunin Yes YesCacti Yes No
[ 11 / 39 ]
General Overview
I ProsI Robust and matureI AdaptableI Extendable
I ConsI UI not flexibleI Data not available for queryingI Except check_postgres, lack of really completedatasource
[ 12 / 39 ]
NagiosSpecific
I Graphing (with third part tool like OPM V2)I PostgreSQL compatibility with
I check_postgres (Bucardo)I check_pg_activity (OPM)
I Hard to configure
[ 13 / 39 ]
ZabbixOverview
I PostgreSQL compatibility withI pg_monz (only PG 9.2+)I postbix (no news since 2013)
[ 14 / 39 ]
MuninOverview
I Native PostgreSQL compatibility, and withI pyMunin
[ 15 / 39 ]
CactiOverview
I PostgreSQL compatibility withI Some workaround with check_postgres (MRTGformat)
[ 16 / 39 ]
What about OPM
I And for Open PostgreSQL Monitoring ?
[ 17 / 39 ]
OPMArchitecture
I NagiosI SchedulerI Alerting
I ProbesI PG specific : check_pgactivityI And monitoring-plugin
I StorageI PostgreSQL :)I 9.3 or more
I GUII Dedicated GUI
[ 18 / 39 ]
OPMOverview
[ 19 / 39 ]
check_pgactivityDesign
I Written from scratch, simpler codeI Provide lots of servicesI Handle a small cacheI Better perfdata
[ 20 / 39 ]
check_pgactivityNew features
I Handle multi-database connections transparentlyI Can compute delta instead of raw valuesI Handle units
[ 21 / 39 ]
check_pgactivityNew service examples
I bgwriter statisticsI better bloat estimationI Most important settings check
[ 22 / 39 ]
check_pgactivityEnhanced service
I better bloat estimationI replication : pg_stat_replication or hot standbyI "Instant" hit ratio
[ 23 / 39 ]
User interface
I Specific ACLI More modern graphsI Custom graph display
[ 24 / 39 ]
EcosystemSpecific solution
Name Maintained Aim Alertingpg_statsinfo Yes Generalist Possiblepg_watch Dead? Generalist NopgObserver Yes Performance NopgCluu Yes Generalist NoPoWA Yes Performance No
[ 25 / 39 ]
pg_statsinfoPros & cons
I ProsI Useful metrics
I ConsI Reports on demandI Needs specific extension and reboot on eachmonitored server
[ 26 / 39 ]
pgObserverPros & cons
I ProsI Useful metricsI And novel metrics (like stored proc)I Dynamic reports
I ConsI Focused on performance
[ 27 / 39 ]
pgCluuPros & cons
I ProsI Useful metricsI General overviewI Easy to use
I ConsI Can be storage greedyI Need to generate reports
[ 28 / 39 ]
PoWA 2.0
I A complete solutionI Handle a lot of datasourceI Requires PG 9.4 or more
I requires queryid exposure, sincepg_stat_statements 1.2
[ 29 / 39 ]
What is PoWA
I A background workerI Dedicated snapshot, aggregation and purgefunctions
I Dedicated UI
[ 30 / 39 ]
PoWAExtensions handled
I Existing extensionsI pg_stat_statementsI pg_proctab (WIP)
I New oneI pg_qualstatsI pg_stat_kcacheI pg_track_settings (WIP)
[ 31 / 39 ]
pg_qualstatsPresentation
I Gather real-time statistics on where clauses perquery
I SeqScan / indexScanI OperatorI Number of executionI Filter ratioI . . .
[ 32 / 39 ]
pg_qualstatsWhat it allows
I Find missing indexesI Which could be partialI Or on several columns if some other queries couldbenefit from it
I . . .
[ 33 / 39 ]
pg_qualstatsWhat it allows
I EXPLAIN normalized queries with real valuesI Most frequentsI Most/Least filteringI . . .
[ 34 / 39 ]
pg_stat_kcachePresentation
I Gather real-time per query system statistics
[ 35 / 39 ]
pg_stat_kcacheWhat it allows
I Compute a real hit ratio (shared_buffers, systemcache and disk)
I Show CPU consumptionI global, per database and/or per query and/or peruser
I over time
[ 36 / 39 ]
Demo
I
[ 37 / 39 ]
For the future
I Nagios optional from OPMI Add ability to handle PoWA in OPMI Better use of metrics
I Trending / statistic analysisI Correlate informations
I Better index suggestion, global database analysisI And much more. . .
[ 38 / 39 ]
Questions?
I julien.rouhaud@dalibo.comI contact@opm.io - powa@dalibo.comI github.com/OPMDG - github.com/dalibo/powaI Thank you !
[ 39 / 39 ]