Managing Resources with PostgreSQL · Managing Resources with PostgreSQL FOSDEM PGDay 2015 Cédric...
Transcript of Managing Resources with PostgreSQL · Managing Resources with PostgreSQL FOSDEM PGDay 2015 Cédric...
Managing Resources with PostgreSQL
FOSDEM PGDay 2015
Cédric Villemain [email protected]
30th January 2015
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 1 / 24
Cédric Villemain
PostgreSQL Expertise
Development, Support, Training
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 2 / 24
What resources ?
• CPU
• RAM
• Disk
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 3 / 24
How ?
• with .... Concurrency
• with .... Quota or Limit
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 4 / 24
Where ?
Operating System
• user process (limits.conf, ...)
• container (cgroup, jail, ...)
• virtualization (VMWare, Xen, ...)
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 5 / 24
Where ?
PostgreSQL
• Main con�guration (postgresql.conf)
• per object
• per role
• per database
• per role in database
• pg_service
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 6 / 24
GUCs with restart
• max_connections
• shared_bu�ers
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 7 / 24
GUCs without restart
• temp_bu�ers
• work_mem
• synchronous_commit
• temp_tablespace
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 8 / 24
More GUCs without restart
• temp_�le_limit
• statement_timeout
• lock_timeout
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 9 / 24
Resources per object
• FUNCTION
• ROLE
• DATABASE
• SYSTEM
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 10 / 24
Function example
ALTER FUNCTION update_datamart()
SET temp_file_limit = -1;
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 11 / 24
Database example
ALTER DATABASE datawarehouse
WITH CONNECTION LIMIT 10; --hard limit
ALTER DATABASE datawarehouse
SET temp_tablespace TO dwh_tblspc;
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 12 / 24
Role example
ALTER ROLE web_user
WITH CONNECTION LIMIT 10; -- hard limit but ...
ALTER ROLE dba
SET work_mem = '128MB';
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 13 / 24
Role in Database example
ALTER ROLE ALL IN DATABASE devel
SET synchronous_commit TO off;
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 14 / 24
System example
ALTER SYSTEM
SET temp_buffers = '12MB';
SELECT pg_reload_conf();
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 15 / 24
Connection Limit
• PostgreSQL �ood & DoS
• pooling & bouncing
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 16 / 24
PgBouncer
• pool_size = 20
• max_client_conn = 2000
my_db = user=web_user host=remote_host pool_size=4
• bonus: PAUSE / RESUME
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 17 / 24
top-like
• pg_activity (python, system only)
• pg_top (C, PostgreSQL extension)
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 18 / 24
pg_proctab
• pg_cputime()
• pg_loadavg()
• pg_memusage()
• pg_proctab()
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 19 / 24
Execution depends on loadavg
if (select load1<1 from pg_loadavg())
then update_datamart();
end if;
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 20 / 24
What about e�ective_cache_size ?
select pg_size_pretty(memcached * 4096)
from pg_memusage;
-- don't forget to count shared_buffers
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 21 / 24
Are we writing a lot on disk?
select pg_size_pretty(wchar) as requested_write,
pg_size_pretty(writes) as really_written,
(writes * 100 / wchar) as percent_really_written
from pg_proctab()
where pid = pg_backend_pid())
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 22 / 24
And next ?
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 23 / 24
Questions ?
Now is the time to ask!
Cédric Villemain [email protected] () Managing Resources with PostgreSQL 30th January 2015 24 / 24