Visuel à insérer ici

WEB SUMMER CAMP 2016

Rovinj

eZ Publish 5: from zero to automated deployment

(and no regressions!) in one afternoon

WHO AM IThe short version

• « the guy who has written a lot of answers

on the eZPublish forums »

• Working at Kaliop in London since 2014

• Principal consultant at eZSystems for 7 years

• Love coffee and bicycles ?

• @gggeek

• Little known fact: today it’s my birthday!

VOTRE SCHEMA

WHAT THIS WORKSHOP IS ABOUTThe short version, too

TALK IS CHEAPShow me the code

https://github.com/kaliop-uk/websummercamp2016

HOUSE RULESMarek and Jakub are the Lethal Enforcers Minions Gentle Assistants

• Interrupting is welcome at any time

• fall behind: the assistants are authorized

to stop me for all the time needed for you

to catch up with the rest

• fall asleep: the assistants are authorized to

kill you empty your pockets pinch your ear

WHO ARE YOU?The short version, please

• Any experience with eZPublish ?

• Any experience with eZPublish Legacy?

• Did anybody attend the Docker workshop

from Marek on Wednesday ?

• Any Docker experience at all ?

• Who has been up all night partying?

VOTRE SCHEMA

SUMMARY

1. INTRO TO DOCKER

2. (MOAR) DOCKER!

3. MANAGING ENVIRONMENTS

4. MANAGING DB CHANGES

5. A DEPLOYMENT SCRIPT

6. TESTING THE SITE

If you have never heard of used

Docker

A. I am living under a rock

B. I am living on the moon

01

What is it, in one phraseDefinition according to Me™

"just like a VM, but lighter"

Virtual Machines vs. ContainersVM on the left

Getting started is easyGetting off the hook on the other hand…

"just like a VM, but lighter"

all it takes to get started:

docker run -ti ubuntu:15.10 bash

Getting started is easy

what has just happened:

• the Docker daemon connected to a public repository to download an

image (https://hub.docker.com/)

• a container was started using that image

• the bash program was started in the container, and its I/O attached to

the shell in execution

Getting started is easy

• ideal for testing a command / script in a different environment

• makes it very easy to deploy (some types of) applications as self-

contained packages

• not created for multi-process applications, nor for Continuous

Integration scenarios

• but it is getting there quickly

Terminology

• container: a container is a runtime instance of a Docker image

• image: Docker images are the basis of containers (recursion?)

• an image is an ordered collection of root filesystem changes and the

corresponding execution parameters for use within a container runtime

• dockerfile: a Dockerfile is a text document that contains all the

commands you would normally execute manually in order to build a

Docker image

• volume: a data volume is a specially-designated directory within one or

more containers that bypasses the Union File System

• in other words: a volume is a directory on the host which gets mounted into the

container

Sample DockerfileReference at: https://docs.docker.com/engine/reference/builder/

FROM UBUNTU:15.10

RUN apt-get update && apt-get install -y php5 php5-curl php5-gd

ADD src/ /var/www/html/

ADD scripts/start.sh /start.sh

EXPOSE 443 80

CMD ["/start.sh"]

Docker philosophyDefinition according to Me™, again

• one process per container

• containers should be immutable (or close to it)

Useful one-liners

• docker images lists all images present on the host computer

• docker ps lists all running containers

• docker ps -a lists all containers, including stopped ones

• docker rm <cont.> removes a container

• docker rmi <image> removes an image

Where the magic stops

• how to share application data with the container?

• how to access the apps in the container (map container ports to the

host)?

• how to start multiple containers together and connect them?

• how to access the log files from the application?

Then things get tricky

• mapping of user ids to properly handle filesystem permissions

• linking containers when there are circular dependencies (solved with

recent Docker versions)

• applying security updates to existing containers

• and more…

• for some more of the fine details, see f.e. this blog post:

http://blog.kaliop.com/en/blog/2015/05/27/docker-in-real-life-the-tricky-

parts/

Docker Compose to the rescue!

• a single file describes multiple containers

• a single command to start/stop them all together

• permits further automation (via eg. introduction of variables)

Sample docker-compose.ymlReference docs: https://docs.docker.com/compose/compose-file/

version: '2'

services:web:

build: images/webhostname: webports:

- "80:80"volumes:

- ./site/:/var/www/siteenv_file:

- docker-compose.env

cli:build: images/clihostname: clienvironment:

- SSH_AUTH_SOCK=/ssh-agentcap_add:

- SYS_PTRACE

A Docker stack for eZPublish

A. I need a break

B. Bring it on!

02

Why use Docker for eZPublish website development?Because you get to go to the Summer Camp, duh!

• easier to share with a distributed dev team than a whole VM

just a bunch of scripts, not multi-MB image

easy to fix/improve and redistribute

• easy to convert to the same version of the OS/Apps used in production

• forces separation of applications to separate environments

closer to a production environment (for non-trivial sites)

good to avoid surprises when deploying

Introducing the stack

• apache

• mysql

• memcached

• solr

• php cli

• varnish

• nginx

plus

• control panels for all of the above

• phpmyadmin

• maildev (an smtp server for development purposes)

• dockerui (a container monitoring the whole container stack [Inception!])

In 1000 wordsAscii art rocks!

How it worksAll the things someone else has figured out for you

• configuration of the applications is mounted as volume, not copied into

the image

a simple service restart is sufficient to take changes into account

• application source code is kept on the host computer

no risk of losing changes when rebuilding containers

no need to have containers running GUI or IDEs

• all log files accessible from the host, in one place

• use ssh-agent to avoid developers storing their own private keys inside

the containers

How it works, IIAll the things someone else has figured out for you

• allow developers to change some variables, while providing good

defaults

a configuration file is provided, which has all the variables that developers

can customize

a 2nd configuration file has to be created with local changes (empty is fine)

ex: id of the user account on the host computer, or github credentials

the config file with local changes is not committed to git

• not tied to a specific project

can be quickly converted for usage of Drupal instead of eZPublish

two of them can be run in parallel

How it works, IIIAll the things someone else has figured out for you

• the containers have to replicate as much as possible a production

environment

the base images used are Debian or Ubuntu (the platforms we generally

deploy on)

the applications are started using service start and service stop

commands

cronjob tasks definitions are kept as part of the sources of the stack

On Disk layout

• config configuration for services running in containers

• data data persisted by the apps in containers (eg: mysql db)

• images Dockerfiles and files used to build the images

• logs log files from services running in containers

• site the website installation folder

• src source code for tools which are part of the stack

On Disk layout and volume mounts( incomplete example )

• config/…

• data/…

• images X

• logs/…

• site

• src X

Mysql container

/etc/mysql/conf.d/

/var/lib/mysql/

/var/log/mysql/

Web container

/etc/apache2/sites-available

/etc/php5/apache2/

/var/log/apache/

/var/www/site/

Cli container

/tmp/cron.d

/etc/php5/cli

/var/log/php/

/var/www/site/

Exercise timeAt last!

Managing environments

A. Not my job

B. DevOps is the new black

03

Where to store per-environment configuration files

• Symfony: excellent native support

• eZPublish legacy: somewhat, hackish support (eg. in extensions?)

• Webserver config etc… : no standard solution

All-in-the-stack solution

• Store eZPublish legacy settings and app. settings in a specific dir

in the source code

• Use symlinks to deploy them

• But not manually!

• kaliop/ezpublish5universalinstaller

• Advantage: only one git repo to take care of

• Advantage: supports an ‘override’ logic

On-Disk layout

Exercise timeAgain!

Managing database changes

A. I connect to the Admin Interface

B. A solved problem

04

Sharing the Database

Many possibilities:

A. All developers connect to the same db

B. The database is committed to git

C. Managing changes via scripts

D. Live DB replication (really ???)

E. More ?

Connecting to a shared database

• good for teams which are *not* geographically distributed

• works when developers do not blow up the database with junk

content: needs some developer discipline

• works only up to the 1st deployment to UAT env;

then 2 dbs have to be managed anyway

Committing the database to git

• good for when the development database does not contain a

huge number of contents and assets

• good for when the development database does not change too

frequently

• developers might be working on different versions

of the db: needs some developer discipline

• works only up to the 1st deployment to UAT env;

then 2 dbs have to be managed anyway

Managing database changes via script

• Safest option

• Good for when the development database does not change too

frequently

• Perfect after deployment to UAT/PROD

• kaliop/ezmigrationbundle

Exercise timeAgain!

Automating deployment

A. FTP is all I need!

B. I put a Jenkins in my Jenkins

05

Automating deployments

• do I even have to explain why it is a good idea ?

• use the same deployment script during development as in

production

so that it gets well tested

• good for automated testing too

eZPublish deployment tasks

1. alter webserver config to disallow access

2. get the latest version of the source code

3. composer install

4. apply changes to database

5. clear Opcache cache (just in case)

6. custom scripts?

7. reindex content (optional)

8. clear shared caches

always: inner to outer

Memcached

Varnish

9. alter webserver config to allow access

eZPublish deployment tasks

extras:

1. run security checks (Symfony has a built-in script)

2. notify monitoring systems of deployment (eg: NewRelic)

Exercise timeAgain!

You do write tests for CMS-

powered websites, don’t you?

A. HAHAHA!

B. Are you joking?

C.No, really?

06

Conventional wisdom on testingIs it really the good choice for sites built on CMS ?

Going against the grainHow much non-project-specific code have you used on this project ?

UI

Unit

???

Going against the grainFunctional testing rocks the world?

• Use Behat if you *really* have customers / PMs / Analysts willing to read

the scenarios

• Functional tests can be written in PHPUnit too

• Functional testing is slow

Run it on a CI platform

On every commit is perfect, but before deployment is good enough

Use Selenium-as-a-service if you can afford it (even if you can’t)

Exercise timeAgain!

THANK YOU

@gggeek

https://github.com/kaliop-uk

https://github.com/kaliop

http://www.kaliop.co.uk/info/ez.html