DevOps for TYPO3 Teams and Projects

TYPO3 Developer Days 2014

Hi. I’m Fedir. I make sites and apps.

@FedirFr

● DevOps● Team ● Project ● Tools● Exchanging

This workshop

What is DevOps ?

Let's take a survey

Popular ideas about DevOps meaning

What is DevOps

DevOps = Dev(elopment) + Op(eration)s

Software Development activities

analysis

coding

documenting

testing

debugging

code maintaining

Information Technology Operations activities

management

envisioning

planning

design

implementation

construction

deployment

distribution

verification

installation

instantiation

execution

maintenance

Dev...

DevPM

DevArchitect

DevDev

DevInt

DevSysAdmin

IntAdmin

DevQA

IntQA

DevOps is not all about automation

DevOps is the confluence of development and operations.

Developers

Developers are living in an abstract world of Design Patterns and perfect IDE.

App Classes are well ranged into MVC containers.

Code is very structured.

SysAdmins

SysAdmins touches reality.

They need to think more about limits of current servers, about versions and compatibility of heterogeneous software.

They don’t knows how Your App is working.

For somebody it’s a cloud, for somebody it’s complex interconnected server infrastructure.

And ... Integrators !

DevOps terminology was created for companies with simple traditional structure.

TYPO3 has Integrators !

Guys, who installs and configures TYPO3 on real servers, who adapts the system and extensions to current infrastructure (often they communicates with Admins).

Communication model

DevOps stages during project life

In reality there is even more communication vectors

Infrastructure : admin, developer, integrator

CMS install : admin, integrator

CMS tuning : developer, integrator

Theming : integrator

Functional coding

Content : editor, integrator

Quality control : QA manager, editor, developer, integrator, admin

Deployment : admin, integrator

Iterative development : admin, developer, integrator

Support : admin, developer, integrator

Communication & methodology

We should communicate effectively

We should work like a team

Different Development Workflows

Models

Different Projects Development Workflows

Different Projects Development Workflows

Different Projects Development Workflows

What is the structure of Your team ?

People

Communication = cooperation

Developing tools to help the project team

● Runtime control and services management● Automated functional tests● Automated visual tests● Deployment hooks

Dedicated interfaces

● QA / PM - Web control panel with tests summary and the access to statistics logs

● SysAdmin - CLI (scriptable), automatically controllable

Self-tested & self-controllable systems

Delivered product (site, extension) delivers not only desired functionality, but QA and SA interfaces

● Tests○ Unit○ Visual○ Functional○ Performance

● Deployment hooks

When we should look for DevOps ?

Projects

Iterative agile development

V.1

V.2

V.3

When the concept of sprints is applied with granular delivery

Single shot project

Simple task

Do and forget

Never meet customer again

But what about code and procedures capitalisation ?

LTS Projects

TYPO3 4.5 LTS

TYPO3 6.2 LTS

...

=>

MyProject LTS

Multiple servers infrastructure

When Your app needs more the web server and web database

● Search engines● Proxies● Caches● Services

DevOps and TYPO3

Closer look

What is special in TYPO3

● TYPO3 CMS is used for large projects, usually developer centred

● TYPO3 has lot’s of data in the database● TYPO3 has multiple cache systems● TYPO3 uses external applications● TYPO3 supports automating testing

TYPO3 project team specific roles

● Extension Developer● Site integrator● Content manager● Server Administrator● QA Tester

As TYPO3 model uses roles separation,

between developer and integrator, an additional level of exchange should

be provided

TestingIf my TYPO3 site works well ?

FE testing tools

● dalek.js● galen framework● casper.js● W3C CSS Validator (local service)● W3C Markup Validator (local service)● W3C Link checker (local service)

FE testing aspects

● Multichannel projections● Using the full spectrum of FE roles● Checking functionalities by module● Automatic checking for HTML and CSS validity● Accessibility

The next step : BE Users & Groups test

Does Your user could make all available operations

Does Your user could make forbidden operations

Errors control

Activity statistics

TYPO3 requirements OK

● Checking, if Your server is TYPO3 ready○ TYPO3 Probe

TYPO3 installation control

● Controlling Your installation○ additional_reports

Performance tests

● WebPageTest○ You could install it on Your server

● JMeter● gor● gatling● Tsung

Extensions testing

1 extension = 1 test package

Easy syntax

Automatic tests generation, based on content data and page structure

Security

Install the system, keep it up to date, upgrade it

Migrations Reality

Migrations is not only about the content

How to ship the site and transfer

● Scriptable environment to automate instances○ Ansible○ Puppet○ Chef

● TYPO3 Flow base + extending by Your app● git + hooks

○ files○ dB○ content projections

Application redeployment

● Flush all if You can and redeploy (backups)

or ...

● Merge it○ Files content○ Databases○ Indexing○ Cache

Content components and projections

● Resource-costly projections of the content also could be synchronised

Content merge

● SearchEngine API● Using caching framework● EXT:t3xutils

Content automated merge

● UUID● Database fixtures● EventSourcing implementation

TYPO3 Neos goes right way !

Projections : should it be versioned ?

● Physical limits of repositories● Time of synchronisation● Granularity● Ability to be merged● Relation to versioned content

● Update scenarios○ Security

■ Critical update detection - Real-time application - Automatical tests scenarios - Verification ASAP

○ Features added■ Notification - Update - Control

● EXT:coreapi

Infrastructure updates

Core InitiativesLet’s make TYPO3 better together

Default TYPO3 CMS package testing

● Core functionality tests● Default template tests

○ Visual tests inclusion○ Functional tests inclusion

● Performance tests○ Single page load○ Multiple page

● Platform configuration tests (pre-deploy)

Big architectural adjustments

● UUID● Event Sourcing● Database fixtures

Thank You for Your attention

Questions / Answers

Ressourceshttp://en.wikipedia.org/wiki/DevOpshttp://en.wikipedia.org/wiki/Software_developmenthttp://en.wikipedia.org/wiki/Computer_programminghttp://en.wikipedia.org/wiki/Information_technology_operationshttps://devcentral.f5.com/articles/devops-is-not-all-about-automationhttp://www.slideshare.net/kfish/deploying-typo3-neos-websites-using-surfhttp://www.slideshare.net/ufried/devops-for-developers-28043923http://www.servermanaged.it/devops/sysadmin-is-changing-cloud-devops/http://www.webpagetest.orghttps://github.com/WPO-Foundation/webpagetest/https://github.com/excilys/gatlinghttp://dalekjs.com/http://galenframework.com/http://typo3.org/extensions/repository/view/additional_reportshttps://github.com/7elix/TYPO3-Probehttp://fr.slideshare.net/etobide/20130704-t3dd13-deploymenthttps://github.com/TYPO3-coreapi/ext-coreapihttp://etobi.de/blog/2013/07/deployment-workshop-on-typo3-devdays-t3dd13-part-110/https://github.com/etobi/Typo3ExtensionUtils

http://jigsaw.w3.org/css-validator/DOWNLOAD.htmlhttp://validator.w3.org/source/#debhttp://tsung.erlang-projects.org/http://search.cpan.org/dist/W3C-LinkChecker/http://java-source.net/open-source/web-testing-toolshttp://phpsec.org/library/http://wapiti.sourceforge.net/http://wiki.typo3.org/Caching_framework