New Magento 2.2 Deployment New Magento 2… · Flexibility in what to do in build vs deploy phases,...

New Magento 2.2 Deployment Capabilities & Patterns

Alan KentMagento Chief ArchitectMagento Commerce, USA

New features to help deployments– Build step no longer needs database access– Faster compilation and asset generation– Compact web assets – reduces bundle and extract time– Read-only directories in production

Improved support for multiple environments– Environment variables– Per environment configuration file (env.php)– Shared configuration file (config.php)– New “sensitive” settings (passwords, PII)

Coming in 2.2

https://12factor.net/

I. Codebase in version controlIII. Store config in environmentV. Build, release, runX. Dev/prod parity

Deployment Flow

A Simple Development/Deployment Lifecycle

Git pull

Develop

Gitcommit/push

Compilation

PHP Code Generation

Database Upgrade

End MaintMode

Web Asset Generation

Git Pull

Maint Mode

Develop Deploy

Production ServerDeveloper Laptop

A Simple Development Deployment Lifecycle

Git pull

Develop

Gitcommit/push

Compilation

PHP Code Generation

Database Upgrade

End MaintMode

Git Pull

Maint Mode

Develop Deploy

Can be slow

Needs DBCan be slow

Can be slow

Production ServerDeveloper Laptop

Adding a Build/Package Phase to Build Pipeline

Git pull

Develop

Gitcommit/push

Extract

Maint Mode

PHP Code Generation

Database Upgrade

End MaintMode

Switch Code

Git pull

Compilation

Test Automation

Develop Build & Bundle Deploy

Bundle (ZIP)

Faster*

Faster*DB not neededCompact

Can be

Production ServerDeveloper Laptop Build Server

* Backported to 2.1 patches

Future: Zero Downtime Deployments (if no DB change)

Git pull

Develop

Gitcommit/push

Extract

Maint Mode

PHP Code Generation

Database Upgrade

End MaintMode

Switch Code

Git pull

Compilation

Test Automation

DB Upgrade

Req’dY N

Switch Code

Develop Build & Bundle Deploy

Bundle (ZIP)

Faster

FasterNo DBCompact

Normal

Developer Laptop Build Server

Summary of Performance Boosts

Code optimization of

compilation

(backported to 2.1.x patches)

Code optimization of

web asset generation

(backported to 2.1.x patches)

New compact deployments, reducing disk

Able to run on build server without DB

(not production server)

3 locales x2 faster; 15 locales x10 faster – due to less disk I/O

Compact Mode for Static Assets

$ magento setup:static-content:deploy --strategy compact ...

$ magento setup:upgrade --keep-generated ...

Compact mode reduces duplication when multiple themes or locales– Creates map.php & requirejs-map.js per area specifying real file locations– PHP & JS functions supplied to map URLs to shared resources

Flexibility in what to do in build vs deploy phases, what is in git, …

To move configuration between environments, 12 Factor App says “configuration is code” (under version control, pushed with code, …)

Configuration as Code

Sections of config.php• Enabled modules• Scopes (websites, stores, store views)

and Themes• Store configuration settings

Enabled Modules Section

Developer 1 Developer 2 Staging Production

config.php

module:enablemodule:disable

config.php config.php config.php

Developer Mode Production Mode

GitBuild

Enabled Modules Section

config.php

module:enablemodule:disable

config.php config.php config.php

GitBuild

magento module:enablemagento module:disable

Updates config.php, pushed with code

• Websites, stores, and store views defined via Admin– In database for foreign key validation

• List of available themes also in database• Needed by build phrase

– Export database content to file for deployment

Scopes and Themes

Scopes and Themes Sections

config.phpapp:config:dump

config.phpapp:config:import

config.php config.php

GitBuild

setup:upgrade setup:upgrade

GitBuild

/admin

Websites, stores, store views defined in Admin

GitBuild

magentoapp:config:dump

Adds scope and theme details to config.php

GitBuild

setup:upgrade setup:upgrademagento

app:config:import

Copies scope and theme details from config.php into database

GitBuild

magento setup:upgrade

Setup upgrade includes import and schema upgrade

GitBuild

System Configuration Settings Fallback

1• Configuration values in environment variables

2• Configuration values in environment-specific config file (env.php)

3• Configuration values in shared config file (config.php)

4• Configuration values in DB

5• Configuration default value (config.xml)

Settings in config.phpand env.php are “locked” - cannot be changed via Admin

System Configuration Settings

env.phpconfig.php

config:set

app:config:dump

env.phpconfig.php env.phpconfig.php

config:set --lock

env.phpconfig.php

config:sensitive:set

/admin

GitBuild

env.phpconfig.php

config:set

app:config:dump

config:set --lock

env.phpconfig.php

/admin

GitBuild

/adminmagento config:set

Settings changed via Admin OR scripted – updates DB

env.phpconfig.php

config:set

app:config:dump

config:set --lock

env.phpconfig.php

/admin

GitBuild

magentoapp:config:dump

Changed settings are written to config.php and env.php

env.phpconfig.php

config:set

app:config:dump

config:set --lock

env.phpconfig.php

/admin

GitBuild

config.php

Configuration settings pushed as code (“locked” in Admin)

env.phpconfig.php

config:set

app:config:dump

config:set --lock

env.phpconfig.php

/admin

GitBuild

magento config:set --lock

Locked values can be overridden in local env.php

env.phpconfig.php

config:set

app:config:dump

config:set --lock

env.phpconfig.php

/admin

GitBuild

magentoconfig:sensitive:set

Interactive mode to prompt for sensitive values for env.php

Other Recommendations

Git Repository

Marketplacecomposer require ...

Source Code Management Recommendations

composer.jsoncomposer.lock

vendor

Git Repository

Build Process

Alternative: publish to a private packagist repo

https://packagist.com/

Production

"config": {"preferred-install": {

"alankent/*": "source","*": "dist"

composer install --prefer-sourceVS

Caches well

• Build process– Think about how to update tool chain, or customize per project– Git clone/pull project repo to get the code – needs repo keys– composer install – keep cache between runs for speed/traffic– Decide if test files should be in deployment bundle

• Test Automation – run functional tests on the production deployment build• Separate Integration Test & Production environments?

– Want environments close to each other– Minimize env.php (keep environments

as close as possible)

Build & Bundle Recommendations

Staging Production

Developer 1 Developer 2

• Read-only file system (/var and /generated now separate)• Plan out your configuration settings per environment• Remember config.php and env.php are executable code• Have build phase create common bundle to share amongst environments• Swap files in – use symlink, change config file, blue/green servers• Consider serving stale content from cache during cache flush• Decide where to deploy

– Magento Cloud– Zerolag, Nexcess, Rackspace, Byte,

Simple Helix - more done for you– Docker / AWS / Azure - set it up yourself

Deployment Recommendations

Staging Production

Conclusions

1. Use Admin to define websites, stores, store views in database2. Enable/disable modules via config.php3. Developer uses Admin or config:set to set store configuration settings4. Developer runs app:config:dump to create config.php and env.php

– Shared à config.php– Environment specific (sandbox vs live endpoints, …) à env.php– Sensitive (passwords, PII, …) setting names à config.php, values ->

env.php5. The config.php file is committed to git for sharing

Workflow Review – Developer Environment

1. Get updated copy of config.php along with rest of latest code2. Run app:config:import to update scopes and themes in database3. config:sensitive:set can be used to set any missing sensitive settings4. config:set will not override “locked” values already in config.php/env.php5. config:set --lock will add value to env.php even if “locked”6. Update database schema, if needed (with store in maintenance mode)7. Flush caches, go live

Workflow Review – Target Environments

• Command to detect if schema changed for optimized deployments• Declarative schemas to make schema changes easier• Further standardization of

– Development environment (DevBox still in beta)– Build pipeline commands– Test automation invocation (and other improvements)– Hosting partner specific deployment processes

Future

Alan Kent@akent99

Pipeline DeploymentDeployment flow

Pipeline DeploymentConfiguration Management

Sample Developer and Production Environments

env.phpconfig.php env.phpconfig.php env.phpconfig.php env.phpconfig.php

GitBuild

Overview of StepsDevelop

Build & Bundle

Deploy to Test/Staging

Deploy to Prod

Identify per environment settings (including those used by extensions)Commit shared configuration (config.php) to git

Depends on production shared production mode settings(Must not depend on dev or specific prod environment settings)

Development Mode Settings

Production Mode Settings

Environment variables, env.php, (config.php), database settings

TODO: CONFUSING SLIDE – REDO?DEFINE LOCKING EARLIER

Build & Bundle

Deploy to Prod

Build & Bundle

Deploy to Prod

Developer controlledShared settings

Merchant controlledPer environment settings

Developer controlledPer environment settings

New Magento 2.2 Deployment New Magento 2… · Flexibility in what to do in build vs deploy phases,...

Documents

Transcript of New Magento 2.2 Deployment New Magento 2… · Flexibility in what to do in build vs deploy phases,...

Securing Your Deployment Pipeline With Docker

(DVO305) Turbocharge YContinuous Deployment Pipeline with Containers

Continuous Integration, Delivery, and Deploy ment ......CSharp Web Shop - Deployment #1 CSharp Web Shop - Deployment 62 ms CSharp Web Shop - Deployment Pipeline #66 CSharp Web Shop

Sunil Shah CONTINUOUS DEPLOYMENT PIPELINE · © 2016 Mesosphere, Inc. All Rights Reserved. 1 BUILDING A CONTAINERI{S|Z}ED CONTINUOUS DEPLOYMENT PIPELINE (using Apache Mesos, Jenkins

Continuous Deployment Pipeline for Systems at Cascadia IT Conference - 2017-03-11

building a containeri{s|z}ed continuous deployment pipeline

Cvent: Building a microservice deployment pipeline – Couchbase Connect 2016

Building a deployment pipeline

Subsea Pipeline Lifting Tool - Hydratight · 2020. 9. 24. · Subsea Pipeline Lifting Tool Deployment of lifting tool Transport storage configuration HT_TL_062_0917_UK Pipeline OD

Chapter 5, “Anatomy of the Deployment Pipeline

Building an Automated Database Deployment Pipeline

Deployment Pipeline for Go Applications on AWS · · 2017-07-13Amazon Web Services – Deployment Pipeline for Go Applications on the AWS Cloud July 2017 Page 2 of 12 Contents Overview

Focal Design Issues Affecting The Deployment Of Wireless Sensor Networks For Pipeline Monitoring

Angrybirds Magento Cloud Deployment

Continuous Integration and Continuous Deployment Pipeline with Apprenda on ONTAP 9

Scalable Data Analytics Pipeline for Real-Time Attack ...publish.illinois.edu/science-of-security-lablet/files/2015/09/10062015-Eric-Badger...Pipeline Design Pipeline Deployment Validation

(GAM404) Gaming DevOps: Scopely's Continuous Deployment Pipeline | AWS re:Invent 2014

AI for Social Impact - teamcore.seas.harvard.edu · Multiagent systems, Data-to-deployment pipeline, Interdisciplinary partnerships Field tests & deployment Prescriptive algorithm

Rock-solid Magento Development and Deployment Workflows

TriopusNet : Automating Wireless Sensor Network Deployment and Replacement in Pipeline Monitoring