Asynchronous processing with PHP and Symfony2

Do it simpleKirill chEbba Chebunin

Kirill chEbba Chebunin

Overview. The Problem

REQUEST RESPONSE

Expected Response

Complete Logic

Kirill chEbba Chebunin

• Data transfer and manipulation !• File upload!• Video conversion!

• Massive business logic !• Billing!• Social!

• 3rd-party communication!• API calls !• Notifications

Overview. Examples

Kirill chEbba Chebunin

Overview. The Solution

REQUEST RESPONSE

Required Logic

Remaining Logic

Kirill chEbba Chebunin

Experience. The Real Project

http://hopeliesat24framespersecond.files.wordpress.com/2011/12/love_actually_movie_image_bill_nighy_01.jpg

Kirill chEbba Chebunin

Experience. Billing

Billing Aggregator

Service Providers

Partners

Kirill chEbba Chebunin

Experience. Billing. 2012

Master

Worker

Worker

Worker

Worker

Worker

Worker

Kirill chEbba Chebunin

Experience. Billing. Revolution

Made by Vasily coyl Kulakov

http://darkroom.baltimoresun.com/wp-content/uploads/2012/05/REU-CUBA-MAYDAY-1.jpg

Kirill chEbba Chebunin

Theory

http://marketmybook.in/wp-content/uploads/2013/05/Library-Books.jpg

Kirill chEbba Chebunin

Theory. Master-Worker

Master

Worker

Worker

Worker

Kirill chEbba Chebunin

• Process control!• Resources in workers!• Long living workers!• Memory control!• Signal support

Theory. Master-Worker. Conclusions

Kirill chEbba Chebunin

Theory. RabbitMQ

http://www.rabbitmq.com/img/tutorials/intro/hello-world-example-routing.png

Kirill chEbba Chebunin

• 1 exchange => multiple queries = topic!

• Query = Worker type!• Durable exchanges/queries!• Persistent messages!• autoAck=false

RabbitMQ

Kirill chEbba Chebunin

Experience. Billing. Review

http://bi.gazeta.pl/im/84/96/d8/z14194308Q,Najglupsze-pomysly-mistrzow-prowizorki.jpg

Kirill chEbba Chebunin

• Master process => supervisor!• Start workers!• Worker count!• Signaled shutdown!• Auto-restart on failure!• Memory control!• ………….

New world. Master

Kirill chEbba Chebunin

Example config ![program:billing_statistics] process_name=%(program_name)s_%(process_num)d numprocs=4 stdout_logfile=/var/log/supervisor/%(program_name)s.out.log stderr_logfile=/var/log/supervisor/%(program_name)s.error.log command=billing rabbitmq:consumer statistics autorestart=true autostart=true

New world. Supervisor

Kirill chEbba Chebunin

• oldsound/rabbitmq-bundle!• Exchange/Queue setup!• Producers!• Consumers!• Command = worker!

• Events!• Special event name!• Producer Listener

New world. Publish/Subscribe

Kirill chEbba Chebunin

Worker

Worker

Worker

Worker

Worker

Worker

Experience. Billing. 2013

Supervisor

Kirill chEbba Chebunin

Profit?

Kirill chEbba Chebunin

• Send any event to async processing!• Use listeners as workers!• Different message bus

Future. Publish/Subscribe. Improve!

Kirill chEbba Chebunin

Future

Kirill chEbba Chebunin

EventBand

EventBand

http://baadu.ru/ph/8/polosy_na_stene_1280x1024.jpg

Kirill chEbba Chebunin

Dispatcher

Band#1

EventBand. Overview

Listener

Listener

Listener

Listener

Listener

Listener

Listener

Listener

Default Band

Listener

Listener

Listener

Listener

Listener

Listener

Listener

Listener

Band#3

Band#2

Band#6

Band#5

Band#4

Event

Current

Kirill chEbba Chebunin

EventBand. Architecture

PublishListener

Listener

Listener

Listener

Listener

T R A N S P O R T

Band#1Listener

Listener

Band#2Listener

Listener

Band#3Listener

Listener

Kirill chEbba Chebunin

• Dispatcher!• Band support!• Abstract Subscribers!• Adapters (Symfony, ZF, …)!

• Transport!• Setup!• Publisher!• Storage!• Routing!• Consumer!• Adapters (RabbitMQ, ActiveMQ, Gearman, …)

EventBand. Architecture

Kirill chEbba Chebunin

• Event Serialization!• Native!• JMS!

• Event Routing!• By name!• By properties (PropertyAccess)

EventBand. Utilities

Kirill chEbba Chebunin

Dispatch: !$dispatcher->dispatch('foo.bar', new FooEvent($foo, $bar)); !………………………………………………………… !Subscribe: !public function logFooBar(FooEvent $event) { $this->service->saveLog($event->getFoo(), $event->getBar()); } !!<service id="log_subscriber" class="FooBarLogSubscriber"> <tag name="kernel.event_subscriber" band=“foo.bar.log" /> </service>

EventBand. Example. Code

Kirill chEbba Chebunin

transports: amqp: driver: amqplib connections: default: ~ exchanges: foo: ~ queues: foo.bar.log: bind: foo: "foo.bar" ! publishers: foo: events: "foo.#" ! consumers: foo.bar.log: ~

EventBand. Example. Config

Kirill chEbba Chebunin

EventBand. Wanna Play?

https://packagist.org/packages/event-band/

Kirill chEbba Chebunin

• Identified Event!• id, time, sequence, process!• logging (sqlite)!

• Delayed Event!• original event, delay!• delay time => queue!• ordered queue!• sleep until delay

Experience. Patterns

Kirill chEbba Chebunin

• Doctrine!• SqlLogger!• Unit of Work!

• flush, detach!• Array properties

Experience. Troubleshooting

Kirill chEbba Chebunin

Questions?

Cool Image with

Cats and Boobs

Kirill chEbba Chebunin iam@chebba.org https://github.com/chEbba https://twitter.com/iamchEbba