Effective Doctrine2: Performance Tips for Symfony2 Developers
-
Upload
marcin-chwedziak -
Category
Technology
-
view
12.837 -
download
6
Transcript of Effective Doctrine2: Performance Tips for Symfony2 Developers
Effective Doctrine2Performance Tips for Symfony2 Developers
Wrocław Symfony Group #2 April 24th, 2014
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Symfony2 ContributorSilex Contributor
Symfony2 Developer
Angular.js D
eveloper
Ag
ile E
nthu
sias
t
„No worries, I
will handle th
is”
App
reci
ated
Con
trac
tor
Marcin Chwedziak github.com/tiraeth
lnkd.in/dp6sPzc
Software Engineer at Spork Labs LLP (Salt Lake City, UT)
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
What is this talk about?
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
plus things that can go wrong while working with Symfony2
107 queries executed in 17.25 seconds
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
While working with databases, the most important strategy is to avoid querying a database.
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
If you must query a database, be wise and do it smarter.
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Rule #2: Code Optimization
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Smart behaviour also means to be explicit in actions.
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Rule #3: Transaction Demarcation
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Metadata
What can be effectively cached?
Query Result
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Parsing model metadata from different sources with each request can have a serious impact on your application’s performance. It is a good practice to cache it.
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Each time you write a DQL query or use the QueryBuilder to create one, Doctrine has to transform that query to SQL. This is an unnecessary waste of available resources.
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Usually, there is no special reason to hit a database for rarely modified data. Configure Result Cache strategy to make sure that your database is not swamped with such queries.
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Be aware!
Make sure you properly create your queries.
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
# Fetch expired subscriptions (non-cachable) SELECT s FROM ASBillingBundle:Subscription s WHERE s.validTo < ? [1397855248] !# Fetch expired subscriptions (cachable) SELECT s FROM ASBillingBundle:Subscription s WHERE s.validTo < CURRENT_TIMESTAMP()
You can cache both DQL and Native SQL queries.
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
How to cache metadata in Symfony2?
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
app/config/config_prod.yml doctrine: orm: # ... metadata_cache_driver: apc # ...
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Can I implement custom caching service?
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
app/config/config_prod.yml doctrine: orm: # ... metadata_cache_driver: type: service id: as.doctrine.cache.couchbase # ...
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
src/AS/CoreBundle/Doctrine/Cache/CouchbaseCache.php <?php !namespace AS/CoreBundle/Doctrine/Cache; !use Doctrine\Common\Cache\CacheProvider; !class CouchbaseCache extends CacheProvider{ // doFetch($id) // doContains($id) // doSave($id, $data, $liteTime = 0) // doDelete($id) // doFlush() // doGetStats() }
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
How to cache queries in Symfony2?
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
app/config/config_prod.yml doctrine: orm: # ... query_cache_driver: apc # ...
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
How to cache query results in Symfony2?
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
app/config/config_prod.yml doctrine: orm: # ... result_cache_driver: apc # ...
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Setting caching driver for Query Result Cache is not sufficient for enabling the strategy. You need to explicitly enable it for each and every query you’re interested in.
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
<?php !// ... !$query = $em->createQuery($dql); $query->useResultCache(true); !// …
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
With great power comes great responsibility.
Voltaire (1694-1778)
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Be aware that high traffic websites can generate races while accessing cached data. This situation is called cache slam. When implementing own cache driver, please refer to your backend’s documentation regarding races avoidance.
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
If anything goes wrong, you can manually clear the cache without restarting/flushing the storage.
php app/console doctrine:cache:clear-metadata php app/console doctrine:cache:clear-query php app/console doctrine:cache:clear-result
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Optimization Guidelines
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
You can begin your journey with code optimization while using Doctrine2 with properly set up domain models.
Please be aware that the following guidelines are for use with Doctrine2, not necessarily with database design in general.
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Step 1: Avoid bidirectional associations
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
If you really need bidirectional associations, try to use lazy fetching strategy, but in most cases bidirectional relations are not necessary and should be avoided.
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Be aware that lazy fetching strategy creates dynamic proxy objects (class is generated on the fly).
Fortunately, Symfony2 handles proxy generation with cache warmup task.
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
For aggregations (with Collections) you can use EXTRA LAZY strategy which will not load the whole collection on first access, as it is done in LAZY strategy.
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Dropping such relations does not mean that you can’t use JOIN with your DQL / Native SQL queries.
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
<?php !namespace AS\BillingBundle\Entity; !class Subscription { /** * @ORM\ManyToOne(targetEntity="User") * @ORM\JoinColumn(name="user_id", referencedColumnName="id") */ protected $user; } !!!class User { // ... }
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
SELECT u FROM ASBillingBundle:User u JOIN ASBillingBundle:Subscription s WITH s.user = u WHERE s.validTo <= CURRENT_DATE()
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Step 2: Avoid unnecessary cascades
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
1. Do not configure cascade operations on every association you have in your domain.
2. Unless required (e.g. additional file removal), use DBMS cascades instead of ORM ones.
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Remember that you do not setup ORM cascades on target entites but on the triggering ones. Opposite to DBMS configuration.
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
<?php !namespace AS\BillingBundle\Entity; !class Subscription { /** * @ORM\OneToOne(targetEntity="User", cascade={"remove"}) * @ORM\JoinColumn(name="user_id", referencedColumnName="id") */ protected $user; }
This is wrong:
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
<?php !namespace AS\BillingBundle\Entity; !class Subscription { /** * @ORM\ManyToOne(targetEntity="User") * @ORM\JoinColumn( * name="user_id", * referencedColumnName="id", * onDelete="CASCADE" * ) */ protected $user; }
This is correct:
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Step 3: Avoid lifecycle events abuse
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Be smart about listening to lifecycle events like persist, update, or remove. Know the difference between Listener and Subscriber.
Do not identify Doctrine2 event subscribers with Symfony2 subscribers, which are more dynamic!
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Avoid requiring EntityManager in your event listeners (it will just not work).
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
<?php !namespace AS\BillingBundle\Listener; !use Symfony\Bridge\Doctrine\RegistryInterface; !abstract class LifecycleListener { protected $doctrine; protected $manager; ! public function __construct( RegistryInterface $doctrine, $manager = 'default' ) { $this->doctrine = $doctrine; $this->manager = $manager; } ! protected function getEntityManager() { return $this->doctrine->getEntityManager($this->manager); } }
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Step 4: Load from database what you need
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Fetch Modes
Return only data you are in need of.
Hydration Modes
Partial Objects
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
When dealing with one-to-one and many-to-one relations, you can switch between EAGER and LAZY fetching modes right in your Query object.
<?php !$query = $em->createQuery( "SELECT s FROM ASBillingBundle:Subscription s" ) ->setFetchMode( "ASBillingBundle:Subscription", "user", \Doctrine\ORM\Mapping\ClassMetadata::FETCH_EAGER );
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
If not necessary, do not fetch a whole object (with relations) to just get its „validTo” field value.
<?php !$expirationDate = $em->createQuery( "SELECT s.validTo " . "FROM ASBillingBundle:Subscription s " . "WHERE s.user = ?1" ) ->setParameter(1, $user) ->getSingleScalarResult();
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Unless you can deal with scalar values or arrays, you may need to wrap data in your model’s class. Use Partial Objects for this purpose.
<?php !$reference = $em->createQuery( "SELECT PARTIAL s.{plan,validTo} " . "FROM ASBillingBundle:Subscription s " . "WHERE s.id = ?1" )->setParameter(1, $id)->getResult(); !$reference->getPartialReference("ASBillingBundle:Subscription", $id);
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Step 4: Load from database when you need
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
If you have to process data for large result sets, think about using DQL UPDATE/DELETE queries or Iterable Result.
When using Iterable Result always remember to detach all existing and not required anymore objects from Doctrine!
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
<?php !$batch = 50; $i = 0; !$iterableResult = $em->createQuery( "SELECT s FROM ASBillingBundle:Subscription s" )->iterate(); !foreach ($iterableResult as $row) { $subscription = $row[0]; $subscription->renew(); if (($i % $batch) === 0) { $em->flush(); // Execute queued updates $em->clear(); // Detach objects and GC them } ++$i; } !$em->flush(); // Execute remainig updates $em->clear(); // Detach remaining objects and GC them
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
<?php !$iterableResult = $em->createQuery( "SELECT s FROM ASBillingBundle:Subscription s " . "WHERE s.validTo = CURRENT_DATE() AND s.trialing = TRUE" )->iterate(); !foreach ($iterableResult as $row) { $mailer->notifyTrialExpiration($row[0]); $em->detach($row[0]); // Detach and GC immediately }
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Transaction Demarcation
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
Proper transaction boundary definition can have positive impact on your application’s performance.
Doctrine2 takes care of proper implicit demarcation for UnitOfWork–handled objects and queue all operations until EntityManager#flush() is invoked.
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
However, Doctrine2 encourages developers to take over and control transaction demarcation tasks themselves.
Please remember to flush the EntityManager prior to transaction commit. In case of any exceptions, always close the EntityManager.
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
<?php !$em->getConnection()->beginTransaction(); !try { $user = new User; $subscription = new Subscription; $subscription->setUser($user); ! // ... do some work ! $em->persist($user); $em->persist($subscription); $em->flush(); $em->getConnection()->commit(); } catch (Exception $e) { $em->getConnection()->rollback(); $em->close(); ! // ... do something about the exception }
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
<?php !$em->transactional(function($em) { $user = new User; $subscription = new Subscription; $subscription->setUser($user); ! // ... do some work ! $em->persist($user); $em->persist($subscription); });
Using EntityManager#transactional() will automatically flush the manager prior to commit.
Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak
<?php !$em->getConnection()->transactional(function($em) { $user = new User; $subscription = new Subscription; $subscription->setUser($user); ! // ... do some work ! $em->persist($user); $em->persist($subscription); $em->flush(); });
Warning! This will not close EntityManager on exception.