Why?
• Current market offerings are slow
• Massive increase expected (IoT and mobile)
• Prices are high and models restricting (per RPS licensing)
First iteration
• Authentication and authorisation in Varnish
• Data stored in JSON
• Built a JSON to VCL compiler and framework
• Didn’t scale past 1000 keys
Second iteration
• F100 media conglomerate
• Move the keys into sqlite, managed on each node
• Enable throttle rules using memcache
• Relies on VCL for configuration
Varnish Cache Varnish Cache Varnish Cache
Management
Accounting
Statistics
API
Consumers
API Engine processing
layer
API endpoints
Administrators
API
API
APIAPI API
APIAPI
SQLite VMOD
Memcached VMOD
VCS
DB
syncd
Client requests
Backendrequests
Statistics Accounting Configuration
VCL
Varnish Cache
Varnish Cache Varnish Cache Varnish Cache
Management
Accounting
Statistics
API
Consumers
API Engine processing
layer
API endpoints
Administrators
API
API
APIAPI API
APIAPI
http://api.example.com/someapiIPv4: 10.0.0.10
[…]Id c9c2f568Type round-robin[…]
$ api-engine create director
SQLite VMOD
Memcached VMOD
VCS
DB
syncd
Client requests
Backendrequests
Statistics Accounting Configuration
VCL
Varnish Cache
SQLite VMOD
Memcached VMOD
VCS
DB
syncd
Client requests
Backendrequests
Statistics Accounting Configuration
VCL
Varnish Cache
$ api-engine create backend --host 10.0.0.10 \--director-id c9c2f568[…]Id 033b7bb2Director Id c9c2f568Host 10.0.0.10Port 80[…]
[…]Id dcd16a26Director Id c9c2f568Url /someapiHost api.example.com[…]
$ api-engine create endpoint --host api.example.com \--url /someapi --director-id c9c2f568
SQLite VMOD
Memcached VMOD
VCS
DB
syncd
Client requests
Backendrequests
Statistics Accounting Configuration
VCL
Varnish Cache
$ api-engine create rule \--key foobar --total 60r10 --endpoint-id dcd16a26[…]Id 641f949eEndpoint Id dcd16a26Key foobarTotal 60r10[…] SQLite
VMOD
Memcached VMOD
VCS
DB
syncd
Client requests
Backendrequests
Statistics Accounting Configuration
VCL
Varnish Cache
GET /someapi/search HTTP/1.1host: api.example.comapikey: foobar
GET /someapi/search?apikey=foobar HTTP/1.1host: api.example.com
What’s next?
• Web interface for management.
• Visualisation of real time statistics.
• Persistent statistics.
Top Related