Automating
tasks with
VARNISH PLUS
Hugo Cruz
Product Manager
About me
• Oslo based developer, from Portugal
• Lots of hats, mostly Front-end
Email: hugo@varnish-software.com
IRC: hugo on irc.linpro.no
Twitter: @hrcru
AGENDA
- Automating tasks with VARNISH PLUS
• The Varnish Plus minimum stack
• Playing with statistics
• VCL version control with Git
• Cache invalidation
• Increase value with VCS
Live demo
ahead
Besides black magic, there is only
automation and mechanisation
- Federico Garcia Lorca
Monitoring Configuration
Business
Statistics
Automate your tasks
Integrate Varnish Plus in your
existing toolset.
The minimum
value stack
BenefitsSetup
Management tools
Realtime monitoring
JSON API for everything
$ Setup Varnish Plus repository
$ Install Varnish Cache Plus
$ Install varnish-agent and vcs-probe
$ Install Varnish Administration Console
$ Install Varnish Custom Statistics
Varnish Cache Plus
Supercharged
version of the
popular open source
HTTP reverse proxy,
Varnish Cache
Varnish
Administration
Console
Cluster controller
for you Varnish(s)
installation
Varnish Custom
Statistics
A real-time
statistics engine
Varnish agents
and VCS probe
Varnish agent and
VCS probe are
responsible for
extracting data for
VAC and VCS
Minimal Value stack
VAC
VCS
4.1
VCS probe
Varnish agent
Varnish Cache Plus
4.1
VCS probe
Varnish agent
4.0
VCS probe
Varnish agent
4.0
VCS probe
Varnish agent
{
UI
API
UI
API
Varnish
Customs Statistics
Varnish
Administration Console
Playing with
stats
Different aggregations available
RRD statistical data
gathered in the VAC
Completely configurable
Varnish Stats through
VAC
More VCL fun
3. automatic deploy and rollover.
1. setup a git hook with VAC API
2. change VCL
VCL + GIT
Purging
Benefits
high performance purging
$ curl -X PURGE -H ‘X-Cache-Group: Production’
-H ‘X-HMAC:
0xf7bc83f430538424b13298e6aa6fb143ef4d59a14946175997479dbc2d1a3cd8
‘ -H ‘X-Timestamp: 1391641007 http://vac-server:8088/purge-url
purge several instances
in a group, fast
$ “ No need for extra setup. Purger is up and running with VAC, on port 8088.”
Issue a purge
acl purgers {
“127.0.0.1”;
“192.168.0.0”/24;
}
sub vcl_recv {
if (req.request == “PURGE”) {
if (!client.ip ~ purgers) {
error 405 “Method not allowed”;
}
if (digest.hmac_sha256(“secret-key”,req.url +
req.http.x-timestamp + req.http.x-cache-group) != req.http.x-hmac) {
error 405 “Method not allowed”;
}
return (lookup);
}
}
Extra security
Super Fast Purger
Fully secure
Increase
value with
VCS
Benefits
Live Production
debugging
Business indicators
Realtime statistics engine
sub vcl_deliver {
# This creates a grouping for each specific value of the Host request
std.log("vcs-key: " + req.http.host);
# This creates a key based on both the Host header and the URL
std.log("vcs-key: URL-“ + req.http.host + req.url);
# This will create two keys 'MISS' and 'HIT', each counting
# stats for requests matching one of the two conditions.
if (obj.hits == 0) {
std.log("vcs-key: MISS");
}
else {
std.log("vcs-key: HIT");
}
}
Time series API
VCS
Basic setup
(counting hits and misses)
A/B testing
Minimum stack
Setup the components
Playing with
stats
VAC powered
Varnish stats
VCL version control
Integrating VCL and GIT
Cache Invalidation
Using the Super Fast Purger
VCS in production
Debugging with VCS
Summing up
Thank you!
Reach out to me if you have further questions
hugo@varnish-software.com

Setting up automated tasks

  • 2.
  • 3.
    Hugo Cruz Product Manager Aboutme • Oslo based developer, from Portugal • Lots of hats, mostly Front-end Email: hugo@varnish-software.com IRC: hugo on irc.linpro.no Twitter: @hrcru
  • 4.
    AGENDA - Automating taskswith VARNISH PLUS • The Varnish Plus minimum stack • Playing with statistics • VCL version control with Git • Cache invalidation • Increase value with VCS
  • 5.
  • 6.
    Besides black magic,there is only automation and mechanisation - Federico Garcia Lorca
  • 7.
    Monitoring Configuration Business Statistics Automate yourtasks Integrate Varnish Plus in your existing toolset.
  • 8.
  • 9.
    BenefitsSetup Management tools Realtime monitoring JSONAPI for everything $ Setup Varnish Plus repository $ Install Varnish Cache Plus $ Install varnish-agent and vcs-probe $ Install Varnish Administration Console $ Install Varnish Custom Statistics
  • 10.
    Varnish Cache Plus Supercharged versionof the popular open source HTTP reverse proxy, Varnish Cache Varnish Administration Console Cluster controller for you Varnish(s) installation Varnish Custom Statistics A real-time statistics engine Varnish agents and VCS probe Varnish agent and VCS probe are responsible for extracting data for VAC and VCS Minimal Value stack
  • 11.
    VAC VCS 4.1 VCS probe Varnish agent VarnishCache Plus 4.1 VCS probe Varnish agent 4.0 VCS probe Varnish agent 4.0 VCS probe Varnish agent { UI API UI API Varnish Customs Statistics Varnish Administration Console
  • 12.
  • 13.
    Different aggregations available RRDstatistical data gathered in the VAC Completely configurable Varnish Stats through VAC
  • 14.
  • 15.
    3. automatic deployand rollover. 1. setup a git hook with VAC API 2. change VCL VCL + GIT
  • 16.
  • 17.
    Benefits high performance purging $curl -X PURGE -H ‘X-Cache-Group: Production’ -H ‘X-HMAC: 0xf7bc83f430538424b13298e6aa6fb143ef4d59a14946175997479dbc2d1a3cd8 ‘ -H ‘X-Timestamp: 1391641007 http://vac-server:8088/purge-url purge several instances in a group, fast $ “ No need for extra setup. Purger is up and running with VAC, on port 8088.” Issue a purge acl purgers { “127.0.0.1”; “192.168.0.0”/24; } sub vcl_recv { if (req.request == “PURGE”) { if (!client.ip ~ purgers) { error 405 “Method not allowed”; } if (digest.hmac_sha256(“secret-key”,req.url + req.http.x-timestamp + req.http.x-cache-group) != req.http.x-hmac) { error 405 “Method not allowed”; } return (lookup); } } Extra security Super Fast Purger Fully secure
  • 18.
  • 19.
    Benefits Live Production debugging Business indicators Realtimestatistics engine sub vcl_deliver { # This creates a grouping for each specific value of the Host request std.log("vcs-key: " + req.http.host); # This creates a key based on both the Host header and the URL std.log("vcs-key: URL-“ + req.http.host + req.url); # This will create two keys 'MISS' and 'HIT', each counting # stats for requests matching one of the two conditions. if (obj.hits == 0) { std.log("vcs-key: MISS"); } else { std.log("vcs-key: HIT"); } } Time series API VCS Basic setup (counting hits and misses) A/B testing
  • 20.
    Minimum stack Setup thecomponents Playing with stats VAC powered Varnish stats VCL version control Integrating VCL and GIT Cache Invalidation Using the Super Fast Purger VCS in production Debugging with VCS Summing up
  • 21.
    Thank you! Reach outto me if you have further questions hugo@varnish-software.com