Caching and tuning fun for high scalability @ FrOSCon 2011

1.
Caching and tuningfun for high scalability Wim Godden Cu.be Solutions

2.
Notes about thispresentation This presentation was part of the FrOSCon 2011 program. It was designed to presented live and as a result many of the slides may seem odd without spoken explanation. The live benchmarks at the conference are ofcourse also not part of these slides.

3.
Who am I? Wim Godden (@wimgtr)

4.
Owner of Cu.beSolutions (http://cu.be)

5.
PHP developer since1997

6.
Developer of OpenX

7.
Zend Certified Engineer

8.
Zend Framework CertifiedEngineer

9.
MySQL Certified Developer

10.
Who are you? Developers ?

11.
System/network engineers ?

12.
Managers ?

13.
Caching experience ?

14.
Caching and tuningfun for high scalability Wim Godden Cu.be Solutions

15.
Goals of thistutorial Everything about caching and tuning

16.
A few techniquesHow-to

17.
How-NOT-to -> Increasereliability, performance and scalability

18.
5 visitors/day ->5 million visitors/day

19.
(Don't expect miraclecure !)

20.
LAMP

21.
LAMP

22.
Architecture

23.
Our test site

24.
Our base benchmarkApachebench = useful enough

25.
Result ?

26.
Caching

27.
What is caching?

28.
What is caching? select * from article join user on article.user_id = user.id order by created desc limit 10

29.
Caching goals Sourceof information (db, file, webservice, …) : Reduce # of request

30.
Reduce the loadLatency : Reduce for visitor

31.
Reduce for Webserverload Network : Send less data to visitor

32.
Hey, that's frontend!

33.
Theory of cachingDB

34.
Theory of cachingDB

35.
Theory of cachingif ($data == false) DB

36.
Caching techniques #1: Store entire pages Company Websites

37.
Blogs

38.
Full pages thatdon't change

39.
Render -> Storein cache -> retrieve from cache

40.
Caching techniques #1: Store entire pages

41.
Caching techniques #2: Store parts of a page Most common technique

42.
Usually a smallblock in a page

43.
Best effect :reused on lots of pages

44.
Caching techniques #2: Store parts of a page

45.
Caching techniques #3: Store SQL queries ↔ SQL query cache Limited in size

46.
Caching techniques #3: Store SQL queries ↔ SQL query cache Limited in size

47.
Resets on everyinsert/update/delete

48.
Server and connectionoverhead Goal : not to get rid of DB

49.
free up DBresources for more hits !

50.
Caching techniques #3: Store SQL queries

51.
Caching techniques #4: Store complex processing results Not just calculations

52.
CPU intensive tasks: Config file parsing

53.
XML file parsing

54.
Loading CSV inan array Save resources -> more resources available

55.
Caching techniques #4: Store complex processing results

56.
Caching techniques #xx: Your call Only limited by your imagination ! When you have data, think : Creating time ?

57.
Modification frequency ?

58.
Retrieval frequency ?

59.
How to findcacheable data New projects : start from 'cache everything'

60.
Existing projects :Look at MySQL slow query log

61.
Make a completequery log (don't forget to turn it off !)

62.
Check page loadingtimes

63.
Caching storage -MySQL query cache Use it

64.
Don't rely onit

65.
Good if youhave : lots of reads

66.
few different queriesBad if you have : lots of insert/update/delete

67.
lots of differentqueries

68.
Caching storage -Disk Data with few updates : good

69.
Caching SQL queries: preferably not

70.
DON'T useNFS or other network file systems especially for sessions

71.
high latency

72.
locking issues !

73.
Caching storage -Disk / ramdisk Overhead : filesystem access

74.
Limited number offiles per directory -> Subdirectories Local 5 Webservers -> 5 local caches

75.
-> Hard toscale

76.
How will youkeep them synchronized ? -> Don't say NFS or rsync !

77.
Caching storage -Memcache Facebook, Twitter, Slashdot, … -> need we say more ?

78.
Distributed memory cachingsystem

79.
Multiple machines ↔1 big memory-based hash-table

80.
Key-value storage systemKeys - max. 250bytes

81.
Values - max.1Mbyte

82.
Caching storage -Memcache Facebook, Twitter, Slashdot, … -> need we say more ?

83.
Distributed memory cachingsystem

84.
Multiple machines ↔1 big memory-based hash-table

85.
Key-value storage systemKeys - max. 250bytes

86.
Values - max.1Mbyte Extremely fast... non-blocking, UDP (!)

87.
Memcache - whereto install

88.
Memcache - whereto install

89.
Memcache - installation& running it Installation Distribution package

90.
PECL

91.
Windows : binariesRunning No config-files

92.
memcached -d -m<mem> -l <ip> -p <port>

93.
ex. : memcached-d -m 2048 -l 127.0.0.1 -p 11211

94.
Caching storage -Memcache - some notes Not fault-tolerant It's a cache !

95.
Lose session data

96.
Lose shopping cartdata

97.
...

98.
Caching storage -Memcache - some notes Not fault-tolerant It's a cache !

99.
Lose session data

100.
Lose shopping cartdata

101.
… Different librariesOriginal : libmemcache

102.
New : libmemcached(consistent hashing, UDP, binary protocol, …) Firewall your Memcache port !

103.
Memcache in code<?php $memcache = new Memcache(); $memcache->addServer( '172.16.0.1' , 11211); $memcache->addServer( '172.16.0.2' , 11211); $myData = $memcache->get( 'myKey' ); if ($myData === false ) { $myData = GetMyDataFromDB(); // Put it in Memcache as 'myKey', without compression, with no expiration $memcache->set( 'myKey' , $myData, false , 0); } echo $myData;

104.
Let's give thata go ! /** * Retrieves the 10 highest rated articles * @return array List of highest rated articles */ static public function getTopRatedArticleList () { if ($articleList = $cache->load( 'topRatedArticleList' ) === false) { $articleList = self :: getTopRatedArticleListUncached (); $cache->save($articleList, 'topRatedArticleList' ); } return $articleList; }

105.
Where's the data? Memcache client decides (!)

106.
2 hashing algorithms: Traditional Server failure -> all data must be rehashed Consistent Server failure -> 1/x of data must be rehashed (x = # of servers) No replication !

107.
Memcache slabs (orwhy Memcache says it's full when it's not) Multiple slabs of different sizes : Slab 1 : 400 bytes

108.
Slab 2 :480 bytes (400 * 1.2)

109.
Slab 3 :576 bytes (480 * 1.2) (and so on...) Multiplier (1.2 here) can be configured

110.
Each larger slabhas room for fewer items (chunks)

111.
-> Store alot of very large objects

112.
-> Large slabsmight be full

113.
-> Rest ofslabs might be free

114.
-> Try tostore more -> eviction of data !

115.
Memcache - Isit working ? Connect to it using telnet "stats" command ->

116.
Use Cacti orother monitoring tools STAT pid 2941 STAT uptime 10878 STAT time 1296074240 STAT version 1.4.5 STAT pointer_size 64 STAT rusage_user 20.089945 STAT rusage_system 58.499106 STAT curr_connections 16 STAT total_connections 276950 STAT connection_structures 96 STAT cmd_get 276931 STAT cmd_set 584148 STAT cmd_flush 0 STAT get_hits 211106 STAT get_misses 65825 STAT delete_misses 101 STAT delete_hits 276829 STAT incr_misses 0 STAT incr_hits 0 STAT decr_misses 0 STAT decr_hits 0 STAT cas_misses 0 STAT cas_hits 0 STAT cas_badval 0 STAT auth_cmds 0 STAT auth_errors 0 STAT bytes_read 613193860 STAT bytes_written 553991373 STAT limit_maxbytes 268435456 STAT accepting_conns 1 STAT listen_disabled_num 0 STAT threads 4 STAT conn_yields 0 STAT bytes 20418140 STAT curr_items 65826 STAT total_items 553856 STAT evictions 0 STAT reclaimed 0

117.
Memcache - backingup

118.
Memcache - deleting<?php $memcache = new Memcache(); $memcache->delete( 'myKey' ); $myData = $memcache->get( 'myKey' ); // $myData === false

119.
Memcache - tipPage with multiple blocks ? -> use Memcached::getMulti() Warning : what if you get some hits and some misses ?

120.
Naming your keysKey names must be unique

121.
Prefix / namespaceyour keys !

122.
Only letters, numbersand underscore

123.
md5() is useful-> BUT : harder to debug Use clear names

124.
Document your keynames !

125.
Updating data

126.
Updating data

127.
Adding/updating data $memcache->delete('ArticleDetails__Toshiba_32C100U_32_Inch' ); $memcache->delete( 'Homepage_Popular_Product_List' );

128.
Adding/updating data

129.
Adding/updating data -Why it crashed

130.

131.

132.
Cache stampeding elePHPants

133.
Cache stampeding

134.
Memcache code ?DB

135.
Cache warmup scriptsUsed to fill your cache when it's empty

136.
Run it before starting Webserver !

137.
2 ways :Visit all URLs Error-prone

138.
Hard to maintainCall all cache-updating methods Make sure you have a warmup script !

139.
Cache stampeding -what about locking ? Seems like a nice idea, but... Lock in place

140.
-> lots ofnew connections

141.
-> memory spike

142.
What if theprocess that created the lock fails ?

143.
Quick word aboutexpiration General rule : don't let things expire

144.
Exception to therule : things that have an end date (calendar items)

145.
So... DON'T DELETEFROM CACHE (and don't expire unless usefull)

146.
LAMP... -> LAMMP-> LANMMP

147.
Nginx Web server

148.
Reverse proxy

149.
Lightweight, fast

150.
7.5% of allWebsites

151.
Nginx No threads,event-driven

152.
Uses epoll /kqueue

153.
Low memory footprint

154.
10000 active connections= normal

155.
Nginx - atrue alternative to Apache ? Not all Apache modules mod_auth_*

156.
mod_dav*

157.
… Basic modulesare available

158.
Some 3 rd party modules (needs recompilation !)

159.
Nginx - InstallationPackages

160.
Win32 binaries

161.
Build from source(./configure; make; make install)

162.
Nginx - Configurationserver { listen 80; server_name www.domain.ext *.domain.ext; index index.html; root /home/domain.ext/www; } server { listen 80; server_name photo.domain.ext; index index.html; root /home/domain.ext/photo; }

163.
Nginx - phase1 Move Apache to a different port (8080)

164.
Put Nginx atport 80

165.
Nginx serves allstatics (images, css, js, …)

166.
Forward dynamic requeststo Apache

167.
Nginx for staticfiles only server { listen 80; server_name www.domain.ext; location ~* ^.*\.(jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|doc|xls|pdf|ppt|txt|tar|rtf|js)$ { expires 30d; root /home/www.domain.ext; } location / { proxy_pass http://www.domain.ext:8080; proxy_pass_header Set-Cookie; proxy_set_header X-Real-IP $remote_addr; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } }

168.
Nginx - let'sgive that a go !

169.
Nginx for PHP? LANMMP to... LNMPP (ok, this is getting ridiculous)

170.
Nginx with PHPIn the past : spawn-fcgi (from Lighttpd)

171.
Now : PHP-FPM(in PHP 5.3 !)

172.
Runs on port9000

173.
Nginx connects usingfastcgi method location / { fastcgi_pass 127.0.0.1:9000; fastcgi_index index.php; include fastcgi_params; fastcgi_param SCRIPT_NAME $fastcgi_script_name; fastcgi_param SCRIPT_FILENAME /home/www.phpbenelux.eu/$fastcgi_script_name; fastcgi_param SERVER_NAME $host; fastcgi_intercept_errors on; }

174.
Nginx + PHP-FPMfeatures Graceful upgrade

175.
Spawn new processesunder high load

176.
Chroot

177.
Slow request log!

178.
Nginx + PHP-FPMfeatures Graceful upgrade

179.
Spawn new processedunder high load

180.
Chroot

181.
Slow request log!

182.
fastcgi_finish_request() -> offlineprocessing

183.
Nginx + PHP-FPM- performance ?

184.
Reverse proxy time...

185.
Varnish Not justa load balancer

186.
Reverse proxy cache/ http accelerator / …

187.
Caches (parts of)pages in memory

188.
Careful : usesthreads

189.
Nginx is fasterand scales better (but doesn't have VCL)

190.
Varnish - Installation& configuration Installation Packages

191.
Source : ./configure&& make && make install Configuration /etc/default/varnish

192.
/etc/varnish/*.vcl

193.
Varnish - backends+ load balancing backend server1 { .host = "192.168.0.10"; } backend server2{ .host = "192.168.0.11"; } director example_director round-robin { { .backend = server1; } { .backend = server2; } }

194.
Varnish - backends+ load balancing backend server1 { .host = "192.168.0.10"; .probe = { .url = "/"; .interval = 5s; .timeout = 1 s; .window = 5; .threshold = 3; } }

195.
Varnish - VCLVarnish Configuration Language

196.
DSL (Domain SpecificLanguage) -> compiled to C Hooks into each request

197.
Defines : Backends(web servers)

198.
ACLs

199.
Load balancing strategyCan be reloaded while running

200.
Varnish - whateveryou want Real-time statistics (varnishtop, varnishhist, ...)

201.
ESI

202.
Varnish - ESIPerfect for caching pages In your article page output : <esi:include src="/latest-news"/>

203.
Varnish with ESI- hold on tight !

204.
Varnish - whatcan/can't be cached ? Can : Static pages

205.
Images, js, css

206.
Pages or partsof pages that don't change often (ESI) Can't : POST requests

207.
Requests with Set-Cookie

208.
Very large files(it's not a file server !)

209.
User-specific content

210.
ESI -> nocaching on user-specific content ? Logged in as : Wim Godden 5 messages TTL = 5min TTL=1h TTL = 0s ?

211.
Coming to anNginx near you soon... Logged in as : Wim Godden 5 messages <esim:include src="/news" ttl="5m" /> <esim:include src="/menu" ttl="1h" /> <esim:include src="/top" usercookie="PHPSESS_ID" ttl="1h" />

212.
New message arrives...Hash using page name and session

213.
Self-chosen key (i.e.'mails_for_' followed by session) DB

214.
Advantages No hitsto backend anymore (except the first one) ! Not for user-specific content

215.
Not even fornon-specific content

216.
Do we needTTLs ? Logged in as : Wim Godden 5 messages <esim:include src="/news" ttl="5m" /> <esim:include src="/menu" ttl="1h" /> <esim:include src="/top" usercookie="PHPSESS_ID" ttl="1h" />

217.

218.
Not even fornon-specific content No TTLs for non-specific content

219.
TTL for user-specificcontent is required (defaults to 5min) No need to specify ESI parameters in configuration file Only needs enabling

220.
How many Memcacherequests ? Logged in as : Wim Godden 5 messages <esim:include src="/news" ttl="5m" /> <esim:include src="/menu" ttl="1h" /> <esim:include src="/top" usercookie="PHPSESS_ID" ttl="1h" />

221.

222.
Not even fornon-specific content No TTLs for non-specific content

223.
TTL for user-specificcontent is required (defaults to 5min) No need to specify ESI parameters in configuration file Only needs enabling Memcache getMulti -> 1 Memcache request per page

224.
Under development Featureset = unclear

225.
Performance = evenmore unclear Debugging code makes it slow Extends ESI standard, but doesn't follow it entirely (what standard ?) Release date ? End 2011 ?

226.
Tuning

227.
Apache - tuningtips Disable unused modules -> fixes 10% of performance issues

228.
Set AllowOverride to None

229.
Disable SymLinksIfOwnerMatchWhy ? Site in /var/www/domain.com/subdomain/html MinSpareServers, MaxSpareServers, StartServers, MaxClients, MPM selection -> a whole session of its own ;-)

230.
Don't mod_proxy ->use Nginx or Varnish !

231.
High load onan SSL-site ? -> put SSL on a reverse proxy

232.
PHP speed -some tips Upgrade PHP - every minor release has 5-15% speed gain !

233.
Use an opcodecache

234.
Caching storage -Opcode caching

235.

236.
Use an opcodecache

237.
Profile your codeXHProf

238.
Xdebug

239.
KCachegrind is yourfriend

240.

241.
Use an opcodecache

242.
Profile your codeXHProf

243.
Xdebug But :turn off profilers on acceptance/production platforms !

244.
Let's see whatdifference opcode caching and profilers make...

245.
DB speed -some tips Avoid NOW() -> use PHP date("Y-m-d") as a parameter Why ? Query cache ! Index, index, index ! (where needed only)

246.
Use same typesfor joins i.e. don't join decimal with int RAND() is evil !

247.
count(*) is evilin InnoDB without a where clause ! (and there are other examples of specific things to avoid) Select the right storage engine

248.
Persistent connect isnot always good !

249.
Caching & Tuning@ frontend http://www.websiteoptimization.com/speed/tweak/average-web-page/

250.
Caching in thebrowser HTTP 304 (Not modified)

251.
Expires/Cache-Control header

252.
HTTP 304 Firstrequest Next requests

253.
HTTP 304 withETag First request Next requests

254.
Expires/Cache-control header Cache-ControlHTTP/1.1

255.
Seconds to expiry

256.
Used by browsersFirst request Next requests No requests until item expires Expires HTTP/1.0

257.
Date to expireon

258.
Used by oldproxies

259.
Requires clock tobe accurate !

260.
Pragma: no-cache =evil "Pragma: no cache" doesn't make it uncacheable

261.
Don't want cachingon a page ? HTTP/1.0 : "Expires : Fri, 30 Oct 1998 00:00:00 GMT" (in the past)

262.
HTTP/1.1 : "Cache-Control:no-store"

263.
Frontend tuning 1.You optimize backend 2. Frontend engineers messes up -> havoc on backend 3. Don't forget : frontend sends requests to backend ! SO... Care about frontend

264.
Test frontend

265.
Check what requestsfrontend sends to backend

266.
Tuning frontend Minimizerequests Combine CSS/JavaScript files

267.
Use inline imagesin CSS/XHTML (not supported on all browsers yet)

268.
Frontend tuning -inline CSS/XHTML images #navbar span { width: 31px; height: 31px; display: inline; float: left; margin-right: 4px; } .home { background-image: url(data:image/gif;base64,R0lGODlhHwAfAPcAAAAAAIxKAKVjCLW1tb29tcbGvc7OxtZ7ANbWztbW1tbe1t7e1uelMefn1ufn3ufn5+fv3u+MAO/v5+/v7/fGCPf35/f37//nY////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////........MEl0nGVUC6tObNnPceSFBaQVMJAxC4lo3gNOrUaFnTHoAxNm3XVxPfRq139e8BEGAjWD5bgIALw287T8AcAXLly2kjOACdc17higXSIKDO/Lpv7Qq4bw7APgBq8eOzX69InrZ6xe3dbxZffyTGkb8tdx8F+b0Xn2sFsCSBAgTM5lp63RHYnoHUudZgRgkGOGCB+43nGk4OGcQTabKx5dyJKJ7ImoUNCaRRAZYN1ppsrT3Y2gIwyjSQBAtUpABml/0IJGYd6VjQUDH9uBFkGxGm5I8dPQaRUAQUMBdhhBV25ZYUJZBcSAtSJBddWZZ5UAGPOTXlgkNVOSZdBxEwIkYu7VhYnAol5GaadRqF0Uaz0TgXnX2umVFyGakJUUAAADs=); margin-left: 4px; } <img border=0 src="data:image/gif;base64,R0lGODlhHwAfAPcAAAAAAIxKAKVjCLW1tb29tcbGvc7OxtZ7ANbWztbW1tbe1t7e1uelMefn1ufn3ufn5+fv3u+MAO/v5+/v7/fGCPf35/f37//nY/......Uaz0TgXnX2umVFyGakJUUAAADs=">

269.

270.
Use inline imagesin CSS (not supported on all browsers yet)

271.
Use CSS Sprites

272.
CSS Sprites

273.
Tuning content -CSS sprites

274.
Tuning content -CSS sprites 11 images 11 HTTP requests 24KByte 1 images 1 HTTP requests 14KByte

275.

276.
Use inline imagesin CSS (not supported on all browsers yet)

277.
Use CSS Sprites(horizontally if possible) Put CSS at top

278.
Put JavaScript atbottom Max. no connections

279.
Especially if JavaScriptdoes Ajax (advertising-scripts, …) ! Avoid iFrames Again : max no. of connections Don't scale images in HTML

280.
Have a favicon.ico(don't 404 it !)

281.
Tuning frontend UseGET for Ajax retrieval requests (and cache them !)

282.
Split requests acrosssubdomains

283.
Put statics ona separate subdomain (without cookies !) www.whatever.com www.whatever.com images.whatever.com

284.
Tuning miscellaneous AvoidDNS lookups Frontend : don't use too many subdomains (2 = ideal)

285.
Backend : Turnoff DNS resolution in Apache : HostnameLookups Off

286.
If your appuses external data Run a local DNS cache (timeout danger !)

287.
Make sure youcan trust DNS servers (preferable run your own) Compress non-binary content (GZIP) mod_deflate in Apache

288.
HttpGzipModule in Nginx(HttpGzipStaticModule for pre-zipped statics !)

289.
No native supportin Varnish

290.
What else cankill your site ? Redirect loops Multiple requests More load on Webserver

291.
More PHP toprocess Additional latency for visitor

292.
Try to avoidredirects anyway

293.
-> InZF : use $this->_forward instead of $this->_redirect Watch your logs, but equally important...

294.
Watch the loggingprocess ->

295.
Logging = diskI/O -> can kill your site !

296.
Slashdot effect

297.
Above all else...be prepared ! Have a monitoring system

298.
Use a cacheabstraction layer (disk -> Memcache)

299.
Don't install forthe worst -> prepare for the worst

300.
Have a test-setup

301.
Have fallbacks Turnoff non-critical functionality

302.
Questions ?

303.
Questions ?

304.
Contact Twitter @wimgtr

305.
Web http://techblog.wimgodden.be

306.
Slides http://www.slideshare.net/wimg

307.
E-mail [email_address]

308.
Please... Rate mytalk : http://tinyurl.com/cachetune

309.
Vote to seeme talk at Confoo : http://www.confoo.ca Caching and tuning fun for high scalability

310.
Keeping old codealive without non-stop hassle

311.
Beyond PHP :it's not (just) about the code !

312.
Who's in controlof your PHP app ?

313.
Creating Fast, DynamicACLs in Zend Framework

314.
Thanks !

Caching and tuning fun for high scalability @ FrOSCon 2011

More Related Content

What's hot

Viewers also liked

Similar to Caching and tuning fun for high scalability @ FrOSCon 2011

More from Wim Godden

Recently uploaded

Caching and tuning fun for high scalability @ FrOSCon 2011