Out of curiosity I decided to invest a few hours and tested three PHP products that offer Bytecode (Opcode) Caching.
There are five products available, of which I tested three:
- Zend Platform
- APC (APC GUI)
- XCache
- eAccelerator
- ionCube Encoder
- PHP Accelerator not tested, does not support PHP 5.x
Since I am a lazy turd, I used my development box for the benchmark, which runs WinXP Pro (SP2), Apache 2.0.59 and PHP 5.1.2.
Installing and setting up either APC and XCache was easy enough. The Zend Platform gave me some headaches. It would not finish its installer, saying “Registering Node with the Zend Central”. After some trial and error (playing with the web server address - localhost, 192.168.1.10, 127.0.0.1), 127.0.0.1 and using “Express Installation” did the trick. Well, sort of. The weird message still doesn’t want to go away, but apparently everything works just fine. And very fine, how you will see soon.
But let me introduce the scripts used in the benchmark. First, a simple “Hello World!” script. Then the “Main”-tab of phpMyAdmin 2.8.2.4 and last but not least the homepage of a freshly installed MediaWiki 1.7.1. Since the latter two require a database, and thus a database server, here is what I currently use: MySQL 4.1.13, more or less out of the box. So nothing fancy here. PHP itself is equipped with two extensions, mbstring and mysqli. On top of that either APC, XCache, eAccelerator or ionCube. For the Zend Platform, the delivered Apache 2.0.50 and PHP 5.0.5 were used.
Each configuration was softly hammered with five tasks via ab:
- Execute helloworld.php 1000 times
- Execute phpMyAdmin/main.php 100 times
- Execute phpMyAdmin/main.php 1000 times
- Execute mediawiki-1.7.1/index.php/Main_Page 100 times
- Execute mediawiki-1.7.1/index.php/Main_Page 1000 times
The guys over at iPerSec did a similar benchmark. They added a timestamp change for the requested files to see how the cachers cope with that. Personally I believe that in a production environment such changes do not happen that often, so I skipped this suggestion.
OK, here the executive summary:
- Use APC or XCache. Both provide a performance gain of over 190%. APC may have an advantage because it belongs to PECL, a controlled set of offical extensions for PHP. XCache is relatively new, but sure worth a look.
- ionCube Encode is not a real opcode cacher. It is used to encode your PHP source to protect it from being read and altered. It offers to optimize the code while encoding it. Since the code needs to be decoded to run, ionCube fails and comes out even below standalone PHP wit just 94% of its performance.
- Zend Platform increases overall performance to 128%. Of course Zend Platform is much more than an opcode cache - it offers debugging, drilling into events and monitoring of all things PHP related. Tight integration with Zend Studio makes these two a very powerful development environment. There are evaluation versions available, to go ahead and find out for yourself. The even greater news: Zend Platform is free for developers!
For all number-addicts, here’s the full chart (and a PDF if it, too).
Any special reason you chose to not include IonCube?
Yeah. Simply skipped that the Encoder actually does bytecode caching. I will update this entry as soon as I get to it, possibly tomorrow evening. Thanks for the hint ;)
By the way, eAccelerator 0.9.5 is now out of beta (released Oct 11 2006) and seems to work okay with PHP 5.1 and 5.2. You might want to add it to your benchmarks too.
Thanks for the info D. Looks like it will be a fun weekend ;)
BTW, I did some first tests with IonCube. It looks promising so far.
Just before the weekend ends ;)
BTW, the +2000% for Zend Platform resulted from my messing up with copy&paste, sorry ;(
You might want to add how you called ab (or ab2?) from the command-line and the output from ab (failed requests, statistical outliers pulling down the average etc.), and try out other values for concurrency as well. From my experience with greater concurrency (up to a point) the difference between opcode cache and plain old php becomes greater. Though granted - this may not be all that relevant for comparing between the different caches.
Thank you for your suggestions, Sencer. This is the first time I publish such a benchmark and surely I can improve its worth by adding more data to it.
For the record: I run ab with
ab -n [100|1000] -c 10 {URL}. I chose a concurrency of ten because I wanted to primarily test the caches and not Apache HTTPD but add a bit of realism to the tests anyway. I will experiment with different settings. I see, this is going to take slightly longer than anticipated ;)PHP 5.2 with eAccelerator 0.9.5 starts to issue errors that it can’t access the protected fields (from within the object itself), and without debug it jsut segfaults.
Also, I think it is not correct to test in windows and made my own benchmarks.
XCache shows to be the same as APC, maybe insignificcantly faster on a single-CPU server.
Though, XCache is reported to give bigger advantage in SMP.
Heute möchte ich euch 2 Blogs von sehr engagierten Bloggern vorstellen. Als erstes haben wir hier Sascha, der in seinem heutigen Beitrag sehr sauer ist. Er hat im Oktober einen, wie ich finde sehr lesenswerten und äußerst informativen…
Would you mind to check/compare the five with PhpExpress by NuSphere?
http://www.nusphere.com/products/phpexpress.htm
To me it appears that PhpExpress is pretty stable and brings no problems with existing scripts over APC and eAcceleartor that were crashing my site time after time when I tried them. Also, I’d mention that APC suffers from problems with include()/require(). I don’t want to rewrite my scripts and have absolute paths everywhere. Otherwise APC seems does not work. At least what works fine without APC does not work with it.
EAccelearator was fast but it appeared that it does not lock shared memory and it’s what brought the performance. It turs out that in many cases it crashes as soon as your site gets any significant load with 100-500 scripts in use. Also, it is not compatible with debugger I use.
Same goes for xcache which is nothing more but yet another derivation work of turk mmcache.
So I still use PhpExpress and have no problems :)
Your chart says that eAccelerator has no “gui” but it does actually have a web management interface (just not installed/activated by default) - google the “eAccelerator control panel”
Thank you for your suggestions, Sencer. This is the first time I publish such a benchmark and surely I can improve its worth by adding more data to it.
For the record: I run ab with
ab -n [100|1000] -c 10 {URL}. I chose a concurrency of ten because I wanted to primarily test the caches and not Apache HTTPD but add a bit of realism to the tests anyway. I will experiment with different settings. I see, this is going to take slightly longer than anticipated ;)I'm still new to this op-code thingy. Quick question, do these accelerators have their own op-code format or do they all follow the Zend way of doing it? Would it be possible to dump APC opcode and have Zend run it? (It may sound stupid, so sorry in advance)
Second question, is it possible to dump Zend op-code and run it straight thereby skipping the lexing, parsing, and compiling stage? If so, how is Zend op-code saved from memory to file and how is it executed?
Thanks!
#1: Actually, I'd rather say - without having tried it - that you might fall into some incampatability traps here.
#2: To obtain the performace increases, Zend (and all OpCode caches) use filestamps to assess whether to compile anew or to run the existing opcode. How to dump Zend's OpCode from memory - I have no clue ;) You might get an answer at the Zend Developer Zone at http://devzone.zend.com/public/view
Thanks Sascha. I'll take a look at the link you gave.
Thanks! I will try APC and eAccelerator.
Im using eAccelerator and glad with a result.
My full set of optimization consist of nginx(front end)+apache+eaccelerate+memcached.
Try out.