Sunny Days – Day 10

I know, quite a lot of time has passed since I last updated the Sunny Days. Two major reasons for that: First, I’ve been quite busy with other work. Second, I tried to test what Sascha suggested: whether using a bytecode cache will boost the delivery of dynamically created pages on the Sun. Since there’s only one product available for Linux and Solaris and for this product happens to be the Zend Optimizer, I settled for that. So, let me present to you what by now I call

The Zend Fiasko

What I did was to install the Zend Optimizer using the installer available for the respective operating system. I didn’t do any optimization. The hardware used in the test was the same you may already be familiar with: My trusty old Compaq Proliant DL380, the Lynx Workgroup Server 4300 and the Sun Fire T2000. Nothing on their setup was changed since the last test except for the installation of the Zend Optimizer. The test run were also the same, using the Apache Benchmark to measure the rate of pages delivered.

Instead of posting all the results as I did last time, I’ll just post the most significant chart for the various runs this time. If you’re interested in the complete data set, you’ll find links at the end of this post.

Static Page, No Keepalive

For the static tests I choose this slide as a reference for it clearly shows the problems I had: completely erratic results for the Sun T2000. While the Lynx’s results almost resemble the runs without Zend Optimizer installed and even the Proliant is well within the bounds of former results, the Sun behaves completely out of order. It won’t reach the performance levels from the previous run of the tests. Also the dropping point isn’t at 500 concurrent connections as it was before but already at 75 concurrent connections.

Static Page, Keepalive active

Now, this looks more like what one would expect after the tests without Zend Optimizer being installed. The Sun is a bit slow to get into the game but then takes the lead and keeps it ’til the end. The Lynx as expected starts strong but drops performance at the end. Ok, so far so good. But have a look at the next picture:

For the life of me I don’t have the slightest idea what happened there. There was no other traffic on the network and re-running the tests didn’t help either. That just gave more erratic results. In fact, what I present here is the last of about 15 benchmarking runs and each of them presented different results for the Sun while results for the Lynx and the Proliant were at least comparable to the runs without Zend Optimizer.

Dynamic Page, No Keepalive

Now, please, have a look at this. Both, the Lynx and the Proliant, are still unaffected by the Zend Optimizier. You couldn’t say that for the sun, however. The performance of the T2000 drops even below the levels of the Proliant when the Zend Optimizer is running. Whoa. This is consistent throughout all the tests. But please also note that both the Lynx and the Proliant didn’t finish the run due to errors when having to serve 1000 concurrent connections. This happened at various points in this run.

Dynamic Page, Keepalive active

Using the keepalive feature the picture doesn’t change much except that both the Lynx and the Proliant are now able to finish the run – although with a heavily degraded performance in case of the Lynx. The results for the Sun don’t show any significant change compared to not using the keepalive feature.

Conclusion

As far as I can tell right now, the best option you have is to stay away from Zend Optimizer at all cost. If time permits I will do some more testing but the previous runs already did take up a lot of time I might have better spent testing other things. Still, the results bother me. So if you happen to know something I might have missed, something that can be tuned to make either the Zend Optimizier “behave” or some setting I may have overlooked: Please tell me. Time *is* running out and I’d love to do more tests while I can. Unfortunately I’ll be on the road a lot for the coming weeks. While I’m writing this, the machines are chewing their ways through a MySQL benchmarks that’s running for three days already … So, well, you’ll at least read the results of that tests before I have to return the T2000.

Oh, and lest I forget them: Here are the spreadsheets in Excel and Open Document format.

[ All posts about my experiences with the SunFire T2000 >>> ]