helion-prime
home about us blogs contacts

Typo3, Joomla!, and Drupal CMSs performance testing

Published by alex.shapovalov on May 19, 2008 under drupal, joomla, typo3  

preamble

Our company above all uses 3 CMSs [http://en.wikipedia.org/wiki/Content_management_system]
Typo3, Joomla!, and Drupal since we find them the most appropriate for most of our tasks.
Check this our article to find more about it [cms-based-web-applications-fast-way-for-creation-of-web-based-solutions.html]

Sometimes our customers asks what is faster, trying to find fair answer I made a short research
in order to compare them and provide our customer as well as you with pretty graphs.

testing conditions

I have created virtual dedicated server with following configuration within our server
to have a picture of average hosting providers.

configuration:
architecture: x86
CPU: 3.0Ghz
RAM: 512Mb
OS: Debian GNU/Linux lenny/sid (testing)
kernel: 2.6.22-3-686

On all CMS returned page contains:
menu, login form, 2 articles with overall size 40Kb

As Joomla! doesn’t have built-in internationalization functionality I made tests with JoomFish 2.0beta extension
that provide that functionality and without it.

Every testing series contains results for 1, 10, and 20 simultaneous users.
Caching functionally:
turned off for Drupal, Joomla!, and Typo3;

apache configuration:
apache web-server API: apache2.0 handler
processing model: mpm-prefork [http://httpd.apache.org/docs/2.0/mod/prefork.html]

software versions:
apache: 2.2.8
php: 5.2.5-3
mysql: 5.0.51a-3

joomla!: 1.5.2
typo3: 4.1.6
drupal:6.2

comparison results

So it’s time to see some results. Keep your eyes widely opened..

note:
results in the tables are presented in milliseconds per page request
and surely smaller result is better

1 user

joomla+joomfish joomla drupal typo3
average 261 185 130 233
median 260 184 129 232
min 253 180 127 225
max 311 209 144 248

10 users

joomla+joomfish joomla drupal typo3
average 2093 1510 1118 2039
median 1965.5 1416 1040 1247.5
min 253 181 126 226
max 36422 31591 30999 29138

20 users

joomla+joomfish joomla drupal typo3
average 4106 2908 2123 3886
median 3689 2306 1552 2075
min 254 182 128 233
max 119187 112612 83831 105285

fast conclusions:

as you see in the tables and on graphs

**Drupal is noticeably faster then competitors
**joomfish is a break for Joomla but it is very important extension and very likely it is installed on your system
**without JoomFish Joomla! can be even faster then Typo3

Be Sociable, Share!

Tags: , ,

26 Responses to “Typo3, Joomla!, and Drupal CMSs performance testing”

  1. [...] Go to the author’s original blog: Typo3, Joomla!, and Drupal CMSs performance testing [...]

  2. WC Leung says:

    Thank you very much for the testing. BTW, is there any result for Drupal I18n L10n plugin?

  3. George says:

    Drupal vs Joomla: performance, tested in 2006:

    http://buytaert.net/drupal-vs-joomla-performance

  4. It’s nice that you’re doing performance testing, but your graphs and tables don’t have and units of measurement. You mention that the numbers are milliseconds, but I can’t see whether they are milliseconds per page request or total milliseconds for the test. You should at least say “Average milliseconds per request”, and then point out that “smaller is better”.

  5. alex.shapovalov says:

    thank you for your comments Robert.
    I’ve made appopriate changes ..

  6. ircmaxell says:

    One thing to note. Drupal does have a built in caching system that cannot be turned off. This is very similar to the Joomla caching option in the global configuration (Excepty Joomla’s can be turned off). Please re-run tests with this caching enabled (not the plugin) to get a fair comparison. Joomla’s cache plugin (included in 1.5) is equivilant to Drupal’s optional caching…

    Don’t believe me? Check out the Drupal database’s cache tables. Even with cache off, they are populated…

  7. alex.shapovalov says:

    Hello ircmaxell.

    First of all thank you for your notes.

    >>Drupal does have a built in caching system that cannot be turned off.

    >>Don’t believe me? Check out the Drupal database’s cache tables.
    >>Even with cache off, they are populated…

    Work of caching can be easily detected: first reply takes much longer then followings.
    Look on this example with Drupal:
    1 user, cache-turned off : 132, 129, 135, 129, 133, … (elapsed time in milliseconds per page request)
    1 user, cache-turned on : 485, 19, 18, 17, 17, … (elapsed time in milliseconds per page request)

    so conclusion is: even if it populates table it is not mean that it use it ..

  8. Codeline says:

    Сравнение производительности Typo3, Joomla и Drupal…

    С ЛОРа занесло ссылку на сравнение CMS.
    Typo3, Joomla!, and Drupal CMSs performance testing
    Мне кажется сравниваются не очень сраниваемые вещи, в виду различного подхода к архитектуре и принципам этих систем. Например, сравнение TYPO3 с Битрикс еще имело бы смысл.

    Но раз такие сравнения делаются. значит это кому-нибудь нужно

  9. can3p says:

    Все правильно сравнивают. Тут смысл в сравнении одинаковой реализации одной и той же функциональности. И то, что выдает drupal, мне нравится

  10. Anton Sidashin says:

    At last, performance testing in production-ready environment. Thank you!

  11. f3ex says:

    Это на сам деле хорошо,

    ЧТо значит среднее, мин\макс ? каким образом это определялось, что-то непонял

    но что будет при тестирование 100-500 пользователей? было бы неплохо еще посмотреть при 300 пользователях и включенном\выключенном кешировании. Кеширование разумеется средствами CMS (в джумле насколько помню – оно по дефолту выключено)

    З.Ы. ЛОР

  12. Drupal vs Joomla article in Dutch.

  13. Andrew Eddie says:

    Thanks for the interesting article – just a couple of questions.

    Where the installations weighed in any way to get a roughly similar template output. For example, did you just use the default installs for each CMS, or did you tune each of them to ensure they are outputting the same page (within reason of course). A screenshot of each site would probably be helpful. Basically what I’m asking is what sort of site are we comparing? If it’s default installs, well, Joomla! is at a disadvantage because it has a very rich set of sample data as compared to at least Drupal (not sure on Typo3).

    Second question is do Drupal and Typo3 have similar on-the-fly translation engines to compare with? That would be good to see.

    Finally, do you have a PHP cache or accelerator running? J!1.5 performs better with one.

  14. alex.shapovalov says:

    Hello Andrew Eddie.
    Thank you for comments.

    >>Where the installations weighed in any way to get a roughly similar template output.
    >>For example, did you just use the default installs for each CMS,
    >>or did you tune each of them to ensure they are outputting the same page (within reason of course).
    Sure I made special pages and redesigned template to have similar pages with approximately same sizes for every CMSs.

    >>Second question is do Drupal and Typo3 have similar on-the-fly translation engines
    >> to compare with? That would be good to see.
    I have not worked with these engines. I will think about such testing.

    >>Finally, do you have a PHP cache or accelerator running?
    >>J!1.5 performs better with one.
    No. I used only pure GNU/Linux Debian php5 package with default settings.
    Anyway any improvement will provide same advantages for every CMSs.
    And without php and internal CMSs caches we have fair comparison.

    Hope I answered your questions..

  15. Dmitry Dulepov says:

    This test is synthetic and does not have much value. No one runs systems like that in real life. Every real web server is optimized for performance of the system it runs. I do not know anything about Joomla or Drupal, but this system is not configured optimally for TYPO3. So TYPO3 results can be discarded as improperly tested.

  16. alex.shapovalov says:

    Hello Dmitry Dulepov.
    Thank you for your comments.

    >>No one runs systems like that in real life.
    >>Every real web server is optimized for performance of the system it runs.
    I fully agree with you no one use systems like that in production environment.

    But I make tests to test effectiveness of pure systems rather then caching sub-system.
    I think that developers should think about effectiveness of their components and
    don’t pass care about performance entirely to caching or other technics.

    Surely cache is good as well as better CPU, bandwidth, and cluster when
    solution is already got all benefits of good software design.

    Do you agree?

  17. Anthony says:

    >>But I make tests to test effectiveness of pure systems rather then caching sub-system.
    >>I think that developers should think about effectiveness of their components and
    >>don’t pass care about performance entirely to caching or other technics.
    I completely don’t agree with that. Sometimes, doing certain things is plain hard (cpu time wise), and the only way to get good performance is to cache it. Joomla! does this with the menus. It’s an xml node based menuing system, which provides a HUGE amount of flexability, but it takes a bunch of memory and time to build. So the system is designed with caching in mind. Disabling it, or honestly testing without it is not testing the system. In some scenarios, cache is an afterthought, but a lot of them caching is esential.

    >>Surely cache is good as well as better CPU, bandwidth, and cluster when
    >>solution is already got all benefits of good software design.
    NOOO… I can do more with cache, than you could do with a cluster. I have a site running right now on a single box that would take at least 8 to 10 servers in a cluster if it wasn’t for caching. If caching is designed well, there are no negatives.

    Also, purposely benchmarking without opcode caching is fine, as long as you realize nobody in their right mind would run a high volume site without it. It’s that important. So I do agree with Dimitry, in that “No one runs systems like that in real life”, but only for reasonably high volume sites. Your tests may be valid for a site hosted in a shared environment, but for the rest of the world, we use things like opcode caches, and memory resident sessions, and tune the daylights out of MySQL, php and (cough)apache(cough)

  18. alex.shapovalov says:

    Hello Anthony.

    Please, find my complete answer in my new post:
    http://blogs.helion-prime.com/alexshapovalov/2008/07/02/battle-n2-typo3-joomla-and-drupal-cmss-performance-testing.html

    Hope I answered your questions..

  19. Tosin says:

    Which tool did you use to measure duration? (milliseconds per page request)

  20. alex.shapovalov says:

    Hello Tosin.

    >>Which tool did you use to measure duration? (milliseconds per page request)
    I have used Jmeter.

  21. typo3naut says:

    Using TYPO3 without caching is like driving a porche with activated handbreak ;-)
    Nobody uses TYPO3 without caching. So, it’s a really strange performance test you’re doing here.

  22. Anthony says:

    Nice CMS comparison,
    But WHY do you use all 3 of these.
    I’d like to see a Pros & Cons review.
    Typo3 seems excessively “guru” intensive, but what about Drupal vs. Joomla ?
    Cheers,
    Anthony

  23. alex.shapovalov says:

    Hello Anthony.

    There are enough comparisons on Internet that compare exactly features.
    But it is hard to compare: what to compare? : features, usability, compare for some platform (php, ruby, java, etc) or lightweight, or only commercial CMSs?
    An important note: in our company we use only open-source software.

    I use my current experience: most wonderful CMSs [enough lightweight, mature, good api] now written in php, although I prefer Java over any other software language:
    joomla – user-friendly, good api, and documentation
    drupal – super lightweight, good documentation
    typo3 – extra features, good documentation

    Sorry if I missed your favorite one, but world is big .. and probably very soon I will like something in Ruby on Rails.

  24. sandrar says:

    Hi! I was surfing and found your blog post… nice! I love your blog. :) Cheers! Sandra. R.

  25. Textnik says:

    Very interesting, I have just switched from Joomla to Drupal.. :)

  26. Furkan Güngör says:

    Thank you for any other excellent post. Where else may anybody get that type of info in such a perfect manner of writing? I’ve a presentation next week, and I’m on the look for such information.

©2010 Helion-Prime Solutions Ltd.
Custom Software Development Agile Company.