Ubuntu ppc64el performance

Question

I installed Ubuntu Linux 14.04.5 on a new IBM Power8 mod. 8286-41a. My goal is to use this partition the application SugarCRM CE 6.5. I’m not been able to install ubuntu 16 because compatibility issues with PHP 5.6. Currently i use sugarcrm on a vmware virtual machine with ubuntu 14.4.1. I was expecting a performance boost on ppc64el platform. Instead I've found, in fact the procedures are considerably slower than the amd64 machine Same lamp version on both system (apache 2.4.7, mysql 5.6.33, php 5.5.9) See attached image for information on the two machines. Any suggestions? ppc64el vs amd64 config

Answer 1

tl;dr: You just happen to be using versions of everything that are before a lot of ppc64el and POWER8 optimizations went in, so there's going to be limits on what performance you can achieve without upgrading one (or more) components.

Around the time Ubuntu 14.04 was released, we went through the whole LAMP stack looking closely at performance on POWER. We found that there were places that were lacking, so we went to work fixing them. Unfortunately, since Ubuntu 14.04 shipped slightly before the first POWER8 systems did, there wasn't enough time to work on the optimizations and get them into upstream before Ubuntu 14.04 shipped.

It was only in May/June 2014 that I had an initial (relatively) stable patchset for MySQL 5.6 working on POWER8. This was a couple of months after Ubuntu 14.04 release. These initial patches weren't too focused on performance (although Ubuntu did pull them in to the distro, as ppc64el is a supported platform for them).

For performance, we were pretty heavily gated on a few things, many of which were being fixed in MySQL 5.7. I had a very proof-of-concept MySQL 5.7 port doing 1 million queries/second on POWER8 back in June 2014. A lot of the MySQL problems are around mutex contention, so it can (strangely enough) with older MySQL improve performance by tying MySQL to fewer CPU cores.

We had other bottlenecks to tackle too, notably CRC32 performance (which if you configure it to do so, InnoDB uses as its on disk checksum), of which emerged a POWER8 optimized CRC32 that is 41 times faster than the non-optimized version. This work was integrated with MariaDB 10.1.

A lot of work was done to get modern MariaDB running really well on POWER8. It does, so if switching to MariaDB is an option, that will help with database end performance. Also, the various _instance variables for MySQL may need increasing - your POWER8 system likely has a lot more CPU cores and threads than an x86 system, and the default values aren't going to be that great.

As for PHP, I was a (lot) less involved in that optimization work. Basically, newer PHP is faster, often by a decent amount. This was due to everything from removing some old powerpc "optimizations" (custom assembly isn't always faster than what a compiler can produce) to contributing code to php, to (i think) improving GCC.

I think my colleague Daniel Axtens is going to reply with some of the PHP details.

Back to your specific configuration, have you assigned 1 CPU core (with 8 threads) or just 1 CPU core (with 1 thread) to the VM on the POWER8? The way virtualization works on POWER, you may as well give the guest the threads, as a the whole core enters the guest at the same time, and SMT4 is usually a big boost for performance over SMT1 (check your VM configuration and you can dynamically change the thread mode using the ppc64_cpu utility in the guest, but not to more threads than the guest has been assigned).

I blogged a bunch on the POWER8 MySQL/MariaDB work over at: https://www.flamingspork.com/blog (see the mysql/power8 tags... askubuntu won't let me post too many links)

Answer 2

Your question is pretty difficult to answer as asked - it would be really helpful to know what parts are slower, how you're measuring speed, etc.

But here are a few pointers:

Firstly, you need to move to 16.04.

• You've said you do that because of PHP issues. I assume by that you mean that SugarCRM relies on PHP5 and 16.04 only ships with PHP7. This is a shame: php7 is much faster than php5 (around 2x, most of the time) on both Intel and Power. But that's not the end of the world. You could move to 16.04 and use HHVM, which is compatible with PHP5 and is usually a lot faster. There's been a lot of great work getting that to work well on Power (some of which I've been involved in). I think most of the work is now upstream, but there's a development repo that should be even more up to date. You'll have to compile it from source, but the instructions are good.

• You really, really need to get an upgraded MySQL/MariaDB, both from a stability and performance point of view. Whatever is shipped with 14.04 is a bad idea.

• As Stewart mentioned, there are a lot of fixes that went in between 14.04 and 16.04. These chips are different to Intel chips: tuning applications for them is different and relies on work we have been doing over a few years. These fixes range from kernel fixes to power scaling to compiler improvements to generate better code.

Secondly, I'm not sure what workload you're testing on SugarCRM. IBM has a (now somewhat dated) white paper on running SugarCRM on Power that shows it sustaining much greater throughput than an Intel system. You need to be clear on the benchmark you're interested in.

Thirdly, you need to check your system settings. Here are some common ones:

• Apache not set to use pthread mutexes instead of file mutexes (fixed in newer Ubuntu versions)

• PHP and Apache set to use a thread pool that is too small for a Power system.

• It looks like you're comparing 1 core on P8 to 4 cores on x86. P8 has both cores and hardware threads (like HyperThreading but instead of 2 hardware threads per core we have up to 8), so this can get a bit confusing. You should:

  • give you Power VM 4 cores. If possible, make sure they're statically allocated in your hypervisor (as in, the cores are dedicated to the guest).
  • Set those 4 cores to SMT-4 mode, not SMT-8 mode; this tends to give a considerable boost to scripting languages.

Lastly, if you have access to any IBM support or sales people from when you bought the machine, please reach out to them; they may be able to get some experts in.

Answer 3

Since you are doing performance measurements, create the partition from the HMC using dedicated processors, with 4 cores as the desired (by using dedicated, you will be separating the IBM i workload from this test workload). As mentioned before, thread count is different from Power than from x86, so you have to make sure you have the right amount of cores, not threads (even setting SMT-4 as recommended, you will still have twice the amount of threads per core on Power).