LLVM performance tests

I’ve been working together with other developers on the ARM backend of LLVM (www.llvm.org) for quite some time. After many improvements and bugfixes, the ARM backend is very reliable and it already can compile large and complex programs.

This week I finished some performance tests of LLVM for ARM. The results were very good. I tested the compilation of two programs: FFMPEG and GTK. Each program was compiled in three ways: gcc, llvm-gcc and llvm-gcc with modifications in the programs build systems to use link-time optimizations. All tests were ran on an Texas Instruments OMAP 1710 board with 32MB of RAM running Debian Linux with kernel 2.6.18-omap1.

In these tests, only the main program was compiled by the llvm-gcc. Any dependency was picked in binary form from the Debian distribution. If we compile all dependencies with llvm-gcc, probably, we can get better results.

Program versions:
gcc – 4.1.2
llvm-gcc – svn revison 40660
ffmpeg – svn revision 9470
gtk – 2.10.12

Average time to convert 1000 frames from mpeg4 to mpeg1.
gcc – 336.87s
llvm-gcc – 323.9 s (4% better than gcc)
llvm-gcc with LTO – 323.7s (4% better than gcc)

GTK test
Average time to run perftest (a gtk internal test) with 1000 iterations.
gcc – 198.29s
llvm-gcc – 192.7s (3% better than gcc)
llvm-gcc with LTO – 194.65s (2% better than gcc)

Even though LLVM has fewer years of development than gcc, it already gets better results in these two tests. In my opinion, the main LLVM strength is its modular and well designed architecture. It makes much easier to contribute to LLVM than to contribute to GCC. This shows how a well designed compiler architecture can improve the community software development speed.

Another interesting thing to be investigated is, at least in this two test cases, the link-time optimizations don’t improve the performance. Maybe in other test cases, we can get better link-time optimizations results.

These results show that we can expect a lot from the LLVM future.


4 Responses to “LLVM performance tests”

  1. diego roversi Says:

    I suggest to compare gcc 4.1 generating eabi code. it generates quite better code then oabi.

    I’ve tested gcc 4.1.x code generated by a custom OpenEmbedded distro, with gcc 4.1 of a debian on StrongArm, and seems generating better code. I’ve benchmarked it using byte bench, and give 2-4% on int bench and much more on float (softlib FTW 🙂 )

  2. laurovenancio Says:

    The tests were done using the EABI.

  3. llvm-newbie Says:

    did you create bitcode from these tests and run it with lli? can you maybe list how you compiled and ran the tests? i am pretty new to llvm, just trying to learn


  4. Nadav Says:

    Hi Lauro. Good job on optimizing LLVM! I would love to read in more details about your changes to the ARM backend. LLVM is a great compiler and I enjoyed using it myself.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: