I took some time to make some tests with compiler optimizations and while trying some settings and triggers I thought it would be interesting to profile the execution times of compiled code to see what the compilers really do.
Tools I used:
- Paul Jackson's in-game-profiling library in C++
- Dev-Cpp using MinGW v3.4.2
- Visual C++ 2008 Express Edition
The following class was created:
1: class ClassProfile
2: { 3: private:
4: float x;
5: float y;
6: char szText[255];
7: public:
8: ClassProfile();
9: ~ClassProfile(void);
10:
11: void ExecDoLoop();
12: void ExecForLoop();
13: };
Listing 1
The following calls were made:
1: ClassProfile m_Class;
2: ClassProfile* m_pClass;
3:
4: dev_ProfileBegin("ExecForLoop_obj_Outer" ); 5: m_Class.ExecForLoop();
6: dev_ProfileEnd("ExecForLoop_obj_Outer"); 7:
8: m_pClass = new ClassProfile();
9: dev_ProfileBegin("ExecForLoop_pointer_Outer" ); 10: m_pClass->ExecForLoop();
11: dev_ProfileEnd("ExecForLoop_pointer_Outer"); 12:
13: delete m_pClass;
Listing 2
The testdrive
The function ExecForLoop consists of two cascaded for loops which perform some basic arithmetic operations. The three candidates are Dev-Cpp vs Visual C++ 6.0 vs Visual C++ 2008 Express Edition. In all the three IDE's an empty console application was created. A release version was compiled. The compiled version does not include compiler optimizations but was compiled using default settings.
Results in direct comparison:
| Version |
Visual C++ 6.0 |
Visual C++ 2008 |
Dev-Cpp |
| |
min |
max |
avg |
min |
max |
avg |
min |
max |
avg |
| ExecForLoop_obj_Outer (10 cycle) |
0.1 |
0.1 |
0.1 |
15.1 |
15.1 |
15.1 |
0.1 |
0.1 |
0.1 |
| ExecForLoop_obj_Inner (25000 cycles) |
50.7 |
50.8 |
50.8 |
24.4 |
24.4 |
24.4 |
50.8 |
50.9 |
50.8 |
| ExecForLoop_pointer_Outer (10 cycles) |
0.2 |
0.2 |
0.2 |
30.6 |
30.6 |
30.6 |
0.1 |
0.1 |
0.1 |
| ExecForLoop_pointer_Inner (25000 cycles) |
49.3 |
49.4 |
49.3 |
26.6 |
26.6 |
26.6 |
48.7 |
48.7 |
48.7 |
What we see is that creating objects on the stack, the code execution time of VC++ 2008 is unbeatable with incredible 24.4 ms. VC++ 6.0 and Dev-Cpp runs equal with about 50.8 ms. But take also a look on the outer loop. Wow 15.1 ms for the VC++ 2008 but only 0.1 ms for VC++ 6.0 and Dev-Cpp either. Why that? To explain this you should have a basic understanding of the memories available on your machine and also compilers. The .NET framework has an incredible memory handler and so the compiler handles the code. While loosing some ms on the outer loop the inner loop with the time critical path takes half the time of the other compilers. Cool isn't it?
Now let's take also a look on our object that is created on the heap. What is the advantage of the heap? The heap provides you nearly endless memory (only limited by your hardware) but is slower than the memory on the stack but we see that all applications are faster on the heap than on the stack. So how can happen this? Quite easy. The allocation for the memory on the heap and the object creation is done only once with the line 8 in listing 2. The memory is already reserved for the function and so it can be executed as often as possible, it will always be fast since the functions calls the entry point in memory where everything is already stored. If we would create the object dynamic within a loop it would take much longer because the allocation handling and accessing is what makes the heap slower than the stack. The difference between VC++ 6.0 and Dev-Cpp is very marginal, again VC++ 9.0 is the winner.
Also interesting is the file size.
| Version |
Visual C++ 6.0 |
Visual C++ 2008 |
Dev-Cpp |
| File size |
73,728 bytes |
10,240 bytes |
1,743,880 bytes |
What do we learn from this table? Nothing we didn't already know but let's go more into detail. The smallest file was produced from the Visual C++ 2008 with about 10 Kbytes followed by Visual C++ 6.0 with 72 Kbytes and on third and last place with over 1.66 Mbytes comes the Dev-Cpp version.
Why are there such great differences? Let's start with the two extremes VC++ 2008 and Dev-Cpp. VC++ 2008 has such a small file size because it uses the .NET framework. That means that you need a framework installed which's download size varies between 22 - 195 Mbyte. The compiler simply stores the most necessary into the application, everything else (memory tables etc) is called from the framework. Dev-Cpp instead has a file size of 1.66 MByte because MinGW links from static libraries as much as possible. So a minimum on additional dll's and runtime environment is required but that increases file size of the application.
In other words you could run the Dev-Cpp console application on a new installed machine without having to install any other runtime or framework. You would have to install the Microsoft Visual C++ 2005 Redistributable Package to run the VC++ 6.0 application and you would even have to install the .NET framework to run the VC++ 2008.
I want to point out that all the three compiled release versions have been compiled without any compiler optimizations. I also want to point out that I used Visual C++ 2008 Express Edition with the .NET framework 3.5. As you know the Visual C++ 2008 Professional Edition provides an optimized compiler and could produce better code quality which would top the already great timings of above.
Didn't we forget something?
No I didn't (the first)!!! Borland fans will complain why I didn't refer to the Borland C++ Builder. Two reasons for that. First is that I didn't want to go through all the registration process on their website to register for a trial of their latest product Borland C++ Builder 2007 and I couldn't find my old CD with Borland C++ Builder 5. The second reason is that Borland doesn't provide a free version of this. This article is for those of you who don't want to buy a compiler for 900 $ when you are able to download one for free.
No I didn't (the second)!!! LCC IS NOT a C++ compiler. Although this is a great project (and I'm using this too) it's a C compiler and not a C++ compiler which makes it for me uninteresting. Don't get me wrong. I like lcc but I'm more the OOP guy.
Conclusion
I love Dev-Cpp and work as often as possible with this tool. Not only you learn to develop with C++ the ANSII "way" but it's free and uses, with MinGW, a very powerful compiler. It doesn't need any runtime or framework to run (when using standard functions). I learned to love Visual C++ 2008 and try to go more into deep with this tool. Microsoft offers a really cool IDE with, as we can see above, a really powerful compiler which produces good and fast code and all this for free. This is an important thing (especially when we thing of the endless discussions in bb's how bad Micro$oft really is... blablabla) when thinking to develop for private use.
Try it for yourself. All you need can be found below in the link section and you need the function timeGetTime() or a timer written in Assembler if you need a more accurate timing method ;)
Links
- Microsoft Visual C++ 2008 Express Edition can be downloaded here
- Dev-Cpp 4.9.9.2 with MinGW 3.4.2 can be downloaded here
Subscribe