The Impact of Disabling Atomic Optimizations on Compiler Behavior

More than half of a typical compiler’s source code is devoted to optimization-related functions. These optimizations aim to improve code efficiency but are difficult to verify against formal specifications. Experiments indicate that advanced compiler optimizations are less dependable than often assumed, with numerous cases of software "bugs" introduced by optimization transformations—affecting both security and performance.

The most frequent issue stems from exploiting undefined behavior. Some compilers, or newer versions of older ones, operate under the assumption that undefined behavior does not occur in compliant C programs. As a result, they apply optimizations that cause programs to behave unexpectedly at higher optimization levels, such as -O2.

Therefore, rigorous testing of compiler optimizations during qualification is essential. Unfortunately, most commercial test suites only check for compliance with the ISO C standard and do not indicate which specific optimizations have been tested.

A typical method to ensure production code behavior remains unaffected by optimizations is to run the test suite first and then decide which optimizations to disable for the production run. This involves identifying optimizations that were not exercised during testing and therefore are considered unverified. Untested optimizations are disabled by adjusting the program’s compilation settings—specifically, by either removing the compiler or linker flag associated with the optimization or by adding its negated form.

Compiler optimizations are applied through a series of complex transformations that produce semantically equivalent programs. Some transformations enable additional optimizations on the modified code. The theoretical dependencies between individual optimizations and optimization groups are well-documented in literature. For example, Strength Reduction and Dead Code Elimination are closely related: strength reduction can introduce dead code, which is subsequently removed by dead code elimination.

The interaction between optimizations and transformations indicates that making changes to the compilation use case, e.g. disabling an optimization by adding the corresponding negative flag, may unintentionally impact a different optimization.

Even though there is prior knowledge that disabling a certain optimization potentially influences the others, understanding the impact of these interactions at a specific optimization level for a specific program requires empirical analysis.

To explore dependencies between optimizations, the "optimize" test suite of SuperTest by SolidSands, which specifically targets optimizations, is compiled with the general -O2 optimization level. By means of GCC compilation options and automation scripts, information about successful optimizations is extracted and analyzed in order to identify the optimizations that are triggered by the test suite.

The goal is to isolate and analyze the impact of a single optimization. Successful optimizations that are observed during the previous compilation run of the suite are disabled manually one at a time, using the corresponding negative command-line option -fno-flag. This means that during each of the next compilation runs on the same test cases, a single atomic optimization is selectively disabled.

After identifying triggered optimizations, the negative form of the corresponding flag is used to selectively disable each optimization and observe its impact on the others.

The -fno-early-inlining flag is used to disable the Early Inlining of Small Functions optimization. The count of triggered optimizations in this disabling run is compared to the reference count when the test suite is compiled with level -O2.

When this optimization is disabled, we observe that the count of successful “Basic Block Vectorization” optimization increases  approximately 8.8% from 771 to 839. Additionally, the Identical Code Folding optimization is completely disabled. 

This data exhibits a scenario in which disabling one atomic optimization results in an optimization of a different type being triggered more frequently.

The figure below offers a summary of the results of the dependencies between atomic optimizations. It depicts the effect of individually disabling each optimization option at the top of the diagram on the atomic optimizations listed on the left.

• M. A. Ertl. What every compiler writer should know about programmers or “Optimization” based on undefined behaviour hurts perform

• X. Wang, N. Zeldovich, M. F. Kaashoek, and A. Solar-Lezama. Towards Optimization-Safe Systems: Analyzing the Impact of Undefined Behavior. In: Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles. SOSP ‘13. Farminton, Pennsylvania: Association for Computing Machinery, 2013, pp. 260–275. doi: 10.1145/2517349.2522728.

• International Organization for Standardization. ISO/IEC 9899:2018(en) – Information technology – Programming languages – C. International Organization for Standardization. ISO/IEC 9899:2018(en). 2018.

• S. S. Muchnick. Advanced Compiler Design and Implementation. San Francisco, CA: Morgan Kaufmann, 1997.

• Solid Sands. Verification of Optimization Correctness. Solid Sands. 2019.

• Free Software Foundation. GCC Documentation. 13.1.0. Free Software Foundation. https://gcc.gnu.org/onlinedocs/, 2023.