Skip to content

Build options to improve the performance and security of Nginx

Without any optimization option, the compiler’s goal is to reduce the cost of compilation and to make debugging produce the expected results. Statements are independent: if you stop the program with a breakpoint between statements, you can then assign a new value to any variable or change the program counter to any other statement in the function and get exactly the results you expect from the source code.

Turning on optimization flags makes the compiler attempt to improve the performance and/or code size at the expense of compilation time and possibly the ability to debug the program.

The compiler performs optimization based on the knowledge it has of the program. Compiling multiple files at once to a single output file mode allows the compiler to use information gained from all of the files when compiling each of them.

Not all optimizations are controlled directly by a flag.

Important flags and macros used:

march – The first and most important option is -march. This tells the compiler what code it should produce for the system’s processor architecture (or arch); it tells GCC that it should produce code for a certain kind of CPU. Different CPUs have different capabilities, support different instruction sets, and have different ways of executing code. The -march flag will instruct the compiler to produce specific code for the system’s CPU, with all its capabilities, features, instruction sets, quirks, and so on provided the source code is prepared to use them.

flto – This option runs the standard link-time optimizer. When invoked with source code, it generates GIMPLE (one of GCC’s internal representations) and writes it to special ELF sections in the object file. When the object files are linked together, all the function bodies are read from these ELF sections and instantiated as if they had been part of the same translation unit.

funsafe-math-optimizations – Allow optimizations for floating-point arithmetic that (a) assume that arguments and results are valid and (b) may violate IEEE or ANSI standards. When used at link-time, it may include libraries or startup files that change the default FPU control word or other similar optimizations.

fstack-protector – Stack buffer overflows are a longstanding problem for C programs that lead to all manner of ills, many of which are security vulnerabilities. The biggest problems have typically been with string buffers on the stack coupled with bad or missing length tests. A programmer who mistakenly leaves open the possibility of overrunning a buffer on a function’s stack may be allowing attackers to overwrite the return pointer pushed onto the stack earlier. Since the attackers may be able to control what gets written, they can control where the function returns—with potentially dire results. GCC, like many compilers, offers features to help detect buffer overflows; the upcoming 4.9 release offers a new stack-protection mode with a different tradeoff between security and performance impact.

Putting stack protection into every function is both overkill and may hurt performance, so one of the GCC options chooses a subset of functions to protect. The existing “-fstack-protector-all” option will protect all functions, while the “-fstack-protector” option chooses any function that declares a character array of eight bytes or more in length on its stack. Some distributions have lowered that threshold (e.g. to four) in their builds by using the —param=ssp-buffer-size=N option. “fstack-protector-strong” offers an improved version of “-fstack-protector” without going all the way to “-fstack-protector-all”. The stack protector feature itself adds a known canary to the stack during function preamble and checks it when the function returns. If it changed, there was a stack overflow, and the program aborts.

D_FORTIFY_SOURCE – The FORTIFY_SOURCE macro provides lightweight support for detecting buffer overflows in various functions that perform operations on memory and strings. Not all types of buffer overflows can be detected with this macro, but it does provide an extra level of validation for some functions that are potentially a source of buffer overflow flaws. It protects both C and C++ code. FORTIFY_SOURCE works by computing the number of bytes that are going to be copied from a source to the destination. In case an attacker tries to copy more bytes to overflow a buffer, the execution of the program is stopped. FORTIFY_SOURCE provides buffer overflow checks for the following functions: memcpy, mempcpy, memmove, memset, strcpy, stpcpy, strncpy, strcat, strncat, sprintf, vsprintf, snprintf, vsnprintf, gets.

Build options used for Nginx:

--with-cc-opt='-O2 -march=westmere -flto -funsafe-math-optimizations -fstack-protector-strong --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2'

Example with other args:

./configure \
--sbin-path=/usr/sbin \
--with-threads \
--with-stream \
--with-http_ssl_module \
--with-http_geoip_module \
--with-cc-opt='-O2 -march=westmere -flto -funsafe-math-optimizations -fstack-protector-strong --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2'

References:
https://wiki.gentoo.org/wiki/GCC_optimization
https://gcc.gnu.org/onlinedocs/gcc-6.3.0/gcc/Optimize-Options.html
https://access.redhat.com/blogs/766093/posts/1976213
https://en.wikipedia.org/wiki/Buffer_overflow_protection#GNU_Compiler_Collection_.28GCC.29

Published inLinuxSecurityWebservers