ModSecurity 3.0 Memory Leak ?

[Menahem1]

Hello

I am currently experiencing a problem with Nginx/ModSecurity (with CRS rule in v3.0) about increasing memory in Nginx.

For now i can't be sure that the increasing memory (or memory leak) is on ModSecurity or other part of Nginx, how to be sure ?

My conf :
nginx version: nginx/1.11.9
built by gcc 4.9.2 (Debian 4.9.2-10)
built with OpenSSL 1.0.1t 3 May 2016
TLS SNI support enabled
configure arguments: --add-module=../ModSecurity-nginx --with-debug --with-cc-opt='-O0 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro' --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --with-pcre-jit --with-http_ssl_module --with-http_realip_module

Usage of Memory

[zimmerle]

Hi @MEN18,

Do you happens to have you summary for the ./configure ? I am interested to know if you are using LMDB or not. Also, what is the last commit in your tree?

[Menahem1]

Yes i am using LMDB

configure:5925: LMDB library found at: /usr/lib/x86_64-linux-gnu//liblmdb.so
configure:5930: LMDB headers found at: /usr/include
configure:5925: LMDB library found at: /usr/lib/x86_64-linux-gnu//liblmdb.so
configure:5930: LMDB headers found at: /usr/include
configure:5925: LMDB library found at: /usr/lib/x86_64-linux-gnu//liblmdb.so
configure:5930: LMDB headers found at: /usr/include
configure:6036: using LMDB v

The last commit i used => 3a41308

[zimmerle]

Do you mind to compile without LMDB ?

$ ./configure --without-lmdb

[Menahem1]

no sorry, i have juste use this command alone

./configure

[zimmerle]

Can you re-compile it using the --without-lmdb parameter?

[Menahem1]

I recompiled with the "--without-lmdb" parameter and the memory (on the whole) is still increasing (sometimes i see an increase of 20mb but then disappears)

[Menahem1]

The only time when my memory is going down it's with a segfault (who send a signal 11 on the worker)

Feb 7 14:29:42 ip-10-65-2-86 kernel: [87308.027588] nginx[445]: segfault at 7fc270936000 ip 00007ff7f1b76c3a sp 00007ffc878da808 error 4 in libc-2.19.so[7ff7f1af5000+1a1000]
Feb 8 13:06:05 ip-10-65-2-86 kernel: [168691.049453] nginx[4854]: segfault at 7fc270936000 ip 00007ff7f1b76c3a sp 00007ffc878da7e8 error 4 in libc-2.19.so[7ff7f1af5000+1a1000]
Feb 8 13:06:06 ip-10-65-2-86 kernel: [168691.653367] nginx[8883]: segfault at 7fc270936000 ip 00007ff7f1b76c3a sp 00007ffc878da7e8 error 4 in libc-2.19.so[7ff7f1af5000+1a1000]

[zimmerle]

Any change to capture the stack for this segfault?

[Menahem1]

It's in an other environment (prod without the debug) without the "--without-lmdb" parameter (but with same version of Nginx/ModSecurity/Rules)

[Menahem1]

An idea of ​​when an upcoming update will be available to correct it ?

[Menahem1]

In precision the machine contains 27 vhosts
Is there a way that modsecurity does not use a resource when this vhost is not active?

[zimmerle]

Hi @MEN18,

It will be great if you can give further details on the segfault.

Here you can find more details on how to get those core dumps:
https://www.nginx.com/resources/wiki/start/topics/tutorials/debugging/#core-dump

[Menahem1]

Hi @zimmerle

Here is the detail of segfault

#0  strlen () at ../sysdeps/x86_64/strlen.S:106
No locals.
#1  0x00007f7ee6f64c28 in std::string::compare(char const*) const () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#2  0x00007f7ee8f8a6cc in operator==<char, std::char_traits<char>, std::allocator<char> > (__rhs="/var/log/nginx/modsec_audit-test.log", __lhs=<optimized out>) at /usr/include/c++/4.9/bits/basic_string.h:2528
No locals.
#3  modsecurity::utils::SharedFiles::find_handler (this=this@entry=0x7f7ee920ece8 <modsecurity::utils::SharedFiles::getInstance()::instance>, fileName="/var/log/nginx/modsec_audit-test.log") at utils/shared_files.cc:42
        current = 0x7f7ee988f000
#4  0x00007f7ee8f8a710 in modsecurity::utils::SharedFiles::close (this=0x7f7ee920ece8 <modsecurity::utils::SharedFiles::getInstance()::instance>, fileName=...) at utils/shared_files.cc:181
        a = <optimized out>
#5  0x00007f7ee8f24119 in modsecurity::audit_log::writer::Serial::~Serial (this=0x5062cb0, __in_chrg=<optimized out>) at audit_log/writer/serial.cc:28
No locals.
#6  0x00007f7ee8f22148 in modsecurity::audit_log::AuditLog::~AuditLog (this=0x1232b30, __in_chrg=<optimized out>) at audit_log/audit_log.cc:69
No locals.
#7  0x00007f7ee8f26c16 in ~RulesProperties (this=0x12326f0, __in_chrg=<optimized out>) at ../headers/modsecurity/rules_properties.h:106
No locals.
#8  modsecurity::Rules::~Rules (this=0x12326f0, __in_chrg=<optimized out>) at rules.cc:80
No locals.
#9  0x00007f7ee8f26f0e in modsecurity::msc_rules_cleanup (rules=0x12326f0) at rules.cc:335
No locals.
#10 0x00000000004e3e27 in ngx_http_modsecurity_config_cleanup (data=0x1230ea8) at ../ModSecurity-nginx/src/ngx_http_modsecurity_module.c:653
        old_pool = 0x0
        t = 0x1230ea8
#11 0x000000000041496e in ngx_destroy_pool (pool=0x1198870) at src/core/ngx_palloc.c:57
        p = 0x7ffe532ac250
        n = 0x11988d8
        l = 0x11989f0
        c = 0x1230f08
#12 0x000000000044e6ba in ngx_worker_process_exit (cycle=0x11988c0) at src/os/unix/ngx_process_cycle.c:999
        i = 1024
        c = 0x5545e80
#13 0x000000000044da0c in ngx_worker_process_cycle (cycle=0x11988c0, data=0x0) at src/os/unix/ngx_process_cycle.c:747
        worker = 0
#14 0x000000000044a18b in ngx_spawn_process (cycle=0x11988c0, proc=0x44d95c <ngx_worker_process_cycle>, data=0x0, name=0x4ea983 "worker process", respawn=-3) at src/os/unix/ngx_process.c:198
        on = 1
        pid = 0
        s = 0
#15 0x000000000044c81d in ngx_start_worker_processes (cycle=0x11988c0, n=3, type=-3) at src/os/unix/ngx_process_cycle.c:358
        i = 0
        ch = {command = 1, pid = 0, slot = 0, fd = 0}
#16 0x000000000044bdda in ngx_master_process_cycle (cycle=0x11988c0) at src/os/unix/ngx_process_cycle.c:130
        title = 0x4bbe554 "master process /usr/local/nginx/sbin/nginx -g pid /run/nginx.pid; daemon on; master_process on;"
        p = 0x4bbe5b3 ""
        size = 96
        i = 3
        n = 0
        sigio = 0
        set = {__val = {0 <repeats 16 times>}}
        itv = {it_interval = {tv_sec = 0, tv_usec = 0}, it_value = {tv_sec = 0, tv_usec = 0}}
        live = 0
        delay = 0
        ls = 0x0
        ccf = 0x119a3f8
#17 0x0000000000410ce0 in main (argc=3, argv=0x7ffe532ac7c8) at src/core/nginx.c:367
        b = 0x7f7ee7fdfd28
        log = 0x71e980 <ngx_log>
---Type <return> to continue, or q <return> to quit---
        i = 140183062643528
        cycle = 0x11988c0
        init_cycle = {conf_ctx = 0x0, pool = 0x1198340, log = 0x71e980 <ngx_log>, new_log = {log_level = 0, file = 0x0, connection = 0, disk_full_time = 0, handler = 0x0, data = 0x0, writer = 0x0, wdata = 0x0, action = 0x0, next = 0x0},
          log_use_stderr = 0, files = 0x0, free_connections = 0x0, free_connection_n = 0, modules = 0x0, modules_n = 0, modules_used = 0, reusable_connections_queue = {prev = 0x0, next = 0x0}, listening = {elts = 0x0, nelts = 0,
            size = 0, nalloc = 0, pool = 0x0}, paths = {elts = 0x0, nelts = 0, size = 0, nalloc = 0, pool = 0x0}, config_dump = {elts = 0x0, nelts = 0, size = 0, nalloc = 0, pool = 0x0}, open_files = {last = 0x0, part = {elts = 0x0,
              nelts = 0, next = 0x0}, size = 0, nalloc = 0, pool = 0x0}, shared_memory = {last = 0x0, part = {elts = 0x0, nelts = 0, next = 0x0}, size = 0, nalloc = 0, pool = 0x0}, connection_n = 0, files_n = 0, connections = 0x0,
          read_events = 0x0, write_events = 0x0, old_cycle = 0x0, conf_file = {len = 32, data = 0x1198390 ""}, conf_param = {len = 49, data = 0x7ffe532acf57 "ng down"}, conf_prefix = {len = 22, data = 0x1198390 ""}, prefix = {len = 17,
            data = 0x4e6106 "/usr/local/nginx/"}, lock_file = {len = 0, data = 0x0}, hostname = {len = 0, data = 0x0}}
        cd = 0x7ffe532ac4d0
        ccf = 0x119a3f8