After modsecurity is enabled on nginx, a large number of TCP connections in the CLOSE-WAIT state will be generated

[Langgeligelang]

Describe the bug

After nginx enables modsecurity, a large number of TCP connections in the CLOSE-WAIT state will be generated, and a large amount of memory will be consumed.
When I run nginx -s reload, the TCP connection in the close-wait state disappears and the memory is freed.

MEM usage

TCP-CLOSE-WAIT

Logs and dumps

Nginx error log

2024/01/03 10:00:52 [alert] 16010#16010: *10257 open socket #2155 left in connection 1073
2024/01/03 10:00:52 [alert] 16010#16010: *10272 open socket #2159 left in connection 1074
2024/01/03 10:00:52 [alert] 16010#16010: *10278 open socket #2160 left in connection 1075
2024/01/03 10:00:52 [alert] 16010#16010: *10271 open socket #2157 left in connection 1076
2024/01/03 10:00:52 [alert] 16010#16010: *10279 open socket #2162 left in connection 1077
2024/01/03 10:00:52 [alert] 16010#16010: *10280 open socket #2165 left in connection 1078
2024/01/03 10:00:52 [alert] 16010#16010: *10310 open socket #2171 left in connection 1079
2024/01/03 10:00:52 [alert] 16010#16010: *10301 open socket #2169 left in connection 1080
2024/01/03 10:00:52 [alert] 16010#16010: *10290 open socket #2166 left in connection 1081
2024/01/03 10:00:52 [alert] 16010#16010: *10317 open socket #2173 left in connection 1082
2024/01/03 10:00:52 [alert] 16010#16010: *10324 open socket #2175 left in connection 1083
2024/01/03 10:00:52 [alert] 16010#16010: *10345 open socket #2177 left in connection 1084

ModSecurity debug.log

[170425146183.514060] [/city] [4] Rule returned 0.
[170425146183.514060] [/city] [4] (Rule: 953013) Executing operator "Lt" with param "2" against TX:EXECUTING_PARANOIA_LEVEL.
[170425146183.514060] [/city] [4] Rule returned 1.
[170425146183.514060] [/city] [5] Setting skipAfter for: END-RESPONSE-953-DATA-LEAKAGES-PHP
[170425146183.514060] [/city] [4] Running (disruptive) action: pass.
[170425146183.514060] [/city] [8] Running action pass
[170425146183.514060] [/city] [4] (Rule: 954011) Executing operator "Lt" with param "1" against TX:EXECUTING_PARANOIA_LEVEL.
[170425146183.514060] [/city] [4] Rule returned 0.
[170425146183.514060] [/city] [4] (Rule: 954013) Executing operator "Lt" with param "2" against TX:EXECUTING_PARANOIA_LEVEL.
[170425146183.514060] [/city] [4] Rule returned 1.
[170425146183.514060] [/city] [5] Setting skipAfter for: END-RESPONSE-954-DATA-LEAKAGES-IIS
[170425146183.514060] [/city] [4] Running (disruptive) action: pass.
[170425146183.514060] [/city] [8] Running action pass
[170425146183.514060] [/city] [4] (Rule: 959011) Executing operator "Lt" with param "1" against TX:EXECUTING_PARANOIA_LEVEL.
[170425146183.514060] [/city] [4] Rule returned 0.
[170425146183.514060] [/city] [4] (Rule: 959013) Executing operator "Lt" with param "2" against TX:EXECUTING_PARANOIA_LEVEL.
[170425146183.514060] [/city] [4] Rule returned 1.
[170425146183.514060] [/city] [5] Setting skipAfter for: END-RESPONSE-959-BLOCKING-EVALUATION
[170425146183.514060] [/city] [4] Running (disruptive) action: pass.
[170425146183.514060] [/city] [8] Running action pass
[170425146183.514060] [/city] [4] Not appending response body. Response Content-Type is . It is not marked to be inspected.
[170425146183.514060] [/city] [4] Starting phase RESPONSE_BODY. (SecRules 4)
[170425146183.514060] [/city] [5] Response Content-Type is . It is not marked to be inspected.
[170425146183.514060] [/city] [8] Content-Type(s) marked to be inspected: text/html text/plain text/xml

Server :

• ModSecurity v3.0.11 with nginx-connector v1.0.3
• WebServer: nginx-1.20.2
• OS : CentOS Linux release 7.6.1810
• coreruleset-3.3.5

nginx.conf

user nginx;
worker_processes 4;
events {
    worker_connections 4096;
}
http {
    include mime.types;
    default_type application/octet-stream;
    tcp_nodelay on;
    modsecurity_rules_file /data/nginx-modsecurity/conf/modsecurity/modsecurity.conf;
    upstream pushgateway {
        server 10.0.0.2:9001 max_fails=1 fail_timeout=2s;
    }
    server {
        listen 1111;
        server_name localhost;
        auth_basic "Please input password";
        auth_basic_user_file /etc/nginx/password;
        modsecurity on;
        absolute_redirect off;
        location /metrics/job {
            add_header 'Access-Control-Allow-Origin' '*';
            add_header 'Access-Control-Allow-Methods' '*';
            add_header 'Access-Control-Allow-Headers' '*';
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Scheme $scheme;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_pass http://pushgateway;
            client_max_body_size 5m;
            keepalive_timeout 1s;
        }
        location / {
            deny all;
        }
    }
    server {
        listen 23231;
        location /nginx_status {
            stub_status on;
        }
    }
}

modsecurity.conf

SecRuleEngine On
SecRequestBodyAccess On
SecRule REQUEST_HEADERS:Content-Type "^(?:application(?:/soap\+|/)|text/)xml" \
     "id:'200000',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=XML"
SecRule REQUEST_HEADERS:Content-Type "^application/json" \
     "id:'200001',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=JSON"
SecRequestBodyLimit 419430400
SecRequestBodyNoFilesLimit 419430400
SecRequestBodyLimitAction Reject
SecRequestBodyJsonDepthLimit 512
SecArgumentsLimit 1000
SecRule &ARGS "@ge 1000" \
"id:'200007', phase:2,t:none,log,deny,status:400,msg:'Failed to fully parse request body due to large argument count',severity:2"
SecRule REQBODY_ERROR "!@eq 0" \
"id:'200002', phase:2,t:none,log,deny,status:400,msg:'Failed to parse request body.',logdata:'%{reqbody_error_msg}',severity:2"
SecRule MULTIPART_STRICT_ERROR "!@eq 0" \
"id:'200003',phase:2,t:none,log,deny,status:400, \
msg:'Multipart request body failed strict validation: \
PE %{REQBODY_PROCESSOR_ERROR}, \
BQ %{MULTIPART_BOUNDARY_QUOTED}, \
BW %{MULTIPART_BOUNDARY_WHITESPACE}, \
DB %{MULTIPART_DATA_BEFORE}, \
DA %{MULTIPART_DATA_AFTER}, \
HF %{MULTIPART_HEADER_FOLDING}, \
LF %{MULTIPART_LF_LINE}, \
SM %{MULTIPART_MISSING_SEMICOLON}, \
IQ %{MULTIPART_INVALID_QUOTING}, \
IP %{MULTIPART_INVALID_PART}, \
IH %{MULTIPART_INVALID_HEADER_FOLDING}, \
FL %{MULTIPART_FILE_LIMIT_EXCEEDED}'"
SecRule MULTIPART_UNMATCHED_BOUNDARY "@eq 1" \
    "id:'200004',phase:2,t:none,log,deny,msg:'Multipart parser detected a possible unmatched boundary.'"
SecPcreMatchLimit 1000
SecPcreMatchLimitRecursion 1000
SecRule TX:/^MSC_/ "!@streq 0" \
        "id:'200005',phase:2,t:none,deny,msg:'ModSecurity internal error flagged: %{MATCHED_VAR_NAME}'"
SecResponseBodyAccess On
SecResponseBodyMimeType text/plain text/html text/xml
SecResponseBodyLimit 524288
SecResponseBodyLimitAction ProcessPartial
SecTmpDir /tmp/
SecDataDir /tmp/
SecDebugLog /data/nginx-modsecurity/logs/debug.log
SecDebugLogLevel 1
SecAuditEngine RelevantOnly
SecAuditLogRelevantStatus "^(?:5|4(?!04))"
SecAuditLogParts ABCDEFHZ
SecAuditLogType Serial
SecAuditLog /data/nginx-modsecurity/logs/modsec_audit.log
SecArgumentSeparator &
SecCookieFormat 0
SecUnicodeMapFile unicode.mapping 20127
SecStatusEngine On
Include /data/nginx-modsecurity/conf/modsecurity/crs-setup.conf
Include /data/nginx-modsecurity/conf/modsecurity/rules/*.conf

[martinhsv]

Hello @Langgeligelang ,

It would be surprising if ModSecurity were the cause of these persistent CLOSE_WAIT states as ModSecurity v3 does not have control over (or even access to) nginx's TCP stack.

The portion of the debug log that you have posted does not show anything unusual except for the absence of the LOGGING phase messages -- and this might be consistent with the CLOSE_WAIT state in those cases.

You could consider reviewing any relevant configuration settings you may be using (for example, nginx's keepalive_timeout setting (and perhaps trying to set it to a very low value like 1). Otherwise some further analysis on your end might be helpful. For example: do only some (but not all) HTTP requests result in the CLOSE_WAIT state? If so, is there something they have in common?

[Langgeligelang]

@martinhsv Thanks for the reply, I tried setting the keepalive_timeout to 1s but it didn't work. Only the curl command will generate a CLOSE-WAIT when accessing the waf.
For example：
curl 127.0.0.1:9100/metrics | curl --data-binary @- http://127.0.0.1:22302

[martinhsv]

So if you have ModSecurity installed with your nginx deployment, that if you have SecAuditEngine On you see that effect when using the curl client, but if you have SecAuditEngine Off you do not see that effect?

[Langgeligelang]

@martinhsv I have disabled SecAuditEngine, but there are still a large number of CLOSE-WAIT TCP connections. Additionally, I have added the modsecurity.conf configuration file to the end of the description of the issue.

[martinhsv]

@Langgeligelang ,

If you have SecAuditEngine Off and you are still seeing your issue, then I have to assume it has nothing to do with ModSecurity.