Can not get data in group when using regular expression.

[anhkhoa14592]

Describe the bug
I tried to extract the value from PHPSESSID with regular pattern (This pattern from Web Application Defender's Cookbook: Battling Hackers and Protecting Users):
(?i:(j?sessionid|(php)?sessid|(asp|jserv|jw)?session[-_]?(id)?|cf(id|token)|sid)=([^\s]+)\;\s?)

But I can not get the value from group 6 (TX:6). I tried in others Text Editor and everything is fine but I don't know these pattern does not work. Maybe I miss somethings?

Logs and dumps

SecRule

SecRule RESPONSE_HEADERS:/Set-Cookie2?/ "(?i:(j?sessionid|(php)?sessid|(asp|jserv|jw)?session[-_]?(id)?|cf(id|token)|sid)\=([^\s]+)\;\s?)" "chain,phase:3,id:'981062',t:none,pass,log,capture,setsid:%{tx.6},setvar:session.sessionid=%{tx.6},setvar:session.valid=1,msg:'%{session.sessionid}, tx.0:%{tx.0},tx.6:%{tx.6}'"
SecRule REMOTE_ADDR "^(\d{1,3}\.\d{1,3}\.\d{1,3}\.)"  "chain,capture,setvar:session.ip_block=%{tx.1}"
SecRule REQUEST_HEADERS:User-Agent ".*" "t:none,t:sha1,t:hexEncode,setvar:session.ua=%{matched_var}"

Output of:
Response
HTTP/1.1 200
Server: nginx/1.18.0
Date: Thu, 11 Jun 2020 11:43:06 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache
X-Powered-By: PHP/7.4.6
Set-Cookie: PHPSESSID=ea101040fa9365d3ad6e921d9e1e04da; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT

AuditLog

ModSecurity: Warning. Matched "Operator Rx' with parameter .*' against variable REQUEST_HEADERS:User-Agent' (Value: curl/7.58.0') [file "/webserver/modsec/modsec.demo.com.conf"] [line "156"] [id "981062"] [rev ""] [msg ", tx.0:PHPSESSID=ea101040fa9365d3ad6e921d9e1e04da; ,tx.6:"] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "127.0.0.1"] [uri "/cookies.php"] [unique_id "159187578641.650199"] [ref "o0,44o0,9o0,3v84,50o0,8o0,8v0,9o40,0o0,40v60,11t:sha1,t:hexEncode"]
ModSecurity: Warning. [file "/webserver/modsec/modsec.conf"] [line "15"] [id "980145"] [rev ""] [msg "'Incoming Anomaly Score: 0'"] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [tag "modsec.demo.com"] [hostname "127.0.0.1"] [uri "/cookies.php"] [unique_id "159187578641.650199"] [ref ""]

Expected Behavior

Based on other TextEditor, Auditlog must have the value of PHPSESSID as below:

[msg ", tx.0:PHPSESSID=ea101040fa9365d3ad6e921d9e1e04da; ,tx.6:ea101040fa9365d3ad6e921d9e1e04da"]

[martinhsv]

Hi @anhkhoa14592 ,

Thank you for the report.

I do see a bug in the handling of matching groups that result in no content. E.g. the matching group '(id)?' where the 'match' occurs because there are 0 occurrences rather than 1.

If you are looking for an immediate workaround, in your case you could consider turning the groups that you do not care about into non-matching groups. I.e. add '?:' at the beginning of each of '(asp|jserv|jw)', '(id)', and '(id|token)|sid)' -- this should then enable you to read the content resulting from the match group '([^\s]+)' using tx.3.

I will work on a code fix for this shortly, though.

[anhkhoa14592]

Hi @martinhsv,

Thanks you for your support. Hope to receive the update for this bug :).