QST: Does the project consider DataFrame.query() arbitrary code execution to be a security vulnerability?

[nickodell]

Research

• I have searched the [pandas] tag on StackOverflow for similar questions.

• I have asked my usage related question on StackOverflow.

Link to question on StackOverflow

https://stackoverflow.com/questions/79304226/should-i-manually-patch-the-pandas-dataframe-query-vulnerability-or-wait-for-a

(To clarify, this question was written by another user.)

Question about pandas

Hi, I saw this question on StackOverflow, which is about a public CVE, CVE-2024-9880.

The basic premise of the CVE is that if an attacker controls the expr argument to DataFrame.query(), then arbitrary code execution can be achieved.

The example given in the CVE is

import pandas as pd


df = pd.DataFrame({'a': [1, 2, 3], 'b': ['error_details', 'confidential_info', 'normal']})


query = '@pd.core.frame.com.builtins.__import__("os").system("""ping google.com #""")'
try:
    engine = "python"
    result = df.query(query,local_dict={},engine="python",).index
except Exception as e:
    print(f'Error: {e}')

However, this is not minimal, and a more minimal construction would be

import pandas as pd

df = pd.DataFrame()

expr = '@pd.compat.os.system("""echo foo""")'
result = df.query(expr, engine='python')

(The report also says that engine='python' is required, but both engine='python' and engine='numexpr' worked in my testing.)

My question is about Pandas's security model. What security guarantees does Pandas make about DataFrame.query() with an attacker-controlled expr?

My intuition about this is "none, don't do that," but I'm wondering what the Pandas project thinks.

[skj-skj]

any updates? Snyk is flagging this as High Severity Vulnerability and is blocking my deployment.

its unfortunate it is happening on Holidays.

[rhshadrach]

My intuition about this is "none, don't do that," but I'm wondering what the Pandas project thinks.

This is indeed my take, both query and eval should be used with string literals and not with strings provided by or derived from untrusted user input.

cc @pandas-dev/pandas-core

[Dr-Irv]

This is indeed my take, both query and eval should be used with string literals and not with strings provided by or derived from untrusted user input.

This could be tricky, because something like this should be OK:

mystr = "something from a user"
df.query(f"a_column == {mystr}")

In other words, it's OK to construct a query string based on user input.

[nickodell]

This could be tricky, because something like this should be OK:

Isn't this technique still vulnerable to injection? Example:

import pandas as pd

df = pd.DataFrame({'a_column': []})

mystr = '@pd.compat.os.system("""echo foo""")'
df.query(f"a_column == {mystr}")

This still prints 'foo' for me.

[Dr-Irv]

Isn't this technique still injectable?

Yes, so it seems that the caller that creates the parameter to query() needs to put a safeguard around that computation.

[rhshadrach]

@Dr-Irv - I do not think it is a reasonable expectation that users are able to implement a safeguard. Preventing query-injection can be very difficult. This is why SQL libraries provide such functionality. It seems to me either pandas needs to provide it or make the recommendation that users do not use query/eval in this manner.

[WillAyd]

The eval documentation has a warning against this (see #59108). I think can be added to query as well

[WillAyd]

As an added note, please do not report security vulnerabilities to our github page. If you suspect or know of one, we have a process in place through our partnership agreement with Tidelift.

See https://tidelift.com/docs/security for more details on how to report going forward. Locking this conversation for now - thanks!