Skip to content

Fix getColumnMeta() for GH-15287 Pdo\Pgsql::setAttribute(PDO::ATTR_PREFETCH, 0) #16249

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

outtersg
Copy link
Contributor

@outtersg outtersg commented Oct 5, 2024

As stated in the UPGRADING, using the passthrough ("single-row") mode of libpq forbids passing a new query while the current one's results have not been entirely consumed.
… But I didn't notice that ext/pdo_pgsql internally used new queries to fetch metadata.

This PR makes those internal calls return NULL for non-essential metadata, instead of letting libpq abort the user-called query.

It moreover includes a small tweak to table oid-to-name translation, with a 1-slot cache.
This may by chance allow the internal call to return something instead of NULL,
but it will nonetheless avoid 30 server calls to get the table name of 30 columns of the same table.

@outtersg
Copy link
Contributor Author

outtersg commented Oct 7, 2024

I noticed this week-end that pgsql_driver.c too was full of PQexec (for internal needs).
So those calls too need to be protected the same way (by ensuring that no running statement has been let in a running state).

So 44afde3 exposes pgsql_stmt_finish(), and makes pgsql_driver.c use it extensively (only when the last run statement is in single-row / unbuffered / lazy fetch mode).

… But then I noticed some patterns that could be factorized in pgsql_stmt_finish() (while((pgsql_result = PQgetResult(H->server))) PQclear(pgsql_result);).
I'm a bit stressed that factorized code could break things that presently work, so for now I just put a comment showing the mutualization opportunity (and the result is quite stable, I'm currently testing it with production cases),
but should I fully explore finishing the code mutualization?

@devnexen
Copy link
Member

devnexen commented Oct 8, 2024

… But then I noticed some patterns that could be factorized in pgsql_stmt_finish() (while((pgsql_result = PQgetResult(H->server))) PQclear(pgsql_result);). I'm a bit stressed that factorized code could break things that presently work, so for now I just put a comment showing the mutualization opportunity (and the result is quite stable, I'm currently testing it with production cases), but should I fully explore finishing the code mutualization?

Sure, you can always explore in your own fork for a while and see how it goes. 8.5 release is in 1 year + :)

@outtersg
Copy link
Contributor Author

@devnexen wrote:

Sure, you can always explore in your own fork for a while and see how it goes. 8.5 release is in 1 year + :)

That's a good point!

Now as I both rebased and amended, the changes are not that clear in the last diff :-\
Here starts the change to pdo_pgsql.

@outtersg outtersg force-pushed the gh15287-columnmeta branch 3 times, most recently from fce61c9 to be8ac25 Compare October 23, 2024 10:51
…lumnMeta()

- each call queried the DB to know the name associated with the table's OID:
  cache the result between two calls
- make pdo_pgsql_translate_oid_to_table higher-level,
  with the last parameter being the handle instead of the raw connection;
  thus the statement is cleaner, letting the handle do all memory handling on the table oid-to-name translation cache
  (which by the way is a driver feature more than a statement one)
doing an (internal) query to fetch metadata from the server broke the currently-running query;
additionnally, fix the condition to interrupt the previous request (we shall test if *it* was lazy, not if the *new* one will)
…l queries

not only the statements, but the driver, love to make internal queries. We make sure no unfinished query still runs when having to pass an internal one.
by the way factorize the loops that consumed the preceding query's results
we unconditionally set it, but conditionally unset it, letting a dangling pointer that was tentatively freed a second time
@outtersg outtersg marked this pull request as ready for review November 26, 2024 21:04
@outtersg
Copy link
Contributor Author

outtersg commented Nov 26, 2024

Now that 8.4 is out, it's time to think of 8.5.

Although the nature of the PR (fixing an incompatibility between prefetch mode and metadata reading) could have made it a candidate for a 8.4.x,
the changes make it sufficently risky that I prefer keeping the 8.5 target (and if necessary document the 8.4 with a warning that "for now, entering the prefetch mode prevents using getColumnMeta() or anythng else meta calling the database's information schema").

This is a first, working pass on the subject, while I found time to code it: it is a technically self-sustainable fix, although we could go further in what pgsql_finish_running_stmt() is a first try at: separating functional PQexecs (the ones from exec(), passing the user queries) from technical ones (called internally by getColumnMeta() and such).
Then:

  • any internal query, depending on a mandatory parameter, would either throw ("Cannot access the meta schema in the middle of a result consumption") or silently return nothing (e.g. in getColumnMeta(), if we cannot resolve a user-defined type to its pretty name, we can still return its OID).
  • later, when implementing libpq's pipeline mode, we could implement in this centralized function the complexity of taking advantage of the pipeline mode, to direct those synchronous meta-select queries to a dedicated "channel" where they could execute even while the user-pulled result is in the middle of its fetch.

But as I am not sure of the time I could devote to this ideal solution,
let's keep this PR focused on its first goal of just preventing unexplainable libpq error codes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants