`target_accept` can be larger than 1

[tomicapretto]

Description

It happened to a friend that they wrote target_accept instead of random_seed. Surprised by how long it was taking to sample, they double-checked the code. A larger surprised was to find that the actual argument was target_accept=121195 and PyMC was still "working".

See the example

import pymc as pm

with pm.Model():
    pm.Normal("x")
    pm.sample(target_accept=121195)

This is with PyMC 5.16.2... Shouldn't this be an error?

[ColCarroll]

That's pretty interesting! The math works formally with any real number, I think, and also the stochastic root finding will do its best to find the value of step size that makes the accept probability that high, mostly by driving the step size down to zero (here is the relevant code - note that a large target accept makes h_bar super large, which makes log_step super negative, which makes the step size close to 0).

I guess some input validation would make sense here!

[Armavica]

Very interesting indeed! I didn't realize that the step size was dynamically adapted to make the exponential average of accepting probabilities tend towards target_accept.
I wonder if additionally to checking the provided value of target_accept, we shouldn't do this target chase in logit space.