I found when using this with Azure OpenAI that the llm responses were coming back as valid json but extract_json_between_markers was rejecting them because it is enforcing a check for the json marker which was not in the response.

I can submit a PR to add a fallback to try parsing the output as json without those markers if you want it.