Improve error response processing in Mark Logic Java Client API

[josvanroosmalen]

Hi,

I am working on some quick POC-code to get data into MarkLogic using Java, and I found out that the response processing is not properly working (or I am not correctly using :-) ).

If I run the PUT-operation (see snippet at the end of this message), I get the following not really informative error. Even if I enable/set request logging on (see snippet), I only get this (The AppServer log does not tell much more than http 409):

com.marklogic.client.FailedRequestException: Local message: failed to write resource at resources/ingest-trade: Conflict. Server Message: null
[..]
LogBuffer 0
null

So I cloned the sourcecode from GitHub and I changed:

JerseyServices.java-> lline 415: client = ApacheHttpClient4.create(config);

to:

line 415: client = ApacheHttpClient4.create(config);
line 416: client.addFilter(new LoggingFilter(System.err));

Now I get suddenly much more interesting info (XPTY0004):

1 * Client out-bound request
1 > HEAD http://localhost:8003/v1/ping
1 * Client in-bound response
1 < 401
1 < Server: MarkLogic
(....)
3 * Client out-bound request
3 > PUT http://localhost:8003/v1/resources/ingest-trade?rs:message-id=1333&rs:source=MX
3 * Client in-bound response
3 < 409
3 < Server: MarkLogic
3 < Content-Type: application/xml; charset=UTF-8
3 < Content-Length: 226
3 < Connection: Keep-Alive
3 < Keep-Alive: timeout=5
3 < 
<?xml version="1.0" encoding="UTF-8"?>
<status elapsed-time="PT0.005S">Exception detected for message with Id=1333 Reason: {"ingestion":"XDMP-AS: (err:XPTY0004) $compiled as node() -- Invalid coercion: () as node()"}</status>

Because the response is XML, I thought it would help to add:

StringHandle result = new StringHandle().withFormat(Format.XML);

But that did not help.

Can someone tell what (1) I did wrong (how to get this (XML) error message in the output handler or (2) when this will be fixed?

Original Test Snippet:

public void ingestTrade(DatabaseClient client,String source,String messageId,String messageContent) throws Exception { 
    RequestParameters params = new RequestParameters();
    params.add("message-id", messageId);
    params.add("source", source);
    StringHandle result = new StringHandle().withFormat(Format.XML);
    StringHandle h = new StringHandle(messageContent).withFormat(Format.XML);
    ByteArrayOutputStream out = new ByteArrayOutputStream();
    try { 
    // trying to get it out with the request logger. 
    RequestLogger logger = client.newLogger(out);
    getServices().startLogging(logger);
    logger.setEnabled(true);
    getServices().put(params,new StringHandle(messageContent).withFormat(Format.XML),result);
    } catch (Exception e) { 
        e.printStackTrace();
    }
    System.err.println("LogBuffer "+out.size()); 
        System.err.println(result);
    }
}

[grechaw]

Hello @josvr , have you read some of the examples or docs about how to ingest documents? This code is very unexpected to me. What is 'source'? It it is a string containing XML, you'll use seomthing like this to writ it to the database.

I don't even see how this code can run -- where is getServices()?

[kcoleman-marklogic]

Sorry, misread what you said, Charles. He calls getServices here:

RequestLogger logger = client.newLogger(out);
    getServices().startLogging(logger);
    logger.setEnabled(true);
    getServices().put(params,new StringHandle(messageContent).withFormat(Format.XML),result);

[sammefford]

@josvr I'm guessing your code is falling into your catch block because it's throwing a FailedRequestException because the response status code is 409. In order to read the response body, your REST service extension should return a status code of 200, 201, or 204.

[josvanroosmalen]

@sammefford Can MarkLogic please consider/mark this as a bug (or enhancement)? Returning 200/201/204 is a workaround. Even if we change our 409 to 204, then if (1) MarkLogic internal or (2) our own custom code by accident still return an different error code then we face in our Java production log:

Message: null

This is a disaster to investigate in a production environment.

In case a FailedRequestException is raised the MarkLogic Client API should simply dump the response as message in a string, so instead:

com.marklogic.client.FailedRequestException: Local message: failed to write resource at resources/ingest-trade: Conflict. Server Message: null

We would get after this bug is fixed something like:

com.marklogic.client.FailedRequestException: Local message: failed to write resource at resources/ingest-trade: Conflict. Server Message: <?xml version="1.0" encoding="UTF-8"?> <status elapsed-time="PT0.005S">Exception detected for message with Id=1333 Reason: {"ingestion":"XDMP-AS: (err:XPTY0004) $compiled as node() -- Invalid coercion: () as node()"}</status>

This is much more informative!!

@grechaw : Yes this snippet is part of a ResourceManager derived class. source/message-id are here parameters for the PUT /v1/config/resources/ingest-trade?rs:source=VALUE1&rs:message-id=VALUE2

[sammefford]

@josvr I'd be glad to look at this as a bug. Could you please share code and sample data so I can reproduce the bug?

[sammefford]

Please consider the following guidelines for helping me reproduce: http://stackoverflow.com/help/mcve

[kcoleman-marklogic]

Is the 409 being returned from your code, @josvr? If so, are you using the error reporting pattern described here:

http://docs.marklogic.com/guide/rest-dev/extensions#id_33892

[josvanroosmalen]

Thanks, I will check this error reporting pattern, and if needed I will post an MWE

[josvanroosmalen]

Please find attached the MWE which demonstrate this bug (or undesired loss of information).

MWEJavaAPINotHandlingErrorMsgCorrectly.java.txt This installs a extension which returns a 409 with a timedstamp status "I DO NOT SEE THIS MESSAGE" and calls this one.
OutputMarkLogicJavaAPI.txt: This is the output of the default MarkLogic Java API. It states "Server Message: null". So I loss my status message and I loss my 409. This is undesireable in a production env.
OutputMarkLogicJavaAPIWithLogging.txt: This is my 'hacked' MarkLogic Java API (described above) which also logs the request/responses. As you can see we receive the 409 with error message.

Thanks,

Jos

MWEJavaAPINotHandlingErrorMsgCorrectly.java.txt
OutputMarkLogicJavaAPIWithLogging.txt
OutputMarkLogicJavaAPI.txt

[kcoleman-marklogic]

This is not the correct way to return an error from an extension:

map:put($context, 'output-status', (409, 'Conflict'))
document { <status elapsed-time='{xdmp:elapsed-time()}'>I DO NOT SEE THIS MESSAGE</status> }

Rather, you should do the following:

fn:error((),"RESTAPI-SRVEXERR", ("409","Conflict","Hopefully, you do see this message"))

This is discussed in http://docs.marklogic.com/guide/rest-dev/extensions#id_33892. As also discussed there, you can pass back a response payload more complex than plain text iff the expected error response MIME type matches. That is, you shouldn't try to return XML if the REST API instance is configured to return JSON errors, or vice versa. The default error format is JSON, I believe.

Does using the fn:error pattern shown above still result in a loss of information for you?

[josvanroosmalen]

Ok using fn:error does work. However this output-status/document construct works perfectly (if we do not use the Java API) so I would consider adding this to the JAVA API as well.