Date storage behavior

[Romaric5M]

Hi,

I have few questions about java.util.Date behavior and support.

My application has java model with few Date attributes (like modifiedAt, birthdate) and UTC format.
Thanks to the existing VPack Date De-/Serializer, I guessed transformation should be automatic and fully integrated but associated keys display "(non representing type)" in the manager.
Then the json exported collection doesn't contains these keys and deserialization fails with "UTC_DATE required" message.

Does actual driver version fully manages java.util.Date or do you have recommandations about using it ?

Thanks,
Note: I'm using java driver 4.1.1 and Database 3.1.0

[mvollmary]

Hi,

VelocyPack is primary used as ArangoDBs internal storage format, the additional VelocyPack types are currently not supported by AQL, the shell or the web-ui. They all have to use json for processing. This is why the web-UI display you "(non representing type)" for utc_date.

Because the Java-driver can directly deserialize from VPack to Java we can parse the additional VPack-types which we have an equivalent in Java. We can parse VPack.UTC_DATE to java.util.Date (or java.sql.Date, java.sql.Timestamp or with the async JDK 8 driver java.time.Instant, LocalDate, LocalDateTime).

So using java.util.Date in POJOs, Map or BaseDocument works fine as long as you do not have to share the stored data with someone (e.g. another application) which use a format with no equivalent for utc_date (like the ArangoDB web-UI).

The same is for parsing VelocyPack to Json within Java, because Json has no type for utc_date.

ArangoDB arangoDB = new ArangoDB.Builder().build();
VPackSlice vpack = new VPackBuilder().add(ValueType.OBJECT).add("date", new Date()).close().slice();
String json = arangoDB.util().deserialize(vpack, String.class);
System.out.println(json);

This short example prints you {"date":"(non-representable type)"} like the ArangoDB web-UI.

A possibility to deal with it would be using a DateFormat and parse the utc_date to a Json String. You can do this with VPackParser by register a custom Deserializer:

VPackParser parser = new VPackParser();
  parser.registerDeserializer(ValueType.UTC_DATE, new VPackJsonDeserializer() {
    @Override
    public void deserialize(VPackSlice parent, String attribute, VPackSlice vpack, StringBuilder json) throws VPackException {
      String dateAsString = SimpleDateFormat.getDateInstance().format(vpack.getAsDate());
      json.append(JSONValue.toJSONString(dateAsString));
    }
});
System.out.println(parser.toJson(vpack));

Every db call in the Java-driver with a document as result use a global instance of VPackParser to parse the result to Json when the parameter type is String.class. e.g. ArangoCollection.getDocument(String, Class). This instance is currently not accessible for configuration, but I can change this if you would like to use it.
If you would like to test the custom VPack to Json parsing for a db call like getDocument und can do so by getting raw VPack from the call and then parse it with your own instance of VPackParser (requires code above):

VPackSlice docAsVpack = arangoDB.db().collection("myCollection").getDocument("myKey", VPackSlice.class);
String docAsJson = parser.toJson(docAsVpack);

I am currently not sure whether a standard implementation of VPack.UTC_DATE to Json String parsing is a good way to deal with utc_date. But a configurable global instance of VPackParser is definitely a needed thing. I will implement it within this week.

I hope this helps you for the moment. It would also be great if you give me feedback on what you think.

Best,
Mark

[mvollmary]

I discuss this with a colleague and implemented a default utc_date parsing to json string with format ISO 8601.

If you need another format you can register your own VPackJsonDeserializer - like in my previous post - on the global VPackParser instance. For that I implemented the method registerJsonDeserializer(ValueType, VPackJsonDeserializer) in ArangoDB.Builder.

The changes will be in version 4.1.2, which I will release within the day.

[Romaric5M]

Thanks for the detailed explanations, its is really clear.

For multiple reasons like export and continuity with previous databases versions, we already decided to register our custom de-/serializer from Date to ISO String.

Good to know we were on the right solution and thanks for the evolution.

Best,
Romaric

[mvollmary]

Hi Romaric,

I am currently releasing version 4.1.2. It should be available within the next hour on maven central repository.

As I said. I implemented deserialization from VPack.UTC_DATE to Json ISO String and configuration methods for custom de-/serializer.

When you need to serialize Json into utc_date you have to use your own serializer, because there is no default serialization from string to utc_date. I do not want to analyze every string whether it corresponds to the date format and then decide to parse it into vpack string or utc_date.

If you have only a few date fields in your documents, you can register a VPackJsonSerializer over ArangoDB.Builder.registerJsonSerializer(String, Class, VPackJsonSerializer) for each affected fieldname.
Otherwise you have to implement it with the mentioned string analysation in a VPackJsonSerializer and register it over ArangoDB.Builder.registerJsonSerializer(Class, VPackJsonSerializer).

Best,
Mark

[grexe]

we also ran into this issue and found it quite confusing that the latest Java driver would allow you to write Dates but not read it back correctly.

I understand your rationale @mpv1989 but it's still very uncomfortable having to write your own (de)serializer for such simple and common use cases imho. If my POJO defines a Date, I'd expect VPack to handle it correctly.

Why not establish the contract that java-arango-driver will always use ISO 8601 Date String for writing to JSON and assume the same for reading it back? (would be consistent).
For cases where this default behavior is not desired, you could always use a VPack specific Annotation on the getter...