Rethink about the runtime encoding of ocaml values in javascript

[bobzhang]

Goal:

1. Less work to patch the compiler
2. OCaml Array is JS Array

Some documentation about the current encoding is [https://github.com/bloomberg/ocamlscript/blob/master/docs%2Fffi.md] here, there is a problem with this encoding is that Pfield i is no longer the same as Parrayref, while the internal OCaml compiler think it is the same, for example stdlib/camlinternalOO.ml, bytecomp/translobj.ml, bytecomp/translclass.ml there might be some other files I am missing. (Recent changes in the trunk of stdlib/camlinternalOO requires us to sync up the change)

So I am thinking of that Obj.tag is not used in too much in the compiler itself(except the GC, which is not relevant in js backend)

So I am proposing that blocks with tag zero is encoded as array, blocks with tag non zero (mostly normal variants) will be encoded as array plus an extra property via Object.defineProperty so that they are polymorphic. and Pfield, Parrayref will behave the same.

for example

type u = 
| B of int * int
|A of int * int * int 

A (1,2,3) will be encoded as Object.defineProperty([1,2,3], {'t', {'value', 1}}
B(1,2) will be encoded as [1,2]

[bobzhang]

Note that Object.defineProperty is slow, and variants in general will not be very big, we can have objects like this for A(1,2,3)

{ t : 1, 0 : 1,1:2,2:3}

This encoding actually be efficient in most cases like option type, list and map, set, since they only have two branches for the internal representation

[m2ym]

As far as I know, ocamlscript currently encodes OCaml string (pure byte array) as JS string (UTF16 string?). I think it may go wrong in particular situations. Sorry if unrelated to this issue.

[bobzhang]

@m2ym indeed, there is a corner case in range pattern

function '\000' .. '\255' -> 1 

in ocaml will be compiled into

function _ -> 1

do you have anything else in mind? thanks! (ocamlscript will require -safe-string)

[bobzhang]

we can walk around this issue by x.charCodeAt(i) & 0xff, but I am not sure whether it is worth the effort (some of them can be optimized away)

[bobzhang]

think twice, it might not be a problem, since string is immutable, and is lexed by ocamllex, there is no way you can encode a string out of range, it only happens in FFI, @alainfrisch ?

[alainfrisch]

Yes, you could get strings from JS through the FFI.

[bobzhang]

another benefit of such encoding is that for the FFI, tuple is quite easy to map, even if we goto typedtree we can still guarantee the mapping of tuple, since the field of tag is singled out, it is easy to tell if it is an ocaml object and js object

[copy]

Float arrays and records could be represented using Float64Array. Maybe integer arrays too (using Int32Array).

[bobzhang]

@copy I have similar ideas, but Float64Array is not widely support across js engines, how does js_of_ocaml do the encoding, do you have any idea why it does not use Float64Array if not?

[copy]

I don't know why they don't use it, maybe they haven't considered it yet.

I agree with engine support. It could be enabled conditionally with a flag or shimmed though.

[bobzhang]

@hhugo do you have any suggestion?

[bobzhang]

For the record, encoding variants as js objects will help debugging a lot, for example

function $Cons(x,y){ this._0 = x; this._1 = y; this.t = "Cons" }

$Cons.prototype.toString = function(){return `${this.t} ( ${this._0}, ${this._1})`}

The benefit is that we can make use of prototype chain in js and get a pretty output when debugging, the challenge is that we need know where constructors are called.

Every call site x::y has to be replaced by new $Cons(x,y)

for the record we can do something similar

type t = { x  : int ; y : int}

function $t(x,y} { this._0 = x; this._1 = y; this.lables = ['x','y']}
$t.prototype.toString= function(){...}

we can still preserve the structure comparison semantics of ocaml, the same challenge as we have for encoding variants, if we go this way, it is less work then compiling from typedtree, but get a reasonable output

[bobzhang]

note that the reason that lazy is broken is also due to ocaml compiler internally treat object and array in a uniform way

[bobzhang]

so currently, record, tuple, array, polymorphic variants are all compiled into array(offset 0) normal variants are compiled into array like object

{ 0 : field_0, 1 : field_1, ... n - 1 : field_n_1, length : n , tag : obj.tag}

type option and list are handled specially,

Some x is compiled into [x]
x::y is compiled into [x, y]