Proposal: Remove initial resolution step for dynamic references

[jdesrosiers]

This issue is a spin-off of #1064. That conversation drifted a bit, so I'm moving some of the proposed amendments to that proposal into a separate issue.

Removing the bookending requirement for dynamic references opens the door for some additional considerations. $recrusiveRef didn't allow non-fragment URI parts. $dynamicRef added that in order to allow using $dynamicRef in a non-recursive schema. However, without the bookending requirement, the initial resolution step is no longer necessary (see #1064 (comment)). Since it's not necessary, there exists the possibility that a $dynamicRef doesn't need to be a URI at all. It makes sense for it to just be the dynamic anchor name. There's no reason to encode it into a URI. (Credit to @ssilverman for this idea) (See also #1064 (comment)).

We could take this a step further and make dynamic anchors URIs instead of plain name identifiers. Because dynamic anchors aren't scoped to the local document like traditional anchors, there's a possibility of name conflicts between schemas, so using URI identifiers would be safer. It's a similar concept to using URIs for link relations (for those who are familiar with those).

[jdesrosiers]

Here's an example that sets a dynamic anchor with the identifier https://json-schema.hyperjump.io/anchor/list and dynamically references it.

{
  "$id": "https://json-schema.hyperjump.io/schema/list",
  "$schema": "https://json-schema.org/draft/2020-12/schema",

  "type": "object",
  "properties": {
    "list": {
      "type": "array",
      "items": { "$dynamicRef": "/anchor/llist" }
    },
    "nextPage": { "type": "integer" },
    "previousPage": { "type": "integer" },
    "perPage": { "type": "integer" },
    "page": { "type": "integer" }
  }
}

{
  "$id": "https://json-schema.hyperjump.io/schema/foo-list",
  "$schema": "https://json-schema.org/draft/2020-12/schema",

  "$ref": "/schema/list",

  "$defs": {
    "foo": {
      "$dynamicAnchor": "/anchor/list",
      "$ref": "/schema/foo"
    }
  }
}

[jdesrosiers]

I couldn't help writing up what the spec might look like without bookending or initial resolution (#1142). It makes the concept much easier to describe. Personally, I think it's easier to understand as well.

[jdesrosiers]

Here is an example using the hyper-schema links schema that shows what is lost by removing the initial resolution step. In this case we want "submissionSchema" to be extendible with a dynamic reference, but we also want it to default to a specific draft. With initial resolution, we can do this.

{
  "$id": "https://json-schema.org/draft/2020-12/links",
  "properties": {
    "submissionSchema": { "$dynamicRef": "/draft/2020-12/hyper-schema#meta" }
  }
}

The dynamic reference first resolves to the 2020-12 dialect where a "meta" dynamic anchor enters the dynamic scope. Since there are no other "meta" dynamic anchors already in this scope, the 2020-12 dialect is used. Without the initial resolution, we can get the same behavior, but it's more verbose.

{
  "$id": "https://json-schema.org/draft/2020-12/links",
  "properties": {
    "submissionSchema": { "$dynamicRef": "meta" }
  },
  "$defs": {
    "defaultDialect": {
      "$dynamicAnchor": "meta",
      "$ref": "/draft/2020-12/hyper-schema"
    }
  }
}

In order for there to be a default place for the dynamic reference to resolve, we have to explicitly set a dynamic anchor somewhere in the schema to define the default.

[handrews]

I just read back through this and the previous issue, as I never responded to your last example there which you have expanded on here.

I agree that this works!

While it's true that this particular case becomes more verbose, I think that's actually a good thing. The mutual recursion case is very hard to wrap your head around, and having explicit starting points helps make it a lot more clear what is happening.

Here's my last example from the other issue, updated based on your current proposal, just to make sure I'm reading things correctly (We'll just pretend like there's a 2020-12 hyper-schema even though there isn't):

{
  "$id": "https://json-schema.org/draft/2020-12/links",
  "$dynamicAnchor": "links-schema",
  "type": "object",
  "properties": {
    "submissionSchema": { "$dynamicRef": "meta" }
  },
  "$defs": {
    "defaultDialect": {
      "$dynamicAnchor": "meta",
      "$ref": "https://json-schema.og/2020-12/hyper-schema#meta"
    }
  }
}

{
  "$id": "https://json-schema.org/draft/2020-12/hyper-schema",
  "$dynamicAnchor": "meta",
  "type": "object",
  "properties": {
    "items": {"$dynamicRef": "meta"},
    "links": {
      "type": "array",
      "items": {"$dynamicRef": "links-schema"}
    }
  },
  "$defs": {
    "defaultLinksSchema": {
      "$dynamicAnchor": "links-schema",
      "$ref": "https://json-schema.og/2020-12/links#links-schema"
    }
  }
}

@jdesrosiers I see that your solution accomplishes several things compared to 2020-12:

• No bookending requirement (which I realize was done as a PR for the previous issue)
• $ref/$anchor/$id and $dynamicRef/$dynamicAnchor are now totally independent sets of functionality
• It is still possible to make a concrete schema involving $dynamicRef, you just have to explicitly bridge the URI-based static reference and identification system to the non-URI-based dynamic one

I definitely find that last point appealing. TBH I'm not sure why I thought the two systems should interact.

I like it! Definitely an improvement 😃

[jdesrosiers]

@handrews Thanks for looking this over. Your timing is perfect because I was hoping to get back to working on this issue soon. Your example is pretty much what I had in mind except there should be no need for the $refs to include the fragment.

What are your thoughts about making dynamic anchors URIs? My main concern is that people will find it confusing even tho there's precedence in the web space (link relations).

We could take this a step further and make dynamic anchors URIs instead of plain name identifiers. Because dynamic anchors aren't scoped to the local document like traditional anchors, there's a possibility of name conflicts between schemas, so using URI identifiers would be safer. It's a similar concept to using URIs for link relations (for those who are familiar with those).

[handrews]

there should be no need for the $refs to include the fragment

Oh, right- at least I think I understand. There are actually two things here:

1. The target of these particular $refs is the resource root, so there's no need for a fragment at all
2. If we did need a fragment, it would need to be declared with $anchor and not $dynamicAnchor because $dynamicAnchor no longer creates fragments (which I think is a good thing).

Are both of those correct? I think in some circumstances it would be fine to declare both a static $anchor target and a dynamic $dynamicAnchor target in the same schema, but agree it is unnecessary and probably confusing in this example.

---

What are your thoughts about making dynamic anchors URIs? My main concern is that people will find it confusing even tho there's precedence in the web space (link relations).

I think it will be confusing simply because you write the same dynamic anchor in multiple places, and that's not how URIs are normally used. It's fine with link relations, because you aren't targeting them in those different locations. They serve as references to or identifiers for the semantics of the link relation type. The target that they identify is the specification, and there's only one of those for each link relation URI.

Making $dynamicAnchor a URI makes that URI both a reference (to the semantics of the dynamic anchor) and a target (because you would then use it as the value of $dynamicRef) at the same time. And that's confusing. It's like making the href from <a href="https://example.com/whatever"> a target. The only way you can make an <a> element a target is <a href="https://example.com/whatever" id="thisIdIsAFragment"> But in that case, it's not the href that's the target, it's the id.

I think it's better to keep $dynamicAnchor and $dynamicRef separate from URIs/IRIs entirely. If the concern is uniqueness in an open ecosystem, there are other ways to handle that such as various sorts of namespacing. There's the Java approach of using reversed domain names, for example.

With link relation types, there is a globally shared notion of semantics. That's not really the same for $dynamicAnchor. While we name them after what they do for convenience, the names are just used mechanically. A schema with all of its human-readable $dynamicAnchor values consistently replaced with gibberish hex strings would work just as well. That is not true of link relation types, hence their need for a global semantic registry identified through URIs.

[jdesrosiers]

1. The target of these particular $refs is the resource root, so there's no need for a fragment at all

Yep

2. If we did need a fragment, it would need to be declared with $anchor and not $dynamicAnchor because $dynamicAnchor no longer creates fragments (which I think is a good thing).

I think we could go either way with this. We could still allow dynamic anchors to be referenced by $ref, but this change would make that unnecessary and it would be cleaner to keep the two concepts separate; you reference anchors with $ref and dynamic anchors with $dynamicRef.

What are your thoughts about making dynamic anchors URIs? My main concern is that people will find it confusing even tho there's precedence in the web space (link relations).

I think it will be confusing ...

Yeah, you make some good points. I think it's not likely to be needed in enough cases to be worth the added complexity and there are plenty of ways to namespace a dynamic anchor that are compatible with plain-name fragments for the cases where it is needed.

[handrews]

I'm in favor of making this change for the next release. We do need to figure out our compatibility story here, as the semantics of $dynamicAnchor change, and both the syntax and semantics of $dynamicRef, even though the core use case remains the same. We could:

• Just change it — we're not guaranteeing compatibility yet anyway
• make $dynamicAnchor2 and $dynamicRef2 or some other new pair of names

Just changing it would be more appealing if we can definitely automatically migrate existing uses to a combination of the new syntax and $ref (as some use cases require both a lexical and dynamic reference working together).

Of course, all of this is relevant to / in the context of #1242 .

[jdesrosiers]

I think we can just change it. We aren't committed to no backwards compatible changes yet. This is definitely something we want to get in before we consider this feature stable.