State that implementations SHOULD accept a schema retrieval IRI / initial base IRI

[handrews]

While we discuss initial base IRIs, at no point do we make it clear that it's advantageous to allow an application to supply a retrieval IRI or other IRI for use as a base IRI in the absence of an absolute $id. Implementations vary with respect to how they handle this (or don't).

This strikes me as a SHOULD requirement rather than a MUST. It's advantageous to allow, but JSON Schema is usable without it, and the exact mechanism is not something we should specify. It's also conceivable that a specialized implementation would have a reason to skip this on the grounds of minimizing code or knowledge that external IRIs of this sort will not be useful in its intended execution context.

CLARIFICATION: "Initial base URI" is not the same thing as RFC 3986 §5.1.4's "Default base URI". An initial base URI can come from any of the sources described in RFC 3986 §5.1.2 - 5.1.4. This issue doesn't have anything to do with setting a broader §5.1.4 default base URI. Several comments related to that concept have been marked off-topic.

[gregsdennis]

For example of this implemented in the wild, see the playground at https://json-everything.net/json-schema. There is an option that allows the user to specify a default base URI.

[mwadams]

Agreed. I have been toying with this thought.

I have a "rebase as root" option which allows you to take a JSON island and rebase it as if it were an isolated document root; and I also have a "don't rebase this but treat this island as a base schema embedded in a non-schema document" (which is essentially the "OpenAPI" case).

But I don't have a "use this iri as the base for a schema, instead of the (potentially local filesystem-based) iri you actually came from.

[gregsdennis]

I definitely think that we need to include language that states this is only a fallback for when $id is absent. $id MUST NOT be overridden.

[handrews]

@gregsdennis good point. Should be obvious, but I am well aware that hardly anyone actually reads RFC 3986 no matter how much I xref it 😛

[handrews]

[EDIT: The requirements I wrote here were wrong, better ones forthcoming in a new comment.]

[handrews]

OK, thinking this through again and hopefully correctly this time, I think the requirements are:

• SHOULD support accepting a retrieval/initial base IRI (RIB IRI) for a schema resource
• MUST consider the retrieval IRI of an embedded resource to be the JSON Pointer fragment IRI using the base IRI of the enclosing resource
• MUST set the RIB IRI as the base IRI for a resource prior to processing its root schema object
• If the resource root has $id, MUST set the base IRI (and canonical IRI if we keep that distinction) for the resource to the resolved $id <-- Greg, this is the "do not override" rule

Note that with this approach, embedded schema resources and schema documents work the same way in terms of retrieval IRIs. Technically, this would mean that we don't need to specify that part of the behavior as part of the $id keyword. It's just normal retrieval-IRI-as-initial-base behavior.

The point I'm less clear on is whether the retrieval IRI needs to be associated with the schema resource root. This would allow it to be used in $ref even if it is not the canonical IRI (or otherwise setting the base IRI for the resource contents).

I would say that it is a bad practice to use such an IRI in $ref in general, but if an implementation does on-demand reference resolution with network fetching, then it's going to be using a retrieval IRI by definition. So I guess there are three cases of resources with differing $id and retrieval IRIs:

• schema document provided to the implementation with a retrieval IRI
• embedded schema resource, in which case the single case of a "shadowed" (resource boundary-crossing) JSON Pointer IRI pointing to the location where the resource is embedded could be usable alongside the canonical IRI, even if shadowed JSON Pointers are otherwise unsupported. This case is really more overlapping resource boundaries rather than crossing them.
• schema document retrieved on demand from a schema that by definition knows the retrieval IRI, but might not be aware of the internal $id value to use in references otherwise

[jdesrosiers]

First of all, I think there's some misunderstanding in parts of this conversation about how retrieval URIs work. A retrieval URI is not the same as a default base URI. If you fetch the schema from the file system or over HTTP, the URI you used to fetch the schema is the retrieval URI. So, when you indicate to an implementation a retrieval URI, you are saying, treat this schema as if I had fetched it from this location. If you have a default base URI that you treat as a retrieval URI, it's like treating all of your schemas as if they were accessed from the location you used as the default base URI. One URI identifying multiple resource is not how URIs are supposed to work. I can provide some examples if that wasn't clear enough.

The point I'm less clear on is whether the retrieval IRI needs to be associated with the schema resource root.

It absolutely makes sense to do this and my implementation does this. However, the way the spec is written now, only canonical URIs MUST be supported so this behavior would be optional.

at no point do we make it clear that it's advantageous to allow an application to supply a retrieval IRI or other IRI for use as a base IRI in the absence of an absolute $id. Implementations vary with respect to how they handle this (or don't).

In UJS - Structuring a Complex Schema I go into detail about how URI resolution works as defined by RFC-3986 including retrieval URIs. I think documentation is a better place for making this recommendation than the spec. The spec should just need to point to the RFC. Documentation is the place to help people understand that RFC and the implications of pointing to that RFC in cases where we see that sort of thing is necessary.

I wrote that page about this time last year, so it's relatively new. I think that most variation in how implementations work with regard to URI resolution is because this guidance didn't exist when they started their implementation rather than because our guidance is insufficient.

For what it's worth, I don't think I've ever used an implementation that does retrieval URIs wrong, it's just that most don't provide a way to simulate retrieval URIs when loading schemas.

[handrews]

If you have a default base URI that you treat as a retrieval URI, it's like treating all of your schemas as if they were accessed from the location you used as the default base URI.

"default base URI" here means "URI you use as a base URI prior to finding a base URI in the resource, or if you don't find a base URI in the resource", not "base URI you use for everything"

[handrews]

I used that term because there are valid RFC 3986 sources for base URIs other than the retrieval URI, and the JSON Schema implementation shouldn't assume that only retrieval URIs will be provided.

[jdesrosiers]

URI you use as a base URI prior to finding a base URI in the resource, or if you don't find a base URI in the resource

I don't believe the first part of that sentence is part of the RFC-3986 process of establishing a base URI. A default base URI applies only if a base URI couldn't be established any other way. That means, if there is no $id and no retrieval URI, then an application defined default base URI could be used. I don't think I mentioned default base URIs in the UJS page I mentioned. If we do want to encourage implementations to support that, I think an update to that page is the best place to make that recommendation.

[handrews]

@jdesrosiers I didn't mean "default base URI" in terms of RFC 3986 §5.1.4. Yes, I occasionally forget RFC wording. I just meant a default value for the base URI to use in the current evaluation. I am marking all of this as off-topic and I'll go fix the wording of the original post.

[handrews]

Actually, wait, I specifically said initial base URI and not default base URI. That was on purpose. That is what we are talking about.