Process for updating browser.webidl.preprocessed.json?

[foolip]

The readme says that browser.webidl.preprocessed.json is a JSON file generated by Microsoft Edge, not to be updated. It looks like it was last updated in April 2019.

addedTypes.json, however, seems to be often updated.

Which brings me to my question: is a new approach needed to get more of the definitions via (semi) automated processes, rather than adding types by hand? I also do some work on webref (scraped IDL from specs) and BCD (browser compat data) and this is what comes to mind:

Match up IDL definitions with BCD entries. For IDL definitions missing in TSJS-lib-generator where BCD says the APIs are supported, propose those for inclusion in TSJS-lib-generator. The precise rules for what level of support is necessary can be debated.

There would be additional work to ensure that BCD keeps getting new entries based on additions to webref, of course, but that's something @vinyldarkscratch and I are already working on separately in https://github.com/foolip/mdn-bcd-collector.

[foolip]

Poking around a bit more, I see that some of the pipeline from reffy+webref also exists in this repository, as idlSources.json and scraped IDL files.

In particular, browser-specs is a similar list to idlSources.json, and idlfetcher.ts is similar to extract-webidl.js.

Perhaps there would be benefit in sharing more of the pipeline here as well. Not because of the code duplication, really, but because there is a manual component to keeping anything like this working over time, and that effort is probably duplicated now. (It's things like fixing broken Web IDL in specs, dealing with specs being renamed/moved, etc.)

Sorry to only throw out ideas so far. If any of this sounds useful and there's a small and low-risk one could start, I'd be willing to also do some work :)

[foolip]

@saschanaz I discovered #915 just now. You're clearly doing some work along similar lines here, so this issue might interest you. I'd love to hear any ideas like this you've been thinking of.

[saschanaz]

idlSources.json is intentionally manual for now, as the initial intention was to slowly migrate from browser.webidl.preprocessed.json to standard IDLs and see how it goes. So far any changes required manual review because there was no automatic way to exclude unimplemented features, which is why idlSources.json only covers a subset of what browser-specs does.

I think we can use browser-specs with confidence after #915 and after we successfully remove browser.webidl.preprocessed.json dependency (#725).

(Edit: I guess a full automation here needs some interest from Microsoft as it would require a bot)

[foolip]

Aha, that sounds promising!

Are there rules for what non-standard features should be included in this repository? I see that browser.webidl.preprocessed.json includes things like window.offscreenBuffering a pointless attribute that nonetheless is in all engines but Gecko. That's a trivial example I knew to look for, presumably there are also other things in there that are also non-standard but widely supported.

In https://github.com/foolip/mdn-bcd-collector/tree/main/custom-idl, @vinyldarkscratch and I have been maintaining a bunch of custom IDL for things that are (perhaps) in some browser but not in any spec IDL. Would something like this also be needed as input to replace browser.webidl.preprocessed.json?

[saschanaz]

README has the rules: https://github.com/microsoft/TSJS-lib-generator#additions

I don't think the WHATWG/W3C differentiation is very useful though. My own rule is simply "at least two stable implementations".

The generated data from EdgeHTML naturally includes EdgeHTML specific types, and such types are removed by removedTypes.json. (The preprocessed.json is not supposed to be edited 👀) #915 will be the successor for this purpose.

The custom IDLs look interesting, I may take a look at them if #725 turns out to lack such types. Thanks for the heads up 👍

Edit: Currently I'm not removing types that are widely supported but nonstandard. I think such things should be specced anyway.

[foolip]

Thanks @saschanaz! I agree that trying to differentiate WHATWG and W3C specs doesn't seem very useful, but hopefully I won't run into that. I hadn't seen removedTypes.json, thanks for pointing that out.

Currently I'm not removing types that are widely supported but nonstandard. I think such things should be specced anyway.

I agree, but don't think there will ever be a point in time when the sum of all spec IDL adds up to something consistent and covering everything shipped in any browser, so some manual curation of IDL seems necessary. I'd like to avoid it though, if the problem becomes small enough.

[foolip]

I'm not sure what the fate of browser.webidl.preprocessed.json is, but there's now https://www.npmjs.com/package/@webref/idl which contains all the IDL in specs. Putting that together with results from https://mdn-bcd-collector.appspot.com/ one could filter that down to only stuff that's known to be in some browser.

[saschanaz]

What I want here is what https://github.com/saschanaz/types-web/ does, which uses webref+bcd to achieve that goal, which again is exactly what you suggest. #915 (comment) implies Microsoft at least wants to accept the bcd part.

[saschanaz]

That said, three-month review cycle is quite painful to make any serious contribution or discussion. Could this repo allow more admins?

[saschanaz]

#1034 removed it

@github-actions close

[github-actions]

Closing because @saschanaz is one of the code-owners of this repository.