coreml : revisit instructions and model generation scripts

[ggerganov]

The CoreML creation process has been flaky from the very beginning (#566), mainly because the python packages were not compatible with each other. Things likely have changed since the initial support was added and hopefully are more stable now.

• Update CoreML model conversion/creation instructions
• Add CI workflows to exercise CoreML conversion (can utilize ggml-ci for Apple Silicon nodes)

[furqan4545]

how to integrate .mlmodelc into my swiftui app? I am stuck here? I want to use coreml support. is there any example that I can follow?

[danbev]

The swiftui example currently does not have support for coreml, but the objective-c example does, and it contains a link to how to convert models. Perhaps these could be helpful to implement in swift.

[furqan4545]

The swiftui example currently does not have support for coreml, but the objective-c example does, and it contains a link to how to convert models. Perhaps these could be helpful to implement in swift.

I am very beginner in obj C... Would it be possible for you guys to make swift ui example? I think a lot of people would appreciate that because as saw in many other repos people were requesting the same thing.

[danbev]

Would it be possible for you guys to make swift ui example?

Perhaps we can update the existing whisper.swiftui example similar to whisper.objc. I've created #2907 to track this.

[ggerganov]

Would it be possible for you guys to make swift ui example?

Perhaps we can update the existing whisper.swiftui example similar to whisper.objc. I've created #2907 to track this.

If I am not mistaken, it might be as simple as adding WHISPER_COREML=1 somewhere in the build settings and providing a CoreML model to the project.

[danbev]

If I am not mistaken, it might be as simple as adding WHISPER_COREML=1 somewhere in the build settings and providing a CoreML model to the project.

Nice, I'll give this a try. Thanks

[danbev]

Hmm, this was not as simple because of the introduction of the xcframework. We are no longer compiling in XCode but instead using the compiled library from the xcframework. And Core ML support requires the above mentioned macro to be set when compiling. My initial throught was that we could provide builds for Core ML for macos and ios and include them in the xcframework, but that does not work as the selection is based on platform and architecture, so trying to create the xcframework results in:

A library with the identifier 'macos-arm64_x86_64' already exists.

I'll need to investigate this a little more to understand what our options are.

[ggerganov]

I think we can enable both WHISPER_COREML=ON and WHISPER_COREML_ALLOW_FALLBACK=ON in the CMake builds and provide only this option in the XCFramework. This way, if at runtime it fails to find a CoreML model, it will fallback to not using it.

[danbev]

I think we can enable both WHISPER_COREML=ON and WHISPER_COREML_ALLOW_FALLBACK=ON in the CMake builds and provide only this option in the XCFramework. This way, if at runtime it fails to find a CoreML model, it will fallback to not using it.

That would be much things much easier. Great, I'll try this out.

[danbev]

@furqan4545 We've added an example of using Core ML to the whisper.swiftui example now.