Skip to content

Use ExternalProject_Add to add extension/llm/tokenizers library #10930

@kirklandsign

Description

@kirklandsign

🚀 The feature, motivation and pitch

Right now we are using add_subdirectory like this:

add_subdirectory(
${EXECUTORCH_ROOT}/extension/llm/tokenizers
${CMAKE_CURRENT_BINARY_DIR}/tokenizers
)

One issue is sub-libraries doesn't have namespace, so if any deps has same name, we get conflict and error. For example, #6767 is blocked.

Let's see if ExternalProject_Add could help.

Alternatives

No response

Additional context

No response

RFC (Optional)

No response

cc @larryliu0820 @jathu

Metadata

Metadata

Assignees

No one assigned

    Labels

    module: build/installIssues related to the cmake and buck2 builds, and to installing ExecuTorchtriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions