Enable install find package

[larryliu0820]

This pull request introduces several enhancements to the CMakeLists.txt file and adds a new configuration file for the tokenizers package. The changes focus on improving build system support, adding installation rules, and providing a package configuration file for easier integration with other projects.

Build system improvements:

• CMakeLists.txt: Included CMakePackageConfigHelpers to enable the use of configure_package_config_file for generating CMake configuration files.
• CMakeLists.txt: Updated target_include_directories to use $<BUILD_INTERFACE> for better handling of include directories during the build process.

Installation rules:

• CMakeLists.txt: Added installation rules for the library, headers, and CMake configuration files, including tokenizers-targets.cmake and tokenizers-config.cmake. These rules ensure the library and its dependencies are correctly installed.

Package configuration:

• cmake/tokenizers-config.cmake.in: Added a new configuration file for the tokenizers package, defining variables such as TOKENIZERS_FOUND, TOKENIZERS_INCLUDE_DIRS, and TOKENIZERS_LIBRARIES. It includes dependency checks and ensures the sentencepiece library is available.

---

For any CMake project we should be able to do the following:

# add tokenizers
ExternalProject_Add(
  tokenizers_external_project
  PREFIX ${CMAKE_CURRENT_BINARY_DIR}/../tokenizers
  SOURCE_DIR ${EXECUTORCH_ROOT}/extension/llm/tokenizers
  CMAKE_ARGS -DSUPPORT_REGEX_LOOKAHEAD=ON -DCMAKE_INSTALL_PREFIX=${CMAKE_CURRENT_BINARY_DIR}/../tokenizers
  INSTALL_DIR ${CMAKE_CURRENT_BINARY_DIR}/../tokenizers
)
add_dependencies(extension_llm_runner tokenizers_external_project)
find_package(tokenizers CONFIG HINTS ${CMAKE_INSTALL_PREFIX} ${CMAKE_CURRENT_BINARY_DIR}/../tokenizers)

[jackzhxng]

Excited to use ExternalProject_Add, I assume you will be refactoring ET to use this as a follow up?

[jackzhxng]

Won't this single include also be found in ${CMAKE_CURRENT_SOURCE_DIR}/include/? Then you can update the pattern to also accept *.hpp?

[larryliu0820]

It's not in ${CMAKE_CURRENT_SOURCE_DIR}/include/