Skip to content

[SYCL] fix set main gpu error, support single/mul gpu mode #6073

Merged
NeoZhangJianyu merged 1 commit intoggml-org:masterfrom
NeoZhangJianyu:fix_set_main_gpu
Mar 15, 2024
Merged

[SYCL] fix set main gpu error, support single/mul gpu mode #6073
NeoZhangJianyu merged 1 commit intoggml-org:masterfrom
NeoZhangJianyu:fix_set_main_gpu

Conversation

@NeoZhangJianyu
Copy link
Copy Markdown
Contributor

  1. fix error to set main gpu as non-zero.

  2. add new APIs: ggml_backend_sycl_set_single_device_mode(), ggml_backend_sycl_set_mul_device_mode() to handle single/multiple cards by split-mode.
    In CI, enable ggml_backend_sycl_set_mul_device_mode() as default.
    If split-mode==layer (default), use all GPUs with top max compute unit.
    Else, use the main-gpu set by user as only device. It supports level_zero:gpu, opencl:gpu.

  3. refactor the shown device list.

  • change the order in shown device list.
    sort the devices by type (level_zero, opencl:gpu, opencl:cpu, opencl:acc) and max compute unit.
  • add type item to follow the output style of tool sycl-ls.
found 6 SYCL devices:
|  |                  |                                             |compute   |Max compute|Max work|Max sub|               |
|ID|       Device Type|                                         Name|capability|units      |group   |group  |Global mem size|
|--|------------------|---------------------------------------------|----------|-----------|--------|-------|---------------|
| 0|[level_zero:gpu:0]|               Intel(R) Arc(TM) A770 Graphics|       1.3|        512|    1024|     32|    16225243136|
| 1|[level_zero:gpu:1]|                    Intel(R) UHD Graphics 770|       1.3|         32|     512|     32|    53651849216|
| 2|    [opencl:gpu:0]|               Intel(R) Arc(TM) A770 Graphics|       3.0|        512|    1024|     32|    16225243136|
| 3|    [opencl:gpu:1]|                    Intel(R) UHD Graphics 770|       3.0|         32|     512|     32|    53651849216|
| 4|    [opencl:cpu:0]|         13th Gen Intel(R) Core(TM) i7-13700K|       3.0|         24|    8192|     64|    67064815616|
| 5|    [opencl:acc:0]|               Intel(R) FPGA Emulation Device|       1.2|         24|67108864|     64|    67064815616|
  1. update examples/sycl/run-llama2.sh
    support to switch in single/multiple cards by set device id as parameter.

@NeoZhangJianyu
Copy link
Copy Markdown
Contributor Author

@ggerganov @slaren I recreate the PR due to the CI always be fault.
It includes same code change of #6022.

Copy link
Copy Markdown
Member

@ggerganov ggerganov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CI using sanitizers is failing due to unrelated problems (some linux kernel update), so it is normal. If the SYCL builds are passing, it's ok to merge

@NeoZhangJianyu
Copy link
Copy Markdown
Contributor Author

The CI using sanitizers is failing due to unrelated problems (some linux kernel update), so it is normal. If the SYCL builds are passing, it's ok to merge

OK! Thank you!

@NeoZhangJianyu
Copy link
Copy Markdown
Contributor Author

@slaren Could you review and approve if it's OK!

@NeoZhangJianyu NeoZhangJianyu merged commit 46acb36 into ggml-org:master Mar 15, 2024
hodlen pushed a commit to hodlen/llama.cpp that referenced this pull request Apr 3, 2024
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
phuongncn pushed a commit to phuongncn/llama.cpp-gx10-dgx-sparks-deepseekv4 that referenced this pull request Apr 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants