Plans to share system configuration information?

[Chris113113]

I'm curious about the image setup for several of the systems and wanted to know if you plan to check these configs / dockerfiles in as well?

[cquil11]

Hi @Chris113113 and welcome

We're not opposed to releasing some system specs, we just think the primary focus for now should be the GPUs.

As for the Dockerfiles, all images used are publicly available and open source.

[sbates130272]

@cquil11 I would like to +1 the system specification component of this. And, if possible, adding to the README any public cloud machines (e.g. AWS, GCP, OCI etc.) that can run some or all of these tests. Thanks for open-sourcing this, it is very cool!

[cquil11]

@sbates130272 hi

I would like to +1 the system specification component of this

What in particular would be the most important to the community to see about the systems, in your optinion?

adding to the README any public cloud machines (e.g. AWS, GCP, OCI etc.) that can run some or all of these tests

Can you clarify what you mean here, please? I can't tell if you mean instructions on how to run the benchmark on the cloud or listing machines that are compatible to run the benchmarks.

[sbates130272]

@cquil11 I would like to see the README updated with a list of public-cloud machines that are compatible to run the benchmarks.

[cquil11]

@sbates130272 you can run this benchmark on any public cloud/neocloud machine you want. Simply fork the repo, add a script for your machine in runners, add the machine as a self-hosted runner on your fork, then use the existing workflow infrastructure we have setup to trigger an action on the runner.

The only requirement for the system is that it has the appropriate compute to run one of the three models we support.