examples: cache hf model when --model not provided#7353
examples: cache hf model when --model not provided#7353ggerganov merged 5 commits intoggml-org:masterfrom
Conversation
|
Also, wouldn't it be nice to have
On windows, the equivalent of Not sure what it is on macos. |
mofosyne
added
enhancement
| cache_directory = std::getenv("HOME") + std::string("/.cache/"); | ||
| } | ||
| #elif defined(__APPLE__) | ||
| cache_directory = std::getenv("HOME") + std::string("/Library/Caches/"); |
There was a problem hiding this comment.
i think on Mac os pretty much all apps also use ~/.cache (so, same as linux)
Just my 2 cents
There was a problem hiding this comment.
I don't use macos very often so I'm not sure. Maybe @ggerganov knows more about this?
My assumption is that ~/.cache widely used by applications ported from linux (maybe?), while ~/Library/Caches is the conventional OS X way (according to this documentation).
In either way, having model cached under user's $HOME is already very nice.
There was a problem hiding this comment.
I see apps caches in both places on my machine - I guess either way is fine. Don't have an opinion
There was a problem hiding this comment.
Let's keep ~/Library/Caches then (as it is referred in apple's documentation)
| cache_directory += "llama.cpp"; | ||
| cache_directory += DIRECTORY_SEPARATOR; | ||
| } | ||
| const bool success = create_directory_with_parents(cache_directory); |
There was a problem hiding this comment.
Maybe you forgot to remove this?
There was a problem hiding this comment.
My bad, it should be removed now :)
ggerganov
left a comment
ggerganov
left a comment
There was a problem hiding this comment.
Should be good to merge after resolving: https://github.com/ggerganov/llama.cpp/pull/7353/files#r1607877007
| } | ||
| params.model = cache_directory + string_split(params.hf_file, '/').back(); | ||
| } | ||
| } else if (!params.model_url.empty()) { |
There was a problem hiding this comment.
Quick question: Do you think we should also use cache dir under this condition params.model_url? @ggerganov
For now only params.hf_repo enables cache dir.
ngxson
left a comment
ngxson
left a comment
There was a problem hiding this comment.
LGTM. I'd prefer to also support cache dir with --model-url. But that's not very important for now.
| #elif defined(__APPLE__) | ||
| cache_directory = std::getenv("HOME") + std::string("/Library/Caches/"); | ||
| #elif defined(_WIN32) | ||
| cache_directory = std::getenv("APPDATA"); |
There was a problem hiding this comment.
Btw, the cache directory on Windows is %LOCALAPPDATA% (see platformdirs). %APPDATA% will be synced synced between computers in an Active Directory domain ("Roaming").
* examples: cache hf model when --model not provided * examples: cache hf model when --model not provided * examples: cache hf model when --model not provided * examples: cache hf model when --model not provided * examples: cache hf model when --model not provided
* examples: cache hf model when --model not provided * examples: cache hf model when --model not provided * examples: cache hf model when --model not provided * examples: cache hf model when --model not provided * examples: cache hf model when --model not provided
* examples: cache hf model when --model not provided * examples: cache hf model when --model not provided * examples: cache hf model when --model not provided * examples: cache hf model when --model not provided * examples: cache hf model when --model not provided
6 participants
Ref: #7252