Initial implementation of upload files api. by christ66 · Pull Request #1703 · mudler/LocalAI

christ66 · 2024-02-13T11:08:30Z

Description

This PR adds the file upload apis.

Part of #1273

Notes for Reviewers
Don't merge yet. Looking for issues and early reviews. Forgot to write tests, but it does work.

Signed commits

Yes, I signed my commits.

netlify · 2024-02-13T11:08:47Z

✅ Deploy Preview for localai ready!

Name	Link
🔨 Latest commit	`a66c432`
🔍 Latest deploy log	https://app.netlify.com/sites/localai/deploys/65d1c694a290330008a32b20
😎 Deploy Preview	https://deploy-preview-1703--localai.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

api/openai/files.go

mudler

This is looking good here, would be nice to have tests but it shouldn't be a blocker. Thanks for the efforts @christ66 !

…use error.

mudler · 2024-02-16T22:53:37Z

@christ66 I'd merge this as-is to avoid any merge conflicts later on, wdyt? I don't think the API as it is now would bring any regression, as we are adding new API endpoints here

mudler · 2024-02-17T22:44:10Z

api/openai/files.go

+func LoadUploadConfig(uploadPath string) {
+	file, err := os.ReadFile(filepath.Join(uploadPath, "uploadedFiles.json"))
+	if err != nil {
+		log.Error().Msgf("Failed to read file: %s", err)


small nit: maybe we should check first if the file does not exist, I guess the first time this is starting it would always display an error message otherwise which could be misleading, but we can address this later

mudler · 2024-02-17T22:45:12Z

This is a very nice contribution, thanks @christ66 !

mudler · 2024-02-18T07:54:06Z

tests seems to fail, but looks like it's just about updating deps with go mod

….0 by renovate (#18546) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [docker.io/localai/localai](https://togithub.com/mudler/LocalAI) | minor | `v2.8.2-cublas-cuda11-ffmpeg-core` -> `v2.9.0-cublas-cuda11-ffmpeg-core` | | [docker.io/localai/localai](https://togithub.com/mudler/LocalAI) | minor | `v2.8.2-cublas-cuda11-core` -> `v2.9.0-cublas-cuda11-core` | | [docker.io/localai/localai](https://togithub.com/mudler/LocalAI) | minor | `v2.8.2-cublas-cuda12-ffmpeg-core` -> `v2.9.0-cublas-cuda12-ffmpeg-core` | | [docker.io/localai/localai](https://togithub.com/mudler/LocalAI) | minor | `v2.8.2-cublas-cuda12-core` -> `v2.9.0-cublas-cuda12-core` | | [docker.io/localai/localai](https://togithub.com/mudler/LocalAI) | minor | `v2.8.2-ffmpeg-core` -> `v2.9.0-ffmpeg-core` | | [docker.io/localai/localai](https://togithub.com/mudler/LocalAI) | minor | `v2.8.2` -> `v2.9.0` | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>mudler/LocalAI (docker.io/localai/localai)</summary> ### [`v2.9.0`](https://togithub.com/mudler/LocalAI/releases/tag/v2.9.0) [Compare Source](https://togithub.com/mudler/LocalAI/compare/v2.8.2...v2.9.0) This release brings many enhancements, fixes, and a special thanks to the community for the amazing work and contributions! We now have sycl images for Intel GPUs, ROCm images for AMD GPUs,and much more: - You can find the AMD GPU images tags between the container images available - look for `hipblas`. For example, [master-hipblas-ffmpeg-core](https://quay.io/repository/go-skynet/local-ai/tag/master-hipblas-ffmpeg-core). Thanks to [@fenfir](https://togithub.com/fenfir) for this nice contribution! - Intel GPU images are tagged with `sycl`. You can find images with two flavors, sycl-f16 and sycl-f32 respectively. For example, [master-sycl-f16](https://quay.io/repository/go-skynet/local-ai/tag/master-sycl-f16-core). Work is in progress to support also diffusers and transformers on Intel GPUs. - Thanks to [@christ66](https://togithub.com/christ66) first efforts in supporting the Assistant API were made, and we are planning to support the Assistant API! Stay tuned for more! - Now LocalAI supports the Tools API endpoint - it also supports the (now deprecated) functions API call as usual. We now also have support for SSE with function calling. See [https://github.com/mudler/LocalAI/pull/1726](https://togithub.com/mudler/LocalAI/pull/1726) for more - Support for Gemma models - did you hear? Google released OSS models and LocalAI supports it already! - Thanks to [@dave-gray101](https://togithub.com/dave-gray101) in [https://github.com/mudler/LocalAI/pull/1728](https://togithub.com/mudler/LocalAI/pull/1728) to put efforts in refactoring parts of the code - we are going to support soon more ways to interface with LocalAI, and not only restful api! ##### Support the project First off, a massive thank you to each and every one of you who've chipped in to squash bugs and suggest cool new features for LocalAI. Your help, kind words, and brilliant ideas are truly appreciated - more than words can say! And to those of you who've been heros, giving up your own time to help out fellow users on Discord and in our repo, you're absolutely amazing. We couldn't have asked for a better community. Just so you know, LocalAI doesn't have the luxury of big corporate sponsors behind it. It's all us, folks. So, if you've found value in what we're building together and want to keep the momentum going, consider showing your support. A little shoutout on your favorite social platforms using [@LocalAI_OSS](https://twitter.com/LocalAI_API) and [@mudler_it](https://twitter.com/mudler_it) or joining our sponsorship program can make a big difference. Also, if you haven't yet joined our Discord, come on over! Here's the link: https://discord.gg/uJAeKSAGDy Every bit of support, every mention, and every star adds up and helps us keep this ship sailing. Let's keep making LocalAI awesome together! Thanks a ton, and here's to more exciting times ahead with LocalAI! 🚀 ##### What's Changed ##### Bug fixes 🐛 - Add TTS dependency for cuda based builds fixes [#1727](https://togithub.com/mudler/LocalAI/issues/1727) by [@blob42](https://togithub.com/blob42) in [https://github.com/mudler/LocalAI/pull/1730](https://togithub.com/mudler/LocalAI/pull/1730) ##### Exciting New Features 🎉 - Build docker container for ROCm by [@fenfir](https://togithub.com/fenfir) in [https://github.com/mudler/LocalAI/pull/1595](https://togithub.com/mudler/LocalAI/pull/1595) - feat(tools): support Tool calls in the API by [@mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1715](https://togithub.com/mudler/LocalAI/pull/1715) - Initial implementation of upload files api. by [@christ66](https://togithub.com/christ66) in [https://github.com/mudler/LocalAI/pull/1703](https://togithub.com/mudler/LocalAI/pull/1703) - feat(tools): Parallel function calling by [@mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1726](https://togithub.com/mudler/LocalAI/pull/1726) - refactor: move part of api packages to core by [@dave-gray101](https://togithub.com/dave-gray101) in [https://github.com/mudler/LocalAI/pull/1728](https://togithub.com/mudler/LocalAI/pull/1728) - deps(llama.cpp): update, support Gemma models by [@mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1734](https://togithub.com/mudler/LocalAI/pull/1734) ##### 👒 Dependencies - deps(llama.cpp): update by [@mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1714](https://togithub.com/mudler/LocalAI/pull/1714) - ⬆️ Update ggerganov/llama.cpp by [@localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1740](https://togithub.com/mudler/LocalAI/pull/1740) ##### Other Changes - ⬆️ Update docs version mudler/LocalAI by [@localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1718](https://togithub.com/mudler/LocalAI/pull/1718) - ⬆️ Update ggerganov/llama.cpp by [@localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1705](https://togithub.com/mudler/LocalAI/pull/1705) - Update README.md by [@lunamidori5](https://togithub.com/lunamidori5) in [https://github.com/mudler/LocalAI/pull/1739](https://togithub.com/mudler/LocalAI/pull/1739) - ⬆️ Update ggerganov/llama.cpp by [@localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1750](https://togithub.com/mudler/LocalAI/pull/1750) ##### New Contributors - [@fenfir](https://togithub.com/fenfir) made their first contribution in [https://github.com/mudler/LocalAI/pull/1595](https://togithub.com/mudler/LocalAI/pull/1595) - [@christ66](https://togithub.com/christ66) made their first contribution in [https://github.com/mudler/LocalAI/pull/1703](https://togithub.com/mudler/LocalAI/pull/1703) - [@blob42](https://togithub.com/blob42) made their first contribution in [https://github.com/mudler/LocalAI/pull/1730](https://togithub.com/mudler/LocalAI/pull/1730) **Full Changelog**: mudler/LocalAI@v2.8.2...v2.9.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "before 10pm on monday" in timezone Europe/Amsterdam, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate).

Initial implementation of upload files api.

93b75d8

mudler reviewed Feb 13, 2024

View reviewed changes

api/openai/files.go Outdated Show resolved Hide resolved

mudler previously approved these changes Feb 13, 2024

View reviewed changes

mudler mentioned this pull request Feb 13, 2024

feat: Assistant API #1273

Closed

christ66 added 8 commits February 14, 2024 01:43

Move sanitize method to utils.

df620c6

Save uploaded data to uploads folder.

dc34716

Avoid loop if we do not have a purpose.

b699d03

Minor cleanup of api and fix bug where deleting duplicate filename ca…

0e0dcb5

…use error.

Revert defer of saving config

f553c06

Moved creation of directory to startup.

5f0d8a4

Make file names unique when storing on disk.

9656089

Merge remote-tracking branch 'upstream/master' into filesApi

1ca8b52

mudler added enhancement New feature or request roadmap labels Feb 14, 2024

Add test for files api.

e90c086

christ66 dismissed mudler’s stale review via e90c086 February 17, 2024 12:37

christ66 marked this pull request as ready for review February 17, 2024 12:39

mudler enabled auto-merge (squash) February 17, 2024 22:41

mudler reviewed Feb 17, 2024

View reviewed changes

mudler previously approved these changes Feb 17, 2024

View reviewed changes

Update dependencies.

472fee1

auto-merge was automatically disabled February 18, 2024 08:23
Head branch was pushed to by a user without write access

christ66 dismissed mudler’s stale review via 472fee1 February 18, 2024 08:23

Merge branch 'master' into filesApi

a66c432

mudler enabled auto-merge (squash) February 18, 2024 09:11

mudler approved these changes Feb 18, 2024

View reviewed changes

mudler merged commit 01205fd into mudler:master Feb 18, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Initial implementation of upload files api.#1703

Initial implementation of upload files api.#1703
mudler merged 12 commits intomudler:masterfrom
christ66:filesApi

christ66 commented Feb 13, 2024 •

edited by mudler

Loading

Uh oh!

netlify bot commented Feb 13, 2024 •

edited

Loading

Uh oh!

Uh oh!

mudler left a comment

Uh oh!

mudler commented Feb 16, 2024

Uh oh!

mudler Feb 17, 2024

Uh oh!

mudler commented Feb 17, 2024

Uh oh!

mudler commented Feb 18, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

christ66 commented Feb 13, 2024 • edited by mudler Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

netlify bot commented Feb 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for localai ready!

Uh oh!

Uh oh!

mudler left a comment

Choose a reason for hiding this comment

Uh oh!

mudler commented Feb 16, 2024

Uh oh!

mudler Feb 17, 2024

Choose a reason for hiding this comment

Uh oh!

mudler commented Feb 17, 2024

Uh oh!

mudler commented Feb 18, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

christ66 commented Feb 13, 2024 •

edited by mudler

Loading

netlify bot commented Feb 13, 2024 •

edited

Loading