Skip to content

Fix server for openai package#877

Merged
awni merged 2 commits intomainfrom
fix_openai
Jul 8, 2024
Merged

Fix server for openai package#877
awni merged 2 commits intomainfrom
fix_openai

Conversation

@awni
Copy link
Copy Markdown
Member

@awni awni commented Jul 7, 2024

  1. Launch server:
mlx_lm.server
  1. Make request:
import openai

openai_client = openai.OpenAI(api_key = "placeholder-api", base_url="http://localhost:8080")

response = openai_client.chat.completions.create(
    model='mlx-community/Meta-Llama-3-8B-Instruct-4bit',
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": f"Say this is a test!"},
    ],
)

Closes #875

Also added a couple fixes for RecurrentGemma 9B

@awni awni requested a review from angeloskath July 7, 2024 22:53
@awni awni mentioned this pull request Jul 8, 2024
Copy link
Copy Markdown
Member

@angeloskath angeloskath left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@awni awni merged commit 68e88d4 into main Jul 8, 2024
@awni awni deleted the fix_openai branch July 8, 2024 19:34
junpeiz pushed a commit to junpeiz/mlx-examples that referenced this pull request Jul 28, 2025
…-explore#877)

* Implementation of mlx.random.multivariate_normal (ml-explore#502)

* Update python/src/random.cpp

Co-authored-by: Awni Hannun <awni.hannun@gmail.com>

* Update python/src/random.cpp

Co-authored-by: Awni Hannun <awni.hannun@gmail.com>

* Update python/src/random.cpp

Co-authored-by: Awni Hannun <awni.hannun@gmail.com>

* Updated typo in docstring

* Restricted multivariate_normal to  float32

* Generic mean and variance shapes

* Review edits

* Update mlx/random.cpp

Co-authored-by: Awni Hannun <awni.hannun@gmail.com>

* Update python/src/random.cpp

Co-authored-by: Awni Hannun <awni.hannun@gmail.com>

* Update python/src/random.cpp

Co-authored-by: Awni Hannun <awni.hannun@gmail.com>

* Update python/src/random.cpp

Co-authored-by: Awni Hannun <awni.hannun@gmail.com>

* Test for ndim of mean and cov

* nits

* smaller size for test

* fix broadcasted sampling

---------

Co-authored-by: Awni Hannun <awni.hannun@gmail.com>
Co-authored-by: Awni Hannun <awni@apple.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants