ClientAuthenticationError when following instructions on provided docs

[mtcogco]

• azure.ai.inference, azure.core.credentials:
• 1.0.0b7, 1.32.0:
• Windows 10:
• 3.12:

Describe the bug
When I specify an endpoint and model, as per the docs here, and send a request as below, I am getting a ClientAuthenticationError.

The following code:

MODEL_NAME = 'Llama-3.3-70B-Instruct'
client = ChatCompletionsClient(
    endpoint = ENDPOINT,
    credential = AzureKeyCredential(API_KEY)
)

response = client.complete(
    messages=[
        SystemMessage(content="You are a helpful assistant."),
        UserMessage(content="Explain Riemann's conjecture in 1 paragraph"),
    ],
    model = "Meta-Llama-3.1-405B-Instruct"
)

Returns the following error:

ClientAuthenticationError: (None) Unauthorized. Access token is missing, invalid, audience is incorrect (https://cognitiveservices.azure.com), or have expired.
Code: None
Message: Unauthorized. Access token is missing, invalid, audience is incorrect (https://cognitiveservices.azure.com), or have expired.

To Reproduce
Steps to reproduce the behavior:

1. Deploy a Llama 3.1 or 3.3 model to an Azure AI Services serverless endpoint
2. Instantiate up a ChatCompletionsClient object as above
3. Send a request using client.complete() as above

Expected behavior
I would like to just send and receive completions for simple requests/prompts.

I have previously had no issues sending requests to serverless Llama 3.1 models using the previous system, using both the ai.inferences package as well as raw HTTPS requests.

Screenshots
My endpoints page looks like this:

Additional context

• I specify the model name using the arg model when calling client.complete(). For example, I have tried to test with Llama-3.3-70B-Instruct as well as Meta-Llama-3.1-405B-Instruct. This was based on the docs
• I've double checked for silly mistakes I can think of, e.g. incorrect endpoints (I have tried both with and without the /models appended to the end of the Azure AI model inference endpoint.
• The docs also indicate that my API key should be 32 characters, but I can confirm that the API key provided by the Azure AI model inference endpoint is longer than this.
• I've double checked for silly mistakes I can think of, e.g. incorrect endpoints (I have tried both with and without the /models appended to the end of the Azure AI model inference endpoint.
• I believe that I meet the version requirement for the azure-ai-inference package (1.0.0b5 per the docs). I can also confirm the following versions for these azure packages are installed in my venv:

Package	Version
azure-ai-inference	1.0.0b7
azure-ai-ml	1.24.0
azure-common	1.1.28
azure-core	1.32.0
azure-core-tracing-opentelemetry	1.0.0b11
azure-identity	1.19.0
azure-mgmt-core	1.5.0
azure-monitor-opentelemetry	1.6.4
azure-monitor-opentelemetry-exporter	1.0.0b33
azure-storage-blob	12.24.0
azure-storage-file-datalake	12.18.0
azure-storage-file-share	12.20.0

Apologies for inappropriately hijacking another issue earlier, I incorrectly thought it may have been relevant, until I realised the culprit of my 404 was the lack of /models in my URI.

Thanks for considering this issue, @dargilco.

[dargilco]

Hi @mtcogco thank you for opening this separate issue!

Indeed, the api-key string length was recently increased. I will update the azure-ai-infernce package README.md and sample code to remove mentioning of the 32-chat length string. Thank you for pointing that out. Please don't hesitate to point out any other documentation and sample issues.
 
The value of model= that you pass to the ChatCompletionsClient "complete" call is what you find in the first column (titled "Name") in the "Models + endpoints" tab in your AI Foundry project page.

The endpoint URL provided to the constructor of ChatCompletions client should look something like https://<your-ai-services-resouce-name>.services.ai.azure.com/models. When a chat completion call is made, the SDK takes this URL, adds the route /chat/completions and the query api-version=2024-05-01-preview, to get the final URL (for my example): https://<your-ai-services-resouce-name>.services.ai.azure.com/models/chat/completions?api-version=2024-05-01-preview

It seems like you already tried different combinations to account for the above. Just wanted to clarify what values are needed.

Can you please take logs and share with me here? They will be redacted (no api-key will be shown)

Add the following at the top of your sample, run your sample and share the console spew. Thanks!

import sys
import logging
logger = logging.getLogger("azure")
logger.setLevel(logging.DEBUG)
logger.addHandler(logging.StreamHandler(stream=sys.stdout))