Thanks for the suggestions! I’ve tried implementing the recommended fixes, and here’s what I found:
What I Tried
Following the previous suggestions, I modified my request by adding HTTPS protocol:
curl -L -X POST https://johnaic.pplus.ai/api/chat/completions \
-H "Authorization: Bearer MY-API-KEY" \
-H "Content-Type: application/json" \
-d '{"model": "llama3.1", "messages": [{"role": "user," "content": "Why is the sky blue?"}]}'
Current Status
The redirect issue is resolved, but I’m now getting a new error:
{"detail":"'llama3.1'"}
I have tried to edit the model names to verify it’s not the model name causing the issue. Following are the model names I tried:
"llama3.1-8b", "llama3.1", "llama3.1:8B"
Also, please enable the swagger documentation. However, the best direction is to find implementation via the OpenAI API compatibility since this approach seems integrative.
Please try this out. I think open-webui project removed the openai compatible API based links in their documentation. But I can still find OpenAI compatible API here: https://johnaic.pplus.ai/openai/docs
It just worked for me. I used the right bearer token in Authorization in the dict above. Here’s what I get printed
{'id': 'chat-4cc865431e6b4f8799a5b4a26dc9d6ac', 'object': 'chat.completion', 'created': 1736506464, 'model': 'neuralmagic/Meta-Llama-3.1-8B-Instruct-FP8', 'choices': [{'index': 0, 'message': {'role': 'assistant', 'content': "The sky appears blue because of a phenomenon called scattering, which is the way light interacts with the tiny molecules of gases in the atmosphere. Here's a simplified explanation:\n\n1. **Sunlight**: When the sun shines, it emits a wide range of electromagnetic radiation, including visible light, ultraviolet (UV) radiation, and infrared (IR) radiation.\n\n2. **Light scattering**: As sunlight enters the Earth's atmosphere, it encounters tiny molecules of gases such as nitrogen (N2) and oxygen (O2). These molecules are much smaller than the wavelength of light, so they scatter the light in all directions.\n\n3. **Blue light scattering**: The smaller molecules scatter the shorter (blue) wavelengths of light more than the longer (red) wavelengths. This is known as Rayleigh scattering, named after the British physicist Lord Rayleigh, who first described the effect in the late 19th century.\n\n4. **Our eyes see blue**: When we look at the sky, our eyes see the scattered blue light more than any other color. This is why the sky appears blue during the daytime, especially in the direction of the sun.\n\n5. **Other factors**: Other factors can affect the color of the sky, such as:\n\n* **Dust and water vapor**: Tiny particles in the atmosphere can scatter light in different ways, making the sky appear more hazy or milky.\n* **Sun's position**: The angle of the sun affects the amount of light that reaches our eyes, making the sky appear more intense or muted.\n* **Atmospheric conditions**: Weather conditions like clouds, fog, or pollution can alter the color of the sky.\n\nIn summary, the sky appears blue because of the scattering of sunlight by tiny molecules of gases in the atmosphere, which favors the shorter (blue) wavelengths of light.", 'tool_calls': []}, 'logprobs': None, 'finish_reason': 'stop', 'stop_reason': None}], 'usage': {'prompt_tokens': 41, 'total_tokens': 408, 'completion_tokens': 367}, 'prompt_logprobs': None}
which container did you add the ENV variable to? I individually added it to ollama and open-webui. Did not see swagger documentation when i did https://chatllama.von-neumann.ai/docs
We updated the model to deepseek r1 distill 8b. You can access the API by changing the model id to JamAndTeaStudios/DeepSeek-R1-Distill-Llama-8B-FP8-Dynamic.
Here’s an example code in bash
# get the API key from johnaic.pplus.ai
MY_API_KEY=sk-xxxx
curl -X 'POST' \
'https://johnaic.pplus.ai/openai/chat/completions?bypass_filter=false' \
-H 'accept: application/json' \
-H "Authorization: Bearer $MY_API_KEY" \
-H 'Content-Type: application/json' \
-d '{
"model": "JamAndTeaStudios/DeepSeek-R1-Distill-Llama-8B-FP8-Dynamic",
"messages": [
{
"role": "user",
"content": "Why is the sky blue?"
}
]
}'
