Update README.md
Browse filesAdded UI compatibility
README.md
CHANGED
@@ -125,6 +125,32 @@ Example response:
|
|
125 |
{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"content":"A static method belongs to the class itself rather than any instance of the class and can be called without creating an object of the class.","role":"assistant"}}],"created":null,"id":"rkllm_chat","object":"rkllm_chat","usage":{"completion_tokens":null,"prompt_tokens":null,"total_tokens":null}}
|
126 |
```
|
127 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
128 |
# License
|
129 |
|
130 |
This conversion follows the license of the source model: [LICENSE · Qwen/Qwen2.5-Coder-3B-Instruct at main](https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct/blob/main/LICENSE)
|
|
|
125 |
{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"content":"A static method belongs to the class itself rather than any instance of the class and can be called without creating an object of the class.","role":"assistant"}}],"created":null,"id":"rkllm_chat","object":"rkllm_chat","usage":{"completion_tokens":null,"prompt_tokens":null,"total_tokens":null}}
|
126 |
```
|
127 |
|
128 |
+
### 4) UI compatibility
|
129 |
+
|
130 |
+
This server exposes an **OpenAI-compatible Chat Completions API**.
|
131 |
+
|
132 |
+
You can connect it to any OpenAI-compatible client or UI (for example: [Open WebUI](https://github.com/open-webui/open-webui?utm_source=chatgpt.com)
|
133 |
+
|
134 |
+
Configure your client with the API base:
|
135 |
+
|
136 |
+
- `http://<SERVER_IP_ADDRESS>:8080`
|
137 |
+
|
138 |
+
|
139 |
+
and use the endpoint:
|
140 |
+
|
141 |
+
- `/rkllm_chat`
|
142 |
+
|
143 |
+
|
144 |
+
Make sure the `model` field matches the converted model’s name, for example:
|
145 |
+
|
146 |
+
```json
|
147 |
+
{
|
148 |
+
"model": "llama-3.1-8b-instruct",
|
149 |
+
"messages": [{"role":"user","content":"Hello!"}],
|
150 |
+
"stream": false
|
151 |
+
}
|
152 |
+
```
|
153 |
+
|
154 |
# License
|
155 |
|
156 |
This conversion follows the license of the source model: [LICENSE · Qwen/Qwen2.5-Coder-3B-Instruct at main](https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct/blob/main/LICENSE)
|