Update README.md
Browse filesAdded UI compatibility
README.md
CHANGED
@@ -114,6 +114,22 @@ Example response:
|
|
114 |
{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"content":"Napoleon Bonaparte was a French military leader and statesman who rose to prominence during the French Revolution. He played a pivotal role in shaping modern Europe through his military campaigns, administrative reforms, and the establishment of new political institutions.","role":"assistant"}}],"created":null,"id":"rkllm_chat","object":"rkllm_chat","usage":{"completion_tokens":null,"prompt_tokens":null,"total_tokens":null}}
|
115 |
```
|
116 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
117 |
# License
|
118 |
|
119 |
This conversion follows the license of the source model: [apache-2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md)
|
|
|
114 |
{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"content":"Napoleon Bonaparte was a French military leader and statesman who rose to prominence during the French Revolution. He played a pivotal role in shaping modern Europe through his military campaigns, administrative reforms, and the establishment of new political institutions.","role":"assistant"}}],"created":null,"id":"rkllm_chat","object":"rkllm_chat","usage":{"completion_tokens":null,"prompt_tokens":null,"total_tokens":null}}
|
115 |
```
|
116 |
|
117 |
+
### 4) UI compatibility
|
118 |
+
|
119 |
+
This server exposes an **OpenAI-compatible Chat Completions API**.
|
120 |
+
|
121 |
+
You can connect it to any OpenAI-compatible client or UI (for example: [Open WebUI](https://github.com/open-webui/open-webui?utm_source=chatgpt.com))
|
122 |
+
- Configure your client with the API base: `http://<SERVER_IP_ADDRESS>:8080` and use the endpoint: `/rkllm_chat`
|
123 |
+
- Make sure the `model` field matches the converted model’s name, for example:
|
124 |
+
|
125 |
+
```json
|
126 |
+
{
|
127 |
+
"model": "llama-3.1-8b-instruct",
|
128 |
+
"messages": [{"role":"user","content":"Hello!"}],
|
129 |
+
"stream": false
|
130 |
+
}
|
131 |
+
```
|
132 |
+
|
133 |
# License
|
134 |
|
135 |
This conversion follows the license of the source model: [apache-2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md)
|