jamescallander commited on
Commit
8a70da4
·
verified ·
1 Parent(s): 014936e

Update README.md

Browse files

Added UI compatibility

Files changed (1) hide show
  1. README.md +26 -0
README.md CHANGED
@@ -125,6 +125,32 @@ Example response:
125
  {"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"content":"A static method belongs to the class itself rather than any instance of the class and can be called without creating an object of the class.","role":"assistant"}}],"created":null,"id":"rkllm_chat","object":"rkllm_chat","usage":{"completion_tokens":null,"prompt_tokens":null,"total_tokens":null}}
126
  ```
127
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
128
  # License
129
 
130
  This conversion follows the license of the source model: [LICENSE · Qwen/Qwen2.5-Coder-3B-Instruct at main](https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct/blob/main/LICENSE)
 
125
  {"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"content":"A static method belongs to the class itself rather than any instance of the class and can be called without creating an object of the class.","role":"assistant"}}],"created":null,"id":"rkllm_chat","object":"rkllm_chat","usage":{"completion_tokens":null,"prompt_tokens":null,"total_tokens":null}}
126
  ```
127
 
128
+ ### 4) UI compatibility
129
+
130
+ This server exposes an **OpenAI-compatible Chat Completions API**.
131
+
132
+ You can connect it to any OpenAI-compatible client or UI (for example: [Open WebUI](https://github.com/open-webui/open-webui?utm_source=chatgpt.com)
133
+
134
+ Configure your client with the API base:
135
+
136
+ - `http://<SERVER_IP_ADDRESS>:8080`
137
+
138
+
139
+ and use the endpoint:
140
+
141
+ - `/rkllm_chat`
142
+
143
+
144
+ Make sure the `model` field matches the converted model’s name, for example:
145
+
146
+ ```json
147
+ {
148
+ "model": "llama-3.1-8b-instruct",
149
+ "messages": [{"role":"user","content":"Hello!"}],
150
+ "stream": false
151
+ }
152
+ ```
153
+
154
  # License
155
 
156
  This conversion follows the license of the source model: [LICENSE · Qwen/Qwen2.5-Coder-3B-Instruct at main](https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct/blob/main/LICENSE)