jamescallander commited on
Commit
2e3ac9a
·
verified ·
1 Parent(s): 457c0d0

Update README.md

Browse files

Added UI compatibility

Files changed (1) hide show
  1. README.md +16 -0
README.md CHANGED
@@ -114,6 +114,22 @@ Example response:
114
  {"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"content":"Napoleon Bonaparte was a French military leader and statesman who rose to prominence during the French Revolution. He played a pivotal role in shaping modern Europe through his military campaigns, administrative reforms, and the establishment of new political institutions.","role":"assistant"}}],"created":null,"id":"rkllm_chat","object":"rkllm_chat","usage":{"completion_tokens":null,"prompt_tokens":null,"total_tokens":null}}
115
  ```
116
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
117
  # License
118
 
119
  This conversion follows the license of the source model: [apache-2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md)
 
114
  {"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"content":"Napoleon Bonaparte was a French military leader and statesman who rose to prominence during the French Revolution. He played a pivotal role in shaping modern Europe through his military campaigns, administrative reforms, and the establishment of new political institutions.","role":"assistant"}}],"created":null,"id":"rkllm_chat","object":"rkllm_chat","usage":{"completion_tokens":null,"prompt_tokens":null,"total_tokens":null}}
115
  ```
116
 
117
+ ### 4) UI compatibility
118
+
119
+ This server exposes an **OpenAI-compatible Chat Completions API**.
120
+
121
+ You can connect it to any OpenAI-compatible client or UI (for example: [Open WebUI](https://github.com/open-webui/open-webui?utm_source=chatgpt.com))
122
+ - Configure your client with the API base: `http://<SERVER_IP_ADDRESS>:8080` and use the endpoint: `/rkllm_chat`
123
+ - Make sure the `model` field matches the converted model’s name, for example:
124
+
125
+ ```json
126
+ {
127
+ "model": "llama-3.1-8b-instruct",
128
+ "messages": [{"role":"user","content":"Hello!"}],
129
+ "stream": false
130
+ }
131
+ ```
132
+
133
  # License
134
 
135
  This conversion follows the license of the source model: [apache-2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md)