Upload folder using huggingface_hub

README.md

@@ -280,6 +280,34 @@ print(f'User: {question}')
 print(f'Assistant: {response}')
 ```
 
+## Deployment
+
+### LMDeploy
+
+LMDeploy is a toolkit for compressing, deploying, and serving LLM, developed by the MMRazor and MMDeploy teams.
+
+```sh
+pip install lmdeploy
+```
+
+You can run batch inference locally with the following python code:
+
+> This model is not yet supported by LMDeploy.
+
+```python
+from lmdeploy.vl import load_image
+from lmdeploy import ChatTemplateConfig, pipeline
+
+model = 'OpenGVLab/InternVL2-4B'
+system_prompt = '我是书生·万象，英文名是InternVL，是由上海人工智能实验室及多家合作单位联合开发的多模态基础模型。人工智能实验室致力于原始技术创新，开源开放，共享共创，推动科技进步和产业发展。'
+image = load_image('https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/tests/data/tiger.jpeg')
+chat_template_config = ChatTemplateConfig('phi-3')
+chat_template_config.meta_instruction = system_prompt
+pipe = pipeline(model, chat_template_config=chat_template_config)
+response = pipe(('describe this image', image))
+print(response)
+```
+
 ## License
 
 This project is released under the MIT license.