Add evaluation results from Github README
Browse filesThis PR adds the evaluation results on APEval, EvalPlus, CanItEdit and OctoPack from the Github README to the model card, making it easier for users to understand the performance of the model.
README.md
CHANGED
@@ -1,11 +1,11 @@
|
|
1 |
---
|
2 |
-
tags:
|
3 |
-
- code
|
4 |
base_model:
|
5 |
- TechxGenus/CursorCore-QW2.5-1.5B
|
6 |
library_name: transformers
|
7 |
-
pipeline_tag: text-generation
|
8 |
license: apache-2.0
|
|
|
|
|
|
|
9 |
---
|
10 |
|
11 |
# CursorCore: Assist Programming through Aligning Anything
|
@@ -48,6 +48,16 @@ CursorCore is a series of open-source models designed for AI-assisted programmin
|
|
48 |
|
49 |
Our models have been open-sourced on Hugging Face. You can access our models here: [CursorCore-Series](https://huggingface.co/collections/TechxGenus/cursorcore-series-6706618c38598468866b60e2"). We also provide pre-quantized weights for GPTQ and AWQ here: [CursorCore-Quantization](https://huggingface.co/collections/TechxGenus/cursorcore-quantization-67066431f29f252494ee8cf3)
|
50 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
51 |
## Usage
|
52 |
|
53 |
Here are some examples of how to use our model:
|
@@ -114,13 +124,27 @@ sample = {
|
|
114 |
{
|
115 |
"type": "code",
|
116 |
"lang": "python",
|
117 |
-
"code": """def quick_sort(arr)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
118 |
}
|
119 |
],
|
120 |
"current": {
|
121 |
"type": "code",
|
122 |
"lang": "python",
|
123 |
-
"code": """def quick_sort(array)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
124 |
},
|
125 |
"user": ""
|
126 |
}
|
@@ -202,7 +226,14 @@ sample = {
|
|
202 |
"current": {
|
203 |
"type": "code",
|
204 |
"lang": "python",
|
205 |
-
"code": """def quick_sort(array)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
206 |
},
|
207 |
"user": "Add Docstring."
|
208 |
}
|
@@ -273,7 +304,14 @@ sample = {
|
|
273 |
"current": {
|
274 |
"type": "code",
|
275 |
"lang": "python",
|
276 |
-
"code": """def quick_sort(array)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
277 |
},
|
278 |
"user": "Add Docstring."
|
279 |
}
|
@@ -342,7 +380,14 @@ sample = {
|
|
342 |
"current": {
|
343 |
"type": "code",
|
344 |
"lang": "python",
|
345 |
-
"code": """def quick_sort(array)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
346 |
},
|
347 |
"user": "Add Docstring."
|
348 |
}
|
@@ -414,4 +459,4 @@ CursorCore is still in a very early stage, and lots of work is needed to achieve
|
|
414 |
|
415 |
## Contribution
|
416 |
|
417 |
-
Contributions are welcome! If you find any bugs or have suggestions for improvements, please open an issue or submit a pull request.
|
|
|
1 |
---
|
|
|
|
|
2 |
base_model:
|
3 |
- TechxGenus/CursorCore-QW2.5-1.5B
|
4 |
library_name: transformers
|
|
|
5 |
license: apache-2.0
|
6 |
+
pipeline_tag: text-generation
|
7 |
+
tags:
|
8 |
+
- code
|
9 |
---
|
10 |
|
11 |
# CursorCore: Assist Programming through Aligning Anything
|
|
|
48 |
|
49 |
Our models have been open-sourced on Hugging Face. You can access our models here: [CursorCore-Series](https://huggingface.co/collections/TechxGenus/cursorcore-series-6706618c38598468866b60e2"). We also provide pre-quantized weights for GPTQ and AWQ here: [CursorCore-Quantization](https://huggingface.co/collections/TechxGenus/cursorcore-quantization-67066431f29f252494ee8cf3)
|
50 |
|
51 |
+
We use the manually written benchmark APEval to assess the model's ability to assist programming. We also utilize [EvalPlus](https://github.com/evalplus/evalplus), [CanItEdit](https://github.com/nuprl/CanItEdit) and [OctoPack](https://github.com/bigcode-project/octopack) to evaluate the model's performance in Python program generation, instructional code editing, and automated program repair. Since we use a custom conversation template, its generation method differs significantly from both instruct models and base models. Please refer to [our paper](http://arxiv.org/abs/2410.07002) for more details.
|
52 |
+
|
53 |
+
Evaluation results on APEval:
|
54 |
+
|
55 |
+
<img src="https://github.com/TechxGenus/CursorCore/blob/main/pictures/APEval.png" alt="APEval" width="75%"/>
|
56 |
+
|
57 |
+
Evaluation results on EvalPlus, CanItEdit and OctoPack:
|
58 |
+
|
59 |
+
<img src="https://github.com/TechxGenus/CursorCore/blob/main/pictures/EvalPlus_CanItEdit_OctoPack.png" alt="EvalPlus_CanItEdit_OctoPack" width="75%">
|
60 |
+
|
61 |
## Usage
|
62 |
|
63 |
Here are some examples of how to use our model:
|
|
|
124 |
{
|
125 |
"type": "code",
|
126 |
"lang": "python",
|
127 |
+
"code": """def quick_sort(arr):
|
128 |
+
if len(arr) <= 1:
|
129 |
+
return arr
|
130 |
+
pivot = arr[len(arr) // 2]
|
131 |
+
left = [x for x in arr if x < pivot]
|
132 |
+
middle = [x for x in arr if x == pivot]
|
133 |
+
right = [x for x in arr if x > pivot]
|
134 |
+
return quick_sort(left) + middle + quick_sort(right)"""
|
135 |
}
|
136 |
],
|
137 |
"current": {
|
138 |
"type": "code",
|
139 |
"lang": "python",
|
140 |
+
"code": """def quick_sort(array):
|
141 |
+
if len(arr) <= 1:
|
142 |
+
return arr
|
143 |
+
pivot = arr[len(arr) // 2]
|
144 |
+
left = [x for x in arr if x < pivot]
|
145 |
+
middle = [x for x in arr if x == pivot]
|
146 |
+
right = [x for x in arr if x > pivot]
|
147 |
+
return quick_sort(left) + middle + quick_sort(right)"""
|
148 |
},
|
149 |
"user": ""
|
150 |
}
|
|
|
226 |
"current": {
|
227 |
"type": "code",
|
228 |
"lang": "python",
|
229 |
+
"code": """def quick_sort(array):
|
230 |
+
if len(arr) <= 1:
|
231 |
+
return arr
|
232 |
+
pivot = arr[len(arr) // 2]
|
233 |
+
left = [x for x in arr if x < pivot]
|
234 |
+
middle = [x for x in arr if x == pivot]
|
235 |
+
right = [x for x in arr if x > pivot]
|
236 |
+
return quick_sort(left) + middle + quick_sort(right)"""
|
237 |
},
|
238 |
"user": "Add Docstring."
|
239 |
}
|
|
|
304 |
"current": {
|
305 |
"type": "code",
|
306 |
"lang": "python",
|
307 |
+
"code": """def quick_sort(array):
|
308 |
+
if len(arr) <= 1:
|
309 |
+
return arr
|
310 |
+
pivot = arr[len(arr) // 2]
|
311 |
+
left = [x for x in arr if x < pivot]
|
312 |
+
middle = [x for x in arr if x == pivot]
|
313 |
+
right = [x for x in arr if x > pivot]
|
314 |
+
return quick_sort(left) + middle + quick_sort(right)"""
|
315 |
},
|
316 |
"user": "Add Docstring."
|
317 |
}
|
|
|
380 |
"current": {
|
381 |
"type": "code",
|
382 |
"lang": "python",
|
383 |
+
"code": """def quick_sort(array):
|
384 |
+
if len(arr) <= 1:
|
385 |
+
return arr
|
386 |
+
pivot = arr[len(arr) // 2]
|
387 |
+
left = [x for x in arr if x < pivot]
|
388 |
+
middle = [x for x in arr if x == pivot]
|
389 |
+
right = [x for x in arr if x > pivot]
|
390 |
+
return quick_sort(left) + middle + quick_sort(right)"""
|
391 |
},
|
392 |
"user": "Add Docstring."
|
393 |
}
|
|
|
459 |
|
460 |
## Contribution
|
461 |
|
462 |
+
Contributions are welcome! If you find any bugs or have suggestions for improvements, please open an issue or submit a pull request.
|