Create README.md

README.md

@@ -0,0 +1,351 @@
+---
+license: apache-2.0
+base_model:
+- microsoft/NextCoder-14B
+- Qwen/Qwen2.5-Coder-14B-Instruct
+language:
+- en
+pipeline_tag: text-generation
+tags:
+- merge
+- programming
+- code generation
+- code
+- codeqwen
+- coding
+- coder
+- qwen2
+- chat
+- qwen
+- qwen-coder
+- code
+- chat
+- microsoft
+- nextcoder
+- selekt
+datasets:
+- microsoft/NextCoderDataset
+- microsoft/NextCoderDataset-Conversational
+- bigcode/commitpackft
+- bigcode/starcoderdata
+library_name: transformers
+---
+
+<h2>Qwen2.5-Microsoft-NextCoder-Instruct-FUSED-CODER-Fast-22B</h2>
+
+This repo contains the full precision source code, in "safe tensors" format to generate GGUFs, GPTQ, EXL2, AWQ, HQQ and other formats.
+The source code can also be used directly.
+
+This model contains Qwen 14b Coder Instruct FUSED with Microsoft's 14B Coder (instruct model) creating an 22B, 73 layers, 879 tensors model.
+
+https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct
+
+https://huggingface.co/microsoft/NextCoder-14B
+
+Information on the models below, and then a complete help section for running LLM / AI models.
+
+The FUSING process enhances model performance and the model has minimal to no "reasoning" blocks.
+
+Ask the model for code, and you get code asap.
+
+Source is in float32 precision to preserve Microsoft Next Coder's 32 bit source.
+
+This model requires:
+- Jinja (embedded) or CHATML template
+- Max context of 32k expanded as per Qwen2.5 methods.
+
+Settings used for testing (suggested):
+- Temp .3 to .7
+- Rep pen 1.05 to 1.1
+- Topp .8 , minp .05
+- Topk 20
+- No system prompt.
+
+This model will respond well to both detailed instructions and step by step refinement and additions to code.
+
+As this is an instruct model, it will also benefit from a detailed system prompt too.
+
+For simpler coding problems, lower quants will work well; but for complex/multi-step problem solving suggest Q6 or Q8.
+
+---
+
+# NextCoder-14B
+
+---
+
+<p align="center">
+        <a href="https://github.com/microsoft/NextCoder">GitHub</a>&nbsp&nbsp | &nbsp&nbsp <a href="https://www.microsoft.com/en-us/research/publication/nextcoder-robust-adaptation-of-code-lms-to-diverse-code-edits/">Paper</a> 
+</p>
+
+> NextCoder: Robust Adaptation of Code LMs to Diverse Code Edits (ICML'2025)
+
+## Introduction
+
+NextCoder is the latest series of Code-Editing large language models developed using the Qwen2.5-Coder Instruct variants as base and trained with novel Selective Knowledge Transfer finetuning methodology as introduced in the paper. NextCoder family model comes in 3 different sizes 7, 14, 32 billion parameters, to meet the needs of different developers.
+Following are the key improvements:
+- Significantly improvements in **code editing**, NextCoder-32B has performing on par with GPT-4o on complex benchmarks like Aider-Polyglot with performance increment of 44% from their base model.
+- No loss of generalizibility, due to our new finetuning method **SeleKT**
+- **Long-context Support** up to 32K tokens.
+
+**This repo contains the NextCoder 14B model**, which has the following features:
+- Type: Causal Language Models
+- Training Stage: Post-training with SeleKT
+- Architecture: transformers with RoPE, SwiGLU, RMSNorm, and Attention QKV bias
+- Number of Parameters: 14.7B
+- Number of Paramaters (Non-Embedding): 13.1B
+- Number of Layers: 48
+- Number of Attention Heads (GQA): 40 for Q and 8 for KV
+  
+For more details, please refer to our [blog](), [GitHub](https://github.com/microsoft/NextCoder), [Paper](https://www.microsoft.com/en-us/research/publication/nextcoder-robust-adaptation-of-code-lms-to-diverse-code-edits/).
+
+## Requirements
+
+The code of NextCoder is based on Qwen2.5 base models which has been in the latest Hugging face `transformers` and we advise you to use the latest version of `transformers`.
+
+With `transformers<4.37.0`, you will encounter the following error:
+```
+KeyError: 'qwen2'
+```
+
+## Quickstart
+
+Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents.
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "microsoft/NextCoder-14B"
+
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype="auto",
+    device_map="auto",
+)
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+
+prompt = """
+Fix the following function that divides two numbers to handle all the edge cases:
+
+def divide(a, b)
+  returm a/b
+"""
+messages = [
+    {"role": "user", "content": prompt}
+]
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
+model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
+
+generated_ids = model.generate(
+    **model_inputs,
+    max_new_tokens=1024
+)
+generated_ids = [
+    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
+]
+
+response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
+```
+## Evaluation and Performance
+
+| Models | HUMANEVALFIX | CANITEDIT | AIDER | POLYGLOT |
+|--------|---------------|-----------|-------|----------|
+| QwenCoder-2.5-3B | 73.2 | 37.1 | 36.8 | - |
+| QwenCoder-2.5-3B-LoRA | 64.6 | 36.2 | 35.8 | - |
+| QwenCoder-2.5-3B-SFT | 76.2 | 32.4 | 30.1 | - |
+| **NextCoder-3B** | 75.6 | 42.4 | 37.6 | - |
+| QwenCoder-2.5-7B | 73.8 | 48.1 | 59.4 | - |
+| QwenCoder-2.5-7B-LoRA | 70.7 | 44.3 | 40.6 | - |
+| QwenCoder-2.5-7B-SFT | 70.1 | 36.7 | 48.9 | - |
+| **NextCoder-7B** | 81.1 | 50.5 | 65.7 | - |
+| QwenCoder-2.5-14B | 87.8 | 58.1 | 66.9 | 9.3 |
+| QwenCoder-2.5-14B-LoRA | 78.0 | 50.9 | 66.2 | 5.3 |
+| QwenCoder-2.5-14B-SFT | 79.9 | 42.4 | 36.8 | 3.1 |
+| **NextCoder-14B** | 89.8 | 60.2 | 72.2 | 12.2 |
+| QwenCoder-2.5-32B | **90.2** | 61.0 | 72.9 | 16.4 |
+| QwenCoder-2.5-32B-LoRA | 82.3 | 52.4 | 60.2 | 6.7 |
+| QwenCoder-2.5-32B-SFT | 81.7 | 49.5 | 66.9 | 8.4 |
+| **NextCoder-32B** | 88.9 | **62.4** | **74.7** | **23.6** |
+
+*Comparison of base QwenCoder-2.5 models of different sizes and their SELEKT-enhanced versions across three code editing benchmarks.*
+
+**Detailed evaluation results are reported in this [📑 paper](https://www.microsoft.com/en-us/research/publication/nextcoder-robust-adaptation-of-code-lms-to-diverse-code-edits/).**
+
+## Responsible AI Use
+The base models (from the QwenCoder-2.5 family) are suspectible to malicious prompts and may generate or execute harmful code. Our finetuning does not enhance or impede such behaviors. The users should use the models and their outputs responsibly and with caution. Model outputs should be subjected to additional analysis, including manual inspection, and sandboxing before execution.
+
+See more here:
+
+https://huggingface.co/microsoft/NextCoder-14B
+
+---
+
+# Qwen2.5-Coder-14B-Instruct
+
+---
+
+# Qwen2.5-Coder-14B-Instruct
+<a href="https://chat.qwenlm.ai/" target="_blank" style="margin: 2px;">
+    <img alt="Chat" src="https://img.shields.io/badge/%F0%9F%92%9C%EF%B8%8F%20Qwen%20Chat%20-536af5" style="display: inline-block; vertical-align: middle;"/>
+</a>
+
+## Introduction
+
+Qwen2.5-Coder is the latest series of Code-Specific Qwen large language models (formerly known as CodeQwen). As of now, Qwen2.5-Coder has covered six mainstream model sizes, 0.5, 1.5, 3, 7, 14, 32 billion parameters, to meet the needs of different developers. Qwen2.5-Coder brings the following improvements upon CodeQwen1.5:
+
+- Significantly improvements in **code generation**, **code reasoning** and **code fixing**. Base on the strong Qwen2.5, we scale up the training tokens into 5.5 trillion including source code, text-code grounding, Synthetic data, etc. Qwen2.5-Coder-32B has become the current state-of-the-art open-source codeLLM, with its coding abilities matching those of GPT-4o.
+- A more comprehensive foundation for real-world applications such as **Code Agents**. Not only enhancing coding capabilities but also maintaining its strengths in mathematics and general competencies.
+- **Long-context Support** up to 128K tokens.
+
+**This repo contains the instruction-tuned 14B Qwen2.5-Coder model**, which has the following features:
+- Type: Causal Language Models
+- Training Stage: Pretraining & Post-training
+- Architecture: transformers with RoPE, SwiGLU, RMSNorm, and Attention QKV bias
+- Number of Parameters: 14.7B
+- Number of Paramaters (Non-Embedding): 13.1B
+- Number of Layers: 48
+- Number of Attention Heads (GQA): 40 for Q and 8 for KV
+- Context Length: Full 131,072 tokens
+  - Please refer to [this section](#processing-long-texts) for detailed instructions on how to deploy Qwen2.5 for handling long texts.
+  
+For more details, please refer to our [blog](https://qwenlm.github.io/blog/qwen2.5-coder-family/), [GitHub](https://github.com/QwenLM/Qwen2.5-Coder), [Documentation](https://qwen.readthedocs.io/en/latest/), [Arxiv](https://arxiv.org/abs/2409.12186).
+
+## Requirements
+
+The code of Qwen2.5-Coder has been in the latest Hugging face `transformers` and we advise you to use the latest version of `transformers`.
+
+With `transformers<4.37.0`, you will encounter the following error:
+```
+KeyError: 'qwen2'
+```
+
+## Quickstart
+
+Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents.
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "Qwen/Qwen2.5-Coder-14B-Instruct"
+
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype="auto",
+    device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+
+prompt = "write a quick sort algorithm."
+messages = [
+    {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."},
+    {"role": "user", "content": prompt}
+]
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
+model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
+
+generated_ids = model.generate(
+    **model_inputs,
+    max_new_tokens=512
+)
+generated_ids = [
+    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
+]
+
+response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
+```
+
+### Processing Long Texts
+
+The current `config.json` is set for context length up to 32,768 tokens.
+To handle extensive inputs exceeding 32,768 tokens, we utilize [YaRN](https://arxiv.org/abs/2309.00071), a technique for enhancing model length extrapolation, ensuring optimal performance on lengthy texts.
+
+For supported frameworks, you could add the following to `config.json` to enable YaRN:
+```json
+{
+  ...,
+  "rope_scaling": {
+    "factor": 4.0,
+    "original_max_position_embeddings": 32768,
+    "type": "yarn"
+  }
+}
+```
+
+For deployment, we recommend using vLLM. 
+Please refer to our [Documentation](https://qwen.readthedocs.io/en/latest/deployment/vllm.html) for usage if you are not familar with vLLM.
+Presently, vLLM only supports static YARN, which means the scaling factor remains constant regardless of input length, **potentially impacting performance on shorter texts**. 
+We advise adding the `rope_scaling` configuration only when processing long contexts is required.
+
+## Evaluation & Performance
+
+Detailed evaluation results are reported in this [📑 blog](https://qwenlm.github.io/blog/qwen2.5-coder-family/).
+
+For requirements on GPU memory and the respective throughput, see results [here](https://qwen.readthedocs.io/en/latest/benchmark/speed_benchmark.html).
+
+See also:
+
+https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct
+
+---
+
+For more information / other Qwen/Mistral Coders / additional settings see:
+
+[ https://huggingface.co/DavidAU/Qwen2.5-MOE-2x-4x-6x-8x__7B__Power-CODER__19B-30B-42B-53B-gguf ]
+
+---
+
+<H2>Help, Adjustments, Samplers, Parameters and More</H2>
+
+---
+
+<B>CHANGE THE NUMBER OF ACTIVE EXPERTS:</B>
+
+See this document:
+
+https://huggingface.co/DavidAU/How-To-Set-and-Manage-MOE-Mix-of-Experts-Model-Activation-of-Experts
+
+<B>Settings: CHAT / ROLEPLAY and/or SMOOTHER operation of this model:</B>
+
+In "KoboldCpp" or  "oobabooga/text-generation-webui" or "Silly Tavern" ;
+
+Set the "Smoothing_factor" to 1.5 
+
+: in KoboldCpp -> Settings->Samplers->Advanced-> "Smooth_F"
+
+: in text-generation-webui -> parameters -> lower right.
+
+: In Silly Tavern this is called: "Smoothing"
+
+
+NOTE: For "text-generation-webui" 
+
+-> if using GGUFs you need to use "llama_HF" (which involves downloading some config files from the SOURCE version of this model)
+
+Source versions (and config files) of my models are here:
+
+https://huggingface.co/collections/DavidAU/d-au-source-files-for-gguf-exl2-awq-gptq-hqq-etc-etc-66b55cb8ba25f914cbf210be
+
+OTHER OPTIONS:
+
+- Increase rep pen to 1.1 to 1.15 (you don't need to do this if you use "smoothing_factor")
+
+- If the interface/program you are using to run AI MODELS supports "Quadratic Sampling" ("smoothing") just make the adjustment as noted.
+
+<B>Highest Quality Settings / Optimal Operation Guide / Parameters and Samplers</B>
+
+This a "Class 1" model:
+
+For all settings used for this model (including specifics for its "class"), including example generation(s) and for advanced settings guide (which many times addresses any model issue(s)), including methods to improve model performance for all use case(s) as well as chat, roleplay and other use case(s) please see:
+
+[ https://huggingface.co/DavidAU/Maximizing-Model-Performance-All-Quants-Types-And-Full-Precision-by-Samplers_Parameters ]
+
+You can see all parameters used for generation, in addition to advanced parameters and samplers to get the most out of this model here:
+
+[ https://huggingface.co/DavidAU/Maximizing-Model-Performance-All-Quants-Types-And-Full-Precision-by-Samplers_Parameters ]