README.md

---
library_name: transformers
tags:
- behavior-tree
- swarm-robotics
- xml
- lora
- self-instruct
- control-systems
- transformers
- text-generation
datasets:
- Inventors-Hub/SwarmChat-BehaviorTree-Dataset
language:
- en
pipeline_tag: text-generation
quantized_version: Inventors-Hub/Falcon3-10B-Instruct-BehaviorTree-3-epochs-GGUF
license: apache-2.0
---

# Falcon3-10B-Instruct-BehaviorTree



A small, efficient LoRA-fine-tuned variant of **Falcon3-10B-Instruct** that generates XML behavior trees for swarm-robotics control. Trained as part of the **SwarmChat** project (made possible by UTTER).

## Model Overview

- **Purpose**  
  Convert a natural-language command into a syntactically valid XML behavior tree, using only a fixed set of safe action and condition nodes.

- **Base model**  
  `tiiuae/Falcon3-10B-Instruct`

- **Fine-tuning method**  
  LoRA adapters (rank=16, α=16) on projection modules, 4-bit quantization via bitsandbytes.

- **Quantized model variant**  
  - `Inventors-Hub/Falcon3-10B-Instruct-BehaviorTree-3-epochs-GGUF`
  - ***F16 and Q4_K_M GGUF-quant available [here](https://huggingface.co/Inventors-Hub/Falcon3-10B-Instruct-BehaviorTree-3-epochs-GGUF)***


## Data

- **Training dataset**  
  - `Inventors-Hub/SwarmChat-BehaviorTree-Dataset` (2 063 synthetic Self-Instruct examples generated with OpenAI’s **o1-mini**).  

    
## Training Details

- **Epochs**: 3  
- **Batch**: 2 per device, gradient accumulation 4  
- **Warmup**: 10% of total steps  
- **Learning rate**: 1 × 10⁻⁴  
- **Optimizer**: `adamw_8bit`  
- **Precision**: BF16 when available, otherwise FP16  
- **Eval**: end of each epoch + every 100 steps  
- **Hardware**: single GPU (e.g. A100)  

## Usage

```python
from transformers import pipeline

model_id = "Inventors-Hub/Falcon3-10B-Instruct-BehaviorTree-3-epochs"
generator = pipeline(
    "text-generation",
    model=model_id,
    tokenizer=model_id,
    trust_remote_code=True,
)

prompt = """
SYSTEM:
<<SYS>>You are a helpful, respectful, and honest AI assistant. Your task is to generate well-structured XML code for behavior trees based on the provided instructions.<</SYS>>

INSTRUCTIONS:
It is CRITICAL to use only the following behaviors structured as a dictionary:
{
say: Action Node: Speak the provided message using text-to-speech if it hasn't been spoken before. Args: message (str): The message to be spoken. Returns: Always returns SUCCESS, indicating the action was executed.
flocking: Action Node: Adjust the agent's move vector by blending alignment and separation forces from nearby agents. Returns: Always returns SUCCESS, indicating the action was executed.
align_with_swarm: Action Node: Align the agent's move vector with the average movement of nearby agents. Returns: Always returns SUCCESS, indicating the action was executed.
is_obstacle_detected: Condition node: Determine if any obstacles are detected in the vicinity of the agent. Returns: SUCCESS if an obstacle is detected, FAILURE otherwise.
form_line: Action node: Direct the agent to form a line towards the center of the window. This function adjuststhe agent's position to align it with the center. Returns: Always returns SUCCESS,
}
to construct behavior tree in XML format to the following command, including in the behaviour tree a behaviour that is not in the provided dictionary can result in damage to the agents, and potentially humans, therefore you are not allowed to do so, AVOID AT ALL COSTS.


USER COMMAND:
generate behavior tree to "form a line". Take a step back and think deeply about the behavior you need for this command. Take another step back and think of the xml structure and the behavior you used.
The output MUST follow this XML structure exactly, including:
- A root element with <root BTCPP_format and main_tree_to_execute attributes.
- A <BehaviorTree> element with an inner structure of Sequences, Fallback, Conditions, and Actions.
- A <TreeNodesModel> section listing all node models.
- No additional text or commentary outside the XML.
Output only the XML behavior tree without extra text.

OUTPUT:
"""
result = generator(prompt, max_new_tokens=512, do_sample=False)
print(result[0]["generated_text"])