Aznaur/tbench-qwen-sft-fix-git-overfit-v7-nat-fixed

T-Bench Qwen SFT Fix-Git Overfit v7 NAT Fixed

Model Description

This model is fine-tuned from Qwen3-8B using fixed Negative-Aware Training (NAT) specifically for the fix-git task.

Training Details

• Base Model: Qwen/Qwen3-8B
• Training Method: Fixed Negative-Aware Training (NAT)
• Task: fix-git only
• Epochs: 300
• Learning Rate: 1e-4
• Batch Size: 2

Dataset Composition

• Total samples: 10 per epoch
• Positive examples: 4 (Claude's successful trajectory)
• Negative examples: 6 (2 per negative type)

NAT Strategy

Negative examples teach universal anti-patterns:

1. Hallucinated arguments: Adding message_title, message_description
2. Looping behavior: Repeating commands after task completion
3. Wrong command format: Using id instead of actual command

Fixes Applied

• Fixed negative example generation
• Improved prompt formatting
• Enhanced training stability

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Aznaur/tbench-qwen-sft-fix-git-overfit-v7-nat-fixed")
tokenizer = AutoTokenizer.from_pretrained("Aznaur/tbench-qwen-sft-fix-git-overfit-v7-nat-fixed")

Performance

Trained for 300 epochs with fixed NAT to improve tool usage and avoid common failure patterns.

Paper Reference

Based on "Learning From Failure: Integrating Negative Examples when Fine-tuning Large Language Models as Agents" (arXiv 2402.11651)

Model Checkpoint

• Epoch: 299
• Global Step: 1199
• Training completed successfully