| # Finetuned FunctionGemma Model: `emredeveloper/functiongemma-tools` |
|
|
| This repository hosts a finetuned version of Google's `functiongemma-270m-it` model, specifically adapted for advanced tool-calling capabilities with integrated reasoning. This model has been enhanced using the `LLM360/TxT360-3efforts` dataset, which focuses on providing detailed thinking processes alongside tool calls. |
|
|
| ## Model Overview |
|
|
| - **Base Model**: `unsloth/functiongemma-270m-it` |
| - **Finetuning Method**: LoRA (Low-Rank Adaptation) using Unsloth library. |
| - **Dataset**: `LLM360/TxT360-3efforts` (agent split, 50,000 examples streamed). |
| - **Key Enhancement**: Improved tool-calling with explicit `<think>...</think>` blocks for internal reasoning, crucial for complex multi-step tasks. |
|
|
| ## Finetuning Details |
|
|
| This model was finetuned in Google Colab using the Unsloth library, which significantly speeds up the finetuning process and reduces VRAM usage. |
|
|
| ### Training Configuration: |
|
|
| - **Max Sequence Length**: 4096 |
| - **LoRA Rank (r)**: 128 |
| - **LoRA Alpha**: 256 |
| - **LoRA Dropout**: 0 |
| - **Gradient Checkpointing**: Enabled with "unsloth" optimization |
| - **Batch Size**: 4 (per device), with gradient accumulation steps = 2 |
| - **Learning Rate**: 2e-4 |
| - **Optimizer**: `adamw_8bit` |
| - **Training Steps**: 100 |
|
|
| Special attention was given to applying a custom chat template that incorporates `<think>` tags for explicit reasoning and aligns with the `functiongemma`'s tool-calling format. The training focused on responses only, masking out the instruction part to enhance the model's generation quality. |
|
|
| ## How to Use |
|
|
| This model can be loaded and used for inference with the Hugging Face `transformers` library, especially when combined with Unsloth for optimized performance. |
|
|
| ### Installation |
|
|
| First, ensure you have the necessary libraries installed: |
|
|
| ```bash |
| pip install transformers unsloth[cuda] torch |
| ``` |
|
|
| ### Loading the Model |
|
|
| ```python |
| from unsloth import FastLanguageModel |
| import torch |
| from transformers import TextStreamer |
| |
| max_seq_length = 4096 |
| |
| model, tokenizer = FastLanguageModel.from_pretrained( |
| model_name = "emredeveloper/functiongemma-tools", # Your finetuned model on Hugging Face Hub |
| max_seq_length = max_seq_length, |
| load_in_4bit = False, |
| load_in_8bit = False, |
| load_in_16bit = True, # Or False if you saved merged_4bit |
| ) |
| ``` |
|
|
| ### Inference Example |
|
|
| To perform inference, you'll need to construct your messages and tools according to the `functiongemma` chat template. Here's an example demonstrating basic tool calling with thinking: |
|
|
| ```python |
| import json |
| |
| tools_example = [ |
| { |
| "type": "function", |
| "function": { |
| "name": "get_amazon_product_details", |
| "description": ( |
| "Retrieves comprehensive product information from Amazon, " |
| "including title, price, description, specifications, and availability." |
| ), |
| "parameters": { |
| "type": "object", |
| "properties": { |
| "asin": { |
| "type": "string", |
| "description": "The Amazon ASIN of the product.", |
| } |
| }, |
| "required": ["asin"], |
| }, |
| }, |
| } |
| ] |
| |
| messages_example = [ |
| { |
| "role": "system", |
| "content": ( |
| "You are a shopping assistant. Use tools when you need detailed " |
| "Amazon product data such as price and specifications." |
| ), |
| }, |
| { |
| "role": "user", |
| "content": "Is the espresso machine with ASIN B0XYZ12345 any good for home use?" |
| }, |
| ] |
| |
| # Apply the chat template for generation |
| text = tokenizer.apply_chat_template( |
| messages_example, |
| tools = tools_example, |
| tokenize = False, |
| add_generation_prompt = True, |
| ).removeprefix('<bos>') # Remove <bos> token if present |
| |
| # Generate a response |
| _ = model.generate( |
| **tokenizer(text, return_tensors = "pt").to("cuda"), |
| max_new_tokens = 1024, |
| streamer = TextStreamer(tokenizer, skip_prompt = False), |
| top_p = 0.95, top_k = 64, temperature = 1.0, |
| ) |
| ``` |
|
|
| This will produce an output similar to: |
|
|
| ``` |
| <start_of_turn>model |
| <think>User is asking for an opinion, but I need factual product details first such as price, features, and reviews. I should call the Amazon product details tool with the provided ASIN.</think><start_function_call>call:get_amazon_product_details{asin:<escape>B0XYZ12345<escape>}<end_function_call> |
| ``` |
|
|
| ## Applications |
|
|
| This finetuned `functiongemma` model is ideal for: |
|
|
| - **Advanced AI Assistants**: Building intelligent agents that can reason about complex tasks and use external tools effectively. |
| - **Tool-Augmented LLMs**: Enhancing LLMs with the ability to dynamically call functions and interpret their results. |
| - **Complex Workflow Automation**: Automating multi-step processes that require logical reasoning and interaction with external systems. |
| - **Research in Tool Learning**: Studying and developing more sophisticated tool-learning mechanisms for LLMs. |
|
|
| ## Feedback and Issues |
|
|
| For any questions, issues, or contributions, please refer to the Unsloth Discord channel or open an issue on the Hugging Face repository. |
