Instructions to use kdunee/IntentGuard-1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use kdunee/IntentGuard-1 with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="kdunee/IntentGuard-1",
	filename="IntentGuard-1.Q8_0.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use kdunee/IntentGuard-1 with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf kdunee/IntentGuard-1:Q8_0
# Run inference directly in the terminal:
llama-cli -hf kdunee/IntentGuard-1:Q8_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf kdunee/IntentGuard-1:Q8_0
# Run inference directly in the terminal:
llama-cli -hf kdunee/IntentGuard-1:Q8_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf kdunee/IntentGuard-1:Q8_0
# Run inference directly in the terminal:
./llama-cli -hf kdunee/IntentGuard-1:Q8_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf kdunee/IntentGuard-1:Q8_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf kdunee/IntentGuard-1:Q8_0

Use Docker

docker model run hf.co/kdunee/IntentGuard-1:Q8_0

LM Studio
Jan
Ollama
How to use kdunee/IntentGuard-1 with Ollama:
```
ollama run hf.co/kdunee/IntentGuard-1:Q8_0
```

Unsloth Studio new

How to use kdunee/IntentGuard-1 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for kdunee/IntentGuard-1 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for kdunee/IntentGuard-1 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for kdunee/IntentGuard-1 to start chatting

Pi new

How to use kdunee/IntentGuard-1 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf kdunee/IntentGuard-1:Q8_0

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "kdunee/IntentGuard-1:Q8_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use kdunee/IntentGuard-1 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf kdunee/IntentGuard-1:Q8_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default kdunee/IntentGuard-1:Q8_0

Run Hermes

hermes

Docker Model Runner
How to use kdunee/IntentGuard-1 with Docker Model Runner:
```
docker model run hf.co/kdunee/IntentGuard-1:Q8_0
```

Lemonade

How to use kdunee/IntentGuard-1 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull kdunee/IntentGuard-1:Q8_0

Run and chat with the model

lemonade run user.IntentGuard-1-Q8_0

List all available models

lemonade list

IntentGuard Model

This repository contains the model powering IntentGuard, a Python library designed for verifying code properties using natural language assertions.

This model is an internal component of IntentGuard and is not intended for direct, standalone use. It is specifically designed to work within the IntentGuard framework and relies on the library's infrastructure for proper execution and integration.

Model Description

The IntentGuard model is a custom 1 billion parameter model, fine-tuned from Llama-3.2-1B-Instruct. It has been specifically trained and optimized for the following tasks:

Understanding Natural Language Assertions about Code: Interpreting plain English statements that describe desired properties of code.
Code Analysis and Property Verification: Analyzing Python code snippets to determine if they satisfy the properties described in the natural language assertions.
Providing Human-Readable Explanations: Generating natural language explanations for why code does or does not satisfy a given assertion.

The model leverages a chain-of-thought approach during evaluation to mimic human-like reasoning about code and natural language, contributing to the deterministic and reliable behavior of IntentGuard.

Training

The model was trained on a curated dataset specifically designed for code property verification. This dataset, available on Hugging Face at kdunee/IntentGuard-1, includes:

Python code snippets representing various programming scenarios.
Natural language assertions describing desired properties of these code snippets (e.g., error handling practices, documentation standards, security considerations).
Chain-of-thought reasoning examples to guide the model in mimicking human-like evaluation processes.

The fine-tuning process focused on optimizing the model's ability to:

Accurately classify whether code satisfies a given natural language assertion.
Generalize to unseen code patterns and assertion types related to code quality and best practices.
Produce coherent and informative explanations.

Intended Use & Limitations

This model is solely intended for use within the IntentGuard library. It is not designed or optimized for general-purpose language tasks or other code-related applications outside of the specific code property verification domain defined by IntentGuard.

Due to its specialized training and internal integration, attempting to use this model directly without the IntentGuard framework is likely to be ineffective. For users interested in code property verification with natural language, please refer to the IntentGuard library repository.

Technical Details

Model Type: Fine-tuned Language Model
Base Model: Llama-3.2-1B-Instruct
Parameters: 1 Billion
Inference Engine: llamafile for local, efficient inference.
License: MIT License - Same as IntentGuard and the training dataset.

Citation

If you use IntentGuard in your research or projects, please cite the IntentGuard library. You can find citation information in the IntentGuard repository.

Downloads last month: 6

GGUF

Model size

1B params

Architecture

llama

Hardware compatibility

8-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for kdunee/IntentGuard-1

Base model

meta-llama/Llama-3.2-1B-Instruct

Quantized

(371)

this model

kdunee
/

IntentGuard-1