Inference Optimization

community

Activity Feed

AI & ML interests

None defined yet.

Recent Activity

nm-research updated a dataset about 9 hours ago

inference-optimization/laguna-xs-ultrachat-responses

nm-research updated a dataset about 9 hours ago

inference-optimization/laguna-xs-ultrachat-conversations

nm-research published a dataset about 9 hours ago

inference-optimization/laguna-xs-ultrachat-responses

View all activity

inference-optimization 's collections 6

HIGGS-per-tensor

meta-llama/Llama-3.2-1B-Instruct

Text Generation • 1B • Updated Oct 24, 2024 • 5.03M • • 1.38k
inference-optimization/Llama-3.2-1B-Instruct-FP8-Dynamic

1B • Updated 17 days ago • 23
inference-optimization/Llama-3.2-1B-Instruct-NVFP4

0.8B • Updated 17 days ago • 37
inference-optimization/Llama-3.2-1B-Instruct-5-bits-mode-heuristic-per-tensor

1B • Updated 1 day ago • 16

test-models

inference-optimization/test_tencentbac_fastmtp

Updated Mar 4 • 2
inference-optimization/test_qwen3_next_mtp

Updated Mar 4 • 1
inference-optimization/Qwen3-Next-80B-A3B-Instruct_mtp_speculator

Text Generation • 2B • Updated Mar 18 • 26
inference-optimization/Qwen3-Next-80B-A3B-Instruct-MTP-ultrachat-epoch3

2B • Updated Mar 19 • 9

NVIDIA-Nemotron-3-Nano-30B-A3B Quantized Models

FP8-dynamic, FP8-block, NVFP4, INT4, versions of nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B

inference-optimization/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8

Text Generation • 32B • Updated Jan 9 • 4
inference-optimization/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4

18B • Updated Jan 15 • 6
inference-optimization/NVIDIA-Nemotron-3-Nano-30B-A3B-quantized.w4a16

6B • Updated Jan 7 • 4
inference-optimization/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8-dynamic

32B • Updated Jan 6 • 4

HIGGS

Mixed Precision Models

meta-llama/Llama-3.1-8B-Instruct

Text Generation • 8B • Updated Sep 25, 2024 • 9.19M • • 5.75k
RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic

Text Generation • 8B • Updated Mar 19 • 45.4k • 9
RedHatAI/Llama-3.1-8B-Instruct-NVFP4

Text Generation • 5B • Updated Nov 21, 2025 • 19.7k • 1
inference-optimization/Llama-3.1-8B-Instruct_5_bits_mode_hybrid

6B • Updated Mar 12 • 11

Granite 4 Small and Tiny Quantized Models

FP8-block, FP8-dynamic, NVFP4, w4a16, w8a8 quantized models of ibm-granite/granite-4.0-h-small and ibm-granite/granite-4.0-h-tiny models

inference-optimization/granite-4.0-h-tiny-FP8-block

Text Generation • 7B • Updated Jan 23 • 4
RedHatAI/granite-4.0-h-tiny-FP8-dynamic

Text Generation • 7B • Updated 13 days ago • 364 • 3
inference-optimization/granite-4.0-h-tiny-quantized.w4a16

Updated Jan 22 • 2
inference-optimization/granite-4.0-h-tiny-NVFP4

Updated Jan 22 • 66

Qwen3-Next-80B-A3B Quantized Models

FP8-dynamic, FP8-block, NVFP4, INT4, INT8 versions of Qwen3-Next-80B-A3B-Instruct and Qwen3-Next-80B-A3B-Thinking Models

inference-optimization/Qwen3-Next-80B-A3B-Instruct-FP8

Text Generation • 81B • Updated Jan 9 • 31
inference-optimization/Qwen3-Next-80B-A3B-Thinking-FP8

Text Generation • 81B • Updated Jan 9 • 9
RedHatAI/Qwen3-Next-80B-A3B-Thinking-FP8-block

Text Generation • 80B • Updated Jan 5 • 11
RedHatAI/Qwen3-Next-80B-A3B-Thinking-FP8-dynamic

Text Generation • 80B • Updated Jan 14 • 13

HIGGS-per-tensor

meta-llama/Llama-3.2-1B-Instruct

Text Generation • 1B • Updated Oct 24, 2024 • 5.03M • • 1.38k
inference-optimization/Llama-3.2-1B-Instruct-FP8-Dynamic

1B • Updated 17 days ago • 23
inference-optimization/Llama-3.2-1B-Instruct-NVFP4

0.8B • Updated 17 days ago • 37
inference-optimization/Llama-3.2-1B-Instruct-5-bits-mode-heuristic-per-tensor

1B • Updated 1 day ago • 16

HIGGS

Mixed Precision Models

meta-llama/Llama-3.1-8B-Instruct

Text Generation • 8B • Updated Sep 25, 2024 • 9.19M • • 5.75k
RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic

Text Generation • 8B • Updated Mar 19 • 45.4k • 9
RedHatAI/Llama-3.1-8B-Instruct-NVFP4

Text Generation • 5B • Updated Nov 21, 2025 • 19.7k • 1
inference-optimization/Llama-3.1-8B-Instruct_5_bits_mode_hybrid

6B • Updated Mar 12 • 11

test-models

inference-optimization/test_tencentbac_fastmtp

Updated Mar 4 • 2
inference-optimization/test_qwen3_next_mtp

Updated Mar 4 • 1
inference-optimization/Qwen3-Next-80B-A3B-Instruct_mtp_speculator

Text Generation • 2B • Updated Mar 18 • 26
inference-optimization/Qwen3-Next-80B-A3B-Instruct-MTP-ultrachat-epoch3

2B • Updated Mar 19 • 9

Granite 4 Small and Tiny Quantized Models

FP8-block, FP8-dynamic, NVFP4, w4a16, w8a8 quantized models of ibm-granite/granite-4.0-h-small and ibm-granite/granite-4.0-h-tiny models

inference-optimization/granite-4.0-h-tiny-FP8-block

Text Generation • 7B • Updated Jan 23 • 4
RedHatAI/granite-4.0-h-tiny-FP8-dynamic

Text Generation • 7B • Updated 13 days ago • 364 • 3
inference-optimization/granite-4.0-h-tiny-quantized.w4a16

Updated Jan 22 • 2
inference-optimization/granite-4.0-h-tiny-NVFP4

Updated Jan 22 • 66

NVIDIA-Nemotron-3-Nano-30B-A3B Quantized Models

FP8-dynamic, FP8-block, NVFP4, INT4, versions of nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B

inference-optimization/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8

Text Generation • 32B • Updated Jan 9 • 4
inference-optimization/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4

18B • Updated Jan 15 • 6
inference-optimization/NVIDIA-Nemotron-3-Nano-30B-A3B-quantized.w4a16

6B • Updated Jan 7 • 4
inference-optimization/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8-dynamic

32B • Updated Jan 6 • 4

Qwen3-Next-80B-A3B Quantized Models

FP8-dynamic, FP8-block, NVFP4, INT4, INT8 versions of Qwen3-Next-80B-A3B-Instruct and Qwen3-Next-80B-A3B-Thinking Models

inference-optimization/Qwen3-Next-80B-A3B-Instruct-FP8

Text Generation • 81B • Updated Jan 9 • 31
inference-optimization/Qwen3-Next-80B-A3B-Thinking-FP8

Text Generation • 81B • Updated Jan 9 • 9
RedHatAI/Qwen3-Next-80B-A3B-Thinking-FP8-block

Text Generation • 80B • Updated Jan 5 • 11
RedHatAI/Qwen3-Next-80B-A3B-Thinking-FP8-dynamic

Text Generation • 80B • Updated Jan 14 • 13

AI & ML interests

Recent Activity

Team members 15

inference-optimization 's collections 6