README.md

---
title: MBench Annotation
emoji: 🎬
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 5.9.1
python_version: "3.11"
app_file: app.py
pinned: false
---

# MBench Annotation Platform (NEW)

Adapted to the new dataset layout (`MBench-V-new` + `MBench-A-New`) on
[`studyOverflow/TempMemoryData`](https://huggingface.co/datasets/studyOverflow/TempMemoryData).

## Tabs

1. **MBench-V Binary** — single video, "is there a memory issue?" (yes/no)
2. **MBench-V Pairwise** — two T2V videos, 5 dimensions
3. **MBench-A Pairwise** — two world-model videos, ≤6 dimensions

## Annotation Sink

Submissions are pushed to `annotations-new/` on the dataset repo every 5 minutes via
`CommitScheduler`. Old `annotations/` is left untouched (legacy).

## Migrated Historical Data

`annotations-new/` already contains:
- `migrated_v_binary.jsonl` (642 records from old `ann_bc109d66.jsonl`)
- `migrated_a_pairwise.jsonl` (821 records from old `ann_mbench_a_*.jsonl`)

These are read on startup so existing annotators don't see already-completed tasks again.