MR0b0t/cpp-to-python-transformer

C++ → Python Transformer (16.4M params)

Encoder-decoder transformer trained from scratch for C++ → Python code translation. Trained on XLCoST on a single GTX 1650 4 GB GPU. Best checkpoint at epoch 19, val_loss 2.0474.

Architecture

• 4 encoder + 4 decoder layers, pre-norm
• d_model 256, 8 heads, d_ff 512
• Sinusoidal positional encoding
• Greedy decoding at inference
• 16.4M parameters

Files

• best_model.pt — full PyTorch checkpoint (model state, optimizer state, src/tgt vocabularies). 189 MB.

Load

from model import build_transformer
import torch
ckpt = torch.load('best_model.pt', map_location='cpu')
model = build_transformer(
    src_vocab_size=len(ckpt['src_vocab']),
    tgt_vocab_size=len(ckpt['tgt_vocab']),
    src_seq_len=300, tgt_seq_len=300,
    d_model=256, N=4, h=8, dropout=0.0, d_ff=512,
)
model.load_state_dict(ckpt['model_state'])
model.eval()

Full training and inference code: github.com/debtirthasaha/cpp-to-python-transformer

Writeup

A transformer that reads C++ and writes Python