AI & ML interests

None defined yet.

Recent Activity

Sunny111Β 
posted an update 22 days ago
view post
Post
1610
Are you familiar with reverse residual connections or looping in language models?

Excited to share my Looped-GPT blog post and codebase πŸš€
https://github.com/sanyalsunny111/Looped-GPT

TL;DR: looping during pre-training improves generalization.

Plot shows GPT2 LMs pre-trained with 15.73B OWT tokens

P.S. This is my first post here β€” I have ~4 followers and zero expectations for reach πŸ˜„
Β·