hp1502/Legal_Text_Summarizer

flan-t5-small-billsum

This model is a fine-tuned version of google/flan-t5-small on the billsum dataset. It achieves the following results on the evaluation set:

• Loss: 1.5675
• Rouge1: 24.0011
• Rouge2: 18.8602
• Rougel: 22.9037
• Rougelsum: 23.1161
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	195	1.6773	22.8625	17.311	21.829	22.0089	19.0
No log	2.0	390	1.6134	23.6942	18.553	22.561	22.8895	19.0
1.9532	3.0	585	1.5882	23.8253	18.7086	22.6519	22.9745	19.0
1.9532	4.0	780	1.5739	24.0178	18.8429	22.9119	23.1471	19.0
1.9532	5.0	975	1.5675	24.0011	18.8602	22.9037	23.1161	19.0

Framework versions

• Transformers 4.33.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3