Commit ·
a88aee5
1
Parent(s): a742c27
Update README.md
Browse files
README.md
CHANGED
|
@@ -20,7 +20,7 @@ SQLCoder is fine-tuned on a base StarCoder model.
|
|
| 20 |
| model | perc_correct |
|
| 21 |
|-|-|
|
| 22 |
| gpt4-2023-10-04 | 82.0 |
|
| 23 |
-
| defog-sqlcoder2 |
|
| 24 |
| gpt4-2023-08-28 | 74.0 |
|
| 25 |
| defog-sqlcoder-7b | 71.0 |
|
| 26 |
| gpt-3.5-2023-10-04 | 66.0 |
|
|
@@ -41,11 +41,11 @@ You can read more about our [training approach](https://defog.ai/blog/open-sourc
|
|
| 41 |
We classified each generated question into one of 5 categories. The table displays the percentage of questions answered correctly by each model, broken down by category.
|
| 42 |
| query_category | gpt-4 | sqlcoder2-15b | sqlcoder-7b | gpt-3.5 | claude-2 | claude-instant | gpt-3 |
|
| 43 |
|:-----------------|--------:|----------------:|--------------:|----------:|-----------:|-----------------:|--------:|
|
| 44 |
-
| date | 72 |
|
| 45 |
-
| group_by | 91.4 |
|
| 46 |
| order_by | 82.9 | 77.1 | 74.3 | 68.6 | 74.3 | 74.3 | 68.6 |
|
| 47 |
-
| ratio | 80 |
|
| 48 |
-
| join | 82.9 |
|
| 49 |
| where | 80 | 77.1 | 74.3 | 74.3 | 62.9 | 60 | 54.3 |
|
| 50 |
|
| 51 |
## Using SQLCoder
|
|
|
|
| 20 |
| model | perc_correct |
|
| 21 |
|-|-|
|
| 22 |
| gpt4-2023-10-04 | 82.0 |
|
| 23 |
+
| defog-sqlcoder2 | 77.5 |
|
| 24 |
| gpt4-2023-08-28 | 74.0 |
|
| 25 |
| defog-sqlcoder-7b | 71.0 |
|
| 26 |
| gpt-3.5-2023-10-04 | 66.0 |
|
|
|
|
| 41 |
We classified each generated question into one of 5 categories. The table displays the percentage of questions answered correctly by each model, broken down by category.
|
| 42 |
| query_category | gpt-4 | sqlcoder2-15b | sqlcoder-7b | gpt-3.5 | claude-2 | claude-instant | gpt-3 |
|
| 43 |
|:-----------------|--------:|----------------:|--------------:|----------:|-----------:|-----------------:|--------:|
|
| 44 |
+
| date | 72 | 80 | 64 | 68 | 52 | 48 | 32 |
|
| 45 |
+
| group_by | 91.4 | 82.9 | 82.9 | 77.1 | 71.4 | 71.4 | 71.4 |
|
| 46 |
| order_by | 82.9 | 77.1 | 74.3 | 68.6 | 74.3 | 74.3 | 68.6 |
|
| 47 |
+
| ratio | 80 | 74.3 | 54.3 | 37.1 | 57.1 | 45.7 | 25.7 |
|
| 48 |
+
| join | 82.9 | 74.3 | 74.3 | 71.4 | 65.7 | 62.9 | 57.1 |
|
| 49 |
| where | 80 | 77.1 | 74.3 | 74.3 | 62.9 | 60 | 54.3 |
|
| 50 |
|
| 51 |
## Using SQLCoder
|