Merge pull request #54 from get-convex/jordan/improvements-for-gpt

Improvements for `GPT-4o`

Author: jordanhunt22

evals/002-queries/000-all_rows/TASK.txt

@@ -12,4 +12,4 @@ export default defineSchema({
 });
 ```
 
-Write a query named `getAllProducts` in `convex/public.ts` that returns all products in the table.
+Write a query named `getAllProducts` in `convex/public.ts` that returns all products in the table, including their system fields.

evals/002-queries/002-userspace_filter/TASK.txt

@@ -19,4 +19,4 @@ Write a query named `getPopularPinnedMessages` in `convex/public.ts` that:
 - Filters IN JAVASCRIPT (not in the database) to find messages that are:
  * Pinned (isPinned === true)
  * Have at least the minimum number of likes
-- Returns the filtered messages sorted by likes in descending order
+- Returns the filtered messages sorted by likes in descending order, including their system fields

evals/002-queries/007-aggregation/TASK.txt

@@ -20,6 +20,7 @@ Write a query named `getCustomerStats` in `convex/public.ts` that:
  * totalItems: sum of all quantities
  * totalSpent: sum of (quantity * pricePerUnit) for all orders
  * averageOrderValue: totalSpent / totalOrders
+- All return value fields should be numbers
 - You can assume that the number of orders for each customer is small.
 - Handle the case where customer has no orders and returns zero for all fields.
 - Format monetary values appropriately (2 decimal places)

runner/models/guidelines.py

@@ -121,6 +121,21 @@ def __init__(self, name: str, children: list):
  });
  ```"""
  ),
+ Guideline(
+ """Here are the valid Convex types along with their respective validators:
+ Convex Type | TS/JS type | Example Usage | Validator for argument validation and schemas | Notes |
+| ----------- | ------------| -----------------------| -----------------------------------------------| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Id | string | `doc._id` | `v.id(tableName)` | |
+| Null | null | `null` | `v.null()` | JavaScript's `undefined` is not a valid Convex value. Functions the return `undefined` or do not return will return `null` when called from a client. Use `null` instead. |
+| Int64 | bigint | `3n` | `v.int64()` | Int64s only support BigInts between -2^63 and 2^63-1. Convex supports `bigint`s in most modern browsers. |
+| Float64 | number | `3.1` | `v.number()` | Convex supports all IEEE-754 double-precision floating point numbers (such as NaNs). Inf and NaN are JSON serialized as strings. |
+| Boolean | boolean | `true` | `v.boolean()` |
+| String | string | `"abc"` | `v.string()` | Strings are stored as UTF-8 and must be valid Unicode sequences. Strings must be smaller than the 1MB total size limit when encoded as UTF-8. |
+| Bytes | ArrayBuffer | `new ArrayBuffer(8)` | `v.bytes()` | Convex supports first class bytestrings, passed in as `ArrayBuffer`s. Bytestrings must be smaller than the 1MB total size limit for Convex types. |
+| Array | Array] | `[1, 3.2, "abc"]` | `v.array(values)` | Arrays can have at most 8192 values. |
+| Object | Object | `{a: "abc"}` | `v.object({property: value})` | Convex only supports "plain old JavaScript objects" (objects that do not have a custom prototype). Objects can have at most 1024 entries. Field names must be nonempty and not start with "$" or "_". |
+| Record | Record | `{"a": "1", "b": "2"}` | `v.record(keys, values)` | Records are objects at runtime, but can have dynamic keys. Keys must be only ASCII characters, nonempty, and not start with "$" or "_". |"""
+ ),
  ],
  ),
  GuidelineSection(
@@ -249,6 +264,9 @@ def __init__(self, name: str, children: list):
  },
  });
  ```
+ Note: `paginationOpts` is an object with the following properties:
+ - `numItems`: the maximum number of documents to return (the validator is `v.number()`)
+ - `cursor`: the cursor to use to fetch the next page of documents (the validator is `v.union(v.string(), v.null())`)
  """
  ),
  Guideline(
@@ -279,7 +297,7 @@ def __init__(self, name: str, children: list):
  Guideline("Always define your schema in `convex/schema.ts`."),
  Guideline("Always import the schema definition functions from `convex/server`:"),
  Guideline(
- "System fields are automatically added to all documents and are prefixed with an underscore."
+ "System fields are automatically added to all documents and are prefixed with an underscore. The two system fields that are automatically added to all documents are `_creationTime` which has the validator `v.number()` and `_id` which has the validator `v.id(tableName)`."
  ),
  Guideline(
  """Always include all index fields in the index name. For example, if an index is defined as `["field1", "field2"]`, the index name should be "by_field1_and_field2"."""