fix: orderable fractional indexing case-sensitivity issue with PostgreSQL

[vladyslavprosolupov]

fix: orderable fractional indexing case-sensitivity issue with PostgreSQL

Problem

When using the orderable feature with PostgreSQL, reordering documents to the "first" position generates keys that break subsequent reordering operations.

Scenario

1. Create two orderable documents → keys are a0, a1
2. Drag a1 before a0 (to make it first)
3. The system generates key Zz for the moved document
4. Result: Ordering is completely broken - cannot reorder any documents anymore

Root Cause

The fractional indexing algorithm uses:

• Uppercase A-Z for "smaller" integer keys
• Lowercase a-z for "larger" integer keys

This relies on ASCII ordering where 'Z' (code 90) < 'a' (code 97), so Zz < a0.

However, PostgreSQL's default collation (en_US.UTF-8) uses case-insensitive comparison, treating 'Z' as 'z'. This means:

• ASCII: Zz < a0 ✓
• PostgreSQL: Zz treated as zz, so zz > a0 ✗

This mismatch causes:

1. Database queries return incorrect adjacent documents
2. The generateKeyBetween function receives arguments in wrong order
3. Subsequent reorder attempts either error out or create more broken keys

Solution

Modified the fractional indexing algorithm to use only characters that sort consistently across all database collations:

Range	Old Encoding	New Encoding
Small keys	A-Z (uppercase)	0-9 (digits)
Large keys	a-z (lowercase)	a-z (lowercase, unchanged)

Key insight: Digits (0-9) always sort before letters in both ASCII ordering and case-insensitive collations.

Before (broken)

decrementInteger('a0') → 'Zz' 'Zz' < 'a0' in ASCII ✓ 'Zz' > 'a0' in PostgreSQL (case-insensitive) ✗ 

After (fixed)

decrementInteger('a0') → '9z' '9z' < 'a0' in ASCII ✓ '9z' < 'a0' in PostgreSQL ✓ '9z' < 'a0' in MongoDB ✓ '9z' < 'a0' in SQLite ✓ 

Changes

packages/payload/src/config/orderable/fractional-indexing.js

• Changed integer part encoding to use digits for "small" range
• Maintains backward compatibility with existing a-z keys
• Legacy A-Z keys are still parsed (for backward compatibility) but won't be generated

Backward Compatibility

• ✅ Existing keys starting with a-z continue to work correctly
• ⚠️ Existing keys starting with A-Z (uppercase) will be parsed but may sort incorrectly in case-insensitive databases. Users with such keys should run a migration to regenerate them.

Testing

Verified the algorithm produces correct ordering:

generateKeyBetween(null, null) // → 'a0' generateKeyBetween(null, 'a0') // → '9z' (previously 'Zz') generateKeyBetween(null, '9z') // → '9y' // Sorting works correctly ['9z', 'a0', 'a1'].sort() // → ['9z', 'a0', 'a1'] ✓

Related

• Fractional Indexing algorithm: https://observablehq.com/@dgreensp/implementing-fractional-indexing
• Original library: https://github.com/rocicorp/fractional-indexing

[DanRibbens]

Thanks for the PR! I actually thought that we fixed this some time ago.

Assuming all existing tests pass, just a few things need to happen before this can be merged.

• remove the excessive comments generated by AI.
• add test in test/sort/int.spec.ts

[vladyslavprosolupov]

Using base 36 because fractional indexing keys use characters 0-9 and a-z.
Base 16 would fail for keys like '9z' (returning just 9) since 'z' is not a valid hex character.

Also changed it in other tests for cohesion, it breaks nothing.