Skip to content

Source postgres: Cursor-based replication reads JSONB[] columns incorrectly #29551

@edgao

Description

@edgao

consider this setup:

create table jsonb_test (id serial primary key, q jsonb[]); insert into jsonb_test (q) values (ARRAY['{"foo": "bar"}' :: jsonb]);

In CDC mode, source-postgres correctly emits a record with this data:

{ "q":["{\"foo\": \"bar\"}"] }

Note that it's a JSON array of serialized JSON objects. (the serialization is up for debate as part of the JSONB project, and is not relevant to this issue).

However, in cursor-based mode, we read something like this:

{ "q": "{\"{\\\"foo\\\": \\\"bar\\\"}\"}" }

Which looks like we took Postgres' string representation of an array ({"{\"foo\":\"bar\"}"}) and dumped it into a JSON string. This is wrong; we should parse out the array elements and build a proper json array.

This is a regression between v1.0.30 and v3.0.2; see #29172 (comment). I did a bisect and found that 1.0.42 is the last version where jsonb[] works correctly in non-cdc mode. (which is surprising, because 1.0.43's changelog notes don't mention anything about jsonb)

Metadata

Metadata

Assignees

Labels

team/db-dw-sourcesBacklog for Database and Data Warehouse Sources team

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions