Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion c-questdb-client
Submodule c-questdb-client updated 49 files
+1 −1 .bumpversion.toml
+7 −1 CMakeLists.txt
+4 −4 README.md
+3 −3 ci/run_all_tests.py
+1 −1 cpp_test/test_line_sender.cpp
+3 −1 doc/C.md
+1 −1 doc/CPP.md
+2 −2 doc/SECURITY.md
+3 −1 examples/line_sender_c_example.c
+3 −1 examples/line_sender_c_example_auth.c
+3 −1 examples/line_sender_c_example_auth_tls.c
+3 −1 examples/line_sender_c_example_from_conf.c
+3 −1 examples/line_sender_c_example_from_env.c
+3 −1 examples/line_sender_c_example_http.c
+3 −1 examples/line_sender_c_example_tls_ca.c
+2 −1 examples/line_sender_cpp_example.cpp
+2 −1 examples/line_sender_cpp_example_auth.cpp
+2 −1 examples/line_sender_cpp_example_auth_tls.cpp
+84 −0 examples/line_sender_cpp_example_decimal_binary.cpp
+138 −0 examples/line_sender_cpp_example_decimal_custom.cpp
+2 −1 examples/line_sender_cpp_example_from_conf.cpp
+2 −1 examples/line_sender_cpp_example_from_env.cpp
+2 −1 examples/line_sender_cpp_example_http.cpp
+2 −1 examples/line_sender_cpp_example_tls_ca.cpp
+51 −0 include/questdb/ingress/line_sender.h
+291 −145 include/questdb/ingress/line_sender.hpp
+269 −0 include/questdb/ingress/line_sender_array.hpp
+178 −118 questdb-rs-ffi/Cargo.lock
+1 −1 questdb-rs-ffi/Cargo.toml
+137 −0 questdb-rs-ffi/src/decimal.rs
+60 −0 questdb-rs-ffi/src/lib.rs
+30 −8 questdb-rs/Cargo.toml
+9 −9 questdb-rs/README.md
+39 −9 questdb-rs/build.rs
+2 −2 questdb-rs/examples/basic.rs
+8 −2 questdb-rs/examples/http.rs
+10 −5 questdb-rs/examples/protocol_version.rs
+3 −0 questdb-rs/src/error.rs
+111 −11 questdb-rs/src/ingress/buffer.rs
+197 −0 questdb-rs/src/ingress/decimal.rs
+31 −1 questdb-rs/src/ingress/mod.rs
+1 −0 questdb-rs/src/ingress/sender/http.rs
+584 −0 questdb-rs/src/tests/decimal.rs
+1 −1 questdb-rs/src/tests/http.rs
+44 −1 questdb-rs/src/tests/interop/ilp-client-interop-test.json
+1 −0 questdb-rs/src/tests/mod.rs
+12 −11 questdb-rs/src/tests/sender.rs
+17 −2 system_test/questdb_line_sender.py
+42 −4 system_test/test.py
55 changes: 55 additions & 0 deletions src/questdb/dataframe.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,13 +93,68 @@ We need to extract:
* 64-bit floats
* UTF-8 string buffers
* Nanosecond-precision UTC unix epoch 64-bit signed int timestamps
* decimals

```python
import pandas as pd
import pyarrow as pa
import datetime as dt
```

### Decimals

Decimals aren't natively supported by pandas nor numpy, they use the `decimal.Decimal` objects.

#### Pandas

Decimals stored as Python objects in an 'object' dtype column.

```python
>>> df = pd.DataFrame({ 'decimals': [Decimal('123.456')] })
>>> df.dtypes
decimals object
dtype: object
```

#### Numpy

Similarly, numpy stores decimals as Python objects.

```python
>>> arr = numpy.array([Decimal('123.456')])
>>> arr
array([Decimal('123.456')], dtype=object)
```

#### PyArrow

PyArrow provides native decimal support with configurable precision and scale.
The data is stored in a fixed-width binary format.

```python
import pyarrow as pa
from decimal import Decimal

# Create decimal array: decimal128(precision, scale)
# precision = total digits, scale = digits after decimal point
decimal_array = pa.array(
[Decimal('123.456'), Decimal('789.012'), Decimal('-456.789'), None],
type=pa.decimal128(10, 3) # 10 total digits, 3 after decimal
)

# Use in DataFrame with ArrowDtype
df = pd.DataFrame({
'prices': pd.array(
[Decimal('123.45'), Decimal('678.90'), None],
dtype=pd.ArrowDtype(pa.decimal128(10, 2))
)
})
```

Notes:
- 4 datatypes: `decimal32`, `decimal64`, `decimal128` and `decimal256`
- Nulls are supported via Arrow's validity bitmap

### Booleans

```python
Expand Down
Loading
Loading