Posted on Sep 22

Apache Polaris Dev List Digest (Sept 15–19, 2025)

#dataengineering #resources #data #opensource

Release 1.1.0-incubating Finalized and Announced

Polaris 1.1.0 Released: The community wrapped up the [RESULT][VOTE] Release Apache Polaris 1.1.0-incubating (rc0)](https://lists.apache.org/thread/7rqqzhyy4fphg82w7fnh7h2d8y5d9gn1) and officially announced the release on September 19. The vote passed with binding +1 votes (e.g. from Dmitri Bourlatchkov) and no objections. Jean-Baptiste Onofré then announced that Apache Polaris 1.1.0-incubating is now released, highlighting several new features and fixes:

Hive Metastore (HMS) support: Polaris can now integrate with Hive Metastore for catalog storage.
Improved authentication & identity: Added an “IMPLICIT” authentication type and expanded external Identity Provider support.
Enhanced S3/MinIO integration: Polaris now supports non-AWS S3-compatible storage (like MinIO, including s3a:// scheme) via STS, and includes fixes for MinIO support.
Federation and CLI updates: A modularized federation architecture and federated catalog support in the Polaris CLI were introduced.
Other updates: Use of Realm objects instead of raw Realm IDs, an official Python client package, documentation improvements, and various bug fixes.

Planning for Apache Polaris 1.2

DISCUSS: 1.2 release – Participants: Prashant Singh (initiator), Yufei Gu, Dmitri Bourlatchkov, Jean-Baptiste Onofré, Alexandre Dutra. The community started brainstorming the scope and timeline for a Polaris 1.2.0 release now that 1.1.0 was nearly complete. Key points:

Release Timing: Targeting end of October for the 1.2 release, maintaining a monthly cadence.
“Release Train” model: Whatever features are ready at branch cut will ship in 1.2. Features not ready roll into future releases.
GitHub Milestone Tracking: Using the 1.2.0 milestone to flag blockers and high-priority PRs.
Automating the Release: The community is working on a GitHub Action to automate tagging and publishing.
Feature Planning: Remote signing (preview mode), Iceberg version bump to 1.10, and other in-progress PRs are being targeted.

Feature Proposals and Improvements

Data Lake Operational Metrics

[PROPOSAL] Add Data Lake operational metrics to Polaris](https://lists.apache.org/thread/x0s9bttz4y6f1fprm2t0r37db3wj6px8) – Pierre Laporte proposed a metrics service in Polaris to collect and serve table-level operational metrics (size, file counts, snapshot counts, etc.).

Use Cases: Monitoring, dashboards, automated compaction, cost analysis, debugging performance.
Scope: Start with point-in-time metrics (counts and timestamps).
Consensus: Start simple (point-in-time), leave historical/time-series tracking to external systems.

Idempotency Key for REST API Mutations

[DISCUSS] Add Idempotency-Key to Polaris REST mutations](https://lists.apache.org/thread/zq1pnqdrjs4wqgx4w48r67lrbpkn4b0q) – Proposed by Huaxin Gao.

Problem: Client retries after a successful mutation can cause inconsistencies.
Solution: Support Idempotency-Key header. Polaris caches results for 30 mins.
Consensus: Positive response. Aligns Polaris with Iceberg REST idempotency work.

OpenAPI Specification Updates

Proposed Public OpenAPI Changes – Adnan Hemani suggested:

Standardizing path parameter names (name vs id).
Ensuring all resource mutations emit events.
General cleanup to improve API clarity.

New PolarisMetaStoreManager APIs

Proposal by Michael Collado – Aimed at exposing new APIs for managing Polaris metadata. Early-stage discussion, with details expected in follow-ups.

Finer-Grained Authorization Checks

Proposal by Travis Bowen – Suggested more granular permissions for UPDATE TABLE operations (schema changes vs. partition spec vs. properties). Aimed at tighter multi-tenant security. Discussion just beginning.

Deprecating EclipseLink Persistence

[DISCUSS] When to remove EclipseLink?](https://lists.apache.org/thread/vm25sbrhkmh3xrr3f0ojkg79rqpx4qdp) – Adam Christian opened the thread.

Consensus: EclipseLink is legacy. JDBC persistence is the future.
Plan: Remove EclipseLink soon (possibly in 1.2 or 1.3). GitHub issue created.

Enabling Additional ErrorProne Rules

[DISCUSS] Enabling New ErrorProne Rules](https://lists.apache.org/thread/wwz2c5q4n7y6fpnfvq9w7mj1mcc4slh8) – Adam Christian proposed expanding static analysis rules.

Goal: Catch more bugs and enforce code quality.
Consensus: Positive. Plan to enable rules gradually and fix warnings.

Using GitHub Discussions for Proposal Tracking

[DISCUSS] Tracking proposals in GH discussions](https://lists.apache.org/thread/1fph8r5z8bgrmc7l7kfjtx4o8sm5z3qq) – Dmitri Bourlatchkov suggested using GitHub Discussions to improve discoverability of design proposals.

Consensus: Positive. Mailing list remains primary, but GitHub Discussions could complement.

Other Notable Discussions

Merging Authentication-Related PRs

Merge of authn-related PRs – Alexandre Dutra listed several approved authn PRs ready to merge:

Include principal name in Polaris tokens.
Remove numeric identifier from PolarisPrincipal.
Remove ActiveRolesProvider.

AWS IAM Authentication for Postgres

Aurora Postgres passwordless DB – Fabio Rizzo asked about using AWS IAM auth with Aurora Postgres instead of static passwords.

Current state: Needs AWS JDBC wrapper integration.
Consensus: Useful improvement. Needs Quarkus changes or contributions.
Outcome: Encouraged Fabio (and others) to raise an issue/PR.

Always Obtaining AccessConfig (MinIO Support)

[DISCUSS] Always obtaining AccessConfig](https://lists.apache.org/thread/l1ztwnw94y6wn9kkv1xpg7hf5ty6trwq) – Dmitri proposed always returning AccessConfig (endpoint info) even if no credentials are requested.

Goal: Simplify MinIO usage.
Outcome: Agreed. Improves compatibility with non-AWS S3. Likely included in 1.2.

Community Updates

Polaris Community Sync – Sept 18, 2025: Regular bi-weekly sync held. Jean-Baptiste reported new community members attending and shared the meeting recording.
General vibe: Very active week. Mix of polish (auth, OpenAPI, metrics) and cleanup (EclipseLink, ErrorProne). Release 1.1.0 finalized. 1.2 planning kicked off with clear direction.

DEV Community