Skip to content

HADOOP-19217. Introduce getTrashPolicy to FileSystem API #8063

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 4 commits into
base: trunk
Choose a base branch
from
Open

HADOOP-19217. Introduce getTrashPolicy to FileSystem API #8063

wants to merge 4 commits into from
+607 −11

Conversation

@ivandika3
Copy link
Contributor

@ivandika3 ivandika3 commented Nov 7, 2025 •
edited
Loading

Description of PR

JIRA: https://issues.apache.org/jira/browse/HADOOP-19217

Hadoop FileSystem supports multiple FileSystem implementations awareness (e.g. client is aware of both hdfs:// and ofs:// protocols).

However, it seems that currently Hadoop TrashPolicy remains the same regardless of the URI scheme. The TrashPolicy is governed by "fs.trash.classname" configuration and stays the same regardless of the FileSystem implementation. For example, HDFS defaults to TrashPolicyDefault and Ozone defaults to TrashPolicyOzone, but only one will be picked since the the configuration will be overwritten by the other.

Therefore, I propose to tie the TrashPolicy implementation to each FileSystem implementation by introducing a new FileSystem#getTrashPolicy interface. TrashPolicy#getInstance can call FileSystem#getTrashPolicy to get the appropriate TrashPolicy.

How was this patch tested?

Unit and contract tests (HDFS and LocalFS).

Disclosure: FileSystem.md part was initially generated by AI, but edited accordingly.
@hadoop-yetus
Copy link

hadoop-yetus commented Nov 7, 2025

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 26s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 markdownlint 0m 0s markdownlint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 5 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 6m 51s Maven dependency ordering for branch
+1 💚 mvninstall 17m 38s trunk passed
+1 💚 compile 10m 6s trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚 compile 9m 57s trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚 checkstyle 1m 41s trunk passed
+1 💚 mvnsite 1m 59s trunk passed
+1 💚 javadoc 1m 33s trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚 javadoc 1m 22s trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
-1 ❌ spotbugs 1m 40s /branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html hadoop-common-project/hadoop-common in trunk has 448 extant spotbugs warnings.
-1 ❌ spotbugs 2m 12s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html hadoop-hdfs-project/hadoop-hdfs in trunk has 291 extant spotbugs warnings.
+1 💚 shadedclient 18m 1s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 22s Maven dependency ordering for patch
+1 💚 mvninstall 1m 30s the patch passed
+1 💚 compile 9m 47s the patch passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚 javac 9m 47s the patch passed
+1 💚 compile 9m 56s the patch passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚 javac 9m 56s the patch passed
-1 ❌ blanks 0m 1s /blanks-eol.txt The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️ checkstyle 1m 44s /results-checkstyle-root.txt root: The patch generated 28 new + 248 unchanged - 0 fixed = 276 total (was 248)
+1 💚 mvnsite 1m 59s the patch passed
+1 💚 javadoc 1m 26s the patch passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚 javadoc 1m 24s the patch passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚 spotbugs 4m 1s the patch passed
+1 💚 shadedclient 17m 38s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 16m 29s /patch-unit-hadoop-common-project_hadoop-common.txt hadoop-common in the patch passed.
-1 ❌ unit 168m 25s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch failed.
+1 💚 asflicense 0m 36s The patch does not generate ASF License warnings.
308m 15s
Reason Tests
Failed junit tests hadoop.fs.contract.localfs.TestLocalFSContractTrash
hadoop.security.ssl.TestDelegatingSSLSocketFactory
hadoop.fs.contract.hdfs.TestHDFSContractTrash
hadoop.hdfs.tools.TestDFSAdmin
Subsystem Report/Notes
Docker ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8063/2/artifact/out/Dockerfile
GITHUB PR #8063
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
uname Linux 438b5dd4e148 5.15.0-156-generic #166-Ubuntu SMP Sat Aug 9 00:02:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 29c99cb
Default Java Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8063/2/testReport/
Max. process+thread count 3971 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8063/2/console
versions git=2.25.1 maven=3.9.11 spotbugs=4.9.7
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.
@hadoop-yetus
Copy link

hadoop-yetus commented Nov 8, 2025

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 25s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+0 🆗 markdownlint 0m 1s markdownlint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 5 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 7m 19s Maven dependency ordering for branch
+1 💚 mvninstall 16m 34s trunk passed
+1 💚 compile 8m 40s trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚 compile 8m 46s trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚 checkstyle 1m 43s trunk passed
+1 💚 mvnsite 2m 3s trunk passed
+1 💚 javadoc 1m 27s trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚 javadoc 1m 27s trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
-1 ❌ spotbugs 1m 39s /branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html hadoop-common-project/hadoop-common in trunk has 448 extant spotbugs warnings.
-1 ❌ spotbugs 2m 0s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html hadoop-hdfs-project/hadoop-hdfs in trunk has 291 extant spotbugs warnings.
+1 💚 shadedclient 16m 31s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 17s Maven dependency ordering for patch
+1 💚 mvninstall 1m 26s the patch passed
+1 💚 compile 8m 14s the patch passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚 javac 8m 14s the patch passed
+1 💚 compile 8m 37s the patch passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚 javac 8m 37s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 1m 34s /results-checkstyle-root.txt root: The patch generated 28 new + 248 unchanged - 0 fixed = 276 total (was 248)
+1 💚 mvnsite 2m 2s the patch passed
+1 💚 javadoc 1m 28s the patch passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚 javadoc 1m 27s the patch passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚 spotbugs 3m 51s the patch passed
+1 💚 shadedclient 16m 20s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 16m 27s /patch-unit-hadoop-common-project_hadoop-common.txt hadoop-common in the patch passed.
-1 ❌ unit 174m 4s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch failed.
+1 💚 asflicense 0m 38s The patch does not generate ASF License warnings.
305m 17s
Reason Tests
Failed junit tests hadoop.security.ssl.TestDelegatingSSLSocketFactory
hadoop.hdfs.tools.TestDFSAdmin
Subsystem Report/Notes
Docker ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8063/4/artifact/out/Dockerfile
GITHUB PR #8063
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
uname Linux c4eea5429b3b 5.15.0-156-generic #166-Ubuntu SMP Sat Aug 9 00:02:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 5af2d3b
Default Java Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8063/4/testReport/
Max. process+thread count 4212 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8063/4/console
versions git=2.25.1 maven=3.9.11 spotbugs=4.9.7
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.
@ivandika3 ivandika3 marked this pull request as ready for review November 8, 2025 11:48
@slfan1989
Copy link
Contributor

slfan1989 commented Nov 10, 2025

@ayushtkn @KeeProMise Could you please review this PR? Thank you very much!
@ivandika3
Copy link
Contributor Author

ivandika3 commented Nov 10, 2025

@steveloughran Please help to take a look when you are available. Thanks in advance.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Common HDFS trunk

3 participants

@ivandika3 @hadoop-yetus @slfan1989