JuiceFS: S3FS & PGFS
How to build a distributed cloud-native filesystem JuiceFS using PostgreSQL and MinIO provided by Pigsty.
JuiceFS is a high-performance, cloud-native distributed filesystem.
This guide demonstrates how to build a production-grade JuiceFS cluster using PostgreSQL as the metadata engine and MinIO as the object storage engine, both provided by Pigsty.
Quick Start
Create a four-node sandbox using the full mode.
./configure -c full ./install.yml Install JuiceFS and configure object storage:
JFSNAME=jfs METAURL=postgres://dbuser_meta:[email protected]:5432/meta DATAURL=( --storage minio --bucket https://sss.pigsty:9000/infra --access-key minioadmin --secret-key minioadmin ) juicefs format "${DATAURL[@]}" ${METAURL} jfs # Format filesystem juicefs mount ${METAURL} ~/jfs -d # Mount in background juicefs umount ~/jfs # Unmount For a more advanced approach: PGFS - Using database as filesystem
JFSNAME=jfs METAURL=postgres://dbuser_meta:[email protected]:5432/meta DATAURL=( --storage postgres --bucket 10.10.10.10:5432/meta --access-key dbuser_meta --secret-key DBUser.Meta ) juicefs format "${DATAURL[@]}" ${METAURL} ${JFSNAME} juicefs mount ${METAURL} ~/jfs -d # Mount in background juicefs umount ~/jfs # Unmount Standalone Mode
Pigsty Infra repository provides the latest JuiceFS RPM/DEB packages. Install directly using your package manager.
The following commands create a local JuiceFS filesystem using local SQLite and filesystem (/var/jfs):
juicefs format sqlite3:///tmp/jfs.db myjfs Format Output
$ juicefs format sqlite3:///jfs.db myjfs 2025/03/19 12:07:56.956222 juicefs[62924] <INFO>: Meta address: sqlite3:///jfs.db [interface.go:504] 2025/03/19 12:07:56.958055 juicefs[62924] <INFO>: Data use file:///var/jfs/myjfs/ [format.go:484] 2025/03/19 12:07:56.966150 juicefs[62924] <INFO>: Volume is formatted as { "Name": "myjfs", "UUID": "1568ee2a-dc4c-4a0e-9788-be0490776dda", "Storage": "file", "Bucket": "/var/jfs/", "BlockSize": 4096, "Compression": "none", "EncryptAlgo": "aes256gcm-rsa", "TrashDays": 1, "MetaVersion": 1, "MinClientVersion": "1.1.0-A", "DirStats": true, "EnableACL": false } [format.go:521] Then mount locally using the following commands:
juicefs mount sqlite3:///tmp/jfs.db ~/jfs # Foreground mount, auto-unmounts on exit juicefs mount sqlite3:///tmp/jfs.db ~/jfs -d # Daemon mount, requires manual unmount juicefs umount ~/jfs # Unmount and exit process Data Cleanup
Clear JuiceFS metadata from PostgreSQL using:
DROP TABLE IF EXISTS jfs_acl,jfs_chunk,jfs_chunk_ref,jfs_counter,jfs_delfile,jfs_delslices,jfs_detached_node,jfs_dir_quota,jfs_dir_stats,jfs_edge,jfs_flock,jfs_node,jfs_plock,jfs_session2,jfs_setting,jfs_sustained,jfs_symlink,jfs_xattr CASCADE; Clear object storage bucket using:
mcli rm --recursive --force infra/jfs PGFS Performance Summary
Benchmark results on second-hand physical hardware:
METAURL=postgres://dbuser_meta:DBUser.Meta@:5432/meta OPTIONS=( --storage postgres --bucket :5432/meta --access-key dbuser_meta --secret-key DBUser.Meta ${METAURL} jfs ) juicefs format "${OPTIONS[@]}" juicefs mount ${METAURL} ~/jfs -d # Mount in background juicefs bench ~/jfs # Run benchmark juicefs umount ~/jfs # Unmount $ juicefs bench ~/jfs # Performance test Write big blocks: 1024/1024 [==============================================================] 178.5/s used: 5.73782533s Read big blocks: 1024/1024 [==============================================================] 31.7/s used: 32.314547037s Write small blocks: 100/100 [==============================================================] 149.2/s used: 670.127171ms Read small blocks: 100/100 [==============================================================] 543.4/s used: 184.109596ms Stat small files: 100/100 [==============================================================] 1723.4/s used: 58.087752ms Benchmark finished! BlockSize: 1.0 MiB, BigFileSize: 1.0 GiB, SmallFileSize: 128 KiB, SmallFileCount: 100, NumThreads: 1 Time used: 42.2 s, CPU: 687.2%, Memory: 179.4 MiB +------------------+------------------+---------------+ | ITEM | VALUE | COST | +------------------+------------------+---------------+ | Write big file | 178.51 MiB/s | 5.74 s/file | | Read big file | 31.69 MiB/s | 32.31 s/file | | Write small file | 149.4 files/s | 6.70 ms/file | | Read small file | 545.2 files/s | 1.83 ms/file | | Stat file | 1749.7 files/s | 0.57 ms/file | | FUSE operation | 17869 operations | 3.82 ms/op | | Update meta | 1164 operations | 1.09 ms/op | | Put object | 356 operations | 303.01 ms/op | | Get object | 256 operations | 1072.82 ms/op | | Delete object | 0 operations | 0.00 ms/op | | Write into cache | 356 operations | 2.18 ms/op | | Read from cache | 100 operations | 0.11 ms/op | +------------------+------------------+---------------+ Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.