JuiceFS 是一个基于Redis和对象存储的POSIX文件系统。
介绍
- 为云环境设计，兼容 POSIX、HDFS 和 S3 协议的分布式文件系统，它全面接管对象存储，可以向使用本地磁盘一样使用 JuiceFS 存储各种文件或为应用提供存储空间。
- JuiceFS 可以为用户提供更高性能的云存储使用体验
- 据说超越本地磁盘性能和无限的存储空间
特点：
- 分布式存储系统
- 企业级开源软件
- 云原生
- 丰富的存储定义
- 支持容量弹性伸缩
- 像本地磁盘一样使用
- 应用开发简单，不需要 SDK
- 最简单最适合 Kubernetes PV
- 完全兼容 HDFS，使用方法也一样
- 还能通过 S3 API 访问 JuiceFS
使用场景：
- 大数据分析
- 机器学习
- 容器平台
- CICD
- 数据共享等
JuiceFS 的组成：
JuiceFS Client 客户端，用来创建、挂载 FS 等Metadata Engine 元数据，通过 redis 提供
Object Storage Service
安装
安装 JuiceFS 客户端
curl -sSL https://d.juicefs.com/install | sh -
# or
wget https://github.com/juicedata/juicefs/releases/download/v1.0.3/juicefs-1.0.3-linux-amd64.tar.gz
tar -zxvf juicefs-1.0.3-linux-amd64.tar.gz
mv juicefs /usr/local/bin/
$ juicefs --help
NAME:
juicefs - A POSIX file system built on Redis and object storage.
USAGE:
juicefs [global options] command [command options] [arguments...]
VERSION:
1.0.3+2022-12-27.e4bf15a
COMMANDS:
ADMIN:
format Format a volume
config Change configuration of a volume
destroy Destroy an existing volume
gc Garbage collector of objects in data storage
fsck Check consistency of a volume
dump Dump metadata into a JSON file
load Load metadata from a previously dumped JSON file
version Show version
INSPECTOR:
status Show status of a volume
stats Show real time performance statistics of JuiceFS
profile Show profiling of operations completed in JuiceFS
info Show internal information of a path or inode
SERVICE:
mount Mount a volume
umount Unmount a volume
gateway Start an S3-compatible gateway
webdav Start a WebDAV server
TOOL:
bench Run benchmarks on a path
objbench Run benchmarks on an object storage
warmup Build cache for target directories/files
rmr Remove directories recursively
sync Sync between two storages
GLOBAL OPTIONS:
--verbose, --debug, -v enable debug log (default: false)
--quiet, -q show warning and errors only (default: false)
--trace enable trace log (default: false)
--no-agent disable pprof (:6060) and gops (:6070) agent (default: false)
--pyroscope value pyroscope address
--no-color disable colors (default: false)
--help, -h show help (default: false)
--version, -V print version only (default: false)
COPYRIGHT:
Apache License 2.0
安装 redis
参考：Redis 安装和常用命令 # Docker 单节点
链接信息：
redis://:123456@127.0.0.1:6379/1
安装 minio
参考：MinIO 介绍
ACCESS_KEY="69EI0OYVOvPOc8eA"
SECRET_KEY="gUVIP5p1WGfzaFdhGkR5D03UrgmANSgN"
格式化 JuiceFS 卷
Format a volume，即创建 JuiceFS 卷，名称为 jfs-volume
$ juicefs format \
--storage s3 \
--bucket "http://172.20.0.142:9000/test-jfs" \
--access-key "69EI0OYVOvPOc8eA" \
--secret-key "gUVIP5p1WGfzaFdhGkR5D03UrgmANSgN" \
redis://:123456@127.0.0.1:6379/1 \
jfs-volume
2023/03/19 19:14:12.654787 juicefs[54823] <INFO>: Meta address: redis://:****@127.0.0.1:6379/1 [interface.go:402]
2023/03/19 19:14:12.662180 juicefs[54823] <INFO>: Ping redis: 1.167964ms [redis.go:2888]
2023/03/19 19:14:12.664400 juicefs[54823] <INFO>: Data use s3://test-jfs/jfs-volume/ [format.go:429]
2023/03/19 19:14:12.696616 juicefs[54823] <INFO>: Volume is formatted as {
"Name": "jfs-volume",
"UUID": "45457a3e-0b07-4bb5-a1e4-5c63d0c2acfb",
"Storage": "s3",
"Bucket": "http://172.20.0.142:9000/test-jfs",
"AccessKey": "69EI0OYVOvPOc8eA",
"SecretKey": "removed",
"BlockSize": 4096,
"Compression": "none",
"KeyEncrypted": true,
"TrashDays": 1,
"MetaVersion": 1
} [format.go:466]
挂载 JuiceFS 卷
$ juicefs mount -d redis://:123456@127.0.0.1:6379/1 /mnt/jfs-volume -d
$ juicefs mount -d redis://:123456@127.0.0.1:6379/1 /mnt/jfs-volume
2023/03/19 19:16:05.621318 juicefs[54835] <INFO>: Meta address: redis://:****@127.0.0.1:6379/1 [interface.go:402]
2023/03/19 19:16:05.628705 juicefs[54835] <INFO>: Ping redis: 800.106µs [redis.go:2888]
2023/03/19 19:16:05.631094 juicefs[54835] <INFO>: Data use s3://test-jfs/jfs-volume/ [mount.go:428]
2023/03/19 19:16:05.633517 juicefs[54835] <INFO>: Disk cache (/var/jfsCache/45457a3e-0b07-4bb5-a1e4-5c63d0c2acfb/): capacity (102400 MB), free ratio (10%), max pending pages (15) [disk_cache.go:94]
2023/03/19 19:16:06.142359 juicefs[54835] <INFO>: OK, jfs-volume is ready at /mnt/jfs-volume [mount_unix.go:45]
$ df -Th
JuiceFS:jfs-volume fuse.juicefs 1.0P 0 1.0P 0% /mnt/jfs-volume
$ echo 123 > abc.md
$ ls -lhart
total 6.0K
dr-xr-xr-x 2 root root 0 .trash
-r--r--r-- 1 root root 0 .stats
-r-------- 1 root root 1.2K .config
-r-------- 1 root root 0 .accesslog
drwxr-xr-x 3 root root 24 ..
drwxrwxrwx 2 root root 4.0K .
-rw-r--r-- 1 root root 4 abc.md
test-jfs/jfs-volume/juicefs_uuid
test-jfs/jfs-volume/chunks/0/0/1_0_4
$ redis-cli
127.0.0.1:6379> auth 123456
OK
127.0.0.1:6379> select 1
OK
127.0.0.1:6379[1]> keys *
1) "lastCleanupFiles"
2) "nextsession"
3) "nextinode"
4) "totalInodes"
5) "i2"
6) "allSessions"
7) "d1"
8) "i1"
9) "usedSpace"
10) "lastCleanupSessions"
11) "nextchunk"
12) "sessionInfos"
13) "c2_0"
14) "i9223372032828243968"
15) "setting"
127.0.0.1:6379[1]>
卸载
juicefs umount /mnt/jfs-volume
fstab 自动挂载
$ cp /usr/local/bin/juicefs /sbin/mount.juicefs
$ cat /etc/fstab
...
redis://:123456@127.0.0.1:6379/1 /mnt/jfs-volume juicefs _netdev,cache-size=10240 0 0
说明：
$ mount -a
2023/03/19 20:23:11.658140 juicefs[55312] <INFO>: Meta address: redis://:****@127.0.0.1:6379/1 [interface.go:402]
2023/03/19 20:23:11.664320 juicefs[55312] <INFO>: Ping redis: 890.216µs [redis.go:2888]
2023/03/19 20:23:11.668049 juicefs[55312] <INFO>: Data use s3://test-jfs/jfs-volume/ [mount.go:428]
2023/03/19 20:23:11.669834 juicefs[55312] <INFO>: Disk cache (/var/jfsCache/45457a3e-0b07-4bb5-a1e4-5c63d0c2acfb/): capacity (10240 MB), free ratio (10%), max pending pages (15) [disk_cache.go:94]
2023/03/19 20:23:12.179200 juicefs[55312] <INFO>: OK, jfs-volume is ready at /mnt/jfs-volume [mount_unix.go:45]
s3 gateway
源码在 https://github.com/juicedata/minio，基于 minio 实现了 JuiceFS 使用作为其网关的后端之一，并支持MinIO网关的全部功能，如多用户管理，同时使用JuiceFS作为后端
kubernetes 使用
JuiceFs CSI Driver PVC 支持两种使用方式：
- 静态配置
- 动态配置，通过 PVC + StorageClass 实现
Sidecar 模式
-
优点：故障排查简单
-
缺点：资源开销大
-
Controller Service
- 以 PV id 为名在 JuiceFS 文件系统中创建子目录
- 向 ApiServer 注册 webhook，在 Pod 中注入 juiceFs 客户端的 sidecar 容器
Mount Pod 模式
- 优点：资源开销小，相同的 PVC 的 Pod 共享一个 JuiceFS 客户端
- 缺点：故障排查困难