When users choose a file system, POSIX semantic compatibility is an indispensable indicator. JuiceFS has always attached great importance to the high compatibility with the POSIX standard, and tried its best to maintain the maximum degree of POSIX compatibility while continuously improving functions and performance.
Recently, in terms of POSIX compatibility, we conducted a test on Tencent Cloud CFS, Alibaba Cloud NAS, Huawei Cloud SFS, GCP Filestore, Amazon EFS, Azure File shares, and JuiceFS, so that users can understand the compatibility performance of these mainstream file systems.
POSIX is the abbreviation of Portable Operating System Interface (Portable Operating System Interface). Simply put, it is the most widely used operating system interface specification in the field of file storage, including file storage. For more discussions on the POSIX standard, you can refer to a question and answer on Quora " What does POSIX conformance/compliance mean in the distributed systems world ?"
Test Methods
For testing the POSIX compatibility of the file system, a popular test case set is pjdfstest , which comes from FreeBSD and is also applicable to systems such as Linux.
Test Results
The test results are shown in the figure above, JuiceFS has 0 failure cases, showing the best compatibility. GCP Filestore followed with two failures; Huawei Cloud SFS, Amazon EFS and Azure File shares failed test cases that were several orders of magnitude larger than other products. For the convenience of comparison, the abscissa in the above figure uses logarithmic coordinates.
Failure Case Analysis
HUAWEI CLOUD SFS, Amazon EFS and Azure File shares' failure cases are much higher than those of other file systems in terms of total number and category, so they cannot be compared in the same chart, and will be analyzed separately later.
GCP Filestore
GCP Filestore failed a total of 2 tests, one each for the categories unlink and utimesat.
The first item is unlink/14.t in the unlink test set , and the corresponding logs are as follows.
/root/pjdfstest/tests/unlink/14.t ...........
not ok 4 - tried 'open pjdfstest_b03f52249a0c653a3f382dfe1237caa1 O_RDONLY : unlink pjdfstest_b03f52249a0c653a3f382dfe1237caa1 : fstat 0 nlink', expected 0, got 1
This test suite ( unlink/14.t ) is used to verify the behavior of a file that is deleted while open:
desc="An open file will not be immediately freed by unlink"
The operation of deleting a file actually corresponds to unlink at the system level, that is, removing the link from the file name to the corresponding inode, and decrementing the corresponding nlink value by 1. This test case is to verify this point.
# A deleted file's link count should be 0
expect 0 open ${n0} O_RDONLY : unlink ${n0} : fstat 0 nlink
The file contents are only actually deleted when the number of links (nlink) is reduced to 0 and there are no open file descriptors (fd) pointing to the file. If nlink is not properly updated, it may cause files that should have been deleted to remain on the system.
The other item is utimesat/09.t in the utimesat test set , and the corresponding logs are as follows:
/root/pjdfstest/tests/utimensat/09.t ........
not ok 5 - tried 'lstat pjdfstest_909f188e5492d41a12208f02402f8df6 mtime', expected 4294967296, got 4294967295
This test case requires support for 64-bit timestamps. GCP Filestore supports 64-bit timestamp, but it will decrease by 1 on this basis, so although it fails in this test case, it should not affect the use .
Tencent Cloud CFS
Tencent Cloud CFS failed 7 items in three categories: utimesat, symlink and unlink. We selected some important failure items for analysis and explanation.
The test log corresponding to the symlink failure case is as follows:
/root/pjdfstest/tests/symlink/03.t ..........
not ok 1 - tried 'symlink 7ea12171c487d234bef89d9d77ac8dc2929ea8ce264150140f02a77fc6dcad7c3b2b36b5ed19666f8b57ad861861c69cb63a7b23bcc58ad68e132a94c0939d5/.../... pjdfstest_57517a47d0388e0c84fa1915bf11fe4a', expected 0, got EINVAL
not ok 2 - tried 'unlink pjdfstest_57517a47d0388e0c84fa1915bf11fe4a', expected 0, got ENOENT
Failed 2/6 subtests
This test set ( symlink/03.t ) is used to test the behavior of symblink when the path exceeds PATH_MAX length.
desc="symlink returns ENAMETOOLONG if an entire length of either path name exceeded {PATH_MAX} characters"
The corresponding code of the failed use case is as follows:
n0=`namegen`nx=`dirgen_max`nxx="${nx}x"
mkdir -p "${nx%/*}"
expect 0 symlink ${nx} ${n0}
expect 0 unlink ${n0}
This test case is to create PATH_MAXa symbolic link of length (including the trailing 0), but it fails to show that a symbolic link PATH_MAXof .
Alibaba Cloud NAS
Alibaba Cloud NAS failed several test cases on chmod, utimesat, unlink.
In the test set of chmod chmod/12.t , Alibaba Cloud NAS failed the following items
/root/pjdfstest/tests/chmod/12.t ............
not ok 3 - tried '-u 65534 -g 65534 open pjdfstest_db85e6a66130518db172a8b6ce6d53da O_WRONLY : write 0 x : fstat 0 mode', expected 0777, got 04777
not ok 4 - tried 'stat pjdfstest_db85e6a66130518db172a8b6ce6d53da mode', expected 0777, got 04777
not ok 7 - tried '-u 65534 -g 65534 open pjdfstest_db85e6a66130518db172a8b6ce6d53da O_RDWR : write 0 x : fstat 0 mode', expected 0777, got 02777
not ok 8 - tried 'stat pjdfstest_db85e6a66130518db172a8b6ce6d53da mode', expected 0777, got 02777
not ok 11 - tried '-u 65534 -g 65534 open pjdfstest_db85e6a66130518db172a8b6ce6d53da O_RDWR : write 0 x : fstat 0 mode', expected 0777, got 06777
not ok 12 - tried 'stat pjdfstest_db85e6a66130518db172a8b6ce6d53da mode', expected 0777, got 06777
Failed 6/14 subtests
This test set ( chmod/12.t ) is used to test the behavior of SUID/SGID bits
desc="verify SUID/SGID bit behaviour"
We select the 11th and 12th test cases to explain in detail, and cover these two permission bits at the same time
# Check whether writing to the file by non-owner clears the SUID+SGID.
expect 0 create ${n0} 06777
expect 0777 -u 65534 -g 65534 open ${n0} O_RDWR : write 0 x : fstat 0 mode
expect 0777 stat ${n0} mode
expect 0 unlink ${n0}
Here, we first create the target file with the authority of 06777, then modify the file content, and check whether the SUID and SGID are cleared correctly. The 777 in the file permission is familiar to everyone. It corresponds to the rwx of owner, group and other, which means readable, writable and executable. The leading 0 indicates an octal number.
The second digit 6 needs to be explained emphatically. This octet (octet) represents a special permission bit, and the first two digits correspond to setuid/setgid (or SUID/SGID), which can be applied to executable files and public directories. When this permission bit is set, any user will run the file as owner (or group). This special attribute allows a user to gain access to files and directories normally only available to the owner. For example, the passwd command sets the setuid authority, which allows ordinary users to modify the password, because the file that saves the password is only allowed to be accessed by root, and the user cannot directly modify it.
The starting point of the design of setuid/setgid is to provide a way for users to access restricted files (not owned by the current user) in a limited way (specifying executable files). Therefore, when the file is modified by a non-owner, this permission bit should be automatically cleared to prevent users from obtaining other permissions through this method.
From the test results, we can see that in the Alibaba Cloud NAS, when the file is modified by a non-owner, the setuid/setgid are not cleared, so that the user can actually perform any operation as the owner by modifying the file content, which will be a problem. safety hazard .
Further reading: Special File Permissions (setuid, setgid and Sticky Bit) (System Administration Guide: Security Services)
Huawei Cloud SFS and Amazon EFS
HUAWEI CLOUD SFS and Amazon Elastic File System (EFS) failed similarly in the pjdfstest test. The failure rate is about 21%, and the failure cases cover almost all categories.
Both support NFS mounting, but support for NFS features is incomplete. For example, block devices and character devices are not supported, which directly leads to the failure of a large number of test cases in pjdfstest. After excluding these two types of files, there are still hundreds of different types of failures, so it must be used with caution in complex scenarios .
Azure File shares
The failure rate of Azure File shares reached 62%, which shows that some basic POSIX scenarios may have incompatibility problems. For example, the default permission of Azure File shares files and folders is 0777, the owner is root, and modification is not supported, that is to say, there is no permission restriction. In addition, Azure File shares do not support hard links and symbolic links. So using Azure File Shares requires careful testing and careful consideration of whether the scenario is simple enough .
Summarize
- JuiceFS performed best in terms of compatibility and passed all the test items.
- Google Filestore was next. There were two types of failures, one of which did not affect actual use.
- Tencent Cloud CFS is similar to Alibaba Cloud NAS, with 7-8 items both failing.
- HUAWEI CLOUD SFS, Amazon EFS, and Azure File Shares have poor compatibility, and a large number of compatibility tests have failed, including several test cases with serious security risks. It is recommended to do a security assessment before use.
If you are helpful, please pay attention to our project Juicedata/JuiceFS ! (0ᴗ0✿)