The reference implementation of the Linux FUSE (Filesystem in Userspace) interface

Related tags

Filesystem libfuse
Overview

libfuse

About

FUSE (Filesystem in Userspace) is an interface for userspace programs to export a filesystem to the Linux kernel. The FUSE project consists of two components: the fuse kernel module (maintained in the regular kernel repositories) and the libfuse userspace library (maintained in this repository). libfuse provides the reference implementation for communicating with the FUSE kernel module.

A FUSE file system is typically implemented as a standalone application that links with libfuse. libfuse provides functions to mount the file system, unmount it, read requests from the kernel, and send responses back. libfuse offers two APIs: a "high-level", synchronous API, and a "low-level" asynchronous API. In both cases, incoming requests from the kernel are passed to the main program using callbacks. When using the high-level API, the callbacks may work with file names and paths instead of inodes, and processing of a request finishes when the callback function returns. When using the low-level API, the callbacks must work with inodes and responses must be sent explicitly using a separate set of API functions.

Development Status

libfuse is shipped by all major Linux distributions and has been in production use across a wide range of systems for many years. However, at present libfuse does not have any active, regular contributors. The current maintainer continues to apply pull requests and makes regular releases, but unfortunately has no capacity to do any development beyond addressing high-impact issues. When reporting bugs, please understand that unless you are including a pull request or are reporting a critical issue, you will probably not get a response. If you are using libfuse, please consider contributing to the project.

Supported Platforms

  • Linux (fully)
  • BSD (mostly/best-effort)
  • For OS-X, please use OSXFUSE

Installation

You can download libfuse from https://github.com/libfuse/libfuse/releases. To build and install, you must use Meson and Ninja. After extracting the libfuse tarball, create a (temporary) build directory and run Meson:

$ mkdir build; cd build
$ meson ..

Normally, the default build options will work fine. If you nevertheless want to adjust them, you can do so with the meson configure command:

$ meson configure # list options
$ meson configure -D disable-mtab=true # set an option

To build, test, and install libfuse, you then use Ninja:

$ ninja
$ sudo python3 -m pytest test/
$ sudo ninja install

Running the tests requires the py.test Python module. Instead of running the tests as root, the majority of tests can also be run as a regular user if util/fusermount3 is made setuid root first:

$ sudo chown root:root util/fusermount3
$ sudo chmod 4755 util/fusermount3
$ python3 -m pytest test/

Security implications

The fusermount3 program is installed setuid root. This is done to allow normal users to mount their own filesystem implementations.

To limit the harm that malicious users can do this way, fusermount3 enforces the following limitations:

  • The user can only mount on a mountpoint for which they have write permission

  • The mountpoint must not be a sticky directory which isn't owned by the user (like /tmp usually is)

  • No other user (including root) can access the contents of the mounted filesystem (though this can be relaxed by allowing the use of the allow_other and allow_root mount options in /etc/fuse.conf)

If you intend to use the allow_other mount options, be aware that FUSE has an unresolved security bug: if the default_permissions mount option is not used, the results of the first permission check performed by the file system for a directory entry will be re-used for subsequent accesses as long as the inode of the accessed entry is present in the kernel cache - even if the permissions have since changed, and even if the subsequent access is made by a different user. This is of little concern if the filesystem is accessible only to the mounting user (which has full access to the filesystem anyway), but becomes a security issue when other users are allowed to access the filesystem (since they can exploit this to perform operations on the filesystem that they do not actually have permissions for).

This bug needs to be fixed in the Linux kernel and has been known since 2006 but unfortunately no fix has been applied yet. If you depend on correct permission handling for FUSE file systems, the only workaround is to use default_permissions (which does not currently support ACLs), or to completely disable caching of directory entry attributes.

Building your own filesystem

FUSE comes with several example file systems in the example directory. For example, the passthrough examples mirror the contents of the root directory under the mountpoint. Start from there and adapt the code!

The documentation of the API functions and necessary callbacks is mostly contained in the files include/fuse.h (for the high-level API) and include/fuse_lowlevel.h (for the low-level API). An autogenerated html version of the API is available in the doc/html directory and at http://libfuse.github.io/doxygen.

Getting Help

If you need help, please ask on the [email protected] mailing list (subscribe at https://lists.sourceforge.net/lists/listinfo/fuse-devel).

Please report any bugs on the GitHub issue tracker at https://github.com/libfuse/libfuse/issues.

Comments
  • Allow passing `/dev/fuse` file descriptor from parent process

    Allow passing `/dev/fuse` file descriptor from parent process

    This adds a new "pre-mounted" mode of operation in which the FUSE file system helper is launched after the /dev/fuse file descriptor has been opened opened and the file system been mounted by a suitably privileged process.

    Pre-mounted mode is requested by passing an empty string as mountpoint, which reflects the fact that the FUSE helper doesn't actually perform or trigger any mounting itself.

    The main benefit of pre-mounted mode is that no privileged operations need to be performed by the file system implementation itself directly or indirectly, so the FUSE process can run with zero privilege. Moreover, mechanisms like securebits and no_new_privs can be used to prevent subprocesses from re-acquiring privilege, which further helps reduce risk for the case the FUSE helper gets exploited by a malicious file system.

    Below is an example that illustrates this. Note that I'm using shell for presentation purposes, the assumption is that there's a privileged daemon that handles mounting and spawning the FUSE helper in a suitable sandbox.

    Make binaries and libs executable by anyone

    $ chmod o+rx build/lib/libfuse3.so* example/hello $ export LD_LIBRARY_PATH=$PWD/build/lib

    example/hello can mount successfully with privilege

    $ sudo sh -c "LD_LIBRARY_PATH=build/lib ./example/hello /mnt/tmp" $ sudo cat /mnt/tmp/hello Hello World! $ sudo umount /mnt/tmp

    example/hello fails to mount without privilege

    $ sudo capsh --drop=all --secbits=0x3f -- -c 'LD_LIBRARY_PATH=build/lib ./example/hello -f /mnt/tmp' fusermount3: mount failed: Operation not permitted

    Pre-mounting allows example/hello to work without privilege

    $ sudo sh -c ' exec 0<>/dev/fuse mount -i -o nodev,nosuid,noexec,fd=0,rootmode=40000,user_id=0,group_id=0 -t fuse hello /mnt/tmp capsh --drop=all --secbits=0x3f -- -c "LD_LIBRARY_PATH=build/lib example/hello -f """ ' & [1] 55491 $ sudo cat /mnt/tmp/hello Hello World! $ sudo umount /mnt/tmp [1]+ Done sudo sh -c ' exec 0<>/dev/fuse mount -i -o nodev,nosuid,noexec,fd=0,rootmode=40000,user_id=0,group_id=0 -t fuse hello /mnt/tmp capsh --drop=all --secbits=0x3f -- -c "LD_LIBRARY_PATH=build/lib example/hello -f """ '

    opened by saittam 41
  • Deadlock in get_path2()

    Deadlock in get_path2()

    We have some testcases that prove this comment correct 😄

    /* FIXME: locking two paths needs deadlock checking */

    (https://github.com/libfuse/libfuse/blob/master/lib/fuse.c#L1258)

    Considering tackling this since I can reproduce the problem. Did you have any thoughts as to how a fix should be implemented?

    bug 
    opened by bnaylor 32
  • fuse-loop/fuse_do_work: Avoid lots of thread creations/destructions

    fuse-loop/fuse_do_work: Avoid lots of thread creations/destructions

    On benchmarking metadata operations with a single threaded bonnie++ and "max_idle_threads" limited to 1, 'top' was showing suspicious 160% cpu usage. Profiling the system with flame graphs showed that an astonishing amount of CPU time was spent in thread creation and destruction.

    After verifying the code it turned out that fuse_do_work() was creating a new thread every time all existing idle threads were already busy. And then just a few lines later after processing the current request it noticed that it had created too many threads and destructed the current thread. I.e. there was a thread creation/destruction ping-pong.

    Code is changed to only create new threads if the max number of threads is not reached.

    Furthermore, thread destruction is disabled, as creation/destruction is expensive in general.

    With this change cpu usage of passthrough_hp went from ~160% to ~80% (with different values of max_idle_threads). And bonnie values got approximately faster by 90%. This is a with single threaded bonnie++ bonnie++ -x 4 -q -s0 -d -n 30:1:1:10 -r 0

    Without this patch, using the default max_idle_threads=10 and just a single bonnie++ the thread creation/destruction code path is not triggered. Just one libfuse and one application thread is just a corner case - the requirement for the issue was just n-application-threads >= max_idle_threads.

    Signed-off-by: Bernd Schubert [email protected]

    opened by aakefbs 27
  • Advise file system developers not to use file handles in nonportable ways

    Advise file system developers not to use file handles in nonportable ways

    Background

    The FUSE protocol allows a server to assign a different filehandle each and every time that a file is opened. Multiple filehandles can be concurrently valid for the same file. Operations like FUSE_READ and FUSE_WRITE are supposed to include the file handle associated with the particular file descriptor that initiated the operation. However, that isn't possible in all cases. The FUSE_WRITE_CACHE bit indicates whether the file handle is valid for a FUSE_WRITE operation, and the FATTR_FH flag indicates whether it's valid for a FUSE_SETATTR operation.

    Problem

    There are other cases, too, when the file handle can be invalid. In fact, the entire concept of a file handle is really a Linuxism. Other operating systems enforce a distinction between the file layer and the vnode layer. That's why OSX, FreeBSD, OpenBSD, and Illumos can almost never guarantee that they're sending the right file handle for any operation. NetBSD takes a more extreme position; it doesn't allow multiple concurrent fuse file handles for a single file (which leads to other problems). But since operations other than FUSE_WRITE and FUSE_SETATTR don't have anything analogous to the FUSE_WRITE_CACHE or FATTR_FH bits, there's no way for those operating systems to indicate that the file handle is guessed.

    Mitigating Circumstances

    The impact of this bug is small simply because few if any fuse servers care about the accuracy of fuse file handles. I surveyed 42 real-world FUSE filesystems* and none of them ever check the writepage bit. Only one of them (catfs) checks FATTR_FH, and it can deal with the absence of that flag. I can't find any evidence that any FUSE filesystem actually wants to assign distinct file handles on each concurrent FUSE_OPEN and requires the FUSE client to keep them straight.

    Solution

    Ideally the FUSE protocol would require servers to assign the same file handle for every concurrent open of a single file. Unfortunately, that's not the current situation and such a change would be backwards-incompatible. The second best solution would be to update the kernel protocol to include some equivalent of the FUSE_WRITE_CACHE bit for every operation that includes a file handle. However, that would require all clients and servers alike to update their kernel API version, so uptake would be extremely slow. The path of least resistance would simply be to document the problem. I recommend the following:

    • Advise FUSE server developers to assign the same file handle for every concurrent open of a single file.
    • Rename the writepage bit to fh_invalid, and add a #define for backwards-compatibility
    • Patch libfuse to unconditionally set fh_invalid for most commands on non-Linux operating systems.

    If the maintainers agree, I'll prepare a PR.

    cc @cemeyer @kendmerry

    *: afuse, chironfs, cryptofs, curlftpfs, encfs, ext2, ext4fuse, funionfs, fusepak, gitfs, gstfs, gunzip, hfsfuse, httpfs, ifuse, ltfs, mhddfs, mp3fs, ntfs, ntfs-compression, pod, rar2fs, s3backer, s3fs, simple-mtpfs, smbnetfs, sqlfs, squashfuse, sshfs, unionfs, wdfs, webdavfs, wikipediafs, zip, MooseFS, CephFS, RedoxFS, gcsf, catfs, qcow2-fuse, shotwellvfs, and one closed-source file system.

    needs-info 
    opened by asomers 20
  • passthrough_ll update

    passthrough_ll update

    This fixes a couple of bugs in the passthrough_ll example filesystem and adds support for lots of missing operations. Options are added to configure a base path and to control caching.

    opened by szmi 20
  • write() after release()

    write() after release()

    This is one of the files being rsynced, roughly in the middle of the overall process. I see following in the debug output:

    unique: 459221, opcode: CREATE (35), nodeid: 4358, insize: 103, pid: 8654
    create flags: 0x80c1 /mega/whatever-dir/whatever-file 0100600 umask=0000
       create[0] flags: 0x80c1 /mega/whatever-dir/whatever-file
    getattr /mega/whatever-dir/whatever-file
       NODEID: 15944
       unique: 459221, success, outsize: 160
    unique: 459222, opcode: FLUSH (25), nodeid: 15944, insize: 64, pid: 8654
    flush[0]
       unique: 459222, success, outsize: 16
    unique: 459223, opcode: RELEASE (18), nodeid: 15944, insize: 64, pid: 0
    unique: 459224, opcode: OPEN (14), nodeid: 15944, insize: 48, pid: 8654
    open flags: 0x8002 /mega/whatever-dir/whatever-file
    release[0] flags: 0x8001
       open[0] flags: 0x8002 /mega/whatever-dir/whatever-file
       unique: 459224, success, outsize: 32
       unique: 459223, success, outsize: 16
    unique: 459225, opcode: SETATTR (4), nodeid: 15944, insize: 128, pid: 8534
    getattr /mega/whatever-dir/whatever-file
       unique: 459225, success, outsize: 120
    unique: 459226, opcode: WRITE (16), nodeid: 15944, insize: 88, pid: 11192
    write[0] 8 bytes to 0 flags: 0x8002
    

    So it seems release is executed after open, and then it tries to write to released file.
    Yes, the program is multithreaded. Is there anything to be done? What are usual workarounds for this?

    bug needs-info 
    opened by Kaned1as 19
  • Pass fuse_file_info in chown, chmod, utimensat, etc

    Pass fuse_file_info in chown, chmod, utimensat, etc

    Hi,

    I would like to get the file handle that was set in open when fchown, fchmod, and futimens are called. This is necessary to distinguish between syscalls, such as, chown and fchown, which make a difference when the underlying implementantion is inode-based and not path-based.

    Is it possible to create new FUSE handlers fchown, fchmod, futimens, that receive the fuse_file_info structure, which contains the file handle created in open?

    Thanks, Jose

    enhancement 
    opened by jabolopes 19
  • Improve/define FreeBSD support

    Improve/define FreeBSD support

    Meson wants to use -ldl but this is not required on FreeBSD, that functionality is built into libc.

    [email protected]:~/libfuse/build % meson ..
    WARNING: You are using 'US-ASCII' which is not a a Unicode-compatible locale.
    WARNING: You might see errors if you use UTF-8 strings as filenames, as strings, or as file contents.
    WARNING: Please switch to a UTF-8 locale for your platform.
    The Meson build system
    Version: 0.40.1
    Source dir: /home/vagrant/libfuse
    Build dir: /home/vagrant/libfuse/build
    Build type: native build
    Project name: libfuse3
    Native c compiler: cc (clang 3.4.1)
    Build machine cpu family: x86_64
    Build machine cpu: x86_64
    Checking for function "fork": YES
    Checking for function "fstatat": YES
    Checking for function "openat": YES
    Checking for function "readlinkat": YES
    Checking for function "pipe2": YES
    Checking for function "splice": NO
    Checking for function "vmsplice": NO
    Checking for function "posix_fallocate": YES
    Checking for function "fdatasync": NO
    Checking for function "utimensat": YES
    Checking for function "setxattr": NO
    Checking for function "iconv": YES
    Checking whether type "struct stat" has member "st_atim": YES
    Checking whether type "struct stat" has member "st_atimespec": YES
    Configuring config.h using configuration
    Dependency threads found: YES
    
    Meson encountered an error in file lib/meson.build, line 20, column 0:
    C library 'dl' not found
    
    bug 
    opened by bnaylor 17
  • 'll' operations fail when writeback cache is enabled

    'll' operations fail when writeback cache is enabled

    Using FUSE3.2.6 to create file system, create a file on the mount directory, the file size is 1G. Format the file and mount it to the directory. After a few seconds, see the directory through the "ll" command.

    #df -h 
    fuse_hello1          1.0G  1.0G     0 100% /root/libfuse-fuse-3.2.6/example/fileDir
    
    #ll -h /root/libfuse-fuse-3.2.6/example/fileDir
    total 512K
    -rw-r--r-- 1 root root 1.0G Nov 16 15:46 file
    
    #mkfs.xfs /root/libfuse-fuse-3.2.6/example/fileDir/file
    #mount /root/libfuse-fuse-3.2.6/example/fileDir/file /root/libfuse-fuse-3.2.6/example/mountDir
    
    • When writeback cache is enabled, the error occurs.
    #ll /root/libfuse-fuse-3.2.6/example/mountDir
    ls: cannot access mountDir: Input/output error
    
    • When writeback cache is disabled, successful.
    # ll mountDir/
    total 0
    

    enable writeback_cache:conn->want |= FUSE_CAP_WRITEBACK_CACHE; Incorrect usage or other reasons?

    needs-info 
    opened by Roay 16
  • Installing failure on Ubuntu 16.04

    Installing failure on Ubuntu 16.04

    On stock Ubuntu 16.04, installing building/installing with meson does not produce working sshfs for example. Building and installing proceeds fine (after manually installing latest meson). However, sshfs --version says:

    shfs: error while loading shared libraries: libfuse3.so.3: cannot open shared object file: No such file or directory

    This happens because installation has placed libfuse to /usr/local/lib/x86_64-linux-gnu/ and that path is not searched for libraries in stock Ubuntu 16.04 system.

    When I built libfuse/sshfs with autotools, this issue did not happen. But autotools support has been dropped so this is no longer an option. I am not 100% sure if this issue is in libfuse or meson or ninja or Ubuntu. But I sure would like to get it fixed.

    bug needs-info 
    opened by tksuoran 16
  • Init script is installed in /usr/local/etc, not /etc

    Init script is installed in /usr/local/etc, not /etc

    Hi all, I'm trying to install fuse and have followed the steps advised: meson ninja sudo python3 -m pytest test/ (all passed) but I receive an error during the ninja install:

    sudo ~/divers_apps/ninja/ninja install returns:

    [0/1] Installing files. Installing lib/libfuse3.so.3.0.0 to /usr/local/lib/x86_64-linux-gnu/libfuse3.so.3.0.0 Installing util/fusermount3 to /usr/local/bin/fusermount3 Installing util/mount.fuse3 to /usr/local/sbin/mount.fuse3 Installing fuse.h to /usr/local/include/fuse3 Installing fuse_common.h to /usr/local/include/fuse3 Installing fuse_lowlevel.h to /usr/local/include/fuse3 Installing fuse_opt.h to /usr/local/include/fuse3 Installing cuse_lowlevel.h to /usr/local/include/fuse3 Installing /home/tom/divers_apps/fuse/fuse-3.0.2/doc/fusermount3.1 to /usr/local/share/man/man1 Installing /home/tom/divers_apps/fuse/fuse-3.0.2/doc/mount.fuse.8 to /usr/local/share/man/man8 Installing /home/tom/divers_apps/fuse/build/meson-private/fuse3.pc to /usr/local/lib/x86_64-linux-gnu/pkgconfig Running custom install script '/home/tom/divers_apps/fuse/fuse-3.0.2/util/install_helper.sh etc bin lib/x86_64-linux-gnu' update-rc.d: error: unable to read /etc/init.d/fuse3

    Would you by chance have any idea? Thanks!

    bug 
    opened by TomBugnon 16
  • Do something about PR_SET_IO_FLUSHER

    Do something about PR_SET_IO_FLUSHER

    From prctl(2):

       PR_SET_IO_FLUSHER (since Linux 5.6)
              If a user process is involved in the block layer or filesystem I/O  path,  and
              can allocate memory while processing I/O requests it must set arg2 to 1.  This
              will put the process in the IO_FLUSHER state, which allows it  special  treat‐
              ment  to make progress when allocating memory. [..]
    
              The calling process must have the CAP_SYS_RESOURCE capability.[...]
    
              Examples  of  IO_FLUSHER  applications are FUSE daemons, SCSI device emulation
              daemons, and daemons that perform error handling like multipath path  recovery
              applications.
    

    As I understand, if this flag is not set and the kernel needs to reclaim memory that is in-use by a FUSE filesystem, this can led to a deadlock.

    libfuse currently does not set this flag. Maybe we should attempt to set it? But then, this seems like an unusual thing for a library to do and requires CAP_SYS_RESOURCE (which we can't ensure because it's a property of the binary that links to libfuse).

    Maybe we should check if this is set and print a warning if not? Filesystems that know that they don't require this could explicitly disable the warning somehow.

    Or just document it somewhere within libfuse?

    enhancement 
    opened by Nikratio 2
  • Remove partial locking of paths

    Remove partial locking of paths

    As described in https://github.com/libfuse/libfuse/issues/695, partial locking of paths can cause a deadlock. Partial locking was added to prevent starvation, but it's unclear what specific cases of starvation were of concern. As far as I was able to determine, since we support reader locks, this means that to starve the queue element, we'd need a constant stream of queued requests that lock the path for write. Write locks are used when the element is being removed, so this stream of requests that starve the rename or lock operations seems unlikely.

    opened by spectral54 8
  • Fix build failure with uclibc using HAVE_SYMVER_ATTRIBUTE macro and

    Fix build failure with uclibc using HAVE_SYMVER_ATTRIBUTE macro and

    moving handling to helper.c file

    Building with uclibc leads to failure:

    FAILED: lib/libfuse3.so.3.12.0.p/helper.c.o.
    /home/giuliobenetti/git/upstream/test-libfuse3/bootlin-armv5-uclibc/host/bin/arm-linux-gcc -Ilib/libf
    In file included from ../lib/fuse_i.h:10,
                     from ../lib/helper.c:14:
    ../include/fuse_lowlevel.h:1921:40: error: redefinition of ‘fuse_parse_cmdline_312’
     1921 | #define fuse_parse_cmdline(args, opts) fuse_parse_cmdline_312(args, opts)
          |                                        ^~~~~~~~~~~~~~~~~~~~~~
    ../lib/helper.c:258:5: note: in expansion of macro ‘fuse_parse_cmdline’
      258 | int fuse_parse_cmdline(struct fuse_args *args,
          |     ^~~~~~~~~~~~~~~~~~
    ../lib/helper.c:208:5: note: previous definition of ‘fuse_parse_cmdline_312’ was here
      208 | int fuse_parse_cmdline_312(struct fuse_args *args,
    

    This happens because uclibc, depending on version, can support symver, so if symver is supported and uclibc is used function fuse_parse_cmdline_312() will be defined twice:

    1. the function itself with symver
    2. fuse_parse_cmdline() as the #define of fuse_parse_cmdline_312() and its prototype This leads to have the redefinition of ‘fuse_parse_cmdline_312’.

    To solve this let's check against HAVE_SYMVER_ATTRIBUTE instead of UCLIBC and APPLE and move all the checks of FUSE_USE_VERSION to helper.c file instead of fuse_lowlevel.h with #define fuse_parse_cmdline that defines fuse_parse_cmdline_312() twice in helper.c leading to the error above.

    Signed-off-by: Giulio Benetti [email protected]

    opened by giuliobenetti 16
  • Deadlock with rmdir and rename

    Deadlock with rmdir and rename

    The partial locking support in queue_element_wakeup can cause a deadlock under the following situation:

    • something triggers a two-path queue entry (such as for a rename operation) to be the first entry and to partially lock
    • an rmdir for a common grandparent directory of the two paths in the queue arrives

    In this situation, the 'rmdir' attempts to acquire a write lock on the directory it wants to remove. This does not succeed, since the reader lock is already held by the partially-locked rename, so the directory is instead marked as "writer waiting" by adding TREELOCK_WAIT_OFFSET (INT_MIN) to the treelock. Every time afterward that the queue attempts to process sees:

    • first queue element (rename):
      • one of the paths is already locked, so it's ignored
      • the other path tries to reader lock the directory that rmdir wants to remove, but fails because there's a writer waiting
      • (no forward progress on this queue element)
    • second queue element (rmdir):
      • attempt to lock the directory, it fails due to it already being locked
      • (no forward progress on this queue element due to treelock already indicating that there's a writer waiting)

    I've been able to reproduce this reliably using a modified version of the hello.c example deadlock.patch.txt, and using the command line (in zsh, but should work in bash too):

    # Build it and run it
    $ mkdir -p /tmp/hello
    $ gcc -Wall hello.c `pkg-config fuse3 --cflags --libs` -o hello
    $ ./hello /tmp/hello
    
    # Reproduce the issue
    $ getfattr -nuser.sleep_5s /tmp/hello/rmthis/foo/file.txt &
    $ sleep 3
    $ getfattr -nuser.sleep_5s /tmp/hello/rmthis/foo/file.txt &
    $ mv /tmp/hello/rmthis/{foo,bar}/file.txt &
    $ sleep 3
    $ rmdir /tmp/hello/rmthis
    

    What this is doing is:

    1. create a situation where /rmthis, /rmthis/foo, and /rmthis/foo/file.txt all have two reader locks on them due to the getxattrs [note: on my machine only one getfattr call is necessary, but if nothing issues spurious getxattr calls for things like the security xattrs, the sleep 3 and extra getfattr should set up things to execute in the proper order, but that's untested]
    2. attempt to rename /rmthis/foo/file.txt -> /rmthis/bar/file.txt. This attempts to grab a write lock on /rmthis/foo/file.txt, fails (it's reader locked), marks /rmthis/foo/file.txt as "writer waiting" [this is not relevant to the issue], unlocks everything, adds it to the queue
    3. when some getxattr call that has /rmthis/foo/file.txt reader locked completes (either the first one we started in the background, or one of the ones for the 'security' attributes) and there's a queue, it tries to process the queue, which currently has one element in it: the rename.
      • /rmthis has only reader locks, so /rmthis/foo/file.txt fails to lock (reader lock still held by the second getxattr call), but locking /rmthis/bar/file.txt succeeds. This adds another reader lock on /rmthis (so it's now held by 2 or more readers: the second getxattr, the rename, and maybe other getxattrs).
    4. sleep 3s to try to ensure that the first getfattr has completed before attempting to do the rmdir (in case there's no extraneous reader locks and queue executions) - we need step 3 of this explanation to have happened, as partial lock acquisition only happens when an item is in the queue, not before it gets added to the queue.
    5. try to rmdir /rmthis. This attempts to grab a write lock on /rmthis, fails (open with reader lock by the second getfattr and by /rmthis/bar/file.txt, and possibly others), marks it as "writer waiting [this time that's important], and adds itself to the queue. -The queue is now two elements: {/rmthis/foo/file.txt [unlocked], /rmthis/bar/file.txt [locked]}, /rmthis
    6. the second getfattr completes, decrements the reader lock count on /rmthis, but this doesn't completely unlock it. Processes the queue: attempt to lock /rmthis/foo/file.txt (fails because /rmthis can't be locked with a reader lock due to a writer waiting), ignores already locked /rmthis/bar/file.txt. Move on to second element: attempt to lock /rmthis, fails because /rmthis/bar/file.txt has /rmthis open with a reader lock.

    Proposed fix: remove this if: https://github.com/libfuse/libfuse/blob/master/lib/fuse.c#L1136, disallowing partial locking. I have not yet understood what starvation issue partial locking is solving, so this might not be a good solution? All other possible solutions involved detecting that the queue makes no forward progress and unlocking the first element if it's partially locked and then reprocessing the queue. I'm concerned that wouldn't completely solve the issue if (a) enough other work is arriving such that the queue as a whole makes progress every time it's processed, but these specific elements do not, or (b) nothing else manages to start queue processing (because the queue is only processed when a write operation unlocks - if no mutation operation comes in that succeeds [due to the deadlock], then I'm worried the queue will never be processed to detect that it doesn't make "forward progress" and thus try the fallback of unlocking the partial lock).

    opened by spectral54 7
  • Consider exposing FUSE-specific errors from `libfuse{2,3}`

    Consider exposing FUSE-specific errors from `libfuse{2,3}`

    fuse_mount_compat25 / fuse_session_mount currently return -1 on error, but provide no other information about what went wrong. If the cause is a system error, the caller can use errno to guess what went wrong, but otherwise the caller just has to abort.

    My motivating example: if the mount options auto_unmount,allow_{root,other} are set but the user does not have user_allow_other set in their fuse.conf, the fusermount / fusermount3 binaries will print an error message to stderr but not set errno. This means that fuse_mount_compat25 / fuse_session_mount (which call fusermount / fusermount3 under the hood) will return -1, but if the caller tries to figure out what error occurred using errno, they will get 0 indicating success. The fuser Rust crate was doing precisely this, and reporting the error as Error: Success which is a very confusing error message. It was addressed by explicitly mapping errno = 0 to an unknown error type (https://github.com/cberner/fuser/pull/178, https://github.com/cberner/fuser/pull/218).

    Perhaps instead the return value itself could be altered, based on whether the error is due to a FUSE-specific config issue or not? fuse_mount_compat25 / fuse_session_mount could be documented as returning -1 on an error from a system call (in which case the caller checks errno for more information), and value < -1 for FUSE-specific errors. The main issue would be how downstream users of libfuse would behave on receiving an error that is not -1 (i.e. if they consider success to be ret >= 0, or ret != -1).

    opened by str4d 13
  • examples/passthrough hangs with max_idle_threads=0 (and maybe other filesystems too)

    examples/passthrough hangs with max_idle_threads=0 (and maybe other filesystems too)

    When max_idle_threads is set to 0,and perform the deletion operation on the mount point,the only worker thread exits. Only the main thread is left, libfuse can not receive and process file op request.

    In fuse_do_work function, when numavail equals 0, should call fuse_loop_start_thread.

    needs-info 
    opened by lan1120 6
Releases(fuse-3.12.0)
Owner
null
Supertag is a tag-based filesystem, written in Rust, for Linux and MacOS

Supertag is a tag-based filesystem, written in Rust, for Linux and MacOS. It provides a tag-based view of your files by removing the hierarchy constraints typically imposed on files and folders. In other words, it allows you to think about your files not as objects stored in folders, but as objects that can be filtered by folders.

Andrew Moffat 516 Sep 19, 2022
A Tauri Plugin to watch the filesystem for changes

Tauri Plugin FSWatch This plugin provides a "classical" Tauri Plugin Interface to watch changes on files and directories through notify. Architecture

Tauri 26 Sep 11, 2022
A virtual filesystem layer for WASI

wasi-vfs A virtual filesystem layer for WASI. NOTICE: This project currently supports only WASI applications on the top of wasi-libc This project prov

Yuta Saito 46 Sep 18, 2022
Spacedrive is an open source cross-platform file explorer, powered by a virtual distributed filesystem written in Rust.

Spacedrive A file explorer from the future. spacedrive.com » Download for macOS · Windows · Linux · iOS · watchOS · Android ~ Links will be added once

Spacedrive 14.2k Sep 25, 2022
Minty is an amazingly fast file deduplication app built in rust with a rust user interface.

minty Project Minty has a new look and feel!!! Minty is an amazingly fast file deduplication app built in rust with a rust user interface. I say super

null 27 Sep 21, 2022
Rust library for filesystems in userspace (FUSE)

Rust FUSE - Filesystem in Userspace About fuse-rs is a Rust library crate for easy implementation of FUSE filesystems in userspace. fuse-rs does not j

Andreas Neuhaus 877 Sep 20, 2022
Special FUSE filesystem to map /etc/resolv.conf to different files depending on Linux network namespace

Linux network namespaces allow separate networking environment for a group of processes (sharing uid or from a separate user). DNS settings (/etc/resolv.conf) are however shared between all those environments, which may be inconvenient in some setups.

Vitaly Shukela 2 May 16, 2022
Fuse filesystem that returns symlinks to executables based on the PATH of the requesting process.

Envfs Fuse filesystem that returns symlinks to executables based on the PATH of the requesting process. This is useful to execute shebangs on NixOS th

Jörg Thalheim 60 Sep 26, 2022
the file filesystem: mount semi-structured data (like JSON) as a Unix filesystem

ffs: the file filesystem ffs, the file filessytem, let's you mount semi-structured data as a fileystem---a tree structure you already know how to work

Michael Greenberg 174 Sep 10, 2022
Userspace WireGuard® Implementation in Rust

BoringTun BoringTun is an implementation of the WireGuard® protocol designed for portability and speed. BoringTun is successfully deployed on millions

Cloudflare 4.6k Sep 22, 2022
Userspace WireGuard® Implementation in Rust

BoringTun BoringTun is an implementation of the WireGuard® protocol designed for portability and speed. BoringTun is successfully deployed on millions

Cloudflare 4.6k Sep 27, 2022
Command line interface for BDSP RNG, primarily used as a reference implementation and as a tool for testing.

BDSP RNG Reference This is a command line interface for BDSP RNG, primarily used as a reference implementation and as a tool for testing. Building Bui

Zak 1 Dec 16, 2021
Userspace libpcap-based tool to mirror your dns traffic

DNS traffic mirroring tool (dns-mirror) Description Userspace libpcap-based tool. dns-mirror sniffs dns packets on the given interface and proxies it

Timofey 1 Mar 15, 2022
Supertag is a tag-based filesystem, written in Rust, for Linux and MacOS

Supertag is a tag-based filesystem, written in Rust, for Linux and MacOS. It provides a tag-based view of your files by removing the hierarchy constraints typically imposed on files and folders. In other words, it allows you to think about your files not as objects stored in folders, but as objects that can be filtered by folders.

Andrew Moffat 516 Sep 19, 2022
Cover your tracks during Linux Exploitation by leaving zero traces on system logs and filesystem timestamps. 👻🐚

moonwalk Cover your tracks during Linux Exploitation / Penetration Testing by leaving zero traces on system logs and filesystem timestamps. ?? Table o

Mufeed VH 1k Sep 23, 2022
Reference implementation for the Poseidon Snark-friendly Hash algorithm.

Dusk-Poseidon Reference implementation for the Poseidon Hashing algorithm. Reference Starkad and Poseidon: New Hash Functions for Zero Knowledge Proof

Dusk Network 81 Sep 9, 2022
StdFuzzer - StdFuzzer is the reference implementation of a generic bit-level fuzzer with LibAFL

StdFuzzer StdFuzzer is the reference implementation of a generic bit-level fuzzer with LibAFL Building Build with $ cargo build --release Compiling a

Advanced Fuzzing League ++ 41 Sep 7, 2022
Reference implementation of a full-stack Rust application

Full-stack Rust with WebAssembly Look Ma, No JavaScript !!! My very first Rust project (implementation of the "Connect 5" game), I used as a learning

Vassil 54 Sep 28, 2022
Reference implementation of a decentralized exchange for custom instruments, risk, and fees

Dexterity What is Dexterity At a high level, Dexterity is a smart contract (or collection of smart contracts) that allow for the creation of a decentr

Solana Foundation 42 Sep 13, 2022
Reference client for NEAR Protocol

Reference implementation of NEAR Protocol About NEAR NEAR's purpose is to enable community-driven innovation to benefit people around the world. To ac

NEAR 1.9k Sep 28, 2022