A file server that supports static serving, uploading, searching, accessing control, webdav...

Overview

Dufs

CI Crates

Dufs is a distinctive utility file server that supports static serving, uploading, searching, accessing control, webdav...

demo

Features

  • Serve static files
  • Download folder as zip file
  • Upload files and folders (Drag & Drop)
  • Create/Edit/Search files
  • Partial responses (Parallel/Resume download)
  • Access control
  • Support https
  • Support webdav
  • Easy to use with curl

Install

With cargo

cargo install dufs

With docker

docker run -v `pwd`:/data -p 5000:5000 --rm -it sigoden/dufs /data -A

With Homebrew

brew install dufs

Binaries on macOS, Linux, Windows

Download from Github Releases, unzip and add dufs to your $PATH.

CLI

Dufs is a distinctive utility file server - https://github.com/sigoden/dufs

Usage: dufs [OPTIONS] [serve_path]

Arguments:
  [serve_path]  Specific path to serve [default: .]

Options:
  -b, --bind <addrs>         Specify bind address or unix socket
  -p, --port <port>          Specify port to listen on [default: 5000]
      --path-prefix <path>   Specify a path prefix
      --hidden <value>       Hide paths from directory listings, separated by `,`
  -a, --auth <rules>         Add auth role
      --auth-method <value>  Select auth method [default: digest] [possible values: basic, digest]
  -A, --allow-all            Allow all operations
      --allow-upload         Allow upload files/folders
      --allow-delete         Allow delete files/folders
      --allow-search         Allow search files/folders
      --allow-symlink        Allow symlink to files/folders outside root directory
      --allow-archive        Allow zip archive generation
      --enable-cors          Enable CORS, sets `Access-Control-Allow-Origin: *`
      --render-index         Serve index.html when requesting a directory, returns 404 if not found index.html
      --render-try-index     Serve index.html when requesting a directory, returns directory listing if not found index.html
      --render-spa           Serve SPA(Single Page Application)
      --assets <path>        Use custom assets to override builtin assets
      --tls-cert <path>      Path to an SSL/TLS certificate to serve with HTTPS
      --tls-key <path>       Path to the SSL/TLS certificate's private key
      --log-format <format>  Customize http log format
      --completions <shell>  Print shell completion script for <shell> [possible values: bash, elvish, fish, powershell, zsh]
  -h, --help                 Print help
  -V, --version              Print version

Examples

Serve current working directory in readonly mode

dufs

Allow all operations like upload/delete/search/create/edit...

dufs -A

Only allow upload operation

dufs --allow-upload

Serve a specific directory

dufs Downloads

Serve a single file

dufs linux-distro.iso

Serve a single-page application like react/vue

dufs --render-spa

Serve a static website with index.html

dufs --render-index

Require username/password

dufs -a admin:123@/:rw

Listen on specific host:ip

dufs -b 127.0.0.1 -p 80

Listen on unix socket

dufs -b /tmp/dufs.socket

Use https

dufs --tls-cert my.crt --tls-key my.key

API

Upload a file

curl -T path-to-file http://127.0.0.1:5000/new-path/path-to-file

Download a file

curl http://127.0.0.1:5000/path-to-file

Download a folder as zip file

curl -o path-to-folder.zip http://127.0.0.1:5000/path-to-folder?zip

Delete a file/folder

curl -X DELETE http://127.0.0.1:5000/path-to-file-or-folder

Create a directory

curl -X MKCOL https://127.0.0.1:5000/path-to-folder

Move the file/folder to the new path

curl -X MOVE https://127.0.0.1:5000/path -H "Destination: https://127.0.0.1:5000/new-path"

List/search directory contents

curl http://127.0.0.1:5000?simple                 # output names only, just like `ls -1`
curl http://127.0.0.1:5000?json                   # output paths in json format
curl http://127.0.0.1:5000?q=Dockerfile&simple    # search for files, just like `find -name Dockerfile`

With authorization

curl --user user:pass --digest http://192.168.8.10:5000/file  # digest auth
curl --user user:pass http://192.168.8.10:5000/file           # basic auth

Advanced topics

Access Control

Dufs supports account based access control. You can control who can do what on which path with --auth/-a.

dufs -a [user:pass]@path[:rw][,path[:rw]...][|...]
  1. Multiple rules are separated by "|"
  2. User and pass are the account name and password, if omitted, it is an anonymous user
  3. One rule can set multiple paths, separated by ","
  4. Add :rw after the path to indicate that the path has read and write permissions, otherwise the path has readonly permissions.
dufs -A -a admin:admin@/:rw

admin has all permissions for all paths.

dufs -A -a admin:admin@/:rw -a guest:guest@/

guest has readonly permissions for all paths.

dufs -A -a admin:admin@/:rw -a @/

All paths is public, everyone can view/download it.

dufs -A -a admin:admin@/:rw -a user1:pass1@/user1:rw -a user2:pass2@/user2
dufs -A -a "admin:admin@/:rw|user1:pass1@/user1:rw|user2:pass2@/user2"

user1 has all permissions for /user1/* path. user2 has all permissions for /user2/* path.

dufs -A -a user:pass@/dir1:rw,/dir2:rw,dir3

user has all permissions for /dir1/* and /dir2/*, has readonly permissions for /dir3/.

dufs -a admin:admin@/

Since dufs only allows viewing/downloading, admin can only view/download files.

Hide Paths

Dufs supports hiding paths from directory listings via option --hidden <glob>,....

dufs --hidden .git,.DS_Store,tmp

The glob used in --hidden only matches file and directory names, not paths. So --hidden dir1/file is invalid.

dufs --hidden '.*'            # hidden dotfiles
dufs --hidden '*/'            # hidden all folders
dufs --hidden '*.log,*.lock'  # hidden by exts

Log Format

Dufs supports customize http log format with option --log-format.

The log format can use following variables.

variable description
$remote_addr client address
$remote_user user name supplied with authentication
$request full original request line
$status response status
$http_ arbitrary request header field. examples: $http_user_agent, $http_referer

The default log format is '$remote_addr "$request" $status'.

2022-08-06T06:59:31+08:00 INFO - 127.0.0.1 "GET /" 200

Disable http log

dufs --log-format=''

Log user-agent

dufs --log-format '$remote_addr "$request" $status $http_user_agent'
2022-08-06T06:53:55+08:00 INFO - 127.0.0.1 "GET /" 200 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36

Log remote-user

dufs --log-format '$remote_addr $remote_user "$request" $status' -a /@admin:admin -a /folder1@user1:pass1
2022-08-06T07:04:37+08:00 INFO - 127.0.0.1 admin "GET /" 200

Environment variables

All options can be set using environment variables prefixed with DUFS_.

  [SERVE_PATH]                DUFS_SERVE_PATH=/dir
  -b, --bind <addrs>          DUFS_BIND=0.0.0.0
  -p, --port <port>           DUFS_PORT=5000
      --path-prefix <path>    DUFS_PATH_PREFIX=/path
      --hidden <value>        DUFS_HIDDEN=*.log
  -a, --auth <rules>          DUFS_AUTH="admin:admin@/:rw|@/" 
      --auth-method <value>   DUFS_AUTH_METHOD=basic
  -A, --allow-all             DUFS_ALLOW_ALL=true
      --allow-upload          DUFS_ALLOW_UPLOAD=true
      --allow-delete          DUFS_ALLOW_DELETE=true
      --allow-search          DUFS_ALLOW_SEARCH=true
      --allow-symlink         DUFS_ALLOW_SYMLINK=true
      --allow-archive         DUFS_ALLOW_ARCHIVE=true
      --enable-cors           DUFS_ENABLE_CORS=true
      --render-index          DUFS_RENDER_INDEX=true
      --render-try-index      DUFS_RENDER_TRY_INDEX=true
      --render-spa            DUFS_RENDER_SPA=true
      --assets <path>         DUFS_ASSETS=/assets
      --tls-cert <path>       DUFS_TLS_CERT=cert.pem
      --tls-key <path>        DUFS_TLS_KEY=key.pem
      --log-format <format>   DUFS_LOG_FORMAT=""

Customize UI

Dufs allows users to customize the UI with your own assets.

dufs --assets my-assets-dir/

Your assets folder must contains a index.html file.

index.html can use the following placeholder variables to retrieve internal data.

  • __INDEX_DATA__: directory listing data
  • __ASSETS_PREFIX__: assets url prefix

License

Copyright (c) 2022 dufs-developers.

dufs is made available under the terms of either the MIT License or the Apache License 2.0, at your option.

See the LICENSE-APACHE and LICENSE-MIT files for license details.

Comments
  • corrupted files in zipped files

    corrupted files in zipped files

    I'm experiencing issues when extracting files form an zip-archive. Downloading the individual .jpg files works fine, but they become corrupted during the zipping process and/or download.

    Version: 0.22.0 OS: Ubuntu 18.04 Browser: FF 99.0

    opened by juliangaal 14
  • Symbolic/hard Link, virtual folders like sftpgo, list directory above if view files by browser support

    Symbolic/hard Link, virtual folders like sftpgo, list directory above if view files by browser support

    Specific Demand

    Symbolic/Hard Link Support

    I found that dufs can't list symbolic link entries.

    Dynamic virtual folders like sftpgo

    When running dufs, you may need add some virtual folders to transfers files.

    Instead of running another instance and using another port.

    list directory above if view files by browser

    We prefer folders above and files below.

    Implement Suggestion

    opened by lynks529 13
  • Can't open Webdav with Kodi

    Can't open Webdav with Kodi

    Problem When use Kodi to mount the dufs's Webdav server,it didn' work. Kodi无法打开dufs的Webdav服务器,但是kodi可以打开其他Webdav服务器。

    Log Dufs's log shows the status code is 401.

    Environment:

    • Dufs version: v0.32.0
    • Browser/Webdav Info: Kodi无法打开
    • OS Info: dufs-v0.32.0-aarch64-unknown-linux-musl.tar.gz
    opened by 289699522 9
  • Full webdav support

    Full webdav support

    Specific Demand

    It would seem that running the litmus test suite (https://github.com/tolsen/litmus / http://webdav.org/neon/litmus/) gives only a very partial webdav support (a lot of errors with copy and move, notably non support of directories, etc.).

    It seems that the partial support causes problems when for example copying from command line in linux on a mounted (with gnome file manager) dav : the copy is stuck...

    Implement Suggestion

    It would be nice that the webdav server complies with (at least), the "http basic copymove locks props" tests of the litmus test suite.

    Best regards.

    opened by nicolaspernoud 8
  • webdav 无法在 Koreader 里面打开

    webdav 无法在 Koreader 里面打开

    您好,我最近发现这个项目挺高兴的,下载了可执行文件后,就可以在我的电脑上开启 web 服务,将里面的文件分享出来。

    然后我平时有一个需求,就是使用开源的阅读器 https://github.com/koreader/koreader 来阅读。这个软件有手机版本,也有 kindle 版本,所以我平时使用频率挺高的。

    里面有一个云存储的功能,可以添加 webdav 地址,这样就可以在手机或者 kindle,直接下载 webdav 服务器里面的书籍了。但是我发现在软件里,无法访问 dufs 开启的 webdav 服务。

    如果我不设置密码,使用手机访问,电脑终端这边会显示

    2022-06-24T13:46:55+08:00 INFO - 192.168.68.211 "PROPFIND //" - 404
    

    如果我添加账号密码,则会显示这样

    $dufs --allow-upload -a /@admin:123
    Listening on:
      http://127.0.0.1:5000/
      http://192.168.68.4:5000/
      http://[::1]:5000/
    
    2022-06-24T13:58:00+08:00 INFO - ::1 "GET /" - 401
    2022-06-24T13:58:06+08:00 INFO - ::1 "GET /" - 200
    2022-06-24T13:58:23+08:00 INFO - 192.168.68.211 "PROPFIND //" - 401
    2022-06-24T14:00:14+08:00 INFO - 192.168.68.211 "PROPFIND //" - 401
    

    我使用过另外一个 webdav 应用 https://github.com/hacdias/webdav 这个可以在 Koreader 里面访问。

    不知道问题出现在哪,期待大佬的解答。

    opened by powersee 8
  • symbolic links to external drives are not listed

    symbolic links to external drives are not listed

    Problem

    as the title suggests, symbolic links linked to external disks are not listed.

    Environment:

    • Dufs version: 0.26.0
    • Browser/Webdav Info: brave on android and librewolf on linux
    • OS Info: Arch Linux
    opened by notjedi 7
  • Rename duf to dufs

    Rename duf to dufs

    There's another tool also called duf whose command is also duf.

    Various Linux distributions (Arch Linux, Debian, Nix, etc) already provide a package named duf that provides the other duf.

    Proposal: Rename this tool to allow users to use it without name/file conflict.

    opened by KSXGitHub 7
  • Cannot handle shortcuts in windows very well

    Cannot handle shortcuts in windows very well

    Just tried shortcuts in Windows and I suppose it cannot deal with symbol link very well in Linux or Mac.

    Error Message: Internal Server Error ENV: Firefox Windows

    image

    help wanted 
    opened by piaoger 7
  • create/delete folder

    create/delete folder

    Hi sigoden, thanks for your development, this tool helps me a lot. Just love it.

    Sometimes, when I upload new files and want to classify them from previous ones, I will create a folder and put them is there. Since dufs does not support creating new folder on web UI, I have to go to the terminal and run mkdir command. So if create/delete folder is supported on web UI, it will be very helpful.

    Hope to hear your opinion. Thanks in advance.

    opened by leolulu 6
  • Possible to resume downloads after Network Disconnect?

    Possible to resume downloads after Network Disconnect?

    Problem

    Thanks for creating this simple tool. It has made sharing large files between people very simple.

    As the title suggests, is it possible to resume a download after the server network has disconnected and the HTTPS connection has terminated. On chrome, when the disconnect happens, the browser marks the download as failed and clicking on resume doesn't seem to restart it from where it left off. Is this the normal behaviour for HTTP ? With SFTP, it has the capability to resume even after a complete termination of connection although i would like to use HTTP as the protocol since the speed doesnt suffer when the RTT is high and the file transfer is happening over the WAN

    Environment:

    • Dufs version: 0.36.0
    • Browser: Version 116.0.5845.188
    • OS Info: Windows 10 Pro 22H2 on Latest Patch as of 21/09/2023
    opened by blackops786187 5
  • has errors when build  release

    has errors when build release

    Problem has errors when build release。

    Log error: failed to run custom build command for quote v1.0.23

    Caused by: process didn't exit successfully: /Users/xxx/Downloads/dufs/target/release/build/quote-9253156aff15e4c7/build-script-build (signal: 9, SIGKILL: kill) warning: build failed, waiting for other jobs to finish... error: failed to run custom build command for proc-macro2 v1.0.51 Caused by: process didn't exit successfully: /Users/xxx/Downloads/dufs/target/release/build/proc-macro2-4f145f136714f2f1/build-script-build (signal: 9, SIGKILL: kill)

    Environment:

    • Dufs version:v0.32.0
    • Browser/Webdav Info:
    • OS Info: Darwin xxx 21.3.0 Darwin Kernel Version 21.3.0: Wed Jan 5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_ARM64_T8101 arm64
    opened by Maplely 5
  • Avoid writing half-uploaded files

    Avoid writing half-uploaded files

    Specific Demand

    Hello,

    Currently, when you upload a file, it is written sequentially. So if an error occurs, you could have only half-file written to the disk. A simple way to reproduce the behaviour is to start uploading a big file, and press F5.

    I don't know if it is a use case that makes sense for you to handle in an other way. Anyway, here is my simple suggestion.

    Implement Suggestion

    Disclaimer - I have literally 0 experiences or practice in Rust. First, time writing something with it.

    So the idea is to write the file in a temporary location until the download is finished. Then, move the file to the correct place. I did a PoC you can find here.

    Technical comments:

    • We cannot create a file to determine if we can write in the directory. Otherwise, the file would appear before it is completely uploaded. So, I used the metadata of the parent folder to determine if the folder is read-only or not.

    • To find where to store the temporary file, I dumbly took the first reference I found. So the solution is based on the crate tempfile. It allows you to create temporary files or folders. Unfortunately, the temporary file is not compatible with tokio::io::copy. Hence, I created a temporary directory in which I create the temporary file using tokio::fs::File::create. And... while writing this feature request, I found that an async_tempfile exists which seems to be compatible with tokio.

    opened by Lukinoh 0
Releases(v0.36.0)
Owner
null
A commmand line tool for uploading homework coded on the dcloud server onto specific google drive course folders.

A commmand line tool for uploading homework coded on the dcloud server onto specific google drive course folders.

Daniel Kogan 2 Sep 8, 2022
CLI and library for uploading files to Arweave.

arloader Command line application and library for effortlessly uploading files to Arweave. Arweave enables you to store documents and applications for

Caleb Everett 79 Nov 20, 2022
A CLI helping with uploading to Redacted, inspired by REDBetter

red_oxide CLI to help uploading to REDacted, inspired by REDBetter. Installing Install intermodal and add it to your PATH Install lame, sox & flac and

Yukine 11 Apr 23, 2023
Simple Secure Static (HTTPS) File Server with embedded certificate

Secure Static File Server Static Files HTTPs server with self signed embedded certificate Installation Install using cargo: cargo install ssfs Or buil

0xor0ne 21 Apr 20, 2023
Command line application for searching

maven_search_rs Command line application for searching in https://search.maven.org Usage Non-interactive $ maven-search -f maven wicket-core The abov

Martin Grigorov 1 Nov 5, 2021
AniTUI is a CLI (and in the future a TUI) app for searching and wathching anime in MPV.

AniTUI is a CLI (and in the future a TUI) app for searching and wathching anime in MPV. This is a Rust rewrite (quite literally a rewrite) of Pystardu

null 7 Oct 31, 2022
REC2 (Rusty External Command and Control) is client and server tool allowing auditor to execute command from VirusTotal and Mastodon APIs written in Rust. 🦀

Information: REC2 is an old personal project (early 2023) that I didn't continue development on. It's part of a list of projects that helped me to lea

Quentin Texier (g0h4n) 104 Oct 7, 2023
A fast static code analyzer & language server for Python

pylyzer ⚡ pylyzer is a static code analyzer / language server for Python written in Rust. Installation cargo (rust package manager) cargo install pyly

Shunsuke Shibayama 78 Jan 3, 2023
Single File Assets is a file storage format for images

SFA (Rust) Single File Assets is a file storage format for images. The packed images are not guaranteed to be of same format because the format while

null 1 Jan 23, 2022
A command-line tool aiming to upload the local image used in your markdown file to the GitHub repo and replace the local file path with the returned URL.

Pup A command line tool aiming to upload the local image used in your markdown file to the GitHub repo and replace the local file path with the return

SteveLau 11 Aug 17, 2022
A tool for determining file types, an alternative to file

file-rs a tool for determining file types, an alternative to file whats done determining file extension determining file type determining file's mime

null 3 Nov 27, 2022
SAORI for UKAGAKA. Convert a image file to resized png file.

Resized Png GitHub repository これは何? デスクトップマスコット、「伺か」で使用できるSAORIの一種です。 機能としては、指定した画像ファイルを拡大または縮小し、pngとして出力します。 「伺か」「SAORI」等の用語については詳しく説明いたしませんのでご了承下さい。

月波 清火 2 Jan 3, 2023
FileSorterX is an automatic file sorting application that sorts your files into folders based on their file extension

FileSorterX is an automatic file sorting application that sorts your files into folders based on their file extension. With FileSorterX, you can easily keep your files organized and find what you need quickly.

Xanthus 22 Apr 4, 2023
My own image file format created for fun! Install the "hif_opener.exe" to open hif files. clone the repo and compile to make your own hif file

Why am i creating this? I wanted to create my own image format since I was 12 years old using Windows 7, tryna modify GTA San Andreas. That day, when

hiftie 3 Dec 17, 2023
A simple program that provides DBus interface to control display temperature and brightness under wayland without flickering

wl-gammarelay-rs Like wl-gammarelay, but written in rust, runs on a single thread, has three times less SLOC and uses DBus (for simplicity). Dbus inte

Max Verevkin 33 Nov 23, 2022
A simple tui to view & control docker containers

oxker A simple tui to view & control docker containers Built in Rust, making heavy use of tui-rs & Bollard Download & install See releases download &

Jack Wills 119 Dec 28, 2022
A tool to control the fan speed by monitoring the temperature of CPU via IPMI.

ipmi-fan-control A tool to control the fan speed by monitoring the temperature of CPU via IPMI. Why Our Dell R730 server's iDRAC is not works as expec

yinheli 9 Dec 29, 2022
A simple interactive OSC (Open Sound Control) debugger for terminal

oscd, a simple interactive OSC debugger for the terminal by using nom as a lexer/parser, oscd offers auto type casting and support sending multiple osc arguments.

Karnpapon Boonput 5 Oct 19, 2022
A command line tool to control the power state of Valve Base Stations 2.0.

lighthousectl A command line tool to control the power state of Valve Base Stations 2.0. Usage Scan All Base Stations It scans endlessly. You can stop

KOBA789 11 Aug 9, 2022