When running `meson ..` I get:
```
WARNING: Running the setup command as `meson [options]` instead of `meson setup [options]` is ambiguous and deprecated.WARNING: Running the setup command as `meson [options]` instead of `meson setup [options]` is ambiguous and deprecated.
```
So let's fix this in the docs.
When building, I get the following warning:
```bash
$ ninja
[18/71] Compiling C object lib/libfuse3.so.3.14.1.p/modules_iconv.c.o
../lib/modules/iconv.c: In function ‘iconv_convpath’:
../lib/modules/iconv.c:85:38: warning: pointer ‘newpath’ may be used after ‘realloc’ [-Wuse-after-free]
85 | p = tmp + (p - newpath);
| ~~~^~~~~~~~~~
../lib/modules/iconv.c:80:31: note: call to ‘realloc’ here
80 | tmp = realloc(newpath, newpathlen + 1);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[71/71] Linking target example/passthrough_hp
```
It's a false positive, I thinks. But it's also easy to silence this
warning with a small refactor.
In
77d662459a/util/fusermount.c (L1219)
`open` is executed as root which does not have access to the mount
point if `allow_other` was not used and the real user id is not 0. Since
`allow_other` usually cannot be specified by unprivileged users,
`auto_unmount` has no effect for unprivileged users.
In this commit, we work around this limitation:
We first try to open the mountpoint as root, and if we get `EACCES`, we
retry as the user who started fusermount, and see if we get `ENOTCONN`.
In my testing, I found that `setfsuid` and `setfsgid` don't work to get
around the lack of `allow_other`. (Sorry, I don't know enough about the
Linux kernel to tell whether that's significant.) As a workaround, I
decided to use `setresuid` and `setresgid` in a forked child process,
and communicate via its exit status.
Please give feedback on correctness, style and suggest tests.
Fixes https://github.com/libfuse/libfuse/issues/586
The fuse_main_real() method doesn't apply the max_threads parameter
parsed through the commandline arguments. This commit fixes the wiring
of max_threads argument.
Right now fuse kernel serializes direct writes on the
same file. This serialization is good for such FUSE
implementations which rely on the inode lock to
avoid any data inconsistency issues but it hurts badly
such FUSE implementations which have their own mechanism
of dealing with cache/data integrity and can handle
parallel direct writes on the same file.
This patch allows parallel direct writes on the same file to be
enabled with the help of a flag FOPEN_PARALLEL_DIRECT_WRITES.
FUSE implementations which want to use this feature can
set this flag during fuse init. Default behaviour remains
same i.e no parallel direct writes on the same file.
Corresponding fuse kernel patch(Merged).
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v6.2&id=153524053bbb0d27bb2e0be36d1b46862e9ce74c
If a program with API before 312 did not set
max_idle_threads the new default from
fuse_parse_cmdline_312() is applied, which sets
UINT_MAX (-1).
Later in compat fuse_session_loop_mt_32 the old
config v1 struct is converted and that conversion
prints a warning if the default unset value was used.
This could have also happened to programs using the current
API, which just apply values struct fuse_cmdline_opts,
without checking if the defaults are set.
Commit d7560cc has split defines into private and public, and passed -DHAVE_LIBFUSE-PRIVATE_CONFIG_H to all C programs. But the arguments of C++ programs have not been changed. This leads to a test failure as reported in issue #734. Pass -DHAVE_LIBFUSE-PRIVATE_CONFIG_H to C++ programs too.
Fixes: #734
Such as for the xfstest-dev's generic/684 test case it will clear
suid and sgid if the fallocate request is commited by an unprivileged
user.
The kernel fuse passed the ATTR_KILL_SUID/ATTR_KILL_SGID flags to
userspace but it will be dropped.
Signed-off-by: Xiubo Li <xiubli@redhat.com>
This addresses https://github.com/libfuse/libfuse/issues/729
commit db35a37def introduced a public
config.h (rename to fuse_config.h to avoid conflicts) that
was installed with the package and included by libfuse users
through fuse_common.h. Probablem is that this file does not have
unique defines so that they are unique to libfuse - on including
the file conflicts with libfuse users came up.
In principle all defines could be prefixed, but then most of them
are internal for libfuse compilation only. So this splits out
publically required defines to a new file 'libfuse_config.h'
and changes back to include of "fuse_config.h" only when
HAVE_LIBFUSE_PRIVATE_CONFIG_H is defined.
This also renames HAVE_LIBC_VERSIONED_SYMBOLS to
LIBFUSE_BUILT_WITH_VERSIONED_SYMBOLS, as it actually
better explains for libfuse users what that variable
is for.
This addresses: https://github.com/libfuse/libfuse/issues/724
HAVE_LIBC_VERSIONED_SYMBOLS configures the library if to use
versioned symbols and is set at meson configuration time.
External filesystems (the main target, actually)
include fuse headers and the preprocessor
then acts on HAVE_LIBC_VERSIONED_SYMBOLS. Problem was now that
'config.h' was not distributed with libfuse and so
HAVE_LIBC_VERSIONED_SYMBOLS was never defined with external
tools and the preprocessor did the wrong decision.
This commit also increases the the minimal meson version,
as this depends on meson feature only available in 0.50
<quote 'meson' >
WARNING: Project specifies a minimum meson_
version '>= 0.42' but uses features which were added
in newer versions:
* 0.50.0: {'install arg in configure_file'}
</quote>
Additionally the config file has been renamed to "fuse_config.h"
to avoid clashes - 'config.h' is not very specific.
when -D_FILE_OFFSET_BITS=64 is defined, the off_t type is 64 bits wide
already. the fuse_common.h header already checks for this, and errors
when it is not, so be consistent with all the other uses of off_t.
some libcs like musl do not have a 32-bit off_t type, and don't define
__off64_t.
dlsym returns the address of the module factory symbol, not the actual function (#722)
pointer. Change the type of `factory` to `fuse_module_factory_t*` to reflect
this and then dereference it when registering the module.
This is a followup to d92bf83, which introduced a NULL pointer dereference
when dlsym returns NULL, and 8ec7fd9, which reverted it back to not
dereferencing the symbol at all.
Fixes: #721
Co-authored-by: Goswin von Brederlow <brederlo@q-leap.de>
The io for FUSE requests and responses can now be further customized by allowing to write custom functions for reading/writing the responses. This includes overriding the splice io.
The reason for this addition is that having a custom file descriptor is not sufficient to allow custom io. Different types of file descriptor require different mechanisms of io interaction. For example, some file descriptor communication has boundaries (SOCK_DGRAM, EOF, etc...), while other types of fd:s might be unbounded (SOCK_STREAMS, ...). For unbounded communication, you have to read the header of the FUSE request first, and then read the remaining packet data. Furthermore, the one read call does not necessarily return all the data expected, requiring further
calls in a loop.
This test is too simple to check for all functionalities of notify_expire as it always successfully passes when libfuse supports the function (even if kernel does not support it - it just takes it as notify_inval)
fuse_loop_mt and fuse_new had not been defined when
HAVE_LIBC_VERSIONED_SYMBOLS had not been set and additionally,
fuse_new_31 was missing in the version script and was therefore
an unusable symbol.
This also adds a test for unset HAVE_LIBC_VERSIONED_SYMBOLS.
In fact only gnu-libc fully supports symbol versioning, so it is
better to have a generic macro for it. This also allows to manually
disable symbol version and allows to run tests with that
configuration on gnu-libc. That testing will still not catch compat
issues, but least ensures the code can compile.
Testing for __APPLE__ and __ULIBC__ is now done by meson. More of such
checks can be added by people using other libcs.
For __APPLE__ and __ULIBC__, which are assumed to not support
versioned symbols, helper.c has a compat ABI symbol for
fuse_parse_cmdline(). However that ABI symbol was conflicting
with the API macro (which redirects to the right API function
for recompilations against current libfuse).
Additionally the parameter 'opts' had a typo and was called
'out_opts'.
As described in https://github.com/libfuse/libfuse/issues/695 and below, partial
locking of paths can cause a deadlock. Partial locking was added to prevent
starvation, but it's unclear what specific cases of starvation were of concern.
As far as I was able to determine, since we support reader locks that give
priority to writers (to prevent starvation), this means that to starve the queue
element, we'd need a constant stream of queued requests that lock the path for
write. Write locks are used when the element is being (potentially) removed, so
this stream of requests that starve the `rename` or `lock` operations seems
unlikely.
### Summarizing issue #695
The high-level API handles locking of the node structures it maintains to
prevent concurrent requests from deleting nodes that are in use by other
requests. This means that requests that might remove these structs (`rmdir`,
`rename`, `unlink`, `link`) need to acquire an (internally managed - not
pthread) exclusive lock before doing so. In the case where the lock is already
held (for read or for write), the operation is placed onto a queue of waiters.
On every unlock, the queue is reinspected for any element that might now be able
to make progress.
Since `rename` and `link` involve two paths, when added to the queue, a single
queue entry requires that we lock two different paths. There was, prior to this
change, support for partially locking the first queue element if it had two
paths to lock. This partial locking can cause a deadlock:
- set up a situation where the first element in the queue is partially locked
(such as by holding a reader lock on one of the paths being renamed, but not
the other). For example: `/rmthis/foo/foo.txt` [not-yet-locked] and
`/rmthis/bar/bar.txt` [locked]
- add an `rmdir` for an ancestor directory of the not-yet-locked path to the
queue. In this example: `/rmthis`
After getting into this situation, we have the following `treelock` values:
- `/rmthis`: 1 current reader (due to the locked `/rmthis/bar/bar.txt`), one
waiting writer (`rmdir`): no new readers will acquire a read lock here.
- `/rmthis/bar`: 1 reader (the locked `/rmthis/bar/bar.txt`)
- `/rmthis/bar/bar.txt`: 1 writer (the locked `/rmthis/bar/bar.txt`)
This is deadlocked, because the partial lock will never be able to be completely
locked, as doing so would require adding a reader lock on `/rmthis`, and that
will be rejected due to write lock requests having priority -- until the writer
succeeds in locking it, no new readers can be added. However, the writer (the
`rmdir`) will never be able to acquire its write lock, as the reader lock will
never be dropped -- there's no support for downgrading a partially locked
element to be unlocked, the only state change that's allowed involves it
becoming completely locked.
autofs uses automount, which calls fuse, during an sshfs call. fuse complains about -n being an unknown option (ref. https://github.com/libfuse/libfuse/issues/715)
this one line edit provides the command to be accepted, and pass through, allowing autofs-automount to operate on the mount, even though it is already in the mtab, given the nature of autofs/automount.
It may happen that none of the worker threads are running
if max_idle_threads is set to 0 although few people will do this.
Adding a limit of keeping at least one worker thread will make
our code more rigorous.
Signed-off-by: Zhansong Gao <zhsgao@hotmail.com>
There was a simple typo and sym1 didn't match the function name
with the older __asm__(".symver " sym1 "," sym2) way to define
ABI compatibility.
Witht the newer "__attribute__ ((symver (sym2)))" sym1 is not used
at all and in manual testing the issue didn't come up therefore.
If we get the interrupt before the fuse op, the fuse_req is deleted without
decrementing the refcount on the cloned file descriptor. This leads to a
leak of the cloned /dev/fuse file descriptor.
On benchmarking metadata operations with a single threaded bonnie++
and "max_idle_threads" limited to 1, 'top' was showing suspicious
160% cpu usage.
Profiling the system with flame graphs showed that an astonishing
amount of CPU time was spent in thread creation and destruction.
After verifying the code it turned out that fuse_do_work() was
creating a new thread every time all existing idle threads
were already busy. And then just a few lines later after processing
the current request it noticed that it had created too many threads
and destructed the current thread. I.e. there was a thread
creation/destruction ping-pong.
Code is changed to only create new threads if the max number of
threads is not reached.
Furthermore, thread destruction is disabled, as creation/destruction
is expensive in general.
With this change cpu usage of passthrough_hp went from ~160% to
~80% (with different values of max_idle_threads). And bonnie
values got approximately faster by 90%. This is a with single
threaded bonnie++
bonnie++ -x 4 -q -s0 -d <path> -n 30:1:1:10 -r 0
Without this patch, using the default max_idle_threads=10 and just
a single bonnie++ the thread creation/destruction code path is not
triggered. Just one libfuse and one application thread is just
a corner case - the requirement for the issue was just
n-application-threads >= max_idle_threads.
Signed-off-by: Bernd Schubert <bschubert@ddn.com>
