GGLinnk/bees
1
0
Fork 0
mirror of https://github.com/Zygo/bees.git synced 2025-05-17 13:25:45 +02:00
Code Packages
bees/docs/btrfs-kernel.md
Zygo Blaxell 34b04f4255 bees: soft-limit computed thread counts to 8 
https://github.com/Zygo/bees/issues/91 describes problems encountered
when running bees on systems with many CPU cores.

Limit the computed number of threads (using --thread-factor or the
default) to a maximum of 8 (i.e. the number of logical cores in a modern
laptop).  Users can override the limit by using --thread-count.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2018-11-21 21:49:16 -05:00

3.4 KiB
Raw Blame History

Recommended kernel version

Linux 4.14.34 or later.

A Brief List Of Btrfs Kernel Bugs

Recent kernel bug fixes:

  • 4.14.29: WARN_ON(ref->count < 0) in fs/btrfs/backref.c triggers almost once per second. The WARN_ON is incorrect, and is now removed.

Unfixed kernel bugs (as of 4.14.71):

  • Bad filesystem destroying interactions with other Linux block layers: bcache and lvmcache can fail spectacularly, and apparently only do so while running bees. This is definitely a kernel bug, either in btrfs or the lower block layers. Avoid using bees with these tools unless your filesystem is disposable and you intend to debug the kernel.

  • Compressed data corruption is possible when using the fallocate system call to punch holes into compressed extents that contain long runs of zeros. The bug results in intermittent corruption during reads, but due to the bug, the kernel might sometimes mistakenly determine data is duplicate, and deduplication will corrupt the data permanently. This bug also affects compressed kvm raw images with the discard feature on btrfs or any compressed file where fallocate -d or fallocate -p has been used.

  • Deadlock when simultaneously using the same files in dedupe and rename. There is no way for bees to reliably know when another process is about to rename a file while bees is deduping it. In the rsync case, bees will dedupe the new file rsync is creating using the old file rsync is copying from, while rsync will rename the new file over the old file to replace it.

Minor kernel problems with workarounds:

  • Slow backrefs (aka toxic extents): Under certain conditions, if the number of references to a single shared extent grows too high, the kernel consumes more and more CPU while holding locks that block access to the filesystem. bees avoids this bug by measuring the time the kernel spends performing LOGICAL_INO operations and permanently blacklisting any extent or hash involved where the kernel starts to get slow. In the bees log, such blocks are labelled as 'toxic' hash/block addresses.

  • btrfs send has various bugs that are triggered when bees is deduping snapshots. bees provides the --workaround-btrfs-send option which should be used whenever btrfs send and bees are run on the same filesystem.

    This issue affects:

    • btrfs send (any mode) and bees active at the same time.
    • btrfs send in incremental mode (using -p option) with bees active at the same or different times.

    Note btrfs receive is not affected. It is OK to run bees with no workarounds on a filesystem that receives btrfs snapshots.

  • Systems with many CPU cores may lock up when bees runs with one worker thread for every core. bees limits the number of threads it will try to create based on detected CPU core count. Users may override this limit with the --thread-count option.

Older kernels:

  • Older kernels have various data corruption and deadlock/hang issues that are no longer listed here, and older kernels are missing important features such as LOGICAL_INO_V2. Using an older kernel is not recommended.