The critical kernel bugs in send have been fixed for years.
The limitations that remain aren't bugs, and bees has no sustainable
workaround for them.
Also update copyright year range.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
https://github.com/Zygo/bees/pull/209
Edited: regenerate docs for the downstream change in index.md.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
The existence of information about known data corruption bugs should be
visible from the top-level page.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
This may help users understand some of the things that happen inside
bees...or it may just be horribly long and confusing.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
Standard crash backtrace collection, plus $BEESSTATUS for the high-level
overview of what bees is doing.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
Split the rather large README into smaller sections with a pitch and
a ToC at the top.
Move the sections into docs/ so that Github Pages can read them.
'make doc' produces a local HTML tree.
Update the kernel bugs and gotchas list.
Add some information that has been accumulating in Github comments.
Remove information about bugs in kernels earlier than 4.14.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
Gentoo has officially merged the ebuild into portage as of:
https://github.com/gentoo/gentoo/pull/9925
Let's update the readme and get rid of the `contrib/gentoo-bees`
directory, so we have no potentially outdated information in the future.
Signed-off-by: Kai Krakow <kai@kaishome.de>
The -g option limits the number of worker threads when the target load
average is exceeded. On some systems the load normally runs high, and
continuous bees operation is required to avoid running out of disk space.
Add a -G/--thread-min option to force at least some threads to continue
running.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
Add -g / --loadavg-target parameter to track system load and add or
remove bees worker threads dynamically to keep system load close to the
loadavg target. Thread count may vary from zero to the maximum
specified by -c or -C, and is adjusted every 5 seconds.
This is better than implementing a similar load average scheme from
outside of the process (though that is still possible) because the
in-process load tracker does not disrupt the performance timing feedback
mechanisms as a freezer cgroup or SIGSTOP would when controlling bees
from outside. The internal load average tracker can also adjust the
number of active threads while an external tracker can only choose from
the maximum or zero.
Also fix a bug where a Task could deadlock waiting for itself to exit
if it tries to insert a new Task after the number of worker threads has
been set to zero.
Also correct usage message for --scan-mode (values are 0..2) since
we are touching adjacent lines anyway.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
Log messages were already labelled with log levels, but there was no
way to filter by log level at run time.
Implement the filter inside the bees process so it can skip evaluation
of the BEESLOG* arguments if the log messages would not be emitted.
Fixes: https://github.com/Zygo/bees/issues/67
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
This commit squashes all the little changes from the previous
integration branch into one, adjusts to the new Makefile changes, and
introduces an overlay layout so that the contrib/gentoo-bees subtree
can be directly added as a Portage overlay to the system.
The following list contains the previous commit descriptions:
sys-fs/bees: Keyword tested architecture ~amd64
Bees was tested on this platform.
sys-fs/bees: Add kernel version checks
Add checking the kernel versions and write some info and/or warnings
before building and installing the package. Running bees on older
kernels may have some serious performance and stability impacts, let's
tell the user about it.
Closes#55
sys-fs/bees: Add metadata.xml
sys-fs/bees: There's no configure script
So, there's no point in calling "default".
sys-fs/bees: Simplify src_configure()
sys-fs/bees: Don't depend on markdown
It makes no sense to install both README.md and README.html, and we can
get rid of one dependency.
Dependencies: btrfs-progs is no longer a buildtime-only dep
It is actually needed by the bees service wrapper script, as pointed out
by Gentoo QA review.
sys-fs/bees: DOCS is not needed
"COPYING" is already covered by the licensing. The ebuild defaults
already include README*
sys-fs/bees: Make warnings exclusive
It was recommended by Gentoo QA to show only either one or another
warning, and change the texts accordingly.
sys-fs/bees: RDEPEND is not implicit
RDEPEND does not implicitly default to DEPEND. Let's explicitly set the
variable.
sys-fs/bees: IUSE=test is only needed for explicit dependencies
Thus, remove it.
Signed-off-by: Kai Krakow <kai@kaishome.de>
Also split "bad feature interactions" into "unknown" (which is what it
really was before) and "bad" (which includes some filesystem-destroying
problems).
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
Add the new WARN_ON bug in v4.14.
Clarify what happens when bees is run on a kernel that is too old.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
Include a brief description of the two algorithms without getting
into too much detail for an ostensibly temporary feature.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
Distribute incoming extents across a thread pool for faster execution
on multi-core, multi-disk environments.
Switch extent enumeration model to scan extent refs consecutively(ish).
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
Bees will someday rely on features available only in kernel v4.14.
Let's start now by removing workarounds for bugs that were fixed in v4.11.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
This commit adds an ebuild for Gentoo. Version 9999 is building live
from current git, currently using kakra:integration because it has some
installation and build fixes important for Gentoo.
Signed-off-by: Kai Krakow <kai@kaishome.de>
If using `scripts/beesd`, we need `blkid` which is part of util-linux.
It should be available on every distribution but let's document it
anyway.
Signed-off-by: Kai Krakow <kai@kaishome.de>
Read-only snapshots have always just worked. Remove them from the
"untested" list.
nodatasum (and therefore nodatacow) inodes are simply ignored. This seems
like the right thing to do since deduping a nodatacow extent turns it
into a datacow extent, which seems contrary to administrator wishes
implied by the nodatacow bit. We probably need an option to
override that assumption.
Clarify why converted ext[234] filesystems may cause problems and
the nature of those problems.
Assorted minor editorial changes.
Discuss calculation of the balance limit parameter when ensuring
sufficient metadata space.
Update kernel version bug/fix/feature lists, including LOGICAL_INO_V2.
Annotate kernel workaround list with known kernel versions that make
the workarounds necessary.
Remove reference to 'DEFRAG_RANGE' as bees requires much more control
over data placement than this interface can offer. It's easy enough to
create a new ioctl to implement bees requirements once it's known what
those requirements are.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
"agent" is a nice generic term for the set of things that userspace
btrfs deduplicators are. Let's call it that.
Throw out the awkward and rambling "About" text and use the announcement
from linux-btrfs instead. Terrible English writing I at am.
