1
0
mirror of https://github.com/Zygo/bees.git synced 2025-06-16 17:46:16 +02:00
Commit Graph

597 Commits

Author SHA1 Message Date
abeb6e74b2 Add scripts to "make all" target
This prevents scripts being generated by "root" during "sudo make
install" phase.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-09 00:48:14 +01:00
6c67ae0d5e Don't zap localconf in "make clean"
When you run "make clean", localconf is being removed. This is probably
in most cases not intentional.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-09 00:48:14 +01:00
ba981c133a Merge remote-tracking branches 'kakra/feature/add-relative-path-option' and 'kakra/integration' 2018-01-07 21:39:01 -05:00
9d295fab4e Makefile: if multiple Markdown utilities are present, use the first one
If two utilities are found, we get commands like

	/usr/bin/markdown /usr/bin/markdown_py README.md > README.html

and that doesn't work.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
v0.5
2018-01-07 20:23:20 -05:00
dc7360397e README: update the state of bees and the kernel for v4.14
Read-only snapshots have always just worked.  Remove them from the
"untested" list.

nodatasum (and therefore nodatacow) inodes are simply ignored.  This seems
like the right thing to do since deduping a nodatacow extent turns it
into a datacow extent, which seems contrary to administrator wishes
implied by the nodatacow bit.  We probably need an option to
override that assumption.

Clarify why converted ext[234] filesystems may cause problems and
the nature of those problems.

Assorted minor editorial changes.

Discuss calculation of the balance limit parameter when ensuring
sufficient metadata space.

Update kernel version bug/fix/feature lists, including LOGICAL_INO_V2.

Annotate kernel workaround list with known kernel versions that make
the workarounds necessary.

Remove reference to 'DEFRAG_RANGE' as bees requires much more control
over data placement than this interface can offer.  It's easy enough to
create a new ioctl to implement bees requirements once it's known what
those requirements are.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2018-01-07 20:23:20 -05:00
305ab5dbfa Merge remote-tracking branch 'nefelim4ag/master' 2018-01-06 22:54:49 -05:00
80e4302958 Update btrfs compression types, add ZSTD, drop LAST
Signed-off-by: Timofey Titovets <nefelim4ag@gmail.com>
2018-01-04 20:32:04 +03:00
07751885d2 error: drop redundant CHECK_CONSTRAINT
CHECK_CONSTRAINT is just THROW_CHECK1 with an inconsistent name.
Remove it.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-12-21 14:00:17 -05:00
77614a0e99 scan: insert toxic matched extents into hash table as they are discovered
When a toxic extent is discovered, insert the offending hash/address/toxic
entry into the hash table.

When a previously discovered toxic extent is encountered, do nothing,
i.e. allow the offending hash/address/toxic entry in the hash table
to expire.

Previously both inserts were removed from the code, but the former one
is required.  The latter prevents bees from forgiving toxic extents
(or any hash matching one) should they be relocated, deleted, or simply
become non-toxic.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-12-21 13:56:15 -05:00
649ae5bb40 makeflags: fix missing -D_FILE_OFFSET_BITS=64 in comment
Interesting things happen when blindly swapping the release-build CCFLAGS
with the debug-build commented-out CCFLAGS.  None of these things that
happen are good.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-12-21 13:28:49 -05:00
40112faf0f Makefile add scripts target for correctly packaging
Currently scheme lead to path like:
/tmp/makepkg/bees-git/pkg/bees-git/usr/lib/bees/bees

While packaging, so allow do:
make
make scripts
make install ...
make install_scripts ...

Signed-off-by: Timofey Titovets <nefelim4ag@gmail.com>
2017-12-21 13:26:45 +03:00
bfb768a079 Fix a fallthrough error in GCC 7+
GCC 7 and higher turn a previous warning into an error for implicit
fallthrough. Let's hint the compiler that this is intentional here.

Signed-off-by: Kai Krakow <kai@kaishome.de>
(cherry picked from commit 270a91cf17)
2017-11-14 11:26:47 -05:00
3024e43355 Fix a fallthrough error in GCC 7+
GCC 7 and higher turn a previous warning into an error for implicit
fallthrough. Let's hint the compiler that this is intentional here.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2017-11-14 07:00:28 +01:00
270a91cf17 Fix a fallthrough error in GCC 7+
GCC 7 and higher turn a previous warning into an error for implicit
fallthrough. Let's hint the compiler that this is intentional here.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2017-11-14 06:58:43 +01:00
93ba0f48de Make clear that options must be supplied in one variable
Previously, expectations may fail when just uncommenting both lines.
2017-11-14 06:58:43 +01:00
d930136484 Remove process forking from frontend script
Now with the patches integrated to filter logging output, we can finally
remove forking a subprocess and stop redirecting file descriptors.

We instead use exec to replace the process with the final daemon.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2017-11-14 06:58:43 +01:00
f7320baa56 Fix indentation/alignment after integration 2017-11-14 06:58:43 +01:00
21212cd3e3 Fix example config for timestamp logging 2017-11-14 06:58:43 +01:00
0c6a4d00c8 Remove filter path logic from frontend script
Now with relative path filtering in place, we can now give sub spawning
subshells in the frontend script.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2017-11-14 01:16:06 +01:00
52997936d5 getopt: Add logic to set relative path from $CWD
This commit adds a new option to set relative path output for name_fd().

Signed-off-by: Kai Krakow <kai@kaishome.de>
2017-11-14 01:16:06 +01:00
755f16a948 crucible: Allow setting a relative path option for name_fd()
This commit adds an option to store a relative path in prepartion for
more human-friendly log output.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2017-11-14 01:16:06 +01:00
71514e7229 main: use static function to control timestamps in log output
Adjust bees to match changes in Chatter's interface.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
(cherry picked from commit 66fd28830d)
2017-11-11 15:18:46 -05:00
78d04b1417 chatter: use static function to control timestamping behavior
Use a static function instead of embedding side-effects in the constructor
of an unrelated class.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
(cherry picked from commit 85106bd9a9)
2017-11-11 15:18:46 -05:00
47805253e6 Make service starter accept bees options
The service starter wasn't able to pass options to the new getopt
parser. This commit fixes it.
2017-10-28 00:14:36 +02:00
629e33b4f3 Fix naming 2017-10-28 00:13:38 +02:00
58157d03dd Add beesd generated script to gitignore 2017-10-28 00:13:23 +02:00
9d67329ef7 Update README after integrating new features
Let's update the README file.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2017-10-27 23:10:14 +02:00
c6be07e158 Add option for prefixing timestamps
To make bees more friendly to use with syslog/systemd, we add an option
to omit timestamps from the log output.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2017-10-27 23:02:47 +02:00
c6bf6bfe1d Implement getopt options parser
This commit adds a simple getopt options parser to show help. This can
be used as a boilerplate for adding more options later.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2017-10-27 22:36:00 +02:00
29d2d51c47 Fix libexec prefix discrepancy
Whoops...
2017-09-20 21:50:58 +02:00
893595190f Allow custom libexec location
To install for different distributions, LIBEXEC_PREFIX can now be set.
It defaults to $(PREFIX)/usr/lib/bees as used in most common
distributions.

Local overrides are possible by setting variables in a "localconf" file
which will be included by the Makefile if it exists.

For some distributions you may want to set it to /usr/libexec or
/usr/libexec/bees.
2017-09-20 21:00:54 +02:00
0455827989 Adjust CPU and IO shares when running under systemd
Let's remove the CPUQuota example and instead give bees a share of
what's available.

128 CPU shares will give it about 12% max CPU under load, give it a
slight boost during startup to allow reading the hash table faster.

100 block shares will give it about 10% max disk bandwidht under load,
give it a slight boost during startup to allow reading the hash table
faster.

Then let's adjust the CPU and IO scheduler to prefer other processes.
This way bees runs completely in the background, barely noticable
during, e.g., gaming.
2017-09-19 21:07:37 +02:00
62626aef7f Adjust service restart and shutdown behavior
Explicitly set control-group kill mode, that is: try SIGTERM first, and
use SIGKILL after a timeout. This exactly defines how bees is running as
a child process within the frontend service starter. Not sure if bees cares
about signals but SIGTERM first seems cleaner. On the way, let bees restart
on abnormal termination.
2017-09-19 21:07:37 +02:00
f59e311809 Explicitly mark systemd unit as Type=simple
Bees does not fork, so let's not rely on systemd defaults.
2017-09-19 21:07:37 +02:00
3bf4e69c4d Make config example more clear
A pre-defined UUID should not be part of the sample config file.

Instead, make it more clear how the config file is intended to be used.
2017-09-19 21:06:18 +02:00
5622ebd411 Bees is meant to be run as root only
As bees is meant to be run as root only, move it to /usr/sbin which is
usually not part of normal users path environment.
2017-09-19 20:32:09 +02:00
04cb25bd04 Move bees to libexec install dir
When bees is meant to be run mainly through the service frontend script,
we should move the bees binary to the libexec directory.
2017-09-19 20:30:51 +02:00
06b8fd8697 Merge remote-tracking branch 'kakra/master' 2017-09-18 22:34:20 -04:00
94ab477b90 Merge remote-tracking branch 'kakra/feature/markdown-detection' 2017-09-18 22:32:41 -04:00
cceb0480a5 Change README.md reflecting nodatacow inode attribute
The previous patch changed behavior regarding nodatacow inode attribute.
Let's document the new behavior.
2017-09-18 01:45:20 +02:00
23749eb634 Enable detect of markdown binary
Some distributions do not provide markdown as "markdown". Let's figure
out which version to use during build.
2017-09-18 01:30:59 +02:00
5afbcb99e3 roots: drop open_root_nocache log entry
After a few hundred subvol threads start running, the inode cache starts
to thrash, and the log gets spammed with messages of the form:

	"open_root_nocache <subvolid>: <path>"

Ideally there would be some way to schedule work to minimize inode
thrashing.  Until that gets done, just silence the messages for now.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-09-16 21:16:40 -04:00
5275249396 roots: trace transid_max calculation
transid_max calculations can take considerable time.  Report their
progress in more detail.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-09-16 17:30:45 -04:00
a07728bc7e tmpfiles: note that kernel race condition is not yet fixed
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-09-16 17:30:36 -04:00
732896b471 log: simplify output for dedup and scan
With many threads it is inconvenient to reassemble the elided parts of
the dedup src/dst and scan filenames output.  Simply output them
unconditionally, and balance the line lengths.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-09-16 17:30:30 -04:00
5cc5a44661 bees: drop unused BeesWorkQueue classes
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-09-16 17:30:22 -04:00
f6a6992ac9 README: update list of currently known kernel bugs
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-09-16 17:28:50 -04:00
ceda8ee6c3 Makefile: add test to PHONY list
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-09-16 17:27:36 -04:00
18ae15658e README: remove stray whitespace
No content changes.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-09-16 14:57:24 -04:00
339579096f roots: move flags check after file identity checks and make error message style consistent
If we lose a race and open the wrong file, we will not retry with the
next path if the file we opened had incompatible flags.  We need to keep
trying paths until we open the correct file or run out of paths.
Fix by moving the inode flag check after the checks for file identity.

Output attributes in hex to be consistent with other attribute error
messages.

There is no need to report root and file paths separately in the error
message for incompatible flags because we have confirmed the identity of
the file before the incompatible flag error is detected.  Other messages
in this loop still output root path and file_path separately because
the identity of 'rv' is unknown at the time these messages are emitted.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-09-16 14:49:09 -04:00