1
0
mirror of https://github.com/Zygo/bees.git synced 2025-05-17 21:35:45 +02:00

241 Commits

Author SHA1 Message Date
Kai Krakow
bfb768a079 Fix a fallthrough error in GCC 7+
GCC 7 and higher turn a previous warning into an error for implicit
fallthrough. Let's hint the compiler that this is intentional here.

Signed-off-by: Kai Krakow <kai@kaishome.de>
(cherry picked from commit 270a91cf17783813edaa75fb9766858dcf8e1689)
2017-11-14 11:26:47 -05:00
Kai Krakow
3024e43355 Fix a fallthrough error in GCC 7+
GCC 7 and higher turn a previous warning into an error for implicit
fallthrough. Let's hint the compiler that this is intentional here.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2017-11-14 07:00:28 +01:00
Kai Krakow
270a91cf17 Fix a fallthrough error in GCC 7+
GCC 7 and higher turn a previous warning into an error for implicit
fallthrough. Let's hint the compiler that this is intentional here.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2017-11-14 06:58:43 +01:00
Kai Krakow
93ba0f48de Make clear that options must be supplied in one variable
Previously, expectations may fail when just uncommenting both lines.
2017-11-14 06:58:43 +01:00
Kai Krakow
d930136484 Remove process forking from frontend script
Now with the patches integrated to filter logging output, we can finally
remove forking a subprocess and stop redirecting file descriptors.

We instead use exec to replace the process with the final daemon.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2017-11-14 06:58:43 +01:00
Kai Krakow
f7320baa56 Fix indentation/alignment after integration 2017-11-14 06:58:43 +01:00
Kai Krakow
21212cd3e3 Fix example config for timestamp logging 2017-11-14 06:58:43 +01:00
Kai Krakow
0c6a4d00c8 Remove filter path logic from frontend script
Now with relative path filtering in place, we can now give sub spawning
subshells in the frontend script.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2017-11-14 01:16:06 +01:00
Kai Krakow
52997936d5 getopt: Add logic to set relative path from $CWD
This commit adds a new option to set relative path output for name_fd().

Signed-off-by: Kai Krakow <kai@kaishome.de>
2017-11-14 01:16:06 +01:00
Kai Krakow
755f16a948 crucible: Allow setting a relative path option for name_fd()
This commit adds an option to store a relative path in prepartion for
more human-friendly log output.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2017-11-14 01:16:06 +01:00
Zygo Blaxell
71514e7229 main: use static function to control timestamps in log output
Adjust bees to match changes in Chatter's interface.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
(cherry picked from commit 66fd28830d42acbd837dfb08b89a1f9c05c6d474)
2017-11-11 15:18:46 -05:00
Zygo Blaxell
78d04b1417 chatter: use static function to control timestamping behavior
Use a static function instead of embedding side-effects in the constructor
of an unrelated class.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
(cherry picked from commit 85106bd9a9fb3be2c574df1a3aed6674e3951465)
2017-11-11 15:18:46 -05:00
Zygo Blaxell
66fd28830d main: use static function to control timestamps in log output
Adjust bees to match changes in Chatter's interface.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-11-11 15:14:52 -05:00
Zygo Blaxell
85106bd9a9 chatter: use static function to control timestamping behavior
Use a static function instead of embedding side-effects in the constructor
of an unrelated class.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-11-11 15:13:41 -05:00
Zygo Blaxell
bb273770c5 Merge remote-tracking branch 'kakra/integration' into subvol-threads 2017-11-11 14:38:17 -05:00
Kai Krakow
47805253e6 Make service starter accept bees options
The service starter wasn't able to pass options to the new getopt
parser. This commit fixes it.
2017-10-28 00:14:36 +02:00
Kai Krakow
629e33b4f3 Fix naming 2017-10-28 00:13:38 +02:00
Kai Krakow
58157d03dd Add beesd generated script to gitignore 2017-10-28 00:13:23 +02:00
Kai Krakow
9d67329ef7 Update README after integrating new features
Let's update the README file.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2017-10-27 23:10:14 +02:00
Kai Krakow
c6be07e158 Add option for prefixing timestamps
To make bees more friendly to use with syslog/systemd, we add an option
to omit timestamps from the log output.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2017-10-27 23:02:47 +02:00
Kai Krakow
c6bf6bfe1d Implement getopt options parser
This commit adds a simple getopt options parser to show help. This can
be used as a boilerplate for adding more options later.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2017-10-27 22:36:00 +02:00
Zygo Blaxell
b7e316b005 roots: clean out dead code around crawl locks
Remove a number of #if 0's.

Remove the redundant thread yield after implementing the same or better
in LockSet.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-10-02 08:33:30 -04:00
Zygo Blaxell
1aa1decd1d lockset: drop unused method wait_unlock
This function is not used and does not appear to be useful.

Remove it.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-10-01 16:18:48 -04:00
Zygo Blaxell
8ea92202fc lockset: avoid starvation using a priority queue
Mutex locks are released and acquired unfairly, causing arbitrary delays
in acquiring locks.  This prevents threads from releasing subvol FD's
which in turn blocks subvol deletes.

Fix by implementing a priority queue in LockSet to ensure fairness.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-10-01 16:18:47 -04:00
Zygo Blaxell
a3cd3ca07f crucible: add cleanup class
Store a function (or closure) in an instance and invoke the function
from the destructor.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-10-01 16:18:47 -04:00
Zygo Blaxell
5a8c655fc4 roots: filter out obsolete extents from extent refs
When an extent ref is modified, all of the refs in the same metadata
page get the same transid in the TREE_SEARCH_V2 header.  All of the
extents are rescanned by later subvol scans.  This causes up to 80%
overhead due to redundant reads of the same extents.

A proper fix for this requires extent-based scanning instead of
extent-ref-based scanning.  Until that happens, filter out new references
to old extents.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-10-01 16:18:47 -04:00
Zygo Blaxell
16432d0bb7 roots: remove open_root_cache correctly
BEESNOTE puts a message on the status message stack.  BEESINFO logs a
message with rate limiting.  The message that was flooding the logs
was coming from BEESINFO not BEESNOTE.

Fix earlier commit which removed the wrong message.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-10-01 16:18:47 -04:00
Zygo Blaxell
b9dc4792bc Merge branch 'master' into subvol-threads 2017-10-01 15:24:02 -04:00
Kai Krakow
29d2d51c47 Fix libexec prefix discrepancy
Whoops...
2017-09-20 21:50:58 +02:00
Kai Krakow
893595190f Allow custom libexec location
To install for different distributions, LIBEXEC_PREFIX can now be set.
It defaults to $(PREFIX)/usr/lib/bees as used in most common
distributions.

Local overrides are possible by setting variables in a "localconf" file
which will be included by the Makefile if it exists.

For some distributions you may want to set it to /usr/libexec or
/usr/libexec/bees.
2017-09-20 21:00:54 +02:00
Kai Krakow
0455827989 Adjust CPU and IO shares when running under systemd
Let's remove the CPUQuota example and instead give bees a share of
what's available.

128 CPU shares will give it about 12% max CPU under load, give it a
slight boost during startup to allow reading the hash table faster.

100 block shares will give it about 10% max disk bandwidht under load,
give it a slight boost during startup to allow reading the hash table
faster.

Then let's adjust the CPU and IO scheduler to prefer other processes.
This way bees runs completely in the background, barely noticable
during, e.g., gaming.
2017-09-19 21:07:37 +02:00
Kai Krakow
62626aef7f Adjust service restart and shutdown behavior
Explicitly set control-group kill mode, that is: try SIGTERM first, and
use SIGKILL after a timeout. This exactly defines how bees is running as
a child process within the frontend service starter. Not sure if bees cares
about signals but SIGTERM first seems cleaner. On the way, let bees restart
on abnormal termination.
2017-09-19 21:07:37 +02:00
Kai Krakow
f59e311809 Explicitly mark systemd unit as Type=simple
Bees does not fork, so let's not rely on systemd defaults.
2017-09-19 21:07:37 +02:00
Kai Krakow
3bf4e69c4d Make config example more clear
A pre-defined UUID should not be part of the sample config file.

Instead, make it more clear how the config file is intended to be used.
2017-09-19 21:06:18 +02:00
Kai Krakow
5622ebd411 Bees is meant to be run as root only
As bees is meant to be run as root only, move it to /usr/sbin which is
usually not part of normal users path environment.
2017-09-19 20:32:09 +02:00
Kai Krakow
04cb25bd04 Move bees to libexec install dir
When bees is meant to be run mainly through the service frontend script,
we should move the bees binary to the libexec directory.
2017-09-19 20:30:51 +02:00
Kai Krakow
b2f000ad7a Change README.md reflecting nodatacow inode attribute
The previous patch changed behavior regarding nodatacow inode attribute.
Let's document the new behavior.
2017-09-18 22:35:07 -04:00
Zygo Blaxell
06b8fd8697 Merge remote-tracking branch 'kakra/master' 2017-09-18 22:34:20 -04:00
Zygo Blaxell
94ab477b90 Merge remote-tracking branch 'kakra/feature/markdown-detection' 2017-09-18 22:32:41 -04:00
Kai Krakow
415756fb99 Enable detect of markdown binary
Some distributions do not provide markdown as "markdown". Let's figure
out which version to use during build.
2017-09-18 22:32:36 -04:00
Kai Krakow
cceb0480a5 Change README.md reflecting nodatacow inode attribute
The previous patch changed behavior regarding nodatacow inode attribute.
Let's document the new behavior.
2017-09-18 01:45:20 +02:00
Kai Krakow
23749eb634 Enable detect of markdown binary
Some distributions do not provide markdown as "markdown". Let's figure
out which version to use during build.
2017-09-18 01:30:59 +02:00
Zygo Blaxell
175d7fc10e roots: drop open_root_nocache log entry
After a few hundred subvol threads start running, the inode cache starts
to thrash, and the log gets spammed with messages of the form:

	"open_root_nocache <subvolid>: <path>"

Ideally there would be some way to schedule work to minimize inode
thrashing.  Until that gets done, just silence the messages for now.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-09-16 21:26:26 -04:00
Zygo Blaxell
5afbcb99e3 roots: drop open_root_nocache log entry
After a few hundred subvol threads start running, the inode cache starts
to thrash, and the log gets spammed with messages of the form:

	"open_root_nocache <subvolid>: <path>"

Ideally there would be some way to schedule work to minimize inode
thrashing.  Until that gets done, just silence the messages for now.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-09-16 21:16:40 -04:00
Zygo Blaxell
5275249396 roots: trace transid_max calculation
transid_max calculations can take considerable time.  Report their
progress in more detail.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-09-16 17:30:45 -04:00
Zygo Blaxell
a07728bc7e tmpfiles: note that kernel race condition is not yet fixed
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-09-16 17:30:36 -04:00
Zygo Blaxell
732896b471 log: simplify output for dedup and scan
With many threads it is inconvenient to reassemble the elided parts of
the dedup src/dst and scan filenames output.  Simply output them
unconditionally, and balance the line lengths.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-09-16 17:30:30 -04:00
Zygo Blaxell
5cc5a44661 bees: drop unused BeesWorkQueue classes
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-09-16 17:30:22 -04:00
Zygo Blaxell
f6a6992ac9 README: update list of currently known kernel bugs
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-09-16 17:28:50 -04:00
Zygo Blaxell
ceda8ee6c3 Makefile: add test to PHONY list
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-09-16 17:27:36 -04:00