To install for different distributions, LIBEXEC_PREFIX can now be set.
It defaults to $(PREFIX)/usr/lib/bees as used in most common
distributions.
Local overrides are possible by setting variables in a "localconf" file
which will be included by the Makefile if it exists.
For some distributions you may want to set it to /usr/libexec or
/usr/libexec/bees.
Let's remove the CPUQuota example and instead give bees a share of
what's available.
128 CPU shares will give it about 12% max CPU under load, give it a
slight boost during startup to allow reading the hash table faster.
100 block shares will give it about 10% max disk bandwidht under load,
give it a slight boost during startup to allow reading the hash table
faster.
Then let's adjust the CPU and IO scheduler to prefer other processes.
This way bees runs completely in the background, barely noticable
during, e.g., gaming.
Explicitly set control-group kill mode, that is: try SIGTERM first, and
use SIGKILL after a timeout. This exactly defines how bees is running as
a child process within the frontend service starter. Not sure if bees cares
about signals but SIGTERM first seems cleaner. On the way, let bees restart
on abnormal termination.
After a few hundred subvol threads start running, the inode cache starts
to thrash, and the log gets spammed with messages of the form:
"open_root_nocache <subvolid>: <path>"
Ideally there would be some way to schedule work to minimize inode
thrashing. Until that gets done, just silence the messages for now.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
After a few hundred subvol threads start running, the inode cache starts
to thrash, and the log gets spammed with messages of the form:
"open_root_nocache <subvolid>: <path>"
Ideally there would be some way to schedule work to minimize inode
thrashing. Until that gets done, just silence the messages for now.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
With many threads it is inconvenient to reassemble the elided parts of
the dedup src/dst and scan filenames output. Simply output them
unconditionally, and balance the line lengths.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
Tune the concurrency model to work a little better with large numbers
of subvols. This is much less than the full rewrite Bees desparately
needs, but it provides a marginal improvement until the new code is ready.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
With many threads it is inconvenient to reassemble the elided parts of
the dedup src/dst and scan filenames output. Simply output them
unconditionally, and balance the line lengths.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
This code has been #if 0 for a long time, and it seems unlikely it
will ever be useful in the future.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
If we lose a race and open the wrong file, we will not retry with the
next path if the file we opened had incompatible flags. We need to keep
trying paths until we open the correct file or run out of paths.
Fix by moving the inode flag check after the checks for file identity.
Output attributes in hex to be consistent with other attribute error
messages.
There is no need to report root and file paths separately in the error
message for incompatible flags because we have confirmed the identity of
the file before the incompatible flag error is detected. Other messages
in this loop still output root path and file_path separately because
the identity of 'rv' is unknown at the time these messages are emitted.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
If you have a lot of or a few big nocow files (like vm images) which
contain a lot of potential deduplication candidates, bees becomes
incredibly slow running through a lot "invalid operation" exceptions.
Let's just skip over such files to get more bang for the buck. I did no
regression testing as this patch seems trivial (and I cannot imagine any
pitfalls either). The process progresses much faster for me now.
If we lose a race and open the wrong file, we will not retry with the
next path if the file we opened had incompatible flags. We need to keep
trying paths until we open the correct file or run out of paths.
Fix by moving the inode flag check after the checks for file identity.
Output attributes in hex to be consistent with other attribute error
messages.
There is no need to report root and file paths separately in the error
message for incompatible flags because we have confirmed the identity of
the file before the incompatible flag error is detected. Other messages
in this loop still output root path and file_path separately because
the identity of 'rv' is unknown at the time these messages are emitted.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
If you have a lot of or a few big nocow files (like vm images) which
contain a lot of potential deduplication candidates, bees becomes
incredibly slow running through a lot "invalid operation" exceptions.
Let's just skip over such files to get more bang for the buck. I did no
regression testing as this patch seems trivial (and I cannot imagine any
pitfalls either). The process progresses much faster for me now.
Keep track of the locking thread so we can see why we are deadlocked
in gdb.
Use a handle type for locks based on shared_ptr. Change the handle type
name to flush out any non-auto local variables.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
(cherry picked from commit aa0b22d445)
This helps identify causes of the "same physical address in dedup"
exception.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
(cherry picked from commit cc7b4f22b5)
BLOCK_SIZE_MIN_EXTENT_DEFRAG, BLOCK_SIZE_MIN_EXTENT_SPLIT, and others
are no longer used. Remove them.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
(cherry picked from commit a3d7032eda)
Add time spent in file create and copy operations to the stats.
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
(cherry picked from commit f01c20f972)
A BEESTRACE closure could throw an exception. Trap those so we don't
end up in terminate().
Signed-off-by: Zygo Blaxell <bees@furryterror.org>
(cherry picked from commit 59660cfc00)
