GGLinnk/bees
1
0
Fork 0
mirror of https://github.com/Zygo/bees.git synced 2025-07-06 02:22:27 +02:00
Code Packages Activity

bees: process each subvol in its own thread

Browse Source 
This is yet another multi-threaded Bees experiment.

This time we are dividing the work by subvol:  one thread is created to
process each subvol in the filesystem.  There is no change in behavior
on filesystems containing only one subvol.

In order to avoid or mitigate the impact of kernel bugs and performance
issues, the btrfs ioctls FILE_EXTENT_SAME, SEARCH_V2, and LOGICAL_INO are
serialized.  Only one thread may execute any of these ioctls at any time.
All three ioctls share a single lock.

In order to simplify the implementation, only one thread is permitted to
create a temporary file during one call to scan_one_extent.  This prevents
multiple threads from racing to replace the same physical extent with
separate physical copies.

The single "crawl" thread is replaced by one "crawl_<root_number>"
for each subvol.

The crawl size is reduced from 4096 items to 1024.  This reduces the
memory requirement per subvol and keeps the data in memory fresher.
It also increases the number of log messages, so turn some of them off.

TODO:  Currently there is no configurable limit on the total number
of threads.  The number of CPUs is used as an upper bound on the number
of active threads; however, we still have one thread per subvol even if
all most of the threads do is wait for locks.

TODO:  Some of the single-threaded code is left behind until I make up
my mind about whether this experiment is successful.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
This commit is contained in:
Zygo Blaxell
2017-01-08 23:50:29 -05:00
parent 4113a171be
commit b22b4ed427
4 changed files with 120 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files

28
src/bees-context.cc
View File

@ -260,9 +260,7 @@ BeesContext::BeesContext(shared_ptr<BeesContext> parent) :
bool
BeesContext::dedup(const BeesRangePair &brp)
{
// TOOLONG and NOTE can retroactively fill in the filename details, but LOG can't
BEESNOTE("dedup " << brp);
// Open the files
brp.first.fd(shared_from_this());
brp.second.fd(shared_from_this());
@ -273,6 +271,12 @@ BeesContext::dedup(const BeesRangePair &brp)
bees_sync(brp.first.fd());
#endif
// To avoid hammering all the cores with long-running ioctls,
// only do one dedup at any given time.
BEESNOTE("Waiting to dedup " << brp);
unique_lock<mutex> lock(bees_ioctl_mutex);
BEESNOTE("dedup " << brp);
BEESTOOLONG("dedup " << brp);
thread_local BeesFileId tl_first_fid, tl_second_fid;
@ -436,6 +440,9 @@ BeesContext::scan_one_extent(const BeesFileRange &bfr, const Extent &e)
BEESTRACE(e << " block_count " << block_count);
string bar(block_count, '#');
// Only one thread may create tmpfiles at any given time
unique_lock<mutex> tmpfile_lock(bees_tmpfile_mutex, defer_lock);
for (off_t next_p = e.begin(); next_p < e.end(); ) {
// Guarantee forward progress
@ -736,6 +743,10 @@ BeesContext::scan_one_extent(const BeesFileRange &bfr, const Extent &e)
// BEESLOG("noinsert_set.count(" << to_hex(p) << ") " << noinsert_set.count(p));
if (noinsert_set.count(p)) {
if (p - last_p > 0) {
if (!tmpfile_lock) {
BEESNOTE("waiting for tmpfile");
tmpfile_lock.lock();
}
rewrite_file_range(BeesFileRange(bfr.fd(), last_p, p));
blocks_rewritten = true;
}
@ -747,6 +758,10 @@ BeesContext::scan_one_extent(const BeesFileRange &bfr, const Extent &e)
}
BEESTRACE("last");
if (next_p - last_p > 0) {
if (!tmpfile_lock) {
BEESNOTE("waiting for tmpfile");
tmpfile_lock.lock();
}
rewrite_file_range(BeesFileRange(bfr.fd(), last_p, next_p));
blocks_rewritten = true;
}
@ -868,12 +883,19 @@ BeesContext::resolve_addr_uncached(BeesAddress addr)
{
THROW_CHECK1(invalid_argument, addr, !addr.is_magic());
THROW_CHECK0(invalid_argument, !!root_fd());
// To avoid hammering all the cores with long-running ioctls,
// only do one resolve at any given time.
BEESNOTE("waiting to resolve addr " << addr);
unique_lock<mutex> lock(bees_ioctl_mutex);
Timer resolve_timer;
// There is no performance benefit if we restrict the buffer size.
BtrfsIoctlLogicalInoArgs log_ino(addr.get_physical_or_zero());
{
BEESNOTE("resolving addr " << addr);
BEESTOOLONG("Resolving addr " << addr << " in " << root_path() << " refs " << log_ino.m_iors.size());
if (log_ino.do_ioctl_nothrow(root_fd())) {
BEESCOUNT(resolve_ok);