From be1aa049c63517721640755b00291fc696076087 Mon Sep 17 00:00:00 2001 From: Zygo Blaxell Date: Thu, 12 Jan 2017 23:01:24 -0500 Subject: [PATCH] context: allow concurrent dedup Dedup was spending a lot of time waiting for the ioctl mutex while it was held by non-dedup ioctls; however, when dedup finally locked the mutex, its average run time was comparatively short and the variance was low. With the various workarounds and kernel fixes in place, FILE_EXTENT_SAME and bees play well enough together that we can allow multiple threads to do dedup at the same time. The extent bytenr lockset should be sufficient to prevent undesirable results (i.e. dup data not removed, or deadlocks on old kernels). Remove the ioctl lock on dedup. LOGICAL_INO and SEARCH_V2 (as used by BeesCrawl) remain under the ioctl mutex because they can still have abitrarily large run times. Signed-off-by: Zygo Blaxell --- src/bees-context.cc | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/bees-context.cc b/src/bees-context.cc index a55e83a..dab36aa 100644 --- a/src/bees-context.cc +++ b/src/bees-context.cc @@ -272,10 +272,16 @@ BeesContext::dedup(const BeesRangePair &brp) bees_sync(brp.first.fd()); #endif + // dedup isn't very long-running compared to LOGICAL_INO. + // Also we are approaching saturation on systems with many cores. + // Hopefully the lock on the extent bytenr will avoid hitting + // kernel deadlocks too often! +#if 0 // To avoid hammering all the cores with long-running ioctls, // only do one dedup at any given time. BEESNOTE("Waiting to dedup " << brp); unique_lock lock(bees_ioctl_mutex); +#endif BEESNOTE("dedup " << brp); BEESTOOLONG("dedup " << brp);