GGLinnk/bees
1
0
Fork 0
mirror of https://github.com/Zygo/bees.git synced 2025-05-18 05:45:45 +02:00
Code Packages

context: allow concurrent dedup

Browse Source 
Dedup was spending a lot of time waiting for the ioctl mutex while it
was held by non-dedup ioctls; however, when dedup finally locked the
mutex, its average run time was comparatively short and the variance
was low.

With the various workarounds and kernel fixes in place, FILE_EXTENT_SAME
and bees play well enough together that we can allow multiple threads
to do dedup at the same time.  The extent bytenr lockset should be
sufficient to prevent undesirable results (i.e. dup data not removed,
or deadlocks on old kernels).

Remove the ioctl lock on dedup.

LOGICAL_INO and SEARCH_V2 (as used by BeesCrawl) remain under the ioctl
mutex because they can still have abitrarily large run times.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
This commit is contained in:
Zygo Blaxell 2017-01-12 23:01:24 -05:00
parent e46b96d23c
commit be1aa049c6
1 changed files with 6 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
src/bees-context.cc
View File

@ -272,10 +272,16 @@ BeesContext::dedup(const BeesRangePair &brp)
bees_sync(brp.first.fd()); bees_sync(brp.first.fd());
#endif #endif
// dedup isn't very long-running compared to LOGICAL_INO.
// Also we are approaching saturation on systems with many cores.
// Hopefully the lock on the extent bytenr will avoid hitting
// kernel deadlocks too often!
#if 0
// To avoid hammering all the cores with long-running ioctls, // To avoid hammering all the cores with long-running ioctls,
// only do one dedup at any given time. // only do one dedup at any given time.
BEESNOTE("Waiting to dedup " << brp); BEESNOTE("Waiting to dedup " << brp);
unique_lock<mutex> lock(bees_ioctl_mutex); unique_lock<mutex> lock(bees_ioctl_mutex);
#endif
BEESNOTE("dedup " << brp); BEESNOTE("dedup " << brp);
BEESTOOLONG("dedup " << brp); BEESTOOLONG("dedup " << brp);