Zygo Blaxell 31d26bcfc6 roots: organize scan workers by inode instead of extent ...

Split crawlers into two separate Tasks:

 1. a Task which locates the next inode with a new data extent.

 2. a Task which scans every new extent in that inode.

This simplifies some lock contention and execution ordering issues.
Files are read sequentially.  Workers dynamically scale up or
down as needed, without creating thousands of deferred Task objects.
Workers obtain inode locks for different inodes in btrfs, so they
can work in parallel instead of waiting for each other.

This change in behavior comes with new names for the worker Tasks:

        "crawl_master" is now "crawl_more", the singular Task which
        creates inode-scanning Tasks.

        "crawl_<subvol>" is now "crawl_<subvol>_<inode>".

Signed-off-by: Zygo Blaxell <bees@furryterror.org>

2022-12-20 20:51:00 -05:00

35 KiB

Raw Blame History

View Raw