GGLinnk/bees
1
0
Fork 0
mirror of https://github.com/Zygo/bees.git synced 2025-05-17 21:35:45 +02:00
Code Packages

task: use exponential backoff algorithm to set thread count

Browse Source 
Tasks are often running longer than 5 seconds (especially extents with
multiple references requiring copy operations), so the load tracking
algorithm needs to average several samples over a longer period of time
than 5 seconds.  If the sample period is 60 seconds, we end up recomputing
the original load average from current_load, so skip the rounding error
and use the original load average value.

Arguably the real fix is to break up the more complex extent operations
over several downstream Task objects, but that's a more significant
design change.

Tweak the attack and decay rates so that threads are started a little
more slowly, but still stopped rapidly when load spikes up.

Remove the hysteresis to provide support for load average targets
below 1, or with fractional components, with a PWM-like effect.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
This commit is contained in:
Zygo Blaxell 2022-12-04 23:56:35 -05:00
parent a85ada3a49
commit b143664747
1 changed files with 7 additions and 11 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
lib/task.cc
View File

@ -499,17 +499,13 @@ namespace crucible {
m_prev_loadavg = loadavg; m_prev_loadavg = loadavg;
// Change the thread target based on the const double load_deficit = m_load_target - loadavg;
// difference between current and desired load if (load_deficit > 0) {
// but don't get too close all at once due to rounding and sample error. // Load is too low, solve by adding another worker
// If m_load_target < 1.0 then we are just doing PWM with one thread. m_thread_target += load_deficit / 3;
} else if (load_deficit < 0) {
if (m_load_target <= 1.0) { // Load is too high, solve by removing all known excess tasks
m_thread_target = 1.0; m_thread_target += load_deficit;
} else if (m_load_target - current_load >= 1.0) {
m_thread_target += (m_load_target - current_load - 1.0) / 2.0;
} else if (m_load_target < current_load) {
m_thread_target += m_load_target - current_load;
} }
m_load_stats = TaskMaster::LoadStats { m_load_stats = TaskMaster::LoadStats {