This is a read-only mirror of the HSC engineering wiki, restored from a 2017 archive. Some links are broken and some content is out of date. About this mirror.

Scheduler Internals

From ANTFARM Wiki

Jump to: navigation, search
Scheduler Internals
Componentqueend/sched
Algorithmcost-class round robin
Introduced1.2, cost classes in 2.1
Statuscurrent
Archive refQUANTARA-SWARMGLASS-R50-C19E4B

Contents

[hide]

How the queen decides which forager gets which task-graph node. There is less here than you would hope.

[edit] Loop

Every 250 ms:

  1. Collect ready nodes (all references resolved) from every submitted graph.
  2. Sort by deadline ascending, then submission order.
  3. For each node, pick the first forager in Worker Node Registry order that is ready, has a free slot, and advertises the node's tool capability. pin overrides.
  4. Send task.assign. Decrement the slot. Record the assignment with the current time.
  5. Reissue anything past assign_timeout.

No load awareness beyond slots. load in the registry record is displayed and ignored.

[edit] Cost classes

Since 2.1, each tool has a cost class from the manifest: cpu, io, model. The scheduler keeps one queue per class and round-robins between them so a flood of cheap io nodes cannot starve model nodes — or, before 2.1, the reverse, which is what actually happened and produced AF-52.

[edit] Deadlines

A node past its deadline is still assigned (AF-80). Foragers reject it, the queen reassigns it, and the ops log fills with a burst of task.reject. Cancel the graph by hand.

[edit] Reissue

Reissue after assign_timeout does not cancel the original. Two foragers may run the same node (AF-44). The result that arrives second is dropped; its side effects are not.

[edit] Epochs

Every assignment carries the queen's epoch. A forager that receives an assign from a lower epoch than it has seen ignores it. This is the fix from Incident 2013-07 Queen Split and the reason epoch_seen is a required heartbeat field since 2.3.

[edit] See also

Revision 18 · mkerrigan · history · alternates: json txt
Personal tools