Test Load Balancer(TLB)

This criterion splits the test suite into a given number of units such that each unit has an equal or optimally equal number of tests.

For example:

• 20 tests and 1 unit => 1 unit with 20 tests.
• 18 tests and 3 units => 3 units with 6 tests.
• 39 tests and 6 units => 3 units with 6 tests and 3 units with 7 tests each.

You need to set TLB_SPLITTER='tlb.splitter.CountBasedTestSplitter' to use count based splitting. But this is not recomended except as a fallback, because in our experience, this is almost always suboptimal.

This criterion splits the test suite into a given number of units such that each unit, when executed, takes the same or optimally same amount of time.

For example:

• 20 tests and 1 unit => 1 unit with 20 tests.
• 10 tests and 2 units with each taking 2 minutes => 2 units with 5 tests each taking 10 minutes each.
• 18 tests and 3 units, with 1 taking 13 minutes and rest taking 1 minute => 1 unit with the 13 minute test, 1 unit with 9 tests taking 9 minutes, 1 unit with 8 tests taking 8 minutes.

You need to set TLB_SPLITTER='tlb.splitter.TimeBasedTestSplitter' to use time based splitting. But this is not recomended either, because the first time your build runs, it won't have data to time balance(which means it will fail).

This criterion delegates to a criteria chain, trying criterion in order, until reaches the last one. This is recomended way of using ‘time based’ setup. We make a chain of ‘time based splitter’ followed by ‘count based splitter’ so if time based splitting fails(which may happen because the server is momenteraly unreachable, or its the first time build is running and time balancing data is not available), it defaults to ‘count balancing’ which doesn’t require the ‘test suite time’ data. If all criterion in the chain fail, the build fails.

Delegating criterion requires TLB_PREFERRED_SPLITTERS to be set to a colon(:) separated list of fully qualified names of criteria classes. For instance, a typical time followed by count criteria would look like ‘tlb.splitter.TimeBasedTestSplitter:tlb.splitter.CountBasedTestSplitter’

The test split criteria is passed in using the environment variable TLB_SPLITTER. If this variable is not set, TLB by default uses a criteria that doesn’t do any balancing at all.

You need to set TLB_SPLITTER='tlb.splitter.DefaultingTestSplitter' TLB_PREFERRED_SPLITTERS='tlb.splitter.TimeBasedTestSplitter:tlb.splitter.CountBasedTestSplitter' to use delegating/defaulting criterion. The setting above tries time based balacing first, and then defaults to count based balacing in case it fails.