Profiling showed a lot of the time was spent in two helper functions:
- glob() which scans the filesystem for filenames that match a pattern
- get_tags() which parses the "@tags" section of a jstest
Printing the arguments of these functions showed that a lot of the calls
are duplicates, because many suites use the same glob pattern, and many
suites want to know what tags a given test contains.
Since we already assume the resmokeconfig is not changing out from under
us, it makes sense to cache these. I don't think memory usage is a
concern because resmoke is either short-lived (as in find-suites) or it
runs subprocesses that already use a lot of memory (mongod, or a whole
sharded cluster).
On my machine these two changes brought the time down from around
1m 20s to 10s.
GitOrigin-RevId: f4538e44aaca8c1ae11c38cc48b6ec76d2a85d1b
Split out the passthrough tests into separate suites. The MongoDB
deployment is started up by resmoke.py so that we can record the
success/failure of each individual test in MCI.
Added support for parallel execution of tests by dispatching to
multiple MongoDB deployments.
Added support for grouping different kinds of tests (e.g. C++ unit
tests, dbtests, and jstests) so that they can be run together. This
allows for customizability in specifying what tests to execute when
changes are made to a particular part of the code.
Juan Gu
Steve McClure
David Percy
Mark Benvenuto
sergey.galtsev
Mathew Robinson
Jonathan Abrahams
Max Hirschhorn