mongo/buildscripts/cost_model
Max Verbinnen 120ee72dd9 SERVER-113632 Update expected COLLSCAN & FETCH filter (#43986)
GitOrigin-RevId: e9dd74c88f9963dc897a77d6d25b9389240d5125
2025-11-14 14:50:02 +00:00
..
.gitignore SERVER-110059 Cleanup cost model calibration scripts (#42008) 2025-10-02 20:27:20 +00:00
BUILD.bazel SERVER-100631 Add ruff into "bazel run lint" (#32166) 2025-06-11 14:11:11 +00:00
OWNERS.yml SERVER-103079 Replace granular QO code owners teams with a single coarse team (#34298) 2025-04-10 06:25:19 +00:00
README.md SERVER-85932 Update README for cost model calibration setup (#40665) 2025-08-28 16:25:03 +00:00
calibration_settings.py SERVER-113679 Take cost of SORT spilling into account (#43934) 2025-11-13 21:23:43 +00:00
ce_data_settings.py SERVER-106620 Remove remaining ABT references as well as unused code from cost model scripts (#42075) 2025-10-07 17:52:55 +00:00
ce_generate_data.py SERVER-106620 Remove remaining ABT references as well as unused code from cost model scripts (#42075) 2025-10-07 17:52:55 +00:00
common.py SERVER-106620 Remove remaining ABT references as well as unused code from cost model scripts (#42075) 2025-10-07 17:52:55 +00:00
config.py SERVER-106620 Remove remaining ABT references as well as unused code from cost model scripts (#42075) 2025-10-07 17:52:55 +00:00
cost_estimator.py SERVER-113632 Create workload for incremental filter leaf cost (#43737) 2025-11-13 16:17:32 +00:00
data_generator.py SERVER-104999 port motor to pymongo async (#36697) 2025-05-30 13:07:39 +00:00
database_instance.py SERVER-107786 Create workload for calibration of COLLSCAN node (#38773) 2025-07-23 19:31:00 +00:00
execution_tree_classic.py SERVER-113632 Create workload for incremental filter leaf cost (#43737) 2025-11-13 16:17:32 +00:00
execution_tree_sbe.py SERVER-106251 Modify cost model calibration code to be able to parse classic execution trees (#37438) 2025-06-20 20:55:36 +00:00
experiment.py SERVER-106620 Remove remaining ABT references as well as unused code from cost model scripts (#42075) 2025-10-07 17:52:55 +00:00
parameters_extractor_classic.py SERVER-113632 Create workload for incremental filter leaf cost (#43737) 2025-11-13 16:17:32 +00:00
qsn_calibrator.py SERVER-113679 Take cost of SORT spilling into account (#43934) 2025-11-13 21:23:43 +00:00
qsn_costing_parameters.py SERVER-113632 Create workload for incremental filter leaf cost (#43737) 2025-11-13 16:17:32 +00:00
query_solution_tree.py SERVER-106251 Modify cost model calibration code to be able to parse classic execution trees (#37438) 2025-06-20 20:55:36 +00:00
random_generator.py SERVER-94077 Use isort in Ruff configs (#27865) 2024-10-10 19:33:49 +00:00
requirements.txt SERVER-104999 port motor to pymongo async (#36697) 2025-05-30 13:07:39 +00:00
start.py SERVER-113632 Update expected COLLSCAN & FETCH filter (#43986) 2025-11-14 14:50:02 +00:00
workload_execution.py SERVER-113679 Take cost of SORT spilling into account (#43934) 2025-11-13 21:23:43 +00:00

README.md

Cost Model Calibrator

Getting Started

1) Setup Mongod

First, prepare the MongoDB server:

  1. Activate the standard virtual environment:
source python3-venv/bin/activate
  1. Build server with optimizations (makes doc insertion faster):
(python3-venv) bazel build --config=opt install-devcore
  1. Run mongod instance:
(python3-venv) bazel-bin/install-mongod/bin/mongod --setParameter internalMeasureQueryExecutionTimeInNanoseconds=true

2) Setup Cost Model Calibrator

In another terminal:

  1. Navigate to the cost model directory:
cd buildscripts/cost_model
  1. Set up Python alias to use MongoDB toolchain:
alias python=/opt/mongodbtoolchain/v4/bin/python3
  1. Deactivate any existing Python environment (if needed):
deactivate
  1. Create new virtual environment:
/opt/mongodbtoolchain/v4/bin/python3 -m venv cm
  1. Activate the new environment:
source cm/bin/activate
  1. Install required packages:
(cm) python -m pip install -r requirements.txt
  1. Run the calibrator:
(cm) python start.py

Note: For the first time it will take a while since it has to generate the data. Afterwards, as long as you aren't modifying the collections, you can comment out await generator.populate_collections() in start.py - this will make it a lot faster.

  1. When done, deactivate the environment:
(cm) deactivate

Install New Packages

  1. Install the package:
(cm) python -m pip install <package_name>
  1. Update requirements.txt:
(cm) python -m pip freeze > requirements.txt