f360d12c33
SERVER-96080: Remove references to bonsai in buildscripts/cost_model ( #32641 )
...
GitOrigin-RevId: 6ed0765190afb716e06a50666e8ba610437414cd
2025-04-09 22:23:09 +00:00
7209d62b77
SERVER-99522 Delete dead py linters ( #31760 )
...
GitOrigin-RevId: 3aedfa559dda734d4d89fefe0fe1154a5a3cf04d
2025-04-09 19:33:45 +00:00
c0effc918b
SERVER-93650 Add people as owners for Query Optimization owned files ( #29093 )
...
GitOrigin-RevId: cf2a2a41b59b7eef3f98e6be0bf443287080a1d8
2024-11-20 14:45:04 +00:00
855dfadef0
SERVER-94077 Use isort in Ruff configs ( #27865 )
...
GitOrigin-RevId: e793d662774ccd3ab6c3f356c2287cf1f7ff9805
2024-10-10 19:33:49 +00:00
2513cc511b
SERVER-94076 Remove unused imports via Ruff rule ( #27337 )
...
GitOrigin-RevId: 8923cc14f276be93355d763cb88c04bdb4dd9000
2024-10-04 01:43:13 +00:00
1d1213a372
SERVER-93311: delete featureFlagCommonQueryFramework ( #27430 )
...
GitOrigin-RevId: 29e4688835f2811d29be3c01e4cb43215f463050
2024-09-26 13:31:07 +00:00
3d252cb83a
SERVER-91954 Add 'queryShapeHash' to explain, rename 'queryHash' to 'planCacheShapeHash' ( #25458 )
...
GitOrigin-RevId: 730bedb1255f7bbbf0238348923aeca66106236e
2024-09-19 17:04:13 +00:00
6857229c18
SERVER-93815: Remove Bonsai-related query knobs ( #27205 )
...
GitOrigin-RevId: c69f8ee3cdfa3897b39806b510085fdd54a93ece
2024-09-18 19:27:12 +00:00
8143f183fc
SERVER-91546 establish initial code ownership for Query Optimization ( #25373 )
...
GitOrigin-RevId: abc8bdb4004433a9df912aa88a9a8900337694d1
2024-07-30 21:19:26 +00:00
b4b23946cd
SERVER-90570: Enable formatting checks for buildscripts directory, excluding idl ( #22254 )
...
GitOrigin-RevId: 9d997a9f44cd43a8dec7c2a17fa2dbcd875e92f6
2024-05-16 22:07:36 +00:00
21e783f5ef
SERVER-86060 Implement Query Solution Node parser in Cost Model Calibration framework ( #18600 )
...
SERVER-86060 Implement Query Solution Node parser in Cost Model
Calibration framework.
* parse Query Solution Node;
* build Cost Model calibration parameters from parsed SQN and SBE tree;
* support new SBE nodes;
* parse SBE seeks node.
GitOrigin-RevId: eb2720e87114fa4505a61982406e135958859386
2024-03-23 03:26:44 +00:00
71f830ce49
SERVER-87034 Initial markdown format ( #19276 )
...
GitOrigin-RevId: 64e388007ec1ac3744537253540995af628bcc00
2024-02-27 19:58:04 +00:00
4cf2ec98fc
SERVER-80756 Expose physical ABT in M2 explain
2023-11-10 20:12:32 +00:00
f6d8fadde7
SERVER-81794 Upgrade cost_model python packages
2023-10-03 20:45:06 +00:00
2380af1b37
SERVER-79143 Bump scipy from 1.9.3 to 1.10.0 in /buildscripts/cost_model
2023-07-20 09:40:42 +00:00
597f3f1037
SERVER-77174: Set CQF query knob values to the production defaults and update fallback mechanism for M2
2023-06-20 13:09:04 +00:00
126ce85a48
SERVER-73691: Require confirmation for upgrade/downgrade
2023-03-28 15:52:51 +00:00
df67dd6db8
SERVER-73030 Generate random array data for CE testing
...
* Ensure all generated fields are non-empty.
* Reduce/modify some data distributions.
2023-02-22 11:19:53 +00:00
9d7d2aedc9
SERVER-73030 Generate random array data for CE testing
...
Fix duplicate field names for array fields.
2023-02-21 13:56:20 +00:00
7979eacf75
SERVER-73979 Partition large random data files into chunks
...
Save randomly generated data into files split into chunks
with a limited number of documents per chunk. This is done
to avoid the 2GB limit on JS files imposed by Node.js.
2023-02-17 08:19:00 +00:00
5af5e4a1e8
SERVER-73031 Generate random data with mixed data types
...
* Added generation of random data with mixed data types
* Generation of random dates and doubles
* Some refactoring of the python generation framework wrt types
2023-02-15 16:32:12 +00:00
8f00ba224e
SERVER-73030 Generate random array data for CE testing
...
* Add generation of random array data of integers and strings.
* Minor change to string generation to limit printable chars.
2023-01-27 13:20:27 +00:00
186ad335bf
SERVER-72662 Generate random string data for CE
...
* Added generation of random string data.
* Remove ce_generate_data_settings.py - no longer needed and used.
2023-01-20 09:44:21 +00:00
d692121e12
SERVER-72663 Visualize distribution of generated data
...
* Visualize generated data via histograms stored as png files in stests/query_golden/libs/data.
* Added a couple of more mixed distributions.
2023-01-17 13:48:13 +00:00
550e070c93
SERVER-72236 Generate random integer data for CE
...
Address final review comments.
2023-01-12 10:36:13 +00:00
3ade0b9fe3
SERVER-72236 Generate random integer data for CE
...
Generate random data with integers. The approach is as follows:
- There is one collection for each different cardinality. All collections contain the same fields.
- Each field contains the data generated from a certain data distribution. The data could be anything - same type, mixed types, same mathematical distribution (e.g. normal), or a mixed distribution.
- The committed configuration file, and the corresponding data file are reduced to only two small collections. For actual experiments one needs to add more data sizes, and re-generate the data locally. This is done so that Evergreen tests can run fast, and to reduce the size of the git repository.
- All data is saved in a single JavaScript file: jstests/query_golden/libs/data/ce_accuracy_test.data, with a corresponding schema file jstests/query_golden/libs/data/ce_accuracy_test.schema.
- The data file is a JavaScript file that can be loaded directly inside a JS test. When loading this file, it creates a global variable dataSet. The reason is that this is the only way to load an external JSON file that doesn't need to install external tools in Evergreen.
2023-01-10 12:51:54 +00:00
38c5aab661
SERVER-72036 Implement data generation and loading into JS CE accuracy tests
...
* Extend the data generation Python framework for cost calibration to support data generation for CE testing as follows:
- the entry point is ce_generate_data.py,
- the configuration of the generated data is in ce_generate_data_settings.py,
- all collection data is exported into a single JSON file stored in 'jstests/query_golden/libs/data', and a schema file stored in the same directory
* Implement a JS data loader function that also creates all indexes specified in the schema file.
* Add a small JS test that shows how to load the generated JSON files into collections.
2022-12-17 10:06:36 +00:00
41f666bd81
SERVER-71576 Cost Model End2End benchmark: calculate r2
2022-12-06 12:47:40 +00:00
39640e543c
SERVER-67161: Add physical NestedLoopJoin and make BinaryJoin logical
2022-12-02 18:22:49 +00:00
7db13d1475
SERVER-71576 Implement Cost Model End2End testing
2022-11-24 11:18:42 +00:00
bfe342d122
SERVER-71363 Use Nanoseconds rather than Microseconds in QueryExecTime and support Nanoseconds in ScopedTimer
2022-11-16 14:11:27 +00:00
ea088bcf55
SERVER-71220 Apply additional filters when calibrating Cost Model
2022-11-11 12:20:20 +00:00
0b52d4cbc8
SERVER-69030 Add python requirements.txt for Cost Model
2022-11-10 11:18:11 +00:00
32545f21f5
SERVER-70500 Calibrate ABT nodes on smaller queries
2022-11-08 21:09:45 +00:00
4acc5a3704
SERVER-70537 Implement end to end cost model benchmark.
2022-11-03 17:04:23 +00:00
eb3badb0ae
SERVER-70491 Calibrate physical ABT nodes in the framework
2022-10-13 17:36:29 +00:00
3b8505623b
SERVER-68984 Implement experiments in Cost Model Framework
2022-09-01 15:50:46 +00:00
5f11658224
SERVER-69031 Move JSON configuration to python file
2022-09-01 09:50:08 +00:00
ce2dbe9a4a
SERVER-67223 Optimize writing to MongoDB data generator
2022-08-30 14:26:01 +00:00
8c0bbf5b25
SERVER-69089: Add ability to define indexes in config
2022-08-30 09:32:51 +00:00
2b69d029a2
SERVER-68983 Do not run ABT calibration workflows with hidden indexes
2022-08-23 16:45:21 +00:00
0c53d0c90b
SERVER-68386 Implement random generator of documents
2022-08-21 15:51:59 +00:00
7e08691fc9
SERVER-68385 Implement random generator of arrays
2022-08-15 16:52:35 +00:00
cdd2775640
SERVER-62042 Consolidate query optimization and execution control into a single knob
2022-08-04 15:39:07 +00:00
757f6f2293
Revert "SERVER-62042 Consolidate query optimization and execution control into a single knob"
...
This reverts commit c9bbd1cfae
2022-08-04 06:09:52 +00:00
c9bbd1cfae
SERVER-62042 Consolidate query optimization and execution control into a single knob
2022-08-03 17:22:30 +00:00
c545bd81b0
SERVER-68349 Support various data distributions in random data generator
2022-07-28 15:26:31 +00:00
30420ef9af
SERVER-67851 Support additional input parameters in Cost Model
2022-07-26 19:46:18 +00:00
304a34e0c9
SERVER-67121 Cost Model and Calibration farmework
2022-07-05 09:31:30 +00:00
Matt Olma
Juan Gu
Anton Korshunov
HanaPearlman
Vlad Dragutoiu
David Storch
Steve McClure
Alexander Ignatyev
Alexander Neben
Militsa Sotirova
dependabot[bot]
XueruiFa
Timour Katchaounov
Ruoxin Xu
Nicholas Zolnierz
auto-revert-processor