mirror
/
mongo
mirror of https://github.com/mongodb/mongo
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

SERVER-106150 Rewrite private links to public links when we sync source code from 10gen/mongo to mongodb/mongo (#37153) 

GitOrigin-RevId: 25f69040178a8b6260cc44ab7fc0f5e576ee7faf
This commit is contained in:
Zack Winter 2025-06-12 10:55:16 -07:00 committed by MongoDB Bot
parent ed87a95bd2
commit 5f223c3ede
28 changed files with 142 additions and 135 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.github/PULL_REQUEST_TEMPLATE/README.md vendored
View File

@ -12,7 +12,7 @@ If you update the default PR template, you also need to update the commit metada
To create a custom template, create a new markdown file in this folder. To create a custom template, create a new markdown file in this folder.
Then create a link of the form `https://github.com/10gen/mongo/compare/main...my-branch?quick_pull=1&template=your_new_template.md` Then create a link of the form `https://github.com/mongodb/mongo/compare/main...my-branch?quick_pull=1&template=your_new_template.md`
Share that link in your team docs to use for creating PRs. By selecting an unused values for `my-branch` it should show a branch selector when following the link. Share that link in your team docs to use for creating PRs. By selecting an unused values for `my-branch` it should show a branch selector when following the link.

4
buildscripts/antithesis/README.md
View File

@ -14,8 +14,8 @@ use Antithesis today.
## Base Images ## Base Images
The `base_images` directory consists of the building blocks for creating a MongoDB test topology. The `base_images` directory consists of the building blocks for creating a MongoDB test topology.
These images are uploaded to the Antithesis Docker registry [nightly](https://github.com/10gen/mongo/blob/6cf8b162a61173eb372b54213def6dd61e1fd684/etc/evergreen_yml_components/variants/ubuntu/test_dev_master_and_lts_branches_only.yml#L28) during the These images are uploaded to the Antithesis Docker registry [nightly](https://github.com/mongodb/mongo/blob/6cf8b162a61173eb372b54213def6dd61e1fd684/etc/evergreen_yml_components/variants/ubuntu/test_dev_master_and_lts_branches_only.yml#L28) during the
[`antithesis image build and push`](https://github.com/10gen/mongo/blob/020632e3ae328f276b2c251417b5a39389af6141/etc/evergreen_yml_components/definitions.yml#L2823) function. [`antithesis image build and push`](https://github.com/mongodb/mongo/blob/020632e3ae328f276b2c251417b5a39389af6141/etc/evergreen_yml_components/definitions.yml#L2823) function.
### mongo_binaries ### mongo_binaries

6
buildscripts/bazel_rules_mongo/README.md
View File

@ -4,13 +4,13 @@ This directory is a bazel rule we use to ship common code between bazel repos
# Using in your repo # Using in your repo
1. Look at the latest version in [this](https://github.com/10gen/mongo/blob/master/buildscripts/bazel_rules_mongo/pyproject.toml) file 1. Look at the latest version in [this](https://github.com/mongodb/mongo/blob/master/buildscripts/bazel_rules_mongo/pyproject.toml) file
2. Get the sha of the latest release at https://mdb-build-public.s3.amazonaws.com/bazel_rules_mongo/{version}/bazel_rules_mongo.tar.gz.sha256 2. Get the sha of the latest release at https://mdb-build-public.s3.amazonaws.com/bazel_rules_mongo/{version}/bazel_rules_mongo.tar.gz.sha256
3. Get the link to the latest version at https://mdb-build-public.s3.amazonaws.com/bazel_rules_mongo/{version}/bazel_rules_mongo.tar.gz 3. Get the link to the latest version at https://mdb-build-public.s3.amazonaws.com/bazel_rules_mongo/{version}/bazel_rules_mongo.tar.gz
4. Add this as a http archive to your repo and implement the dependencies listed in the [WORKSPACE](https://github.com/10gen/mongo/blob/master/buildscripts/bazel_rules_mongo/WORKSPACE.bazel) file. It will look something like this 4. Add this as a http archive to your repo and implement the dependencies listed in the [WORKSPACE](https://github.com/mongodb/mongo/blob/master/buildscripts/bazel_rules_mongo/WORKSPACE.bazel) file. It will look something like this
``` ```
# Poetry rules for managing Python dependencies # Poetry rules for managing Python dependencies
@ -61,5 +61,5 @@ alias(
# Deploying # Deploying
When you are ready for a new version to be released, bump the version in the [pyproject.toml](https://github.com/10gen/mongo/blob/master/buildscripts/bazel_rules_mongo/pyproject.toml) file. When you are ready for a new version to be released, bump the version in the [pyproject.toml](https://github.com/mongodb/mongo/blob/master/buildscripts/bazel_rules_mongo/pyproject.toml) file.
This will be deployed the next time the `package_bazel_rules_mongo` task runs (nightly). You can schedule this earlier in the waterfall when your pr is merged if you want it quicker. This will be deployed the next time the `package_bazel_rules_mongo` task runs (nightly). You can schedule this earlier in the waterfall when your pr is merged if you want it quicker.

14
buildscripts/resmokelib/hang_analyzer/README.md
View File

@ -35,15 +35,15 @@ sequenceDiagram
Hang Analyzer ->> Core Dumps: Attach to pid and generate core dumps Hang Analyzer ->> Core Dumps: Attach to pid and generate core dumps
``` ```
When a task times out, it hits the [timeout](https://github.com/10gen/mongo/blob/a6e56a8e136fe554dc90565bf6acf5bf86f7a46e/etc/evergreen_yml_components/definitions.yml#L2694) section in the defined evergreen config. When a task times out, it hits the [timeout](https://github.com/mongodb/mongo/blob/a6e56a8e136fe554dc90565bf6acf5bf86f7a46e/etc/evergreen_yml_components/definitions.yml#L2694) section in the defined evergreen config.
In this timeout section, we run [this](https://github.com/10gen/mongo/blob/a6e56a8e136fe554dc90565bf6acf5bf86f7a46e/etc/evergreen_yml_components/definitions.yml#L2302) task which runs the hang-analyzer with the following invocation: In this timeout section, we run [this](https://github.com/mongodb/mongo/blob/a6e56a8e136fe554dc90565bf6acf5bf86f7a46e/etc/evergreen_yml_components/definitions.yml#L2302) task which runs the hang-analyzer with the following invocation:
``` ```
python3 buildscripts/resmoke.py hang-analyzer -o file -o stdout -m exact -p python python3 buildscripts/resmoke.py hang-analyzer -o file -o stdout -m exact -p python
``` ```
This tells the hang-analyzer to look for all of the python processes (we are specifically looking for resmoke) on the machine and to signal them. This tells the hang-analyzer to look for all of the python processes (we are specifically looking for resmoke) on the machine and to signal them.
When resmoke is [signaled](https://github.com/10gen/mongo/blob/08a99b15eea7ae0952b2098710d565dd7f709ff6/buildscripts/resmokelib/sighandler.py#L25), it again invokes the hang analyzer with the specific pids of it's child processes. When resmoke is [signaled](https://github.com/mongodb/mongo/blob/08a99b15eea7ae0952b2098710d565dd7f709ff6/buildscripts/resmokelib/sighandler.py#L25), it again invokes the hang analyzer with the specific pids of it's child processes.
It will look similar to this most of the time: It will look similar to this most of the time:
``` ```
@ -61,7 +61,7 @@ After investigation of the above issue, we found that compressing and uploading
2. Gzip is single threaded. 2. Gzip is single threaded.
3. Uploading a big file synchronously is not fast. 3. Uploading a big file synchronously is not fast.
We made a [script](https://github.com/10gen/mongo/blob/master/buildscripts/fast_archive.py) that gzips all of the core dumps in parallel and uploads them to S3 individually asynchronously. We made a [script](https://github.com/mongodb/mongo/blob/master/buildscripts/fast_archive.py) that gzips all of the core dumps in parallel and uploads them to S3 individually asynchronously.
This solved all of the problems listed above. This solved all of the problems listed above.
### Generating the core analyzer task ### Generating the core analyzer task
@ -75,8 +75,8 @@ sequenceDiagram
Generated Task ->> Core Analyzer Output: Overwrite output with<br/> core dump analysis Generated Task ->> Core Analyzer Output: Overwrite output with<br/> core dump analysis
``` ```
In the [post task](https://github.com/10gen/mongo/blob/709e3f4efc04b42e5d29a8ad2417a01d3610fc3f/etc/evergreen_yml_components/definitions.yml#L2665) section, we [define](https://github.com/10gen/mongo/blob/709e3f4efc04b42e5d29a8ad2417a01d3610fc3f/etc/evergreen_yml_components/definitions.yml#L2184) the evergreen function used to generate the core analyzer task. In the [post task](https://github.com/mongodb/mongo/blob/709e3f4efc04b42e5d29a8ad2417a01d3610fc3f/etc/evergreen_yml_components/definitions.yml#L2665) section, we [define](https://github.com/mongodb/mongo/blob/709e3f4efc04b42e5d29a8ad2417a01d3610fc3f/etc/evergreen_yml_components/definitions.yml#L2184) the evergreen function used to generate the core analyzer task.
This [script](https://github.com/10gen/mongo/blob/709e3f4efc04b42e5d29a8ad2417a01d3610fc3f/buildscripts/resmokelib/hang_analyzer/gen_hang_analyzer_tasks.py) runs on every task (passing or failing) and is independent of anything else that happened prior in the task and does all of the checks to ensure it should run. This [script](https://github.com/mongodb/mongo/blob/709e3f4efc04b42e5d29a8ad2417a01d3610fc3f/buildscripts/resmokelib/hang_analyzer/gen_hang_analyzer_tasks.py) runs on every task (passing or failing) and is independent of anything else that happened prior in the task and does all of the checks to ensure it should run.
These checks include: These checks include:
1. The task is being run on an operating system supported by the core analyzer. 1. The task is being run on an operating system supported by the core analyzer.
@ -86,7 +86,7 @@ These checks include:
The output from this script is a json file in the format evergreen expects. The output from this script is a json file in the format evergreen expects.
We then pass this json file into the `generate.tasks` evergreen command to generate the task. We then pass this json file into the `generate.tasks` evergreen command to generate the task.
After the task is generated, we have [another script](https://github.com/10gen/mongo/blob/709e3f4efc04b42e5d29a8ad2417a01d3610fc3f/etc/evergreen_yml_components/definitions.yml#L2213) that finds the task that was just generated and attaches it to the current task being ran. After the task is generated, we have [another script](https://github.com/mongodb/mongo/blob/709e3f4efc04b42e5d29a8ad2417a01d3610fc3f/etc/evergreen_yml_components/definitions.yml#L2213) that finds the task that was just generated and attaches it to the current task being ran.
The reason we upload a temporary file to the original task is to attach that s3 file link to the task. The reason we upload a temporary file to the original task is to attach that s3 file link to the task.
Evergreen does not currently have a way to attach files to a task after it was ran so we need to upload something while the original task is in progress. Evergreen does not currently have a way to attach files to a task after it was ran so we need to upload something while the original task is in progress.

7
copy.bara.sky
View File

@ -58,5 +58,12 @@ core.workflow(
# If there are no trailers, then there will only be a single line of text, with no newline # If there are no trailers, then there will only be a single line of text, with no newline
# chars, and so the pattern will fail to match. # chars, and so the pattern will fail to match.
metadata.scrubber("(^.*?\n)((?:\n|.)*)", replacement = "$1\n$2"), metadata.scrubber("(^.*?\n)((?:\n|.)*)", replacement = "$1\n$2"),
#
# STEP 5: Replace any private links with their public equivalent.
core.replace(
before = "https://github.com/10gen/mongo",
after = "https://github.com/mongodb/mongo",
paths = glob(["**/*.md"]),
),
], ],
) )

4
docs/baton.md
View File

@ -80,6 +80,6 @@ must be non-blocking and preferably short-lived to ensure forward progress.
[defaultBaton]: https://github.com/mongodb/mongo/blob/9cfe13115e92a43d1b9273ee1d5817d548264ba7/src/mongo/db/default_baton.h#L46-L75 [defaultBaton]: https://github.com/mongodb/mongo/blob/9cfe13115e92a43d1b9273ee1d5817d548264ba7/src/mongo/db/default_baton.h#L46-L75
[networkingBaton]: https://github.com/mongodb/mongo/blob/9cfe13115e92a43d1b9273ee1d5817d548264ba7/src/mongo/transport/baton.h#L61-L96 [networkingBaton]: https://github.com/mongodb/mongo/blob/9cfe13115e92a43d1b9273ee1d5817d548264ba7/src/mongo/transport/baton.h#L61-L96
[asioNetworkingBaton]: https://github.com/mongodb/mongo/blob/9cfe13115e92a43d1b9273ee1d5817d548264ba7/src/mongo/transport/baton_asio_linux.h#L60-L529 [asioNetworkingBaton]: https://github.com/mongodb/mongo/blob/9cfe13115e92a43d1b9273ee1d5817d548264ba7/src/mongo/transport/baton_asio_linux.h#L60-L529
[asioNetworkingBatonScheduling]: https://github.com/10gen/mongo/blob/46b8c49b4e13cc4c8389b2822f9e30dd73b81d6e/src/mongo/executor/network_interface_tl.cpp#L910 [asioNetworkingBatonScheduling]: https://github.com/mongodb/mongo/blob/46b8c49b4e13cc4c8389b2822f9e30dd73b81d6e/src/mongo/executor/network_interface_tl.cpp#L910
[asioNetworkingBatonPollingSetup]: https://github.com/10gen/mongo/blob/eab4ec41cc2b28bf0a38eb813f9690e1bfa6c9a6/src/mongo/transport/asio/asio_session_impl.cpp#L666-L696 [asioNetworkingBatonPollingSetup]: https://github.com/mongodb/mongo/blob/eab4ec41cc2b28bf0a38eb813f9690e1bfa6c9a6/src/mongo/transport/asio/asio_session_impl.cpp#L666-L696
[example]: https://github.com/mongodb/mongo/blob/262e5a961fa7221bfba5722aeea2db719f2149f5/src/mongo/s/multi_statement_transaction_requests_sender.cpp#L91-L99 [example]: https://github.com/mongodb/mongo/blob/262e5a961fa7221bfba5722aeea2db719f2149f5/src/mongo/s/multi_statement_transaction_requests_sender.cpp#L91-L99

4
docs/branching/README.md
View File

@ -34,7 +34,7 @@ git checkout -b vX.Y-branching-task
The reason they should be pushed as separate commits is in the case of needing to revert one aspect of this entire task. The reason they should be pushed as separate commits is in the case of needing to revert one aspect of this entire task.
- [8.1 Branching commit](https://github.com/10gen/mongo/commit/359cbe95216ffaf1a6173884f6519b3d408f1fb5) for reference. - [8.1 Branching commit](https://github.com/mongodb/mongo/commit/359cbe95216ffaf1a6173884f6519b3d408f1fb5) for reference.
### Copybara configuration ### Copybara configuration
@ -104,7 +104,7 @@ The reason they should be pushed as separate commits is in the case of needing t
- `buildscripts/generate_version_expansions.py` - `buildscripts/generate_version_expansions.py`
- Update [suffixes](https://github.com/10gen/mongo/blob/41ebdd14567ee35bdda0942958a5dc193f97dd5f/buildscripts/generate_version_expansions.py#L64-L65) from "latest" to "vX.Y-latest", where `vX.Y` is a new branch name - Update [suffixes](https://github.com/mongodb/mongo/blob/41ebdd14567ee35bdda0942958a5dc193f97dd5f/buildscripts/generate_version_expansions.py#L64-L65) from "latest" to "vX.Y-latest", where `vX.Y` is a new branch name
## 3. Test changes ## 3. Test changes

2
docs/command_dispatch.md
View File

@ -85,4 +85,4 @@ For details on transport internals, including ingress networking, see [this docu
[template_method_pattern]: https://en.wikipedia.org/wiki/Template_method_pattern [template_method_pattern]: https://en.wikipedia.org/wiki/Template_method_pattern
[transport_internals]: ../src/mongo/transport/README.md [transport_internals]: ../src/mongo/transport/README.md
[ACReadMe]: ../src/mongo/db/admission/README.md [ACReadMe]: ../src/mongo/db/admission/README.md
[IngressControl]: https://github.com/10gen/mongo/blob/a86c7f5de2a5de4d2f49e40e8970754ec6a5ba6c/src/mongo/db/service_entry_point_shard_role.cpp#L1803 [IngressControl]: https://github.com/mongodb/mongo/blob/a86c7f5de2a5de4d2f49e40e8970754ec6a5ba6c/src/mongo/db/service_entry_point_shard_role.cpp#L1803

2
docs/cpp_style.md
View File

@ -334,7 +334,7 @@ in a `.h` file and provide `= default;` as the implementation in a `.cpp` file.
The `noexcept` feature is easy to overuse. Do not use it solely as "documentation" The `noexcept` feature is easy to overuse. Do not use it solely as "documentation"
since it affects runtime behavior. It's a large topic, covered in the [Exception since it affects runtime behavior. It's a large topic, covered in the [Exception
Architecture](https://github.com/10gen/mongo/blob/master/docs/exception_architecture.md#using-noexcept) Architecture](https://github.com/mongodb/mongo/blob/master/docs/exception_architecture.md#using-noexcept)
document. document.
### Overriding Virtuals ### Overriding Virtuals

28
docs/evergreen-testing/multiversion.md
View File

@ -178,13 +178,13 @@ fixture the suite is running with. These are the recommended version combination
- `last-continuous new-new-old` - `last-continuous new-new-old`
- `last-continuous new-old-new` - `last-continuous new-old-new`
- `last-continuous old-new-new` - `last-continuous old-new-new`
- Ex: [change_streams](https://github.com/10gen/mongo/blob/88d59bfe9d5ee2c9938ae251f7a77a8bf1250a6b/buildscripts/resmokeconfig/suites/change_streams.yml) uses a [`ReplicaSetFixture`](https://github.com/10gen/mongo/blob/88d59bfe9d5ee2c9938ae251f7a77a8bf1250a6b/buildscripts/resmokeconfig/suites/change_streams.yml#L50) so the corresponding multiversion suites are - Ex: [change_streams](https://github.com/mongodb/mongo/blob/88d59bfe9d5ee2c9938ae251f7a77a8bf1250a6b/buildscripts/resmokeconfig/suites/change_streams.yml) uses a [`ReplicaSetFixture`](https://github.com/mongodb/mongo/blob/88d59bfe9d5ee2c9938ae251f7a77a8bf1250a6b/buildscripts/resmokeconfig/suites/change_streams.yml#L50) so the corresponding multiversion suites are
- [`change_streams_last_continuous_new_new_old`](https://github.com/10gen/mongo/blob/612814f4ce56282c47d501817ba28337c26d7aba/buildscripts/resmokeconfig/matrix_suites/mappings/change_streams_last_continuous_new_new_old.yml) - [`change_streams_last_continuous_new_new_old`](https://github.com/mongodb/mongo/blob/612814f4ce56282c47d501817ba28337c26d7aba/buildscripts/resmokeconfig/matrix_suites/mappings/change_streams_last_continuous_new_new_old.yml)
- [`change_streams_last_continuous_new_old_new`](https://github.com/10gen/mongo/blob/612814f4ce56282c47d501817ba28337c26d7aba/buildscripts/resmokeconfig/matrix_suites/mappings/change_streams_last_continuous_new_old_new.yml) - [`change_streams_last_continuous_new_old_new`](https://github.com/mongodb/mongo/blob/612814f4ce56282c47d501817ba28337c26d7aba/buildscripts/resmokeconfig/matrix_suites/mappings/change_streams_last_continuous_new_old_new.yml)
- [`change_streams_last_continuous_old_new_new`](https://github.com/10gen/mongo/blob/612814f4ce56282c47d501817ba28337c26d7aba/buildscripts/resmokeconfig/matrix_suites/mappings/change_streams_last_continuous_old_new_new.yml) - [`change_streams_last_continuous_old_new_new`](https://github.com/mongodb/mongo/blob/612814f4ce56282c47d501817ba28337c26d7aba/buildscripts/resmokeconfig/matrix_suites/mappings/change_streams_last_continuous_old_new_new.yml)
- [`change_streams_last_lts_new_new_old`](https://github.com/10gen/mongo/blob/612814f4ce56282c47d501817ba28337c26d7aba/buildscripts/resmokeconfig/matrix_suites/mappings/change_streams_last_lts_new_new_old.yml) - [`change_streams_last_lts_new_new_old`](https://github.com/mongodb/mongo/blob/612814f4ce56282c47d501817ba28337c26d7aba/buildscripts/resmokeconfig/matrix_suites/mappings/change_streams_last_lts_new_new_old.yml)
- [`change_streams_last_lts_new_old_new`](https://github.com/10gen/mongo/blob/612814f4ce56282c47d501817ba28337c26d7aba/buildscripts/resmokeconfig/matrix_suites/mappings/change_streams_last_lts_new_old_new.yml) - [`change_streams_last_lts_new_old_new`](https://github.com/mongodb/mongo/blob/612814f4ce56282c47d501817ba28337c26d7aba/buildscripts/resmokeconfig/matrix_suites/mappings/change_streams_last_lts_new_old_new.yml)
- [`change_streams_last_lts_old_new_new`](https://github.com/10gen/mongo/blob/612814f4ce56282c47d501817ba28337c26d7aba/buildscripts/resmokeconfig/matrix_suites/mappings/change_streams_last_lts_old_new_new.yml) - [`change_streams_last_lts_old_new_new`](https://github.com/mongodb/mongo/blob/612814f4ce56282c47d501817ba28337c26d7aba/buildscripts/resmokeconfig/matrix_suites/mappings/change_streams_last_lts_old_new_new.yml)
- Sharded cluster fixture combinations: - Sharded cluster fixture combinations:
@ -193,17 +193,17 @@ fixture the suite is running with. These are the recommended version combination
replica sets per shard where the 1st node of the 1st shard is the `latest`, 2nd node of 1st replica sets per shard where the 1st node of the 1st shard is the `latest`, 2nd node of 1st
shard - `last-lts`, 1st node of 2nd shard - `last-lts`, 2nd node of 2nd shard - `latest`, etc.) shard - `last-lts`, 1st node of 2nd shard - `last-lts`, 2nd node of 2nd shard - `latest`, etc.)
- `last-continuous new-old-old-new` - `last-continuous new-old-old-new`
- Ex: [change_streams_downgrade](https://github.com/10gen/mongo/blob/a96b83b2fa7010a5823fefac2469b4a06a697cf1/buildscripts/resmokeconfig/suites/change_streams_downgrade.yml) uses a [`ShardedClusterFixture`](https://github.com/10gen/mongo/blob/a96b83b2fa7010a5823fefac2469b4a06a697cf1/buildscripts/resmokeconfig/suites/change_streams_downgrade.yml#L408) so the corresponding multiversion suites are - Ex: [change_streams_downgrade](https://github.com/mongodb/mongo/blob/a96b83b2fa7010a5823fefac2469b4a06a697cf1/buildscripts/resmokeconfig/suites/change_streams_downgrade.yml) uses a [`ShardedClusterFixture`](https://github.com/mongodb/mongo/blob/a96b83b2fa7010a5823fefac2469b4a06a697cf1/buildscripts/resmokeconfig/suites/change_streams_downgrade.yml#L408) so the corresponding multiversion suites are
- [`change_streams_downgrade_last_continuous_new_old_old_new`](https://github.com/10gen/mongo/blob/612814f4ce56282c47d501817ba28337c26d7aba/buildscripts/resmokeconfig/matrix_suites/mappings/change_streams_downgrade_last_continuous_new_old_old_new.yml) - [`change_streams_downgrade_last_continuous_new_old_old_new`](https://github.com/mongodb/mongo/blob/612814f4ce56282c47d501817ba28337c26d7aba/buildscripts/resmokeconfig/matrix_suites/mappings/change_streams_downgrade_last_continuous_new_old_old_new.yml)
- [`change_streams_downgrade_last_lts_new_old_old_new`](https://github.com/10gen/mongo/blob/612814f4ce56282c47d501817ba28337c26d7aba/buildscripts/resmokeconfig/matrix_suites/mappings/change_streams_downgrade_last_lts_new_old_old_new.yml) - [`change_streams_downgrade_last_lts_new_old_old_new`](https://github.com/mongodb/mongo/blob/612814f4ce56282c47d501817ba28337c26d7aba/buildscripts/resmokeconfig/matrix_suites/mappings/change_streams_downgrade_last_lts_new_old_old_new.yml)
- Shell fixture combinations: - Shell fixture combinations:
- `last-lts` (i.e. suite runs the shell fixture that spins up `last-lts` as the `old` versions, - `last-lts` (i.e. suite runs the shell fixture that spins up `last-lts` as the `old` versions,
etc.) etc.)
- `last-continuous` - `last-continuous`
- Ex: [initial_sync_fuzzer](https://github.com/10gen/mongo/blob/908625ffdec050a71aa2ce47c35788739f629c60/buildscripts/resmokeconfig/suites/initial_sync_fuzzer.yml) uses a Shell Fixture, so the corresponding multiversion suites are - Ex: [initial_sync_fuzzer](https://github.com/mongodb/mongo/blob/908625ffdec050a71aa2ce47c35788739f629c60/buildscripts/resmokeconfig/suites/initial_sync_fuzzer.yml) uses a Shell Fixture, so the corresponding multiversion suites are
- [`initial_sync_fuzzer_last_lts`](https://github.com/10gen/mongo/blob/612814f4ce56282c47d501817ba28337c26d7aba/buildscripts/resmokeconfig/matrix_suites/mappings/initial_sync_fuzzer_last_lts.yml) - [`initial_sync_fuzzer_last_lts`](https://github.com/mongodb/mongo/blob/612814f4ce56282c47d501817ba28337c26d7aba/buildscripts/resmokeconfig/matrix_suites/mappings/initial_sync_fuzzer_last_lts.yml)
- [`initial_sync_fuzzer_last_continuous`](https://github.com/10gen/mongo/blob/612814f4ce56282c47d501817ba28337c26d7aba/buildscripts/resmokeconfig/matrix_suites/mappings/initial_sync_fuzzer_last_continuous.yml) - [`initial_sync_fuzzer_last_continuous`](https://github.com/mongodb/mongo/blob/612814f4ce56282c47d501817ba28337c26d7aba/buildscripts/resmokeconfig/matrix_suites/mappings/initial_sync_fuzzer_last_continuous.yml)
If `last-lts` and `last-continuous` versions happen to be the same, or last-continuous is EOL, we skip `last-continuous` If `last-lts` and `last-continuous` versions happen to be the same, or last-continuous is EOL, we skip `last-continuous`
and run multiversion suites with only `last-lts` combinations in Evergreen. and run multiversion suites with only `last-lts` combinations in Evergreen.
@ -244,7 +244,7 @@ In addition to disabling multiversion tests based on FCV, there is no need to ru
have not implemented this feature. For multiversion tasks, we pass the `--runNoFeatureFlagTests` flag to avoid these have not implemented this feature. For multiversion tasks, we pass the `--runNoFeatureFlagTests` flag to avoid these
failures on `all feature flag` variants. failures on `all feature flag` variants.
For more info on FCV, take a look at [FCV_AND_FEATURE_FLAG_README.md](https://github.com/10gen/mongo/blob/master/src/mongo/db/repl/FCV_AND_FEATURE_FLAG_README.md). For more info on FCV, take a look at [FCV_AND_FEATURE_FLAG_README.md](https://github.com/mongodb/mongo/blob/master/src/mongo/db/repl/FCV_AND_FEATURE_FLAG_README.md).
Another common case could be that the changes on master branch are breaking multiversion tests, Another common case could be that the changes on master branch are breaking multiversion tests,
but with those changes backported to the older branches the multiversion tests should work. but with those changes backported to the older branches the multiversion tests should work.

2
docs/owners/owners_format.md
View File

@ -140,7 +140,7 @@ If someone changes `a/b/c/file.yaml` the owner resolution will not find a team.
### v2.0.0 ### v2.0.0
See the [previous version](https://github.com/10gen/mongo/blob/79590effe86c471cc15d91c6785599ec2085d7c0/docs/owners/owners_format.md) of this documentation for details on v1.0.0. See the [previous version](https://github.com/mongodb/mongo/blob/79590effe86c471cc15d91c6785599ec2085d7c0/docs/owners/owners_format.md) of this documentation for details on v1.0.0.
Patterns without a slash are no longer prepended with `**/` to make them apply recursively. If you want your pattern you apply recursively you must add the `**/` yourself now. Patterns without a slash are no longer prepended with `**/` to make them apply recursively. If you want your pattern you apply recursively you must add the `**/` yourself now.

8
docs/testing/otel_resmoke.md
View File

@ -18,7 +18,7 @@ To see this visually navigate to the [resmoke dataset](https://ui.honeycomb.io/m
### Configuration ### Configuration
The bulk of configuration is done in the The bulk of configuration is done in the
`_set_up_tracing(...)` method in [configure_resmoke.py#L164](https://github.com/10gen/mongo/blob/976ce50f6134789e73c639848b35f10040f0ff4a/buildscripts/resmokelib/configure_resmoke.py#L164). This method includes documentation on how it works. `_set_up_tracing(...)` method in [configure_resmoke.py#L164](https://github.com/mongodb/mongo/blob/976ce50f6134789e73c639848b35f10040f0ff4a/buildscripts/resmokelib/configure_resmoke.py#L164). This method includes documentation on how it works.
## BatchedBaggageSpanProcessor ## BatchedBaggageSpanProcessor
@ -26,11 +26,11 @@ See documentation [batched_baggage_span_processor.py#L8](https://github.com/mong
## FileSpanExporter ## FileSpanExporter
See documentation [file_span_exporter.py#L16](https://github.com/10gen/mongo/blob/976ce50f6134789e73c639848b35f10040f0ff4a/buildscripts/resmokelib/utils/file_span_exporter.py#L16) See documentation [file_span_exporter.py#L16](https://github.com/mongodb/mongo/blob/976ce50f6134789e73c639848b35f10040f0ff4a/buildscripts/resmokelib/utils/file_span_exporter.py#L16)
## Capturing Data ## Capturing Data
We mostly capture data by using a decorator on methods. Example taken from [job.py#L200](https://github.com/10gen/mongo/blob/6d36ac392086df85844870eef1d773f35020896c/buildscripts/resmokelib/testing/job.py#L200) We mostly capture data by using a decorator on methods. Example taken from [job.py#L200](https://github.com/mongodb/mongo/blob/6d36ac392086df85844870eef1d773f35020896c/buildscripts/resmokelib/testing/job.py#L200)
``` ```
TRACER = trace.get_tracer("resmoke") TRACER = trace.get_tracer("resmoke")
@ -41,7 +41,7 @@ def func_name(...):
span.set_attribute("attr1", True) span.set_attribute("attr1", True)
``` ```
This system is nice because the decorator captures exceptions and other failures and a user can never forget to close a span. On occasion we will also start a span using the `with` clause in python. However, the decorator method is preferred since the method below makes more of a readability impact on the code. This example is taken from [job.py#L215](https://github.com/10gen/mongo/blob/6d36ac392086df85844870eef1d773f35020896c/buildscripts/resmokelib/testing/job.py#L215) This system is nice because the decorator captures exceptions and other failures and a user can never forget to close a span. On occasion we will also start a span using the `with` clause in python. However, the decorator method is preferred since the method below makes more of a readability impact on the code. This example is taken from [job.py#L215](https://github.com/mongodb/mongo/blob/6d36ac392086df85844870eef1d773f35020896c/buildscripts/resmokelib/testing/job.py#L215)
``` ```
with TRACER.start_as_current_span("func_name", attributes={}): with TRACER.start_as_current_span("func_name", attributes={}):

4
jstests/with_mongot/cross_repo_testing_requirements.md
View File

@ -10,10 +10,10 @@ Unfortunately, evergreen doesn't support multiple alias options. For that reason
evergreen patch -p mongodb-mongo-master --trigger-alias search-integration --alias required-and-mongot-e2e-tests evergreen patch -p mongodb-mongo-master --trigger-alias search-integration --alias required-and-mongot-e2e-tests
If your evergreen patch shows your changes failed a search e2e test defined on `10gen/mongod`, you can follow [these instructions](https://github.com/10gen/mongo/blob/master/jstests/with_mongot/e2e/mongot_testing_instructions.md) for running that test locally on your VM. If your evergreen patch shows your changes failed a search e2e test defined on `10gen/mongod`, you can follow [these instructions](https://github.com/mongodb/mongo/blob/master/jstests/with_mongot/e2e/mongot_testing_instructions.md) for running that test locally on your VM.
If your evergreen patch shows your changes failed an e2e test defined on `10gen/mongot`, please reach out in #search-query-engineering for assistance from mongot engineers in translating and addressing the failure. If your evergreen patch shows your changes failed an e2e test defined on `10gen/mongot`, please reach out in #search-query-engineering for assistance from mongot engineers in translating and addressing the failure.
### Didn't Find What You're Looking For? ### Didn't Find What You're Looking For?
Visit [the landing page](https://github.com/10gen/mongo/blob/master/src/mongo/db/query/search/README.md) for all `$search`/`$vectorSearch`/`$searchMeta` related documentation for server contributors. Visit [the landing page](https://github.com/mongodb/mongo/blob/master/src/mongo/db/query/search/README.md) for all `$search`/`$vectorSearch`/`$searchMeta` related documentation for server contributors.

4
jstests/with_mongot/e2e/mongot_testing_instructions.md
View File

@ -126,7 +126,7 @@ This order is required to ensure correctness. This is due to the nature of data
You can download the mongot binary that a specific evergreen patch or version utilized, which can be useful for trying to replicate errors. You can download the mongot binary that a specific evergreen patch or version utilized, which can be useful for trying to replicate errors.
You can download the mongot binary from any build variant that compiles mongot--i.e., variants which include the expansion `build_mongot: true` ([example](https://github.com/10gen/mongo/blob/848b5264be2d0f93d21ffe2e4058e810f8ea18f2/etc/evergreen_yml_components/variants/amazon/test_dev_master_branch_only.yml#L194)). More specifically, that includes: You can download the mongot binary from any build variant that compiles mongot--i.e., variants which include the expansion `build_mongot: true` ([example](https://github.com/mongodb/mongo/blob/848b5264be2d0f93d21ffe2e4058e810f8ea18f2/etc/evergreen_yml_components/variants/amazon/test_dev_master_branch_only.yml#L194)). More specifically, that includes:
- Compile variants that are depended upon by variants which run the search end to end tests, such as the variant `amazon-linux2023-arm64-static-compile` _(! Amazon Linux 2023 arm64 Enterprise Shared Library Compile & Static Analysis)_, which is depended upon by _! Amazon Linux 2023 arm64 Atlas Enterprise (all feature flags)_ - Compile variants that are depended upon by variants which run the search end to end tests, such as the variant `amazon-linux2023-arm64-static-compile` _(! Amazon Linux 2023 arm64 Enterprise Shared Library Compile & Static Analysis)_, which is depended upon by _! Amazon Linux 2023 arm64 Atlas Enterprise (all feature flags)_
- Variants that compile mongot **and** run the search end to end tests, such as: `amazon-linux2023-arm64-mongot-integration-patchable` _(AL2023 arm64 mongot integration tasks)_ - Variants that compile mongot **and** run the search end to end tests, such as: `amazon-linux2023-arm64-mongot-integration-patchable` _(AL2023 arm64 mongot integration tasks)_
@ -158,4 +158,4 @@ General information about the `setup-repro-env` command can be found in its [REA
### Didn't Find What You're Looking For? ### Didn't Find What You're Looking For?
Visit [the landing page](https://github.com/10gen/mongo/blob/master/src/mongo/db/query/search/README.md) for all $search/$vectorSearch/$searchMeta related documentation for server contributors. Visit [the landing page](https://github.com/mongodb/mongo/blob/master/src/mongo/db/query/search/README.md) for all $search/$vectorSearch/$searchMeta related documentation for server contributors.

18
src/mongo/db/admission/README.md
View File

@ -233,12 +233,12 @@ are expected to fall under this category**.
To apply admission control selectively, override `isSubjectToIngressAdmissionControl` and implement To apply admission control selectively, override `isSubjectToIngressAdmissionControl` and implement
selective logic to determine [when it should be applied][subjectAdmissionFind]. selective logic to determine [when it should be applied][subjectAdmissionFind].
[initiateCommand]: https://github.com/10gen/mongo/blob/a86c7f5de2a5de4d2f49e40e8970754ec6a5ba6c/src/mongo/db/service_entry_point_shard_role.cpp#L1588 [initiateCommand]: https://github.com/mongodb/mongo/blob/a86c7f5de2a5de4d2f49e40e8970754ec6a5ba6c/src/mongo/db/service_entry_point_shard_role.cpp#L1588
[admissionServerParam]: https://github.com/10gen/mongo/blob/291b72ec4a8364208d7633d881cddc98787832b8/src/mongo/db/service_entry_point_shard_role.cpp#L1804 [admissionServerParam]: https://github.com/mongodb/mongo/blob/291b72ec4a8364208d7633d881cddc98787832b8/src/mongo/db/service_entry_point_shard_role.cpp#L1804
[admissionPriority]: https://github.com/10gen/mongo/blob/291b72ec4a8364208d7633d881cddc98787832b8/src/mongo/db/service_entry_point_shard_role.cpp#L1809 [admissionPriority]: https://github.com/mongodb/mongo/blob/291b72ec4a8364208d7633d881cddc98787832b8/src/mongo/db/service_entry_point_shard_role.cpp#L1809
[tryAcquire]: https://github.com/10gen/mongo/blob/0ed24f52f011fc16cd968368ace216fe7e747723/src/mongo/util/concurrency/ticketholder.cpp#L130 [tryAcquire]: https://github.com/mongodb/mongo/blob/0ed24f52f011fc16cd968368ace216fe7e747723/src/mongo/util/concurrency/ticketholder.cpp#L130
[subjectAdmissionExTrue]: https://github.com/10gen/mongo/blob/0ed24f52f011fc16cd968368ace216fe7e747723/src/mongo/db/commands/query_cmd/bulk_write.cpp#L1311 [subjectAdmissionExTrue]: https://github.com/mongodb/mongo/blob/0ed24f52f011fc16cd968368ace216fe7e747723/src/mongo/db/commands/query_cmd/bulk_write.cpp#L1311
[subjectAdmissionFind]: https://github.com/10gen/mongo/blob/0ed24f52f011fc16cd968368ace216fe7e747723/src/mongo/db/commands/query_cmd/find_cmd.cpp#L385 [subjectAdmissionFind]: https://github.com/mongodb/mongo/blob/0ed24f52f011fc16cd968368ace216fe7e747723/src/mongo/db/commands/query_cmd/find_cmd.cpp#L385
[subjectVirtualFalse]: https://github.com/10gen/mongo/blob/0ed24f52f011fc16cd968368ace216fe7e747723/src/mongo/db/commands.h#L956 [subjectVirtualFalse]: https://github.com/mongodb/mongo/blob/0ed24f52f011fc16cd968368ace216fe7e747723/src/mongo/db/commands.h#L956
[ticketRelease]: https://github.com/10gen/mongo/blob/0ed24f52f011fc16cd968368ace216fe7e747723/src/mongo/db/service_entry_point_shard_role.cpp#L519 [ticketRelease]: https://github.com/mongodb/mongo/blob/0ed24f52f011fc16cd968368ace216fe7e747723/src/mongo/db/service_entry_point_shard_role.cpp#L519
[ingressACidl]: https://github.com/10gen/mongo/blob/cbb6b8543feeb6e110f646bbeb44d8779d838db1/src/mongo/db/admission/ingress_admission_control.idl#L43 [ingressACidl]: https://github.com/mongodb/mongo/blob/cbb6b8543feeb6e110f646bbeb44d8779d838db1/src/mongo/db/admission/ingress_admission_control.idl#L43

18
src/mongo/db/auth/README.md
View File

@ -196,7 +196,7 @@ the storage mechanism is listed as a sub-bullet below.
authentication. GSSAPI is the communication method used to communicate with Kerberos servers and authentication. GSSAPI is the communication method used to communicate with Kerberos servers and
with clients. When initializing this auth mechanism, the server tries to acquire its credential with clients. When initializing this auth mechanism, the server tries to acquire its credential
information from the KDC by calling information from the KDC by calling
[`tryAcquireServerCredential`](https://github.com/10gen/mongo-enterprise-modules/blob/r4.4.0/src/sasl/mongo_gssapi.h#L36). [`tryAcquireServerCredential`](https://github.com/mongodb/mongo-enterprise-modules/blob/r4.4.0/src/sasl/mongo_gssapi.h#L36).
If this is not approved, the server fasserts and the mechanism is not registered. On Windows, If this is not approved, the server fasserts and the mechanism is not registered. On Windows,
SChannel provides a `GSSAPI` library for the server to use. On other platforms, the Cyrus SASL SChannel provides a `GSSAPI` library for the server to use. On other platforms, the Cyrus SASL
library is used to make calls to the KDC (Kerberos key distribution center). library is used to make calls to the KDC (Kerberos key distribution center).
@ -770,30 +770,30 @@ The user must supply roles when running the `createUser` command. Roles are stor
LDAP authorization is an external method of getting roles. When a user authenticates using LDAP, LDAP authorization is an external method of getting roles. When a user authenticates using LDAP,
there are roles stored in the User document specified by the LDAP system. The LDAP system relies on there are roles stored in the User document specified by the LDAP system. The LDAP system relies on
the the
[`AuthorizationBackendLDAP`](https://github.com/10gen/mongo-enterprise-modules/blob/r4.4.0/src/ldap/authz_manager_external_state_ldap.h) [`AuthorizationBackendLDAP`](https://github.com/mongodb/mongo-enterprise-modules/blob/r4.4.0/src/ldap/authz_manager_external_state_ldap.h)
to make external requests to the LDAP server. The `AuthorizationBackendLDAP` overrides the to make external requests to the LDAP server. The `AuthorizationBackendLDAP` overrides the
`AuthorizationBackendLocal` for the current process, initially attempting to route all `AuthorizationBackendLocal` for the current process, initially attempting to route all
Authorization requests to LDAP and falling back on Local Authorization. LDAP queries are generated Authorization requests to LDAP and falling back on Local Authorization. LDAP queries are generated
from from
[`UserRequest`](https://github.com/10gen/mongo-enterprise-modules/blob/r4.4.0/src/ldap/authz_manager_external_state_ldap.cpp#L75-L113) [`UserRequest`](https://github.com/mongodb/mongo-enterprise-modules/blob/r4.4.0/src/ldap/authz_manager_external_state_ldap.cpp#L75-L113)
objects, passing just the username into the query. If a user has specified the `userToDNMapping` objects, passing just the username into the query. If a user has specified the `userToDNMapping`
server parameter, the `AuthorizationManager` calls the LDAPManager to transform the usernames into server parameter, the `AuthorizationManager` calls the LDAPManager to transform the usernames into
names that the LDAP server can understand. The LDAP subsystem relies on a complicated string names that the LDAP server can understand. The LDAP subsystem relies on a complicated string
escaping sequence, which is handled by the LDAPQuery class. After LDAP has returned the `User` escaping sequence, which is handled by the LDAPQuery class. After LDAP has returned the `User`
document, it resolves role names into privileges by dispatching a call to document, it resolves role names into privileges by dispatching a call to
[`Local::getUserObject`](https://github.com/10gen/mongo-enterprise-modules/blob/r4.7.0/src/ldap/authz_manager_external_state_ldap.cpp#L110-L123) [`Local::getUserObject`](https://github.com/mongodb/mongo-enterprise-modules/blob/r4.7.0/src/ldap/authz_manager_external_state_ldap.cpp#L110-L123)
with a `UserRequest` struct containing a set of roles to be resolved. with a `UserRequest` struct containing a set of roles to be resolved.
Connections to LDAP servers are made by the `LDAPManager` through the Connections to LDAP servers are made by the `LDAPManager` through the
[`LDAPRunner`](https://github.com/10gen/mongo-enterprise-modules/blob/r4.4.0/src/ldap/ldap_runner.h) [`LDAPRunner`](https://github.com/mongodb/mongo-enterprise-modules/blob/r4.4.0/src/ldap/ldap_runner.h)
by calling `bindAsUser()`. `BindAsUser()` attempts to set up a connection to the LDAP server using by calling `bindAsUser()`. `BindAsUser()` attempts to set up a connection to the LDAP server using
connection parameters specified through the command line when starting the process.The connection parameters specified through the command line when starting the process.The
[`LDAPConnectionFactory`](https://github.com/10gen/mongo-enterprise-modules/blob/r4.4.0/src/ldap/connections/ldap_connection_factory.h) [`LDAPConnectionFactory`](https://github.com/mongodb/mongo-enterprise-modules/blob/r4.4.0/src/ldap/connections/ldap_connection_factory.h)
is the class that is actually tasked with establishing a connection and sending raw bytes over the is the class that is actually tasked with establishing a connection and sending raw bytes over the
wire to the LDAP server, all other classes decompose the information to send and use the factory to wire to the LDAP server, all other classes decompose the information to send and use the factory to
actually send the information. The `LDAPConnectionFactory` has its own thread pool and executor to actually send the information. The `LDAPConnectionFactory` has its own thread pool and executor to
drive throughput for authorization. LDAP has an drive throughput for authorization. LDAP has an
[`LDAPUserCacheInvalidator`](https://github.com/10gen/mongo-enterprise-modules/blob/r4.4.0/src/ldap/ldap_user_cache_invalidator_job.h) [`LDAPUserCacheInvalidator`](https://github.com/mongodb/mongo-enterprise-modules/blob/r4.4.0/src/ldap/ldap_user_cache_invalidator_job.h)
that periodically sweeps the `AuthorizationManager` and deletes user entries that have `$external` as that periodically sweeps the `AuthorizationManager` and deletes user entries that have `$external` as
their authentication database. their authentication database.
@ -801,7 +801,7 @@ There are a few thread safety concerns when making connections to the LDAP serve
LibLDAP to make connections to the LDAP server. LibLDAP comes without any thread safety guarantees, LibLDAP to make connections to the LDAP server. LibLDAP comes without any thread safety guarantees,
so all the calls to libLDAP are wrapped with mutexes to ensure thread safety when connecting to LDAP so all the calls to libLDAP are wrapped with mutexes to ensure thread safety when connecting to LDAP
servers on certain distros. The logic to see whether libLDAP is thread-safe lives servers on certain distros. The logic to see whether libLDAP is thread-safe lives
[here](https://github.com/10gen/mongo-enterprise-modules/blob/r4.4.0/src/ldap/connections/openldap_connection.cpp#L348-L378). [here](https://github.com/mongodb/mongo-enterprise-modules/blob/r4.4.0/src/ldap/connections/openldap_connection.cpp#L348-L378).
#### <a name="x509azn"></a>X.509 Authorization #### <a name="x509azn"></a>X.509 Authorization
@ -884,7 +884,7 @@ Refer to the following links for definitions of the Classes referenced in this d
| `AuthorizationManager` | [mongo/db/auth/authorization_manager.h](https://github.com/mongodb/mongo/blob/r4.4.0/src/mongo/db/auth/authorization_manager.h) | Interface to external state providers | | `AuthorizationManager` | [mongo/db/auth/authorization_manager.h](https://github.com/mongodb/mongo/blob/r4.4.0/src/mongo/db/auth/authorization_manager.h) | Interface to external state providers |
| `AuthorizationSession` | [mongo/db/auth/authorization_session.h](https://github.com/mongodb/mongo/blob/r4.4.0/src/mongo/db/auth/authorization_session.h) | Representation of currently authenticated and authorized users on the `Client` connection | | `AuthorizationSession` | [mongo/db/auth/authorization_session.h](https://github.com/mongodb/mongo/blob/r4.4.0/src/mongo/db/auth/authorization_session.h) | Representation of currently authenticated and authorized users on the `Client` connection |
| `AuthzManagerExternalStateLocal` | [.../authorization_baackend_local.h](https://github.com/mongodb/mongo/blob/master/src/mongo/db/auth/authorization_backend_local.h) | `Local` implementation of user/role provider | | `AuthzManagerExternalStateLocal` | [.../authorization_baackend_local.h](https://github.com/mongodb/mongo/blob/master/src/mongo/db/auth/authorization_backend_local.h) | `Local` implementation of user/role provider |
| `AuthzManagerExternalStateLDAP` | [.../authorization_baackend_ldap.h](https://github.com/10gen/mongo/blob/master/src/mongo/db/modules/enterprise/src/ldap/authorization_backend_ldap.h) | `LDAP` implementation of users/role provider | | `AuthzManagerExternalStateLDAP` | [.../authorization_baackend_ldap.h](https://github.com/mongodb/mongo/blob/master/src/mongo/db/modules/enterprise/src/ldap/authorization_backend_ldap.h) | `LDAP` implementation of users/role provider |
| `Client` | [mongo/db/client.h](https://github.com/mongodb/mongo/blob/r4.4.0/src/mongo/db/client.h) | An active client session, typically representing a remote driver or shell | | `Client` | [mongo/db/client.h](https://github.com/mongodb/mongo/blob/r4.4.0/src/mongo/db/client.h) | An active client session, typically representing a remote driver or shell |
| `Privilege` | [mongo/db/auth/privilege.h](https://github.com/mongodb/mongo/blob/r4.4.0/src/mongo/db/auth/privilege.h) | A set of `ActionType`s permitted on a particular `resource' | | `Privilege` | [mongo/db/auth/privilege.h](https://github.com/mongodb/mongo/blob/r4.4.0/src/mongo/db/auth/privilege.h) | A set of `ActionType`s permitted on a particular `resource' |
| `ResourcePattern` | [mongo/db/auth/resource_pattern.h](https://github.com/mongodb/mongo/blob/r4.4.0/src/mongo/db/auth/resource_pattern.h) | A reference to a namespace, db, collection, or cluster to apply a set of `ActionType` privileges to | | `ResourcePattern` | [mongo/db/auth/resource_pattern.h](https://github.com/mongodb/mongo/blob/r4.4.0/src/mongo/db/auth/resource_pattern.h) | A reference to a namespace, db, collection, or cluster to apply a set of `ActionType` privileges to |

2
src/mongo/db/catalog/README.md
View File

@ -848,7 +848,7 @@ However, legacy indexes (indexes that were created pre-3.4) still need to be han
where a customer with legacy indexes upgrades to MongoDB 3.4+. The server treats any non-negative where a customer with legacy indexes upgrades to MongoDB 3.4+. The server treats any non-negative
numerical index key and non-numerical index key value as an ascending index, and treats negative numerical index key and non-numerical index key value as an ascending index, and treats negative
numerical values as descending. The exception to this is any form of a negative 0 (-0, -0.0, etc); numerical values as descending. The exception to this is any form of a negative 0 (-0, -0.0, etc);
these are treated as ascending. Details on how these values are treated can be found in [ordering.h](https://github.com/10gen/mongo/blob/master/src/mongo/bson/ordering.h), and examples of legacy indexes can be found in [this test](https://github.com/10gen/mongo/blob/master/src/mongo/bson/ordering_test.cpp). these are treated as ascending. Details on how these values are treated can be found in [ordering.h](https://github.com/mongodb/mongo/blob/master/src/mongo/bson/ordering.h), and examples of legacy indexes can be found in [this test](https://github.com/mongodb/mongo/blob/master/src/mongo/bson/ordering_test.cpp).
# Glossary # Glossary

14
src/mongo/db/exec/README.md
View File

@ -11,7 +11,7 @@ important that they are periodically released and reacquired.
### DocumentSource ### DocumentSource
[`mongo::DocumentSource`](https://github.com/10gen/mongo/blob/master/src/mongo/db/pipeline/document_source.h#L198) [`mongo::DocumentSource`](https://github.com/mongodb/mongo/blob/master/src/mongo/db/pipeline/document_source.h#L198)
execution model currently doesn't account for longer lived storage resources. execution model currently doesn't account for longer lived storage resources.
Due to lack of yielding support, all acquired resources must have a strictly bounded lifecycle, e.g. Due to lack of yielding support, all acquired resources must have a strictly bounded lifecycle, e.g.
@ -21,9 +21,9 @@ there.
### DocumentSourceCursor ### DocumentSourceCursor
[`mongo::DocumentSourceCursor`](https://github.com/10gen/mongo/blob/master/src/mongo/db/pipeline/document_source_cursor.h#L73) [`mongo::DocumentSourceCursor`](https://github.com/mongodb/mongo/blob/master/src/mongo/db/pipeline/document_source_cursor.h#L73)
acts as an adapter from [`mongo::PlanExecutor`](https://github.com/10gen/mongo/blob/master/src/mongo/db/query/plan_executor.h#L129) acts as an adapter from [`mongo::PlanExecutor`](https://github.com/mongodb/mongo/blob/master/src/mongo/db/query/plan_executor.h#L129)
to [`mongo::DocumentSource`](https://github.com/10gen/mongo/blob/master/src/mongo/db/pipeline/document_source.h#L198). It to [`mongo::DocumentSource`](https://github.com/mongodb/mongo/blob/master/src/mongo/db/pipeline/document_source.h#L198). It
is responsible for acquiring and releasing the storage resources (snapshots, locks, etc.), so that is responsible for acquiring and releasing the storage resources (snapshots, locks, etc.), so that
they don't leave the `mongo::DocumentSourceCursor` scope. Some of the storage resources are acquired they don't leave the `mongo::DocumentSourceCursor` scope. Some of the storage resources are acquired
and released by `mongo::PlanExecutor` via `mongo::PlanExecutor::restoreState` and and released by `mongo::PlanExecutor` via `mongo::PlanExecutor::restoreState` and
@ -42,9 +42,9 @@ are released. Buffered results are then used in subsequent calls to
Classic plan executor performs yields outside of calls to work(). Classic plan stages also use Classic plan executor performs yields outside of calls to work(). Classic plan stages also use
interrupt style yielding. When yield is needed work() can be interruped and a interrupt style yielding. When yield is needed work() can be interruped and a
[`mongo::PlanStage::StageState::NEED_YIELD`](https://github.com/10gen/mongo/blob/master/src/mongo/db/exec/plan_stage.h#L166-L184) [`mongo::PlanStage::StageState::NEED_YIELD`](https://github.com/mongodb/mongo/blob/master/src/mongo/db/exec/plan_stage.h#L166-L184)
is used to unwind the plan stage call stack. After that classic plan executor is used to unwind the plan stage call stack. After that classic plan executor
([`mongo::PlanExecutorImpl`](https://github.com/10gen/mongo/blob/master/src/mongo/db/query/plan_executor_impl.h#L118`)) ([`mongo::PlanExecutorImpl`](https://github.com/mongodb/mongo/blob/master/src/mongo/db/query/plan_executor_impl.h#L118`))
will release and reaquire storage resources. will release and reaquire storage resources.
### SBE yielding ### SBE yielding
@ -94,7 +94,7 @@ Such values must be either discarded, when no longer needed, or converted to own
their copy before the storage resources are released. their copy before the storage resources are released.
This is commongly done by using This is commongly done by using
[mongo::sbe::CanChangeState::prepareForYielding](https://github.com/10gen/mongo/blob/master/src/mongo/db/exec/sbe/stages/stages.h#L126) [mongo::sbe::CanChangeState::prepareForYielding](https://github.com/mongodb/mongo/blob/master/src/mongo/db/exec/sbe/stages/stages.h#L126)
utility. utility.
When stage slots are no longer needed, a `mongo::sbe::CanTrackStats::disableSlotAccess` method can When stage slots are no longer needed, a `mongo::sbe::CanTrackStats::disableSlotAccess` method can

54
src/mongo/db/exec/runtime_planners/classic_runtime_planner/README.md
View File

@ -12,31 +12,31 @@ The Query Optimization team is currently developing a cost-based plan ranker as
> ### Aside: Classic Runtime Planner for SBE > ### Aside: Classic Runtime Planner for SBE
> >
> This document outlines the multiplanning process that uses "classic" query plans. MongoDB also supports a Slot-Based Execution (SBE) engine that converts the result of classic multiplanning into SBE plans. For more details on this process, refer to the [Classic Runtime Planner for SBE README](https://github.com/10gen/mongo/blob/95d1830ce1acffd0108932d04538ed9dc995ade5/src/mongo/db/query/classic_runtime_planner_for_sbe/README.md). > This document outlines the multiplanning process that uses "classic" query plans. MongoDB also supports a Slot-Based Execution (SBE) engine that converts the result of classic multiplanning into SBE plans. For more details on this process, refer to the [Classic Runtime Planner for SBE README](https://github.com/mongodb/mongo/blob/95d1830ce1acffd0108932d04538ed9dc995ade5/src/mongo/db/query/classic_runtime_planner_for_sbe/README.md).
## [`CachedPlanner`](https://github.com/10gen/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/query/classic_runtime_planner/planner_interface.h#L164) ## [`CachedPlanner`](https://github.com/mongodb/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/query/classic_runtime_planner/planner_interface.h#L164)
If a query has been executed previously, its winning query plan might already be stored in the [plan cache](../plan_cache/README.md). If so, we can entirely bypass multiplanning by retrieving the winning query plan from the corresponding `PlanCacheEntry` via a `CachedPlanner`. If a query has been executed previously, its winning query plan might already be stored in the [plan cache](../plan_cache/README.md). If so, we can entirely bypass multiplanning by retrieving the winning query plan from the corresponding `PlanCacheEntry` via a `CachedPlanner`.
Note that if execution of a cached plan fails because the cached plan is less efficient that expected, we [`replan()`](https://github.com/10gen/mongo/blob/0088f90ce188b8bc32575e7e4b75149ceab5da0e/src/mongo/db/exec/cached_plan.cpp#L213) by falling back to standard multiplanning. For more information on replanning, see [here](../plan_cache/README.md#aside-replanning) Note that if execution of a cached plan fails because the cached plan is less efficient that expected, we [`replan()`](https://github.com/mongodb/mongo/blob/0088f90ce188b8bc32575e7e4b75149ceab5da0e/src/mongo/db/exec/cached_plan.cpp#L213) by falling back to standard multiplanning. For more information on replanning, see [here](../plan_cache/README.md#aside-replanning)
## [`MultiPlanner`](https://github.com/10gen/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/query/classic_runtime_planner/planner_interface.h#L182) ## [`MultiPlanner`](https://github.com/mongodb/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/query/classic_runtime_planner/planner_interface.h#L182)
If we have [more than one](#singlesolutionpassthroughplanner) `QuerySolution`, multiplanning begins in [`buildMultiPlan()`](https://github.com/10gen/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/query/get_executor.cpp#L586). This function initializes a `MultiPlanner` that manages the multiplanning process. For each `QuerySolution`, [`buildExecutableTree()`](https://github.com/10gen/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/query/classic_runtime_planner/multi_planner.cpp#L46) is called which constructs a tree of `PlanStage`s, where each stage corresponds to the tree in the `QuerySolution`. The `MultiPlanner` initiates planning within [`MultiPlanStage::pickBestPlan()`](https://github.com/10gen/mongo/blob/6b012bcbe4610ef1e88f9f75d171faa017503713/src/mongo/db/exec/multi_plan.cpp#L246-L253). If we have [more than one](#singlesolutionpassthroughplanner) `QuerySolution`, multiplanning begins in [`buildMultiPlan()`](https://github.com/mongodb/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/query/get_executor.cpp#L586). This function initializes a `MultiPlanner` that manages the multiplanning process. For each `QuerySolution`, [`buildExecutableTree()`](https://github.com/mongodb/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/query/classic_runtime_planner/multi_planner.cpp#L46) is called which constructs a tree of `PlanStage`s, where each stage corresponds to the tree in the `QuerySolution`. The `MultiPlanner` initiates planning within [`MultiPlanStage::pickBestPlan()`](https://github.com/mongodb/mongo/blob/6b012bcbe4610ef1e88f9f75d171faa017503713/src/mongo/db/exec/multi_plan.cpp#L246-L253).
During multiplanning, each `QuerySolution` is evaluated by running each candidate for a trial period in a round-robin fashion. This round-robin execution successively calls `PlanStage::work()` on each query, performing one unit of work for each. During multiplanning, each `QuerySolution` is evaluated by running each candidate for a trial period in a round-robin fashion. This round-robin execution successively calls `PlanStage::work()` on each query, performing one unit of work for each.
> ### Aside: Works > ### Aside: Works
> >
> Each `PlanStage` defines its own [`doWork()`](https://github.com/10gen/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/exec/plan_stage.h#L383) method, which is called from [`PlanStage::work()`](https://github.com/10gen/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/exec/plan_stage.h#L207). The content of this function represents one unit of work for each stage. For example: > Each `PlanStage` defines its own [`doWork()`](https://github.com/mongodb/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/exec/plan_stage.h#L383) method, which is called from [`PlanStage::work()`](https://github.com/mongodb/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/exec/plan_stage.h#L207). The content of this function represents one unit of work for each stage. For example:
> >
> - [`IndexScan::doWork()`](https://github.com/10gen/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/exec/index_scan.cpp#L147) retrieves the next `IndexKeyEntry` from the index, if one exists. > - [`IndexScan::doWork()`](https://github.com/mongodb/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/exec/index_scan.cpp#L147) retrieves the next `IndexKeyEntry` from the index, if one exists.
> - [`FetchStage::doWork()`](https://github.com/10gen/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/exec/fetch.cpp#L76) consumes one element of the input stream and returns the result. This means the `FetchStage` may need more than one work call to fetch an entire document. > - [`FetchStage::doWork()`](https://github.com/mongodb/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/exec/fetch.cpp#L76) consumes one element of the input stream and returns the result. This means the `FetchStage` may need more than one work call to fetch an entire document.
> >
> Some stages, like `GROUP` and `SORT`, require many successive calls to `work()` before generating a result. > Some stages, like `GROUP` and `SORT`, require many successive calls to `work()` before generating a result.
> For more information on `work()` refer to `plan_stage.h`'s [header comment](https://github.com/10gen/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/exec/plan_stage.h#L61-L122). > For more information on `work()` refer to `plan_stage.h`'s [header comment](https://github.com/mongodb/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/exec/plan_stage.h#L61-L122).
`PlanStage::work()` returns a [`StageState`](https://github.com/10gen/mongo/blob/4f39d22a701b931862b1f1d9ba97011bcacb9f98/src/mongo/db/exec/plan_stage.h#L151), which indicates that after this incremental unit of work, query execution is in one of these states: `PlanStage::work()` returns a [`StageState`](https://github.com/mongodb/mongo/blob/4f39d22a701b931862b1f1d9ba97011bcacb9f98/src/mongo/db/exec/plan_stage.h#L151), which indicates that after this incremental unit of work, query execution is in one of these states:
1. `ADVANCED`: A query result was returned. 1. `ADVANCED`: A query result was returned.
1. `IS_EOF`: Query execution is complete. 1. `IS_EOF`: Query execution is complete.
@ -47,7 +47,7 @@ During multiplanning, each `QuerySolution` is evaluated by running each candidat
> >
> "Yielding" refers to releasing locks on storage resources during execution to prevent long-running queries from blocking other queries. By default, queries are intended to yield every 10ms or 1000 iterations. > "Yielding" refers to releasing locks on storage resources during execution to prevent long-running queries from blocking other queries. By default, queries are intended to yield every 10ms or 1000 iterations.
> >
> During multiplanning, [`tryYield()`](https://github.com/10gen/mongo/blob/804e6f131091af60245b44ebada26cbdd2d34307/src/mongo/db/exec/multi_plan.cpp#L207) is called both between `work()` calls and when a `work()` returns `NEED_YIELD`, in order to ensure yielding is performed adequately. `tryYield()` takes a [`PlanYieldPolicy`](https://github.com/10gen/mongo/blob/804e6f131091af60245b44ebada26cbdd2d34307/src/mongo/db/query/plan_yield_policy.h#L78), which is passed through `work()` to indicate to the executor whether to yield under the current configuration. > During multiplanning, [`tryYield()`](https://github.com/mongodb/mongo/blob/804e6f131091af60245b44ebada26cbdd2d34307/src/mongo/db/exec/multi_plan.cpp#L207) is called both between `work()` calls and when a `work()` returns `NEED_YIELD`, in order to ensure yielding is performed adequately. `tryYield()` takes a [`PlanYieldPolicy`](https://github.com/mongodb/mongo/blob/804e6f131091af60245b44ebada26cbdd2d34307/src/mongo/db/query/plan_yield_policy.h#L78), which is passed through `work()` to indicate to the executor whether to yield under the current configuration.
As the query plans execute, statistics are recorded on their performance. These statistics will eventually be used to [determine a winner](#plan-ranking). As the query plans execute, statistics are recorded on their performance. These statistics will eventually be used to [determine a winner](#plan-ranking).
@ -57,11 +57,11 @@ Multiplanning continues until at least one candidate plan does one of the follow
1. Returns one full default `batchsize` of results. 1. Returns one full default `batchsize` of results.
- Currently, a default `batchsize` of results is 101 documents. - Currently, a default `batchsize` of results is 101 documents.
- This value can be modified using the [`internalQueryPlanEvaluationMaxResults`](https://github.com/10gen/mongo/blob/e60552830a722dc52422f39bf6be407933934392/src/mongo/db/query/query_knobs.idl#L156-L165) query knob. - This value can be modified using the [`internalQueryPlanEvaluationMaxResults`](https://github.com/mongodb/mongo/blob/e60552830a722dc52422f39bf6be407933934392/src/mongo/db/query/query_knobs.idl#L156-L165) query knob.
1. Hits "EOF", meaning the `QuerySolution` has finished execution during the trial period and returned all its results. 1. Hits "EOF", meaning the `QuerySolution` has finished execution during the trial period and returned all its results.
1. Makes the maximum number of [`work()`](#aside-works) calls. 1. Makes the maximum number of [`work()`](#aside-works) calls.
- Currently, the maximum number of `work()` calls is [calculated](https://github.com/10gen/mongo/blob/e60552830a722dc52422f39bf6be407933934392/src/mongo/db/exec/trial_period_utils.cpp#L44-L55) as `max(10K, 0.3 * collection_size)`. - Currently, the maximum number of `work()` calls is [calculated](https://github.com/mongodb/mongo/blob/e60552830a722dc52422f39bf6be407933934392/src/mongo/db/exec/trial_period_utils.cpp#L44-L55) as `max(10K, 0.3 * collection_size)`.
- These values can be modified using the [`internalQueryPlanEvaluationWorks`](https://github.com/10gen/mongo/blob/e60552830a722dc52422f39bf6be407933934392/src/mongo/db/query/query_knobs.idl#L101-L112) and [`internalQueryPlanEvaluationCollFraction`](https://github.com/10gen/mongo/blob/e60552830a722dc52422f39bf6be407933934392/src/mongo/db/query/query_knobs.idl#L129-L139) query knobs. - These values can be modified using the [`internalQueryPlanEvaluationWorks`](https://github.com/mongodb/mongo/blob/e60552830a722dc52422f39bf6be407933934392/src/mongo/db/query/query_knobs.idl#L101-L112) and [`internalQueryPlanEvaluationCollFraction`](https://github.com/mongodb/mongo/blob/e60552830a722dc52422f39bf6be407933934392/src/mongo/db/query/query_knobs.idl#L129-L139) query knobs.
Consider this setup, where `a` and `b` are random integers evenly distributed between 0 and 100: Consider this setup, where `a` and `b` are random integers evenly distributed between 0 and 100:
@ -82,7 +82,7 @@ Because the predicate on `a` is significantly more selective than the predicate
## Plan Ranking ## Plan Ranking
After the trial period is complete, [`plan_ranker::pickBestPlan()`](https://github.com/10gen/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/query/plan_ranker_util.h#L210) assigns each `QuerySolution` a score based on its performance during the trial period. The [formula for the scores](https://github.com/10gen/mongo/blob/23fcb16382bb8962d5648adfa7a7a2a828c555ec/src/mongo/db/query/plan_ranker.h#L160) is as follows: After the trial period is complete, [`plan_ranker::pickBestPlan()`](https://github.com/mongodb/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/query/plan_ranker_util.h#L210) assigns each `QuerySolution` a score based on its performance during the trial period. The [formula for the scores](https://github.com/mongodb/mongo/blob/23fcb16382bb8962d5648adfa7a7a2a828c555ec/src/mongo/db/query/plan_ranker.h#L160) is as follows:
``` ```
score = 1 score = 1
@ -116,14 +116,14 @@ Because the productivity ratio and tie-breakers will never surpass the EOF Bonus
In the event of a tie between plans, there are a few metrics regarding the shape of the candidate `QuerySolution`s that can be used to determine which is marginally better. In the event of a tie between plans, there are a few metrics regarding the shape of the candidate `QuerySolution`s that can be used to determine which is marginally better.
[Some such metrics](https://github.com/10gen/mongo/blob/95d1830ce1acffd0108932d04538ed9dc995ade5/src/mongo/db/query/plan_ranker.h#L120-L158) include: [Some such metrics](https://github.com/mongodb/mongo/blob/95d1830ce1acffd0108932d04538ed9dc995ade5/src/mongo/db/query/plan_ranker.h#L120-L158) include:
- `noFetchBonus`: Bonus for covered plans (i.e. plans that do not have a `FETCH` stage) - `noFetchBonus`: Bonus for covered plans (i.e. plans that do not have a `FETCH` stage)
- `noSortBonus`: Bonus for plans that do not have a [blocking `SORT`](#aside-blocking-sort) stage. - `noSortBonus`: Bonus for plans that do not have a [blocking `SORT`](#aside-blocking-sort) stage.
- `noIxisectBonus`: Bonus for plans that do not have an `AND_HASH` or `AND_SORTED` index intersection stage. - `noIxisectBonus`: Bonus for plans that do not have an `AND_HASH` or `AND_SORTED` index intersection stage.
- `groupByDistinctBonus`: Bonus for using a `DISTINCT_SCAN` in an aggregation context, where the bonus is [proportional](https://github.com/10gen/mongo/blob/340adb94bc0d348f42f7b427a06418dfd27f4bfc/src/mongo/db/query/plan_ranker.h#L156) to the productivity of the `DISTINCT_SCAN`. - `groupByDistinctBonus`: Bonus for using a `DISTINCT_SCAN` in an aggregation context, where the bonus is [proportional](https://github.com/mongodb/mongo/blob/340adb94bc0d348f42f7b427a06418dfd27f4bfc/src/mongo/db/query/plan_ranker.h#L156) to the productivity of the `DISTINCT_SCAN`.
Standard bonus is a value of [`epsilon`](https://github.com/10gen/mongo/blob/340adb94bc0d348f42f7b427a06418dfd27f4bfc/src/mongo/db/query/plan_ranker.h#L115), unless noted otherwise. Standard bonus is a value of [`epsilon`](https://github.com/mongodb/mongo/blob/340adb94bc0d348f42f7b427a06418dfd27f4bfc/src/mongo/db/query/plan_ranker.h#L115), unless noted otherwise.
Each `QuerySolution` that meets any of these specifications is awarded an additional point value to break the tie. Each `QuerySolution` that meets any of these specifications is awarded an additional point value to break the tie.
@ -169,25 +169,25 @@ flowchart TD
## Alternative Planners ## Alternative Planners
Although `MultiPlanner` is our "standard" case, not all queries utilize a `MultiPlanner`. Under certain conditions, we may use a different planner that is a subclass of the abstract class [`ClassicPlannerInterface`](https://github.com/10gen/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/query/classic_runtime_planner/planner_interface.h#L59); [`CachedPlanner`](#plan-cache-consultation) is one such example. Each subclass of `ClassicPlannerInterface` overrides [`doPlan()`](https://github.com/10gen/mongo/blob/6b012bcbe4610ef1e88f9f75d171faa017503713/src/mongo/db/query/classic_runtime_planner/planner_interface.h#L117). `MultiPlanner`'s override, for example, calls [`MultiPlanStage::pickBestPlan()`](https://github.com/10gen/mongo/blob/6b012bcbe4610ef1e88f9f75d171faa017503713/src/mongo/db/query/classic_runtime_planner/multi_planner.cpp#L55). Although `MultiPlanner` is our "standard" case, not all queries utilize a `MultiPlanner`. Under certain conditions, we may use a different planner that is a subclass of the abstract class [`ClassicPlannerInterface`](https://github.com/mongodb/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/query/classic_runtime_planner/planner_interface.h#L59); [`CachedPlanner`](#plan-cache-consultation) is one such example. Each subclass of `ClassicPlannerInterface` overrides [`doPlan()`](https://github.com/mongodb/mongo/blob/6b012bcbe4610ef1e88f9f75d171faa017503713/src/mongo/db/query/classic_runtime_planner/planner_interface.h#L117). `MultiPlanner`'s override, for example, calls [`MultiPlanStage::pickBestPlan()`](https://github.com/mongodb/mongo/blob/6b012bcbe4610ef1e88f9f75d171faa017503713/src/mongo/db/query/classic_runtime_planner/multi_planner.cpp#L55).
Each subclass is detailed below: Each subclass is detailed below:
### [`SingleSolutionPassthroughPlanner`](https://github.com/10gen/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/query/classic_runtime_planner/planner_interface.h#L146) ### [`SingleSolutionPassthroughPlanner`](https://github.com/mongodb/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/query/classic_runtime_planner/planner_interface.h#L146)
If only one `QuerySolution` exists for a query, no decision needs to be made about which plan to use. In this case, we initialize a `SingleSolutionPassthroughPlanner`, which [does no planning](https://github.com/10gen/mongo/blob/6b012bcbe4610ef1e88f9f75d171faa017503713/src/mongo/db/query/classic_runtime_planner/single_solution_passthrough_planner.cpp#L44) and creates a classic plan executor for its plan immediately. If only one `QuerySolution` exists for a query, no decision needs to be made about which plan to use. In this case, we initialize a `SingleSolutionPassthroughPlanner`, which [does no planning](https://github.com/mongodb/mongo/blob/6b012bcbe4610ef1e88f9f75d171faa017503713/src/mongo/db/query/classic_runtime_planner/single_solution_passthrough_planner.cpp#L44) and creates a classic plan executor for its plan immediately.
### [`IdHackPlanner`](https://github.com/10gen/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/query/classic_runtime_planner/planner_interface.h#L132) ### [`IdHackPlanner`](https://github.com/mongodb/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/query/classic_runtime_planner/planner_interface.h#L132)
An `IDHACK` query is a special query of the form `{_id: {$eq: <value>}}`. If a query meets this specification, we can bypass query planning and construct a "find-by-\_id" plan. This is a ["fast path" optimization](https://github.com/10gen/mongo/blob/2ceaa45d0b28d1a057c7538327958bdd3c7222db/src/mongo/db/query/get_executor.cpp#L1170) and therefore does not consult the [plan cache](#cachedplanner). It is a guarantee that we will only use the default `_id` index to execute the query. An `IDHACK` query is a special query of the form `{_id: {$eq: <value>}}`. If a query meets this specification, we can bypass query planning and construct a "find-by-\_id" plan. This is a ["fast path" optimization](https://github.com/mongodb/mongo/blob/2ceaa45d0b28d1a057c7538327958bdd3c7222db/src/mongo/db/query/get_executor.cpp#L1170) and therefore does not consult the [plan cache](#cachedplanner). It is a guarantee that we will only use the default `_id` index to execute the query.
### [`SubPlanner`](https://github.com/10gen/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/query/classic_runtime_planner/planner_interface.h#L199) ### [`SubPlanner`](https://github.com/mongodb/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/query/classic_runtime_planner/planner_interface.h#L199)
Rooted `$or` queries are eligible for subplanning. Rooted `$or` queries are eligible for subplanning.
> **Rooted `$or`**: A query that contains an `$or` operator at its top level. > **Rooted `$or`**: A query that contains an `$or` operator at its top level.
In these cases, a [`SubplanStage`](https://github.com/10gen/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/exec/subplan.h#L82) calls [`buildSubPlan()`](https://github.com/10gen/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/query/get_executor.cpp#L720), which constructs a special `PlanStage` tree. Here, each branch undergoes multiplanning separately. After a winner has been determined from each branch, the branches are ranked against each other, until a winning plan prevails for the entire rooted `$or`. In these cases, a [`SubplanStage`](https://github.com/mongodb/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/exec/subplan.h#L82) calls [`buildSubPlan()`](https://github.com/mongodb/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/query/get_executor.cpp#L720), which constructs a special `PlanStage` tree. Here, each branch undergoes multiplanning separately. After a winner has been determined from each branch, the branches are ranked against each other, until a winning plan prevails for the entire rooted `$or`.
For example: For example:
@ -197,9 +197,9 @@ db.c.find({$or: [{a: 1, x: 1}, {b: 1, y: 2}]})
In this query, the two branches: `{a: 1, x: 1}` and `{b: 1, y: 2}` would be multiplanned separately. Once a winning plan exists for each branch, the plan with the better score becomes the plan for the whole query. In this query, the two branches: `{a: 1, x: 1}` and `{b: 1, y: 2}` would be multiplanned separately. Once a winning plan exists for each branch, the plan with the better score becomes the plan for the whole query.
Note that subplanning's interaction with the [plan cache](../plan_cache/README.md#subplanning) is done on a _per-clause basis_. For details on this interaction, see [here](https://github.com/10gen/mongo/blob/21e7b8cfb79f3a7baae651055d5e1b3549dbdfda/src/mongo/db/exec/subplan.h#L70-L80). Note that subplanning's interaction with the [plan cache](../plan_cache/README.md#subplanning) is done on a _per-clause basis_. For details on this interaction, see [here](https://github.com/mongodb/mongo/blob/21e7b8cfb79f3a7baae651055d5e1b3549dbdfda/src/mongo/db/exec/subplan.h#L70-L80).
If, for any reason, subplanning fails on one of the children individually, [`choosePlanWholeQuery()`](https://github.com/10gen/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/exec/subplan.cpp#L115) is called which falls back to multiplanning on the query as a whole. If, for any reason, subplanning fails on one of the children individually, [`choosePlanWholeQuery()`](https://github.com/mongodb/mongo/blob/12390d154c1d06b6082a03d2410ff2b3578a323e/src/mongo/db/exec/subplan.cpp#L115) is called which falls back to multiplanning on the query as a whole.
--- ---

4
src/mongo/db/exec/sbe/README.md
View File

@ -52,12 +52,12 @@ overview of the different EExpression types:
As the name suggests, this expression type stores a single, immutable SBE value. An `EConstant` As the name suggests, this expression type stores a single, immutable SBE value. An `EConstant`
manages the value's lifetime (that is, it releases the value's memory on destruction if manages the value's lifetime (that is, it releases the value's memory on destruction if
necessary). necessary).
- [EPrimUnary, EPrimBinary and EPrimNary](https://github.com/10gen/mongo/blob/a04e7eea7dea44ee536703dbd98e7f832a495d11/src/mongo/db/exec/sbe/expressions/expression.h#L331-L457): - [EPrimUnary, EPrimBinary and EPrimNary](https://github.com/mongodb/mongo/blob/a04e7eea7dea44ee536703dbd98e7f832a495d11/src/mongo/db/exec/sbe/expressions/expression.h#L331-L457):
These expressions represent basic logical, arithmetic, and comparison operations that take one, These expressions represent basic logical, arithmetic, and comparison operations that take one,
two, and two or more arguments, respectively. two, and two or more arguments, respectively.
- [EIf](https://github.com/mongodb/mongo/blob/06a931ffadd7ce62c32288d03e5a38933bd522d3/src/mongo/db/exec/sbe/expressions/expression.h#L440-L461): - [EIf](https://github.com/mongodb/mongo/blob/06a931ffadd7ce62c32288d03e5a38933bd522d3/src/mongo/db/exec/sbe/expressions/expression.h#L440-L461):
Represents an 'if then else' expression. Represents an 'if then else' expression.
- [ESwitch](https://github.com/10gen/mongo/blob/a04e7eea7dea44ee536703dbd98e7f832a495d11/src/mongo/db/exec/sbe/expressions/expression.h#L509-L567): - [ESwitch](https://github.com/mongodb/mongo/blob/a04e7eea7dea44ee536703dbd98e7f832a495d11/src/mongo/db/exec/sbe/expressions/expression.h#L509-L567):
Represents a multi-conditional switch expression (a.k.a. if-then-elif-...-else expression). Represents a multi-conditional switch expression (a.k.a. if-then-elif-...-else expression).
- [EFunction](https://github.com/mongodb/mongo/blob/06a931ffadd7ce62c32288d03e5a38933bd522d3/src/mongo/db/exec/sbe/expressions/expression.h#L416-L438): - [EFunction](https://github.com/mongodb/mongo/blob/06a931ffadd7ce62c32288d03e5a38933bd522d3/src/mongo/db/exec/sbe/expressions/expression.h#L416-L438):
Represents a named, built-in function supported natively by the engine. At the time of writing, there are over [150 such Represents a named, built-in function supported natively by the engine. At the time of writing, there are over [150 such

4
src/mongo/db/pipeline/search/vectorSearch_technical_overview.md
View File

@ -57,7 +57,7 @@ Vector indexes are managed through the existing [search index management command
### Explain ### Explain
'$vectorSearch' explains follow how $search/$searchMeta explains work. Check out [search_technical_overview.md](https://github.com/10gen/mongo/blob/master/src/mongo/db/query/search/search_technical_overview.md) for more information. '$vectorSearch' explains follow how $search/$searchMeta explains work. Check out [search_technical_overview.md](https://github.com/mongodb/mongo/blob/master/src/mongo/db/query/search/search_technical_overview.md) for more information.
### Testing ### Testing
@ -65,4 +65,4 @@ The `vectorSearch` command is supported by [`mongotmock`](https://github.com/mon
### Didn't Find What You're Looking For? ### Didn't Find What You're Looking For?
Visit [the landing page](https://github.com/10gen/mongo/blob/master/src/mongo/db/query/search/README.md) for all $search/$vectorSearch/$searchMeta related documentation for server contributors. Visit [the landing page](https://github.com/mongodb/mongo/blob/master/src/mongo/db/query/search/README.md) for all $search/$vectorSearch/$searchMeta related documentation for server contributors.

12
src/mongo/db/query/query_stats/README.md
View File

@ -315,9 +315,9 @@ output one document per query stats key - output in the "key" field.
<!-- Links --> <!-- Links -->
[query stats store]: https://github.com/10gen/mongo/blob/3cc7cd2a439e25fff9dd26fb1f94057d837a06f9/src/mongo/db/query/query_stats/query_stats.h#L100-L104 [query stats store]: https://github.com/mongodb/mongo/blob/3cc7cd2a439e25fff9dd26fb1f94057d837a06f9/src/mongo/db/query/query_stats/query_stats.h#L100-L104
[partition calculation comment]: https://github.com/10gen/mongo/blob/3cc7cd2a439e25fff9dd26fb1f94057d837a06f9/src/mongo/db/query/query_stats/query_stats.cpp#L173-179 [partition calculation comment]: https://github.com/mongodb/mongo/blob/3cc7cd2a439e25fff9dd26fb1f94057d837a06f9/src/mongo/db/query/query_stats/query_stats.cpp#L173-179
[register request]: https://github.com/10gen/mongo/blob/3cc7cd2a439e25fff9dd26fb1f94057d837a06f9/src/mongo/db/query/query_stats/query_stats.h#L196-L199 [register request]: https://github.com/mongodb/mongo/blob/3cc7cd2a439e25fff9dd26fb1f94057d837a06f9/src/mongo/db/query/query_stats/query_stats.h#L196-L199
[write query stats]: https://github.com/10gen/mongo/blob/3cc7cd2a439e25fff9dd26fb1f94057d837a06f9/src/mongo/db/query/query_stats/query_stats.h#L253-L258 [write query stats]: https://github.com/mongodb/mongo/blob/3cc7cd2a439e25fff9dd26fb1f94057d837a06f9/src/mongo/db/query/query_stats/query_stats.h#L253-L258
[write query stats comments]: https://github.com/10gen/mongo/blob/3cc7cd2a439e25fff9dd26fb1f94057d837a06f9/src/mongo/db/query/query_stats/query_stats.h#L243-L252 [write query stats comments]: https://github.com/mongodb/mongo/blob/3cc7cd2a439e25fff9dd26fb1f94057d837a06f9/src/mongo/db/query/query_stats/query_stats.h#L243-L252
[query stats will never exhaust]: https://github.com/10gen/mongo/blob/8be794e1983e2b24938489ad2b018b630ea9b563/src/mongo/db/clientcursor.h#L510 [query stats will never exhaust]: https://github.com/mongodb/mongo/blob/8be794e1983e2b24938489ad2b018b630ea9b563/src/mongo/db/clientcursor.h#L510

18
src/mongo/db/query/search/README.md
View File

@ -6,17 +6,17 @@ This README serves as a landing page for all search\* related documentation to m
## Technical Details ## Technical Details
To read about the high-level technical implementation of $search and $searchMeta, please check out [search_technical_overview.md](https://github.com/10gen/mongo/blob/master/src/mongo/db/query/search/search_technical_overview.md). To read about the high-level technical implementation of $search and $searchMeta, please check out [search_technical_overview.md](https://github.com/mongodb/mongo/blob/master/src/mongo/db/query/search/search_technical_overview.md).
To read about a high-level overview of $vectorSearch, please check out [vectorSearch_technical_overview.md](https://github.com/10gen/mongo/blob/master/src/mongo/db/pipeline/search/vectorSearch_technical_overview.md) To read about a high-level overview of $vectorSearch, please check out [vectorSearch_technical_overview.md](https://github.com/mongodb/mongo/blob/master/src/mongo/db/pipeline/search/vectorSearch_technical_overview.md)
## Testing ## Testing
Testing search features is a bit different than other aggregation stages! You will fall into one of three categories: Testing search features is a bit different than other aggregation stages! You will fall into one of three categories:
1. If your search feature requires currently in-progress/incomplete changes to 10gen/mongot, you will need to test your mongod changes with mongot-mock. To find out more about writing a jstest that uses mongot-mock, please follow this [wiki](https://wiki.corp.mongodb.com/display/~zixuan.zhuang@mongodb.com/How+to+run+%24search+locally+using+Mongot+Mock). You can peruse more examples in [jstests/with_mongot/mongotmock/](https://github.com/10gen/mongo/blob/master/jstests/with_mongot/mongotmock/). 1. If your search feature requires currently in-progress/incomplete changes to 10gen/mongot, you will need to test your mongod changes with mongot-mock. To find out more about writing a jstest that uses mongot-mock, please follow this [wiki](https://wiki.corp.mongodb.com/display/~zixuan.zhuang@mongodb.com/How+to+run+%24search+locally+using+Mongot+Mock). You can peruse more examples in [jstests/with_mongot/mongotmock/](https://github.com/mongodb/mongo/blob/master/jstests/with_mongot/mongotmock/).
2. If your search feature is fully supported on the release mongot binary (eg the mongot required changes have been merged to 10gen/mongot and it's been released), you can write a standard jstest for your search feature. However you will need a mongot binary on your machine to run your jstest locally. To learn how to acquire a mongot binary locally and get more details on end-to-end testing search features locally and on evergreen, please checkout [mongot_testing_instructions.md](https://github.com/10gen/mongo/blob/master/jstests/with_mongot/e2e/mongot_testing_instructions.md). 2. If your search feature is fully supported on the release mongot binary (eg the mongot required changes have been merged to 10gen/mongot and it's been released), you can write a standard jstest for your search feature. However you will need a mongot binary on your machine to run your jstest locally. To learn how to acquire a mongot binary locally and get more details on end-to-end testing search features locally and on evergreen, please checkout [mongot_testing_instructions.md](https://github.com/mongodb/mongo/blob/master/jstests/with_mongot/e2e/mongot_testing_instructions.md).
3. If your search feature is supported on the latest mongot binary (from 10gen/mongot) but not on the released version, you can use tags to ensure your test only runs on the appropriate version. Mongot versions follow the format X.Y.Z, where X is the API version, Y updates for functionality changes, and Z updates for bug fixes. You can use the following tag formats to disallow tests from running on the released version of mongot. 3. If your search feature is supported on the latest mongot binary (from 10gen/mongot) but not on the released version, you can use tags to ensure your test only runs on the appropriate version. Mongot versions follow the format X.Y.Z, where X is the API version, Y updates for functionality changes, and Z updates for bug fixes. You can use the following tag formats to disallow tests from running on the released version of mongot.
@ -31,7 +31,7 @@ Testing search features is a bit different than other aggregation stages! You wi
Once the released version matches or exceeds the tag on the test, the test will also run for the released version. Once the released version matches or exceeds the tag on the test, the test will also run for the released version.
Regardless of the category you find yourself in, you are required to run all e2e tests defined on both 10gen/mongod and 10gen/mongot repos. To learn how to run all cross-repo e2e tests, please check out [jstests/with_mongot/cross_repo_testing_requirements.md](https://github.com/10gen/mongo/blob/master/jstests/with_mongot/cross_repo_testing_requirements.md). Regardless of the category you find yourself in, you are required to run all e2e tests defined on both 10gen/mongod and 10gen/mongot repos. To learn how to run all cross-repo e2e tests, please check out [jstests/with_mongot/cross_repo_testing_requirements.md](https://github.com/mongodb/mongo/blob/master/jstests/with_mongot/cross_repo_testing_requirements.md).
## Hybrid Search ## Hybrid Search
@ -49,21 +49,21 @@ The scoreDetails field is built up in 4 phases.
- `{value: {$meta: "score"}, details: []}` if the input pipeline generates score metadata, but not scoreDetails, then the incoming score is set, and the details array is set to empty - `{value: {$meta: "score"}, details: []}` if the input pipeline generates score metadata, but not scoreDetails, then the incoming score is set, and the details array is set to empty
- `{details: []}` if the input pipeline generates neither score nor scoreDetails metadata - `{details: []}` if the input pipeline generates neither score nor scoreDetails metadata
See the [addScoreDetails function](https://github.com/10gen/mongo/blob/f3eed1b0cc3860aae1d524de205f7fe3b28e2a3d/src/mongo/db/pipeline/document_source_hybrid_scoring_util.cpp#L765) for the exact implementation. See the [addScoreDetails function](https://github.com/mongodb/mongo/blob/f3eed1b0cc3860aae1d524de205f7fe3b28e2a3d/src/mongo/db/pipeline/document_source_hybrid_scoring_util.cpp#L765) for the exact implementation.
2. **Phase 2** 2. **Phase 2**
The second phase starts after all input pipelines have executed and begins the process of combining all the results into a total ranked set. This consists of grouping the newly added scoreDetails fields (remember that there’s 1 for each input pipeline) across all the documents. The grouping is needed because after processing the N input pipelines, there can be up to N repeats of the same document. Each document will have its own fields and any added fields for that pipeline (ex: the searchPipe_scoreDetails is an example of an added field specific to the searchPipe pipeline. Only 1 of the N repeated documents will have this field.) The result of this step is that each unique document should have a document field named <input_pipeline_name>\_scoreDetails, for each input pipeline this document appeared in. The second phase starts after all input pipelines have executed and begins the process of combining all the results into a total ranked set. This consists of grouping the newly added scoreDetails fields (remember that there’s 1 for each input pipeline) across all the documents. The grouping is needed because after processing the N input pipelines, there can be up to N repeats of the same document. Each document will have its own fields and any added fields for that pipeline (ex: the searchPipe_scoreDetails is an example of an added field specific to the searchPipe pipeline. Only 1 of the N repeated documents will have this field.) The result of this step is that each unique document should have a document field named <input_pipeline_name>\_scoreDetails, for each input pipeline this document appeared in.
See the [groupEachScore function](https://github.com/10gen/mongo/blob/f3eed1b0cc3860aae1d524de205f7fe3b28e2a3d/src/mongo/db/pipeline/document_source_rank_fusion.cpp#L461) for the exact implementation. See the [groupEachScore function](https://github.com/mongodb/mongo/blob/f3eed1b0cc3860aae1d524de205f7fe3b28e2a3d/src/mongo/db/pipeline/document_source_rank_fusion.cpp#L461) for the exact implementation.
3. **Phase 3** 3. **Phase 3**
The third phase calculates a new field called `calculatedScoreDetails` per document that combines all the input pipeline's scoreDetails into an array, with one entry per input pipeline scoreDetails. Each entry in the array contains the following scoreDetails subfields: `input pipeline name`, `rank, weight`, and `details` (`details` contains the value of the input pipeline's `scoreDetails`). The two optional subfields are `value` (if the pipeline sets the score metadata field, then this is the value of `$meta: score`) and `description` (if the input pipeline generates a `description` as part of `scoreDetails`, `description` contains that value). The third phase calculates a new field called `calculatedScoreDetails` per document that combines all the input pipeline's scoreDetails into an array, with one entry per input pipeline scoreDetails. Each entry in the array contains the following scoreDetails subfields: `input pipeline name`, `rank, weight`, and `details` (`details` contains the value of the input pipeline's `scoreDetails`). The two optional subfields are `value` (if the pipeline sets the score metadata field, then this is the value of `$meta: score`) and `description` (if the input pipeline generates a `description` as part of `scoreDetails`, `description` contains that value).
The non-hybrid search stages ($search and $score) only set the outermost scoreDetails fields (`value`, `description`, and `details`); The non-hybrid search stages ($search and $score) only set the outermost scoreDetails fields (`value`, `description`, and `details`);
See the [calculatedScoreDetails function](https://github.com/10gen/mongo/blob/f3eed1b0cc3860aae1d524de205f7fe3b28e2a3d/src/mongo/db/pipeline/document_source_hybrid_scoring_util.cpp#L120) for the exact implementation. See the [calculatedScoreDetails function](https://github.com/mongodb/mongo/blob/f3eed1b0cc3860aae1d524de205f7fe3b28e2a3d/src/mongo/db/pipeline/document_source_hybrid_scoring_util.cpp#L120) for the exact implementation.
4. **Phase 4** 4. **Phase 4**
The fourth phase simply sets the `scoreDetails` metadata which represents the final scoreDetails value that gets returned to the user in the final results. Its fields include the `value` field which is the calculated score (for example: in the case of $rankFusion, the Reciprocal Rank Fusion score), the `description` field which is the stage’s description for how the score was calculated, and the `details` field which is the fully assembled calculatedScoreDetails from the previous step. The fourth phase simply sets the `scoreDetails` metadata which represents the final scoreDetails value that gets returned to the user in the final results. Its fields include the `value` field which is the calculated score (for example: in the case of $rankFusion, the Reciprocal Rank Fusion score), the `description` field which is the stage’s description for how the score was calculated, and the `details` field which is the fully assembled calculatedScoreDetails from the previous step.
See the [buildScoreAndMergeStages function](https://github.com/10gen/mongo/blob/f3eed1b0cc3860aae1d524de205f7fe3b28e2a3d/src/mongo/db/pipeline/document_source_rank_fusion.cpp#L565) for the exact implementation. See the [buildScoreAndMergeStages function](https://github.com/mongodb/mongo/blob/f3eed1b0cc3860aae1d524de205f7fe3b28e2a3d/src/mongo/db/pipeline/document_source_rank_fusion.cpp#L565) for the exact implementation.

6
src/mongo/db/query/search/mongot_queries_on_views.md
View File

@ -28,7 +28,7 @@ myMatchView.aggregate([{$search: { index: "matchIndex", exists: {path: "_id"}}},
1. mongod receives the query on the requested nss (`myMatchView`) which is the view nss. As this is non-sharded, mongod has the views catalog and resolves the view itself. 1. mongod receives the query on the requested nss (`myMatchView`) which is the view nss. As this is non-sharded, mongod has the views catalog and resolves the view itself.
2. mongod recursively calls `_runAggregate` on the resolved view. 2. mongod recursively calls `_runAggregate` on the resolved view.
3. In `parsePipelineAndRegisterQueryStats()`, we call `search_helpers::checkAndAddResolvedNamespaceForSearch()` before parsing the raw bson obj vector into a Pipeline. This is essential because the search helper will add the resolved view to the expression context. 3. In `parsePipelineAndRegisterQueryStats()`, we call `search_helpers::checkAndAddResolvedNamespaceForSearch()` before parsing the raw bson obj vector into a Pipeline. This is essential because the search helper will add the resolved view to the expression context.
4. When `parsePipelineAndRegisterQueryStats()` parses the raw bson obj vector, createFromBson() is called on every stage. When `DocumentSourceSearch::createFromBson()` is called, it will call a helper, `search_helpers::getViewFromBSONObj()`, that [checks the `expCtx` for the view ns](https://github.com/10gen/mongo/blob/e2a70df2954e20568c8a1c6fa08c6aa7ffee0d39/src/mongo/db/pipeline/search/search_helper.cpp#L680-L685) that was stored in step 3 to determine if the request is on a view or a normal collection. Since the request is indeed on a view, this helper will return a boost::optional pointer to a custom struct, `MongotQueryViewInfo`, that contains the view information. 4. When `parsePipelineAndRegisterQueryStats()` parses the raw bson obj vector, createFromBson() is called on every stage. When `DocumentSourceSearch::createFromBson()` is called, it will call a helper, `search_helpers::getViewFromBSONObj()`, that [checks the `expCtx` for the view ns](https://github.com/mongodb/mongo/blob/e2a70df2954e20568c8a1c6fa08c6aa7ffee0d39/src/mongo/db/pipeline/search/search_helper.cpp#L680-L685) that was stored in step 3 to determine if the request is on a view or a normal collection. Since the request is indeed on a view, this helper will return a boost::optional pointer to a custom struct, `MongotQueryViewInfo`, that contains the view information.
5. `DocumentSourceSearch::createFromBson()` will then pass the `MongotQueryViewInfo` obj to the `DocumentSourceSearch` constructor call in order to store the view name + pipeline in a special struct on the `DocumentSourceSearch` object itself. 5. `DocumentSourceSearch::createFromBson()` will then pass the `MongotQueryViewInfo` obj to the `DocumentSourceSearch` constructor call in order to store the view name + pipeline in a special struct on the `DocumentSourceSearch` object itself.
6. When `$search` is desugared, it will store the view name + pipeline on the `$_internalSearchMongotRemote`[^2] and the view pipeline on `$_internalSearchIdLookup`. This allows us to pass the view name + pipeline in the request to mongot and it allows the server to run the view pipeline as part of `$_internalSearchIdLookup`'s subpieline. The desugared pipeline becomes roughly: 6. When `$search` is desugared, it will store the view name + pipeline on the `$_internalSearchMongotRemote`[^2] and the view pipeline on `$_internalSearchIdLookup`. This allows us to pass the view name + pipeline in the request to mongot and it allows the server to run the view pipeline as part of `$_internalSearchIdLookup`'s subpieline. The desugared pipeline becomes roughly:
@ -72,7 +72,7 @@ myMatchView.aggregate([{$search: { index: "matchIndex", exists: {path: "_id"}}},
3. The shards bump this request back to mongos via throwing a `CommandOnShardedViewNotSupportedOnMongod` expection with the resolved view info from the primary shard 3. The shards bump this request back to mongos via throwing a `CommandOnShardedViewNotSupportedOnMongod` expection with the resolved view info from the primary shard
4. Mongos does not append the view pipeline to the beginning of the request. Mongos retains the `resolvedView` that is passed back from the shards in `ClusterAggregate::runAggregate()` to be able to persist it through `parsePipelineAndRegisterQueryStats`, as the latter creates the `expCtx`, which will temporarily store the view name and pipeline. 4. Mongos does not append the view pipeline to the beginning of the request. Mongos retains the `resolvedView` that is passed back from the shards in `ClusterAggregate::runAggregate()` to be able to persist it through `parsePipelineAndRegisterQueryStats`, as the latter creates the `expCtx`, which will temporarily store the view name and pipeline.
5. Mongos then puts the view name and pipeline on its `expCtx`. 5. Mongos then puts the view name and pipeline on its `expCtx`.
6. When the raw bson obj vector is parsed into a pipeline and `DocumentSourceSearch::createFromBson` is consequently called, it will call a [helper that check the `expCtx` for the view ns](https://github.com/10gen/mongo/blob/e2a70df2954e20568c8a1c6fa08c6aa7ffee0d39/src/mongo/db/pipeline/search/search_helper.cpp#L680-L685) that was stored in step 5. 6. When the raw bson obj vector is parsed into a pipeline and `DocumentSourceSearch::createFromBson` is consequently called, it will call a [helper that check the `expCtx` for the view ns](https://github.com/mongodb/mongo/blob/e2a70df2954e20568c8a1c6fa08c6aa7ffee0d39/src/mongo/db/pipeline/search/search_helper.cpp#L680-L685) that was stored in step 5.
7. `DocumentSourceSearch::createFromBson()` will then pass this view to the `DocumentSourceSearch` constructor call in order to store the view name + pipeline in a special struct on the `DocumentSourceSearch` object itself. 7. `DocumentSourceSearch::createFromBson()` will then pass this view to the `DocumentSourceSearch` constructor call in order to store the view name + pipeline in a special struct on the `DocumentSourceSearch` object itself.
8. When mongos serializes the query to perform shard targeting, it serializes the view name and pipeline inside the `$search` stage. 8. When mongos serializes the query to perform shard targeting, it serializes the view name and pipeline inside the `$search` stage.
@ -89,7 +89,7 @@ myMatchView.aggregate([{$search: { index: "matchIndex", exists: {path: "_id"}}},
} }
``` ```
9. The targeted shard receives the BSON request from mongos and [if the `$search` spec includes a view object](https://github.com/10gen/mongo/blob/e2a70df2954e20568c8a1c6fa08c6aa7ffee0d39/src/mongo/db/pipeline/search/search_helper.cpp#L674-L678), `DocumentSourceSearch::createFromBson()` will pass this view to the `DocumentSourceSearch` constructor call in order to store the view name + pipeline in a special struct on the `DocumentSourceSearch` object itself. 9. The targeted shard receives the BSON request from mongos and [if the `$search` spec includes a view object](https://github.com/mongodb/mongo/blob/e2a70df2954e20568c8a1c6fa08c6aa7ffee0d39/src/mongo/db/pipeline/search/search_helper.cpp#L674-L678), `DocumentSourceSearch::createFromBson()` will pass this view to the `DocumentSourceSearch` constructor call in order to store the view name + pipeline in a special struct on the `DocumentSourceSearch` object itself.
10. When the shard desugars `$search` to <`$_internalSearchMongotRemote`, `$_internalSearchIdLookup`> it will store the view name + pipeline on the former and the view pipeline on the latter. This allows us to pass the view name in the request to mongot and it allows the shard to run the view pipeline as part of `$_internalSearchIdLookup`'s subpieline. The desugared pipeline will look identical to the one presented in `Non-Sharded` 10. When the shard desugars `$search` to <`$_internalSearchMongotRemote`, `$_internalSearchIdLookup`> it will store the view name + pipeline on the former and the view pipeline on the latter. This allows us to pass the view name in the request to mongot and it allows the shard to run the view pipeline as part of `$_internalSearchIdLookup`'s subpieline. The desugared pipeline will look identical to the one presented in `Non-Sharded`
[^1]: For returnStoredSource queries, the documents returned by mongot already include the fields transformed by the view pipeline. As such, mongod doesn't need to apply the view pipeline after `idLookup`. [^1]: For returnStoredSource queries, the documents returned by mongot already include the fields transformed by the view pipeline. As such, mongod doesn't need to apply the view pipeline after `idLookup`.

8
src/mongo/db/query/search/search_technical_overview.md
View File

@ -63,15 +63,15 @@ Within this stage, the underlying [TaskExecutorCursor](https://github.com/mongod
Each batch returned from mongot includes a batch of BSON documents and metadata about the query results. Each document contains an \_id and a relevancy score. The relevancy score indicates how well the document’s indexed values matched the user query. Metadata is a user-specified group of fields with information about the result set as a whole, mostly including counts of various groups (or facets). Each batch returned from mongot includes a batch of BSON documents and metadata about the query results. Each document contains an \_id and a relevancy score. The relevancy score indicates how well the document’s indexed values matched the user query. Metadata is a user-specified group of fields with information about the result set as a whole, mostly including counts of various groups (or facets).
We try to optimize time spent communicating with and waiting on mongot by tuning the `batchSize` option on mongot requests and toggling "prefetch-ing" of GetMore requests. This batchSize-tuning and prefetch-enablement logic is based on an attempt at inferring how many documents need to be requested from mongot (the upper and lower bounds of [`DocsNeededBounds`](https://github.com/10gen/mongo/blob/03222ee4d38696f293302d0d322b7dac2ccb1e1d/src/mongo/db/pipeline/visitors/docs_needed_bounds.h/#L39)). See [`extractDocsNeededBounds()`](https://github.com/10gen/mongo/blob/07c5da765d36bfc2bb6bc6d9b101a90cf09e82f7/src/mongo/db/pipeline/visitors/document_source_visitor_docs_needed_bounds.h/#L94) for details on how we traverse the full user pipeline to compute those bounds. We try to optimize time spent communicating with and waiting on mongot by tuning the `batchSize` option on mongot requests and toggling "prefetch-ing" of GetMore requests. This batchSize-tuning and prefetch-enablement logic is based on an attempt at inferring how many documents need to be requested from mongot (the upper and lower bounds of [`DocsNeededBounds`](https://github.com/mongodb/mongo/blob/03222ee4d38696f293302d0d322b7dac2ccb1e1d/src/mongo/db/pipeline/visitors/docs_needed_bounds.h/#L39)). See [`extractDocsNeededBounds()`](https://github.com/mongodb/mongo/blob/07c5da765d36bfc2bb6bc6d9b101a90cf09e82f7/src/mongo/db/pipeline/visitors/document_source_visitor_docs_needed_bounds.h/#L94) for details on how we traverse the full user pipeline to compute those bounds.
Once the bounds are computed and stored in the document source, we follow a set of heuristics to compute a batchSize for the initial mongot request based on those bounds ([here](https://github.com/10gen/mongo/blob/03222ee4d38696f293302d0d322b7dac2ccb1e1d/src/mongo/db/query/search/mongot_cursor.cpp/#L110)). The heuristics include applying "oversubscription" logic for non-storedSource queries, to account for the possibility that $\_internalSearchIdLookup may discard some of the documents returned by mongot. Once the bounds are computed and stored in the document source, we follow a set of heuristics to compute a batchSize for the initial mongot request based on those bounds ([here](https://github.com/mongodb/mongo/blob/03222ee4d38696f293302d0d322b7dac2ccb1e1d/src/mongo/db/query/search/mongot_cursor.cpp/#L110)). The heuristics include applying "oversubscription" logic for non-storedSource queries, to account for the possibility that $\_internalSearchIdLookup may discard some of the documents returned by mongot.
#### Mongot GetMore Stretegy #### Mongot GetMore Stretegy
We customize GetMore-related behaviors of the TaskExecutorCursor (enabling prefetching and tuning the batchSize option) with mongot-specific logic via the `MongotTaskExecutorCursorGetMoreStrategy`. We customize GetMore-related behaviors of the TaskExecutorCursor (enabling prefetching and tuning the batchSize option) with mongot-specific logic via the `MongotTaskExecutorCursorGetMoreStrategy`.
For example, if we know that we will need all documents from mongot in order to satisfy the query (for example, if the post-$search pipeline has a blocking stage like $sort or $group), then we'll immediately pre-fetch all GetMore requests and will follow a exponential batchSize growth strategy per batch. On the other hand, if the query has an extractable limit N, we attempt to retrieve all N documents in the first batch by tuning the initial batchSize; in that case, we'll never pre-fetch, and if a GetMore is actually needed, we'll tune the batchSize to try to request all still-needed documents in the next batch. See the [`MongotTaskExecutorCursorGetMoreStrategy`](https://github.com/10gen/mongo/blob/07c5da765d36bfc2bb6bc6d9b101a90cf09e82f7/src/mongo/db/query/search/mongot_cursor_getmore_strategy.h/#L47) for all heuristics and implementation details. For example, if we know that we will need all documents from mongot in order to satisfy the query (for example, if the post-$search pipeline has a blocking stage like $sort or $group), then we'll immediately pre-fetch all GetMore requests and will follow a exponential batchSize growth strategy per batch. On the other hand, if the query has an extractable limit N, we attempt to retrieve all N documents in the first batch by tuning the initial batchSize; in that case, we'll never pre-fetch, and if a GetMore is actually needed, we'll tune the batchSize to try to request all still-needed documents in the next batch. See the [`MongotTaskExecutorCursorGetMoreStrategy`](https://github.com/mongodb/mongo/blob/07c5da765d36bfc2bb6bc6d9b101a90cf09e82f7/src/mongo/db/query/search/mongot_cursor_getmore_strategy.h/#L47) for all heuristics and implementation details.
### $\_internalSearchIdLookup ### $\_internalSearchIdLookup
@ -91,4 +91,4 @@ Older versions of mongot do not support returning an explain object alongside th
### Didn't Find What You're Looking For? ### Didn't Find What You're Looking For?
Visit [the landing page](https://github.com/10gen/mongo/blob/master/src/mongo/db/query/search/README.md) for all $search/$vectorSearch/$searchMeta related documentation for server contributors. Visit [the landing page](https://github.com/mongodb/mongo/blob/master/src/mongo/db/query/search/README.md) for all $search/$vectorSearch/$searchMeta related documentation for server contributors.

4
src/mongo/db/s/README_transactions_and_ddl.md
View File

@ -88,7 +88,7 @@ For **untracked** collections, the protocol is as follows:
1. The targeted shard checks the attached metadata. All the following conditions must be 1. The targeted shard checks the attached metadata. All the following conditions must be
met for the request to be considered valid: met for the request to be considered valid:
1. The received database version must match the current (latest) database version. 1. The received database version must match the current (latest) database version.
1. The received `atClusterTime` must not be earlier than the latest database version's [timestamp field](https://github.com/10gen/mongo/blob/eeef1763cb0ff77757bb60eabb8ad1233c990786/src/mongo/db/s/README_versioning_protocols.md#database-version) known by the shard. 1. The received `atClusterTime` must not be earlier than the latest database version's [timestamp field](https://github.com/mongodb/mongo/blob/eeef1763cb0ff77757bb60eabb8ad1233c990786/src/mongo/db/s/README_versioning_protocols.md#database-version) known by the shard.
This field represents the commit timestamp of the latest reincarnation (drop/create) or movePrimary operation for this database. This field represents the commit timestamp of the latest reincarnation (drop/create) or movePrimary operation for this database.
1. The received placement version is UNSHARDED, and the shard checks the latest version matches. 1. The received placement version is UNSHARDED, and the shard checks the latest version matches.
1. The collection in the snapshot must be the same incarnation (same UUID) as in the latest CollectionCatalog. 1. The collection in the snapshot must be the same incarnation (same UUID) as in the latest CollectionCatalog.
@ -167,7 +167,7 @@ For **untracked** collections, the protocol is as follows:
1. The targeted shard checks the attached metadata. All the following conditions must be 1. The targeted shard checks the attached metadata. All the following conditions must be
met for the request to be considered valid: met for the request to be considered valid:
1. The received database version must match the current (latest) database version. 1. The received database version must match the current (latest) database version.
1. `placementConflictTime` must not be earlier than the database version's [timestamp field](https://github.com/10gen/mongo/blob/eeef1763cb0ff77757bb60eabb8ad1233c990786/src/mongo/db/s/README_versioning_protocols.md#database-version). 1. `placementConflictTime` must not be earlier than the database version's [timestamp field](https://github.com/mongodb/mongo/blob/eeef1763cb0ff77757bb60eabb8ad1233c990786/src/mongo/db/s/README_versioning_protocols.md#database-version).
This field represents the commit timestamp of the latest reincarnation (drop/create) or movePrimary operation for this database. This field represents the commit timestamp of the latest reincarnation (drop/create) or movePrimary operation for this database.
1. The received placement version is UNSHARDED, and the shard checks the latest version matches. 1. The received placement version is UNSHARDED, and the shard checks the latest version matches.
1. The collection in the snapshot must be the same incarnation (same UUID) as in the latest CollectionCatalog. 1. The collection in the snapshot must be the same incarnation (same UUID) as in the latest CollectionCatalog.

10
src/mongo/db/storage/README.md
View File

@ -495,7 +495,7 @@ Change stream collection that uses CollectionTruncateMarkers
dedicated environments. dedicated environments.
The change stream pre-images collections has a periodic remover thread The change stream pre-images collections has a periodic remover thread
([ChangeStreamExpiredPreImagesRemover](https://github.com/10gen/mongo/blob/r7.1.0-rc3/src/mongo/db/pipeline/change_stream_expired_pre_image_remover.cpp#L71). ([ChangeStreamExpiredPreImagesRemover](https://github.com/mongodb/mongo/blob/r7.1.0-rc3/src/mongo/db/pipeline/change_stream_expired_pre_image_remover.cpp#L71).
The remover thread: The remover thread:
1. Creates the tenant's initial CollectionTruncateMarkers for the tenant if they do not yet exist 1. Creates the tenant's initial CollectionTruncateMarkers for the tenant if they do not yet exist
@ -518,7 +518,7 @@ Each tenant has 1 pre-images collection. Each pre-images collection contains pre
the tenant's pre-image enabled collections. the tenant's pre-image enabled collections.
A pre-images collection is clustered by A pre-images collection is clustered by
[ChangeStreamPreImageId](https://github.com/10gen/mongo/blob/r7.1.0-rc3/src/mongo/db/pipeline/change_stream_preimage.idl#L69), [ChangeStreamPreImageId](https://github.com/mongodb/mongo/blob/r7.1.0-rc3/src/mongo/db/pipeline/change_stream_preimage.idl#L69),
which implicitly orders pre-images first by their `'nsUUID'` (the UUID of the collection the which implicitly orders pre-images first by their `'nsUUID'` (the UUID of the collection the
pre-image is from), their `'ts'` (the timestamp associated with the pre-images oplog entry), and pre-image is from), their `'ts'` (the timestamp associated with the pre-images oplog entry), and
then by their `'applyOpsIndex'` (the index into the applyOps oplog entry which generated the then by their `'applyOpsIndex'` (the index into the applyOps oplog entry which generated the
@ -545,13 +545,13 @@ record to the the minimum RecordId for the nsUUID when there is an expired trunc
class](https://github.com/mongodb/mongo/blob/r7.1.0-rc3/src/mongo/db/storage/collection_truncate_markers.h#L78) class](https://github.com/mongodb/mongo/blob/r7.1.0-rc3/src/mongo/db/storage/collection_truncate_markers.h#L78)
- The main api for CollectionTruncateMarkers. - The main api for CollectionTruncateMarkers.
- [The OplogTruncateMarkers - [The OplogTruncateMarkers
class](https://github.com/10gen/mongo/blob/r7.1.0-rc3/src/mongo/db/storage/wiredtiger/wiredtiger_record_store_oplog_truncate_markers.h) class](https://github.com/mongodb/mongo/blob/r7.1.0-rc3/src/mongo/db/storage/wiredtiger/wiredtiger_record_store_oplog_truncate_markers.h)
- Oplog specific truncate markers. - Oplog specific truncate markers.
- [The PreImagesTruncateMarkersPerNsUUID - [The PreImagesTruncateMarkersPerNsUUID
class](https://github.com/10gen/mongo/blob/r7.1.0-rc3/src/mongo/db/change_stream_pre_images_truncate_markers_per_nsUUID.h#L62) class](https://github.com/mongodb/mongo/blob/r7.1.0-rc3/src/mongo/db/change_stream_pre_images_truncate_markers_per_nsUUID.h#L62)
- Truncate markers for a given nsUUID captured within a pre-images collection. - Truncate markers for a given nsUUID captured within a pre-images collection.
- [The PreImagesTruncateManager - [The PreImagesTruncateManager
class](https://github.com/10gen/mongo/blob/r7.1.0-rc3/src/mongo/db/change_stream_pre_images_truncate_manager.h#L70) class](https://github.com/mongodb/mongo/blob/r7.1.0-rc3/src/mongo/db/change_stream_pre_images_truncate_manager.h#L70)
- Manages pre image truncate markers for each tenant. - Manages pre image truncate markers for each tenant.
# Oplog Collection # Oplog Collection

14
src/mongo/db/timeseries/README.md
View File

@ -215,7 +215,7 @@ parameter.
## Batched Inserts ## Batched Inserts
When a batch of inserts is processed by the write path, this typically comes from an `insertMany` command, When a batch of inserts is processed by the write path, this typically comes from an `insertMany` command,
the batch is [organized by meta value and sorted](https://github.com/10gen/mongo/blob/5e5a0d995995fc4404c6e32546ed0580954b1e39/src/mongo/db/timeseries/bucket_catalog/bucket_catalog.h#L506-L527) in time ascending order to ensure efficient bucketing the batch is [organized by meta value and sorted](https://github.com/mongodb/mongo/blob/5e5a0d995995fc4404c6e32546ed0580954b1e39/src/mongo/db/timeseries/bucket_catalog/bucket_catalog.h#L506-L527) in time ascending order to ensure efficient bucketing
and insertion. Prior versions would perform bucket targeting for each measurement in the order the user and insertion. Prior versions would perform bucket targeting for each measurement in the order the user
specified. This could lead to poor insert performance and very sub-optimal bucketing behavior if the user's specified. This could lead to poor insert performance and very sub-optimal bucketing behavior if the user's
batch was in time descending order. This change in batching also significantly reduces the number of stripe batch was in time descending order. This change in batching also significantly reduces the number of stripe
@ -375,7 +375,7 @@ some reasonable presets of "seconds", "minutes" and "hours".
| _Minutes_ | 3,600 (1 hour) | 86,400 (1 day) | | _Minutes_ | 3,600 (1 hour) | 86,400 (1 day) |
| _Hours_ | 86,400 (1 day) | 2,592,000 (30 days) | | _Hours_ | 86,400 (1 day) | 2,592,000 (30 days) |
Chart sources: [bucketRoundingSeconds](https://github.com/10gen/mongo/blob/279417f986c9792b6477b060dc65b926f3608529/src/mongo/db/timeseries/timeseries_options.cpp#L368-L381) and [bucketMaxSpanSeconds](https://github.com/10gen/mongo/blob/279417f986c9792b6477b060dc65b926f3608529/src/mongo/db/timeseries/timeseries_options.cpp#L259-L273). Chart sources: [bucketRoundingSeconds](https://github.com/mongodb/mongo/blob/279417f986c9792b6477b060dc65b926f3608529/src/mongo/db/timeseries/timeseries_options.cpp#L368-L381) and [bucketMaxSpanSeconds](https://github.com/mongodb/mongo/blob/279417f986c9792b6477b060dc65b926f3608529/src/mongo/db/timeseries/timeseries_options.cpp#L259-L273).
If the user does not specify any bucketing parameters when creating a collection, the default value If the user does not specify any bucketing parameters when creating a collection, the default value
is `{granularity: "seconds"}`. is `{granularity: "seconds"}`.
@ -472,7 +472,7 @@ We have three phases of a time-series write. Each phase of the time-series write
### Calculating Memory Usage ### Calculating Memory Usage
Memory usage for Buckets, the BucketCatalog, and other aspects of time-series collection internals (Stripes, BucketMetadata, etc) Memory usage for Buckets, the BucketCatalog, and other aspects of time-series collection internals (Stripes, BucketMetadata, etc)
is calculated using the [Timeseries Tracking Allocator](https://github.com/10gen/mongo/blob/f726b6db3a361122a87555dbea053d98b01685a3/src/mongo/db/timeseries/timeseries_tracking_allocator.h). is calculated using the [Timeseries Tracking Allocator](https://github.com/mongodb/mongo/blob/f726b6db3a361122a87555dbea053d98b01685a3/src/mongo/db/timeseries/timeseries_tracking_allocator.h).
### Freezing Buckets ### Freezing Buckets
@ -491,7 +491,7 @@ each phase described above in [Errors When Staging and Committing Measurements](
<img width="1364" alt="bucket_catalog_stripe" src="https://github.com/user-attachments/assets/c6016335-34e3-4f62-b46a-0f0a970f2c05" /> <img width="1364" alt="bucket_catalog_stripe" src="https://github.com/user-attachments/assets/c6016335-34e3-4f62-b46a-0f0a970f2c05" />
Recall that each bucket contains up to [timeseriesBucketMaxCount](https://github.com/10gen/mongo/blob/0f3a0dfd67b05e8095c70a03c7d7406f9e623277/src/mongo/db/timeseries/timeseries.idl#L58) (1,000) measurements within a span of time, `bucketMaxSpanSeconds`. And further, the bucket Recall that each bucket contains up to [timeseriesBucketMaxCount](https://github.com/mongodb/mongo/blob/0f3a0dfd67b05e8095c70a03c7d7406f9e623277/src/mongo/db/timeseries/timeseries.idl#L58) (1,000) measurements within a span of time, `bucketMaxSpanSeconds`. And further, the bucket
catalog maintains an invariant of at most 1 open bucket per unique meta value. This diagram shows catalog maintains an invariant of at most 1 open bucket per unique meta value. This diagram shows
how measurements map to each open bucket, which map to a specific stripe. The hashing function may not distribute the how measurements map to each open bucket, which map to a specific stripe. The hashing function may not distribute the
client's writes to stripes evenly, but on the whole there is a good chance that stripe contention won't be a bottleneck since client's writes to stripes evenly, but on the whole there is a good chance that stripe contention won't be a bottleneck since
@ -499,8 +499,8 @@ there are often more or the same number of stripes to cores on the database serv
if the workload accesses many buckets (meta values) that map to the same stripe. In 8.2+, while holding the stripe lock during if the workload accesses many buckets (meta values) that map to the same stripe. In 8.2+, while holding the stripe lock during
a write, only a small amount of work is done intentionally. This has lessened the impact of stripe contention. a write, only a small amount of work is done intentionally. This has lessened the impact of stripe contention.
The [Stripe](https://github.com/10gen/mongo/blob/0f3a0dfd67b05e8095c70a03c7d7406f9e623277/src/mongo/db/timeseries/bucket_catalog/bucket_catalog.h#L152-L186) struct stores most of the core components of the [Bucket Catalog](https://github.com/10gen/mongo/blob/0f3a0dfd67b05e8095c70a03c7d7406f9e623277/src/mongo/db/timeseries/bucket_catalog/bucket_catalog.h#L198-L228). The structures within it generally compose The [Stripe](https://github.com/mongodb/mongo/blob/0f3a0dfd67b05e8095c70a03c7d7406f9e623277/src/mongo/db/timeseries/bucket_catalog/bucket_catalog.h#L152-L186) struct stores most of the core components of the [Bucket Catalog](https://github.com/mongodb/mongo/blob/0f3a0dfd67b05e8095c70a03c7d7406f9e623277/src/mongo/db/timeseries/bucket_catalog/bucket_catalog.h#L198-L228). The structures within it generally compose
the memory usage of the Bucket Catalog (see: [bucket_catalog::getMemoryUsage](https://github.com/10gen/mongo/blob/0f3a0dfd67b05e8095c70a03c7d7406f9e623277/src/mongo/db/timeseries/bucket_catalog/bucket_catalog.cpp#L203-L219)). the memory usage of the Bucket Catalog (see: [bucket_catalog::getMemoryUsage](https://github.com/mongodb/mongo/blob/0f3a0dfd67b05e8095c70a03c7d7406f9e623277/src/mongo/db/timeseries/bucket_catalog/bucket_catalog.cpp#L203-L219)).
# References # References
@ -514,7 +514,7 @@ See:
- **active bucket**: A bucket that is either archived or open. It forms the cardinality managed by the - **active bucket**: A bucket that is either archived or open. It forms the cardinality managed by the
BucketCatalog. BucketCatalog.
- **archived bucket**: A bucket that resides on-disk with a crumb of info [still in memory](https://github.com/10gen/mongo/blob/883a40fdd73056c88221aa668a627cd2e6c621a6/src/mongo/db/timeseries/bucket_catalog/bucket_catalog.h#L166-L175) - **archived bucket**: A bucket that resides on-disk with a crumb of info [still in memory](https://github.com/mongodb/mongo/blob/883a40fdd73056c88221aa668a627cd2e6c621a6/src/mongo/db/timeseries/bucket_catalog/bucket_catalog.h#L166-L175)
to support efficient reopening without querying. Adding new measurements to this to support efficient reopening without querying. Adding new measurements to this
bucket will require materializing data from disk. bucket will require materializing data from disk.