Compare commits

...

207 Commits

Author SHA1 Message Date
Binbin 8ab0152bef
Check for duplicate nodeids when loading nodes.conf (#2852)
For corrupted (human-made) or program-error-recovered nodes.conf files,
check for duplicate nodeids when loading nodes.conf. If a duplicate is
found, panic is triggered to prevent nodes from starting up
unexpectedly.

The node ID is used to identify every node across the whole cluster,
we do not expect to find duplicate nodeids in nodes.conf.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-12-17 11:23:38 +08:00
Ping Xie 6b60e6bfc7
Update MAINTAINERS list and add committee chair section (#2939) 2025-12-16 14:06:26 -08:00
Ping Xie 50ff460007
Make 1/3 TSC organization limit explicitly inclusive (#2930) 2025-12-16 07:15:38 -08:00
Binbin 51f871ae52
Strictly check CRLF when parsing querybuf (#2872)
Currently, when parsing querybuf, we are not checking for CRLF,
instead we assume the last two characters are CRLF by default,
as shown in the following example:
```
telnet 127.0.0.1 6379
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
*3
$3
set
$3
key
$5
value12
+OK
get key
$5
value

*3
$3
set
$3
key
$5
value12345
+OK
-ERR unknown command '345', with args beginning with:
```

This should actually be considered a protocol error. When a bug
occurs in the client-side implementation, we may execute incorrect
requests (writing incorrect data is the most serious of these).

---------

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-12-14 09:10:57 +02:00
Ping Xie cd6faaa726
Refine major decision process and update TSC composition rules (#2927)
- Require a 2/3 supermajority vote for all Governance Major Decisions.
- Update Technical Major Decision voting to prioritize simple majority, limiting the use of "+2" approval.
- Define remediation steps for when the 1/3 organization limit is exceeded.

---------

Signed-off-by: Ping Xie <pingxie@outlook.com>
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-12-12 08:53:50 -08:00
Vitah Lin 79fff5dbac
Upgrade macos version in actions (#2920)
GitHub has deprecated older macOS runners, and macos-13 is no longer supported.

1. The latest version of cross-platform-actions/action does allow
running on ubuntu-latest (Linux runner) and does not strictly require macOS.
2. Previously, cross-platform-actions/action@v0.22.0 used runs-on:
macos-13. I checked the latest version of cross-platform-actions, and
the official examples now use runs-on: ubuntu. I think we can switch from macOS to Ubuntu.

---------

Signed-off-by: Vitah Lin <vitahlin@gmail.com>
2025-12-11 23:50:26 +01:00
Viktor Söderqvist 5940dbfb0b
Revert "Allow partial sync after loading AOF with preamble (#2366)" (#2925)
This reverts commit 2da21d9def.

The implementation in #2366 made it possible to perform a partial
resynchronization after loading an AOF file, by storing the replication
offset in the AOF preamble and counting the bytes in the AOF command
stream in the same way as we count byte offset in a replication stream.

However, this approach isn't safe because some commands are replicated
but not stored in the AOF file. This includes the commands REPLCONF,
PING, PUBLISH and module-implemented commands where a module can control
to propagate a command to the replication stream only, to AOF only or to
both. This oversight led to data inconsistency, where the wrong
replication offset is used for partial resynchronization as explained in
issue #2904.

The revert caused small merge conflicts with 3c3a1966ec which are
solved.

Fixes #2904.

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-12-11 08:39:53 +02:00
Vadym Khoptynets 44dc58181d
Fix a typo in aof-multi-part.tcl (#2922)
A small PR to fix a smal typo: `util` --> `until`

Signed-off-by: Vadym Khoptynets <1099644+poiuj@users.noreply.github.com>
2025-12-09 21:30:28 +02:00
bandalgomsu 2a9a1731ee
Fix CLUSTER SLOTS crash when called from module timer callback (#2915)
The CLUSTER SLOTS reply depends on whether the client is connected over
IPv6, but for a fake client there is no connection and when this command
is called from a module timer callback or other scenario where no real
client is involved, there is no connection to check IPv6 support on.
This fix handles the missing case by returning the reply for IPv4
connected clients.

Fixes #2912.

---------

Signed-off-by: Su Ko <rhtn1128@gmail.com>
Signed-off-by: Binbin <binloveplay1314@qq.com>
Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
Co-authored-by: Su Ko <rhtn1128@gmail.com>
Co-authored-by: KarthikSubbarao <karthikrs2021@gmail.com>
Co-authored-by: Binbin <binloveplay1314@qq.com>
Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-12-09 14:02:35 +01:00
Binbin 48e0cbbb41
Fix commandlog large-reply when using reply copy avoidance (#2652)
In #2078, we did not report large reply when copy avoidance is allowed.
This results in replies larger than 16384 not being recorded in the
commandlog large-reply. This 16384 is controlled by the hidden config
min-string-size-avoid-copy-reply.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-12-09 10:20:45 +08:00
Binbin 29d3244937
Make the COB soft limit also use repl-backlog-size when its value is smaller (#2866)
We have the same settings for the hard limit, and we should apply them to the soft
limit as well. When the `repl-backlog-size` value is larger, all replication buffers
can be handled by the replication backlog, so there's no need to worry about the
client output buffer soft limit in here. Furthermore, when `soft_seconds` is 0, in
some ways, the soft limit behaves the same (mostly) as the hard limit.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-12-08 12:21:34 +08:00
zhaozhao.zz 04d0bba398
support whole cluster info for INFO command in cluster section (#2876)
Allow users to more easily get cluster information.

Signed-off-by: zhaozhao.zz <zhaozhao.zz@alibaba-inc.com>
2025-12-04 09:34:52 -08:00
Ouri Half 3d65a4aecd
Fix deadlock in IO thread shutdown during panic (#2898)
## Problem
IO thread shutdown can deadlock during server panic when the main thread
calls `pthread_cancel()` while the IO thread holds its mutex, preventing
the thread from observing the cancellation.

## Solution  
Release the IO thread mutex before cancelling to ensure clean thread
termination.

## Testing
Reproducer:
```
bash
./src/valkey-server --io-threads 2 --enable-debug-command yes
./src/valkey-cli debug panic
```

Before: Server hangs indefinitely
After: Server terminates cleanly

Signed-off-by: Ouri Half <ourih@amazon.com>
2025-12-04 09:10:09 -08:00
Roshan Khatri c90e634f11
Add PR and Release benchmark with new changes in framework (#2871)
This adds the workflow improvements for PR and Release benchmark where
it runs on `c8g.metal-48xl` for `ARM64` and `c7i.metal-48xl` for `X86`.

```
Cluster mode: disabled
TLS: disabled
io-threads: 1, 9
Pipelining: 1, 10
Clients: 1600
Benchmark Treads: 90
Data size: 16 ,96
Commands: SET, GET
```

c8g.metal-48xl Spec: https://aws.amazon.com/ec2/instance-types/c8g/
c7i.metal.48xl Spec: https://aws.amazon.com/ec2/instance-types/c7i/

```
vCPU: 192
NUMA nodes: 2
Memory (GiB): 384
Network Bandwidth (Gbps): 50
```

PR benchmarking will be executed on **ARM64** machine as it has been
seen to be more consistent.
Additionally, it runs 5 iterations for each tests and posts the average
and other statistical metrics like
- CI99%: 99% Confidence Interval - range where the true population mean
is likely to fall
- PI99%: 99% Prediction Interval - range where a single future
observation is likely to fall
- CV: Coefficient of Variation - relative variability (σ/μ × 100%)

_Note: Values with (n=X, σ=Y, CV=Z%, CI99%=±W%, PI99%=±V%) indicate
averages from X runs with standard deviation Y, coefficient of variation
Z%, 99% confidence interval margin of error ±W% of the mean, and 99%
prediction interval margin of error ±V% of the mean. CI bounds [A, B]
and PI bounds [C, D] show the actual interval ranges._

For comparing between versions, it adds a workflow which runs on both
**ARM64** and **X86** machine. It will also post the comparison between
the versions like this:
https://github.com/valkey-io/valkey/issues/2580#issuecomment-3399539615

---------

Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
Signed-off-by: Roshan Khatri <117414976+roshkhatri@users.noreply.github.com>
2025-12-04 17:33:21 +01:00
Rain Valentine 70196ee20f
Add safe iterator tracking in hashtable to prevents invalid access on hashtable delete (#2807)
This makes it safe to delete hashtable while a safe iterator is
iterating it. This currently isn't possible, but this improvement is
required for fork-less replication #1754 which is being actively
worked on.

We discussed these issues in #2611 which guards against a different but
related issue: calling hashtableNext again after it has already returned
false.

I implemented a singly linked list that hashtable uses to track its
current safe iterators. It is used to invalidate all associated safe
iterators when the hashtable is released. A singly linked list is
acceptable because the list length is always very small - typically zero
and no more than a handful.

Also, renames the internal functions:

    hashtableReinitIterator -> hashtableRetargetIterator
    hashtableResetIterator -> hashtableCleanupIterator

---------

Signed-off-by: Rain Valentine <rsg000@gmail.com>
Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-12-03 18:15:45 +01:00
Harkrishn Patro 8ec9381974
Send duplicate multi meet packet only for node which supports it (#2840)
This prevents crashes on the older nodes in mixed clusters where some
nodes are running 8.0 or older. Mixed clusters often exist temporarily
during rolling upgrades.

Fixes: #2341 

Signed-off-by: Harkrishn Patro <harkrisp@amazon.com>
2025-12-03 17:17:26 +01:00
Murad Shahmammadli c8341d4165
Replace strcmp with byte-by-byte comparison in valkey-check-aof (#2809)
Fixes #2792

Replace strcmp with byte-by-byte comparison to avoid accidental
heap-buffer-overflow errors.

Signed-off-by: murad shahmammadli <shmurad@amazon.com>
Co-authored-by: murad shahmammadli <shmurad@amazon.com>
2025-12-03 17:07:04 +01:00
Roshan Khatri 2e8fba49a2
[Test Fix] flaky benchmark test for warmup (#2890)
Fixes: https://github.com/valkey-io/valkey/issues/2859
Increased the warmup to 2 sec so we can verify that it runs more number
of commands than the actual benchmark.

Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
2025-12-02 21:40:20 +01:00
Jim Brunner 1b5f245eae
Refactor of LFU/LRU code for modularity (#2857)
General cleanup on LRU/LFU code. Improve modularity and maintainability.

Specifically:
* Consolidates the mathematical logic for LRU/LFU into `lrulfu.c`, with
an API in `lrulfu.h`. Knowledge of the LRU/LFU implementation was
previously spread out across `db.c`, `evict.c`, `object.c`, `server.c`,
and `server.h`.
* Separates knowledge of the LRU from knowledge of the object containing
the LRU value. `lrulfu.c` knows about the LRU/LFU algorithms, without
knowing about the `robj`. `object.c` knows about the `robj` without
knowing about the details of the LRU/LFU algorithms.
* Eliminated `server.lruclock`, instead using `server.unixtime`. This
also eliminates the periodic need to call `mstime()` to maintain the lru
clock.
* Fixed a minor computation bug in the old `LFUTimeElapsed` function
(off by 1 after rollover).
* Eliminate specific IF checks for rollover, using defined behavior for
unsigned rollover instead.
* Fixed a bug in `debug.c` which would perform LFU modification on an
LRU value.

---------

Signed-off-by: Jim Brunner <brunnerj@amazon.com>
Co-authored-by: Ran Shidlansik <ranshid@amazon.com>
2025-12-02 10:14:33 -08:00
Zhijun Liao 853d111f47
Build: Support `make test` when PROG_SUFFIX is used (#2885)
Closes #2883

Support a new environment variable `VALKEY_PROG_SUFFIX` in the test
framework, which can be used for running tests if the binaries are
compiled with a program suffix. For example, if the binaries are
compiled using `make PROG_SUFFIX=-alt` to produce binaries named
valkey-server-alt, valkey-cli-alt, etc., run the tests against these
binaries using `VALKEY_PROG_SUFFIX=-alt ./runtest` or simply using `make
test`.

Now the test with the make variable `PROG_SUFFIX` works well.

```
% make PROG_SUFFIX="-alt"
		...
		...
		CC trace/trace_aof.o
    LINK valkey-server-alt
    INSTALL valkey-sentinel-alt
    CC valkey-cli.o
    CC serverassert.o
    CC cli_common.o
    CC cli_commands.o
    LINK valkey-cli-alt
    CC valkey-benchmark.o
    LINK valkey-benchmark-alt
    INSTALL valkey-check-rdb-alt
    INSTALL valkey-check-aof-alt

Hint: It's a good idea to run 'make test' ;)
% 
% make test                                                                
cd src && /Library/Developer/CommandLineTools/usr/bin/make test
    CC Makefile.dep
    CC release.o
    LINK valkey-server-alt
    INSTALL valkey-check-aof-alt
    INSTALL valkey-check-rdb-alt
    LINK valkey-cli-alt
    LINK valkey-benchmark-alt
Cleanup: may take some time... OK
Starting test server at port 21079
[ready]: 39435
Testing unit/pubsub
```

Signed-off-by: Zhijun <dszhijun@gmail.com>
2025-12-02 15:03:47 +01:00
Zhijun Liao 3fd0942279
Refactor TCL reference to support running tests with CMake (#2816)
Historically, Valkey’s TCL test suite expected all binaries
(src/valkey-server, src/valkey-cli, src/valkey-benchmark, etc.) to exist
under the src/ directory. This PR enables Valkey TCL tests to run
seamlessly after a CMake build — no manual symlinks or make build
required.

The test framework accepts a new environment variable `VALKEY_BIN_DIR`
to look for the binaries.

CMake will copy all TCL test entrypoints (runtest, runtest-cluster,
etc.) into the CMake build dir (e.g. `cmake-build-debug`) and insert
`VALKEY_BIN_DIR` into these. Now we can either do
./cmake-build-debug/runtest at the project root or ./runtest at the
Cmake dir to run all tests.

A new CMake post-build target prints a friendly reminder after
successful builds, guiding developers on how to run tests with their
CMake binaries:

```
Hint: It is a good idea to run tests with your CMake-built binaries ;)
      ./cmake-build-debug/runtest

Build finished
```

A helper TCL script `tests/support/set_executable_path.tcl` is added to
support this change, which gets called by all test entrypoints:
`runtest`, `runtest-cluster`, `runtest-sentinel`.

---------

Signed-off-by: Zhijun <dszhijun@gmail.com>
2025-12-02 14:14:28 +01:00
Daniil Kashapov 825d19fb09
Make all ACL categories explicit in JSON files (#2887)
Resolves #417

---------

Signed-off-by: Daniil Kashapov <daniil.kashapov.ykt@gmail.com>
2025-12-02 13:33:20 +01:00
Binbin 4a0e20bbc9
Handle failed psync log when there is no replication backlog (#2886)
This crash was introduced in #2877, we will crash when there is no
replication backlog.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-12-01 10:20:32 +08:00
Binbin a087cc1132
Add psync offset range information to the failed psync log (#2877)
Although we can infer this infomartion from the replica logs,
i think this still would be useful to see this information directly
in the primary logs.

So now we can see the psync offset range when psync fails and then
we can analyze and adjust the value of repl-backlog-size.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-11-29 12:20:34 +08:00
Viktor Söderqvist 761fba2e9d
Fix persisting missing make variables (#2881)
Persist USE_FAST_FLOAT and PROG_SUFFIX to prevent a complete rebuild
next time someone types make or make test without specifying variables.

Fixes #2880

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-11-28 10:35:53 +01:00
Binbin d16788e52d
Fix discarded-qualifiers warnings reported by fedorarawhide (#2874)
fedorarawhide CI reports these warnings:
```
networking.c: In function 'afterErrorReply':
networking.c:821:30: error: initialization discards 'const' qualifier from pointer target type [-Werror=discarded-qualifiers]
  821 |             char *spaceloc = memchr(s, ' ', len < 32 ? len : 32);
```

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-11-27 10:39:42 +08:00
Zhijun Liao faac14ab9c
Cluster: Optimize slot bitmap iteration from per-bit to 64-bit word scan (#2781)
In functions `clusterSendFailoverAuthIfNeeded` and
`clusterProcessPacket`, we iterate through **every slot bit**
sequentially in the form of `for (int j = 0; j < CLUSTER_SLOTS; j++)`,
performing 16384 checks even when only a few bits were set, and thus
causing unnecessary loop overhead.

This is particularly wasteful in function
`clusterSendFailoverAuthIfNeeded` where we need to ensure the sender's
claimed slots all have up-to-date config epoch. Usually healthy senders
would meet such condition, and thus we normally need to exhaust the for
loop of 16384 checks.

The proposed new implementation loads 64 bits (8 byte word) at a time
and skips empty words completely, therefore only performing 256 checks.

---------

Signed-off-by: Zhijun <dszhijun@gmail.com>
2025-11-26 15:52:17 +02:00
Roshan Khatri 56ab3c4a81
Adds HGETDEL Support to Valkey (#2851)
Fixes this: https://github.com/valkey-io/valkey/issues/2850
Adds support for HGETDEL to Valkey and aligns with Redis 8.0 feature.
Maintains syntax compatibility
Retrieves all the values, and null if fields dont exists and deletes
once retrieved.
```
127.0.0.1:6379> HGETDEL key FIELDS numfields field [ field ... ]
```
```
127.0.0.1:6379> HSET foo field1 bar1 field2 bar2 field3 bar3
(integer) 3
127.0.0.1:6379> HGETDEL foo FIELDS 1 field2
1) "bar2"
127.0.0.1:6379> HGETDEL foo FIELDS 1 field2
1) (nil)
127.0.0.1:6379> HGETALL foo
1) "field1"
2) "bar1"
3) "field3"
4) "bar3"
127.0.0.1:6379>  HGETDEL foo FIELDS 2 field2 field3
1) (nil)
2) "bar3"
127.0.0.1:6379> HGETALL foo
1) "field1"
2) "bar1"
127.0.0.1:6379> HGETDEL foo FIELDS 3 field1 non-exist-field
(error) ERR syntax error
127.0.0.1:6379> HGETDEL foo FIELDS 2 field1 non-exist-field
1) "bar1"
2) (nil)
127.0.0.1:6379> HGETALL foo
(empty array)
127.0.0.1:6379> 
```

---------

Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
Signed-off-by: Ran Shidlansik <ranshid@amazon.com>
Co-authored-by: Ran Shidlansik <ranshid@amazon.com>
2025-11-26 15:19:53 +02:00
Ran Shidlansik 9562bdc0ab
Align the complexity description for all multi field HFE commands docs (#2875)
When we added the Hash Field Expiration feature in Valkey 9.0, some of
the new command docs included complexity description of O(1) even tough
they except multiple arguments.
(see discussion in
https://github.com/valkey-io/valkey/pull/2851#discussion_r2535684985)
This PR does:
1. align all the commands to the same description
2. fix the complexity description of some commands (eg HSETEX and
HGETEX)

---------

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>
2025-11-26 10:31:28 +02:00
Zhijun Liao da3c43d76b
Additional log information for cluster accept handler and message processing (#2815)
Enhance debugging for cluster logs

[1] Add human node names in cluster tests so that we can easily
understand which nodes we are interacting with:

```
pong packet received from: 92a960ffd62f1bd04efeb260b30fe9ca6b9294ed (R4) from client: :0
node 92a960ffd62f1bd04efeb260b30fe9ca6b9294ed (R4) announces that it is a primary in shard c6d1152caee49a5e70cb4b77d1549386078be603
Reconfiguring node 92a960ffd62f1bd04efeb260b30fe9ca6b9294ed (R4) as primary for shard c6d1152caee49a5e70cb4b77d1549386078be603
Configuration change detected. Reconfiguring myself as a replica of node 92a960ffd62f1bd04efeb260b30fe9ca6b9294ed (R4) in shard c6d1152caee49a5e70cb4b77d1549386078be603
```



[2] Currently there are logs showing the address of incoming
connections:

```
Accepting cluster node connection from 127.0.0.1:59956
Accepting cluster node connection from 127.0.0.1:59957
Accepting cluster node connection from 127.0.0.1:59958
Accepting cluster node connection from 127.0.0.1:59959
```

but we have no idea which nodes these connections refer to. I added a
logging statement when the node is set to the inbound link connection.

```
Bound cluster node 92a960ffd62f1bd04efeb260b30fe9ca6b9294ed (R4) to connection of client 127.0.0.1:59956
```



[3] Add a debug log when processing a packet to show the packet type,
sender node name, and sender client port (this also has the benefit of
telling us whether this is an inbound or outbound link).

```
pong packet received from: 92a960ffd62f1bd04efeb260b30fe9ca6b9294ed (R4) from client: :0
ping packet received from: 92a960ffd62f1bd04efeb260b30fe9ca6b9294ed (R4) from client: 127.0.0.1:59973
fail packet received from: 92a960ffd62f1bd04efeb260b30fe9ca6b9294ed (R4) from client: 127.0.0.1:59973
auth-req packet received from: 92a960ffd62f1bd04efeb260b30fe9ca6b9294ed (R4) from client: 127.0.0.1:59973
```

---------

Signed-off-by: Zhijun <dszhijun@gmail.com>
2025-11-25 17:07:21 -08:00
Leon Anavi e5de417f1e
Fix build on 32-bit ARM by only using NEON on AArch64 (#2873)
Only enable `HAVE_ARM_NEON` on AArch64 because it supports vaddvq and
all needed compiler intrinsics.

Fixes the following error when building for machine `qemuarm` using the
Yocto Project and OpenEmbedded:

```
| bitops.c: In function 'popcountNEON':
| bitops.c:219:23: error: implicit declaration of function 'vaddvq_u16'; did you mean 'vaddq_u16'? [-Wimplicit-function-declaration]
|   219 |         uint32_t t1 = vaddvq_u16(sc);
|       |                       ^~~~~~~~~~
|       |                       vaddq_u16
| bitops.c:225:14: error: implicit declaration of function 'vaddvq_u8'; did you mean 'vaddq_u8'? [-Wimplicit-function-declaration]
|   225 |         t += vaddvq_u8(vcntq_u8(vld1q_u8(p)));
|       |              ^~~~~~~~~
|       |              vaddq_u8
```

More details are available in the following log:
https://errors.yoctoproject.org/Errors/Details/889836/

Signed-off-by: Leon Anavi <leon.anavi@konsulko.com>
2025-11-25 21:23:32 +01:00
jiegang0219 dd2827a14e
Add support for asynchronous release to replicaKeysWithExpire on writable replica (#2849)
## Problem
When executing `FLUSHALL ASYNC` on a **writable replica** that has
a large number of expired keys directly written to it, the main thread
gets blocked for an extended period while synchronously releasing the
`replicaKeysWithExpire` dictionary. 

## Root Cause
`FLUSHALL ASYNC` is designed for asynchronous lazy freeing of core data
structures, but the release of `replicaKeysWithExpire` (a dictionary tracking
expired keys on replicas) still happens synchronously in the main thread.
This synchronous operation becomes a bottleneck when dealing with massive
key volumes, as it cannot be offloaded to the lazyfree background thread.

This PR addresses the issue by moving the release of `replicaKeysWithExpire`
to the lazyfree background thread, aligning it with the asynchronous design
of `FLUSHALL ASYNC` and eliminating main thread blocking.

## User scenarios
In some operations, people often need to do primary-replica switches.
One goal is to avoid noticeable impact on the business—like key loss
or reduced availability (e.g., write failures).

Here is the process: First, temporarily switch traffic to writable replicas.
Then we wait for the primary pending replication data to be fully synced
(so primry and replicas are in sync), before finishing the switch. We don't
usually need to do the flush in this case, but it's an optimization that can
be done.

Signed-off-by: Scut-Corgis <512141203@qq.com>
Signed-off-by: jiegang0219 <512141203@qq.com>
Co-authored-by: Binbin <binloveplay1314@qq.com>
2025-11-25 19:25:39 +08:00
Binbin 8ea7f1330c
Update dual channel replication conf to mention the local buffer is imited by COB (#2824)
After introducing the dual channel replication in #60, we decided in #915
not to add a new configuration item to limit the replica's local replication
buffer, just use "client-output-buffer-limit replica hard" to limit it.

We need to document this behavior and mention that once the limit is reached,
all future data will accumulate in the primary side.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-11-23 23:27:50 +08:00
Binbin 8189fe5c42
Add rdb_transmitted to replstateToString so that we can see it in INFO (#2833)
In dual channel replication, when the rdb channel client finish
the RDB transfer, it will enter REPLICA_STATE_RDB_TRANSMITTED
state. During this time, there will be a brief window that we are
not able to see the connection in the INFO REPLICATION.

In the worst case, we might not see the connection for the
DEFAULT_WAIT_BEFORE_RDB_CLIENT_FREE seconds. I guess there is no
harm to list this state, showing connected_slaves but not showing
the connection is bad when troubleshooting.

Note that this also affects the `valkey-cli --rdb` and `--functions-rdb`
options. Before the client is in the `rdb_transmitted` state and is
released, we will now see it in the info (see the example later).

Before, not showing the replica info
```
role:master
connected_slaves:1
```

After, for dual channel replication:
```
role:master
connected_slaves:1
slave0:ip=xxx,port=xxx,state=rdb_transmitted,offset=0,lag=0,type=rdb-channel
```

After, for valkey-cli --rdb-only and --functions-rdb:
```
role:master
connected_slaves:1
slave0:ip=xxx,port=xxx,state=rdb_transmitted,offset=0,lag=0,type=replica
```

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-11-21 18:31:31 +08:00
Ricardo Dias 05540af405
Add script function flags in the module API (#2836)
This commit adds script function flags to the module API, which allows
function scripts to specify the function flags programmatically.

When the scripting engine compiles the script code can extract the flags
from the code and set the flags on the compiled function objects.

---------

Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
2025-11-20 10:23:00 +00:00
Hanxi Zhang ed8856bdfc
Fix cluster slot migration flaky test (#2756)
The original test code only checks:

The original test code only checks:

1. wait_for_cluster_size 4, which calls cluster_size_consistent for every node.
Inside that function, for each node, cluster_size_consistent queries cluster_known_nodes,
which is calculated as (unsigned long long)dictSize(server.cluster->nodes). However, when
a new node is added to the cluster, it is first created in the HANDSHAKE state, and
clusterAddNode adds it to the nodes hash table. Therefore, it is possible for the new
node to still be in HANDSHAKE status (processed asynchronously) even though it appears
that all nodes “know” there are 4 nodes in the cluster.

2. cluster_state for every node, but when a new node is added, server.cluster->state remains FAIL.


Some handshake processes may not have completed yet, which likely causes the flakiness.
To address this, added a --cluster check to ensure that the config state is consistent.

Fixes #2693.

Signed-off-by: Hanxi Zhang <hanxizh@amazon.com>
Co-authored-by: Binbin <binloveplay1314@qq.com>
2025-11-20 15:07:16 +08:00
aradz44 e19ceb7a6d
deflake "Hash field TTL and active expiry propagates correctly" (#2856)
Fix a little miss in "Hash field TTL and active expiry propagates
correctly through chain replication" test in `hashexpire.tcl`.
The test did not wait for the initial sync of the chained replica and thus  made the test flakey

Signed-off-by: Arad Zilberstein <aradz@amazon.com>
2025-11-19 11:33:55 +02:00
Venkat Pamulapati 3c3a1966ec
Perform data cleanup during RDB load on successful version/signature validation (#2600)
Addresses: https://github.com/valkey-io/valkey/issues/2588

## Overview
Previously we call `emptyData()` during a fullSync before validating the
RDB version is compatible.

This change adds an rdb flag that allows us to flush the database from
within `rdbLoadRioWithLoadingCtx`. THhis provides the option to only
flush the data if the rdb has a valid version and signature. In the case
where we do have an invalid version and signature, we don't emptyData,
so if a full sync fails for that reason a replica can still serve stale
data instead of clients experiencing cache misses.

## Changes
- Added a new flag `RDBFLAGS_EMPTY_DATA` that signals to flush the
database after rdb validation
- Added logic to call `emptyData` in `rdbLoadRioWithLoadingCtx` in
`rdb.c`
- Added logic to not clear data if the RDB validation fails in
`replication.c` using new return type `RDB_INCOMPATIBLE`
- Modified the signature of `rdbLoadRioWithLoadingCtx` to return RDB
success codes and updated all calling sites.

## Testing
Added a tcl test that uses the debug command `reload nosave` to load
from an RDB that has a future version number. This triggers the same
code path that full sync's will use, and verifies that we don't flush
the data until after the validation is complete.

A test already exists that checks that the data is flushed:
https://github.com/valkey-io/valkey/blob/unstable/tests/integration/replication.tcl#L1504

---------

Signed-off-by: Venkat Pamulapati <pamuvenk@amazon.com>
Signed-off-by: Venkat Pamulapati <33398322+ChiliPaneer@users.noreply.github.com>
Co-authored-by: Venkat Pamulapati <pamuvenk@amazon.com>
Co-authored-by: Harkrishn Patro <bunty.hari@gmail.com>
2025-11-18 17:08:10 -08:00
yzc-yzc 57892663be
Fix SCAN consistency test to only test what we guarantee (#2853)
Test the SCAN consistency by alternating SCAN
calls to primary and replica.
We cannot rely on the exact order of the elements and the returned
cursor number.

---------

Signed-off-by: yzc-yzc <96833212+yzc-yzc@users.noreply.github.com>
Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-11-18 16:06:20 +01:00
chzhoo 33bfac37ba
Optimize zset memory usage by embedding element in skiplist (#2508)
By default, when the number of elements in a zset exceeds 128, the
underlying data structure adopts a skiplist. We can reduce memory usage
by embedding elements into the skiplist nodes. Change the `zskiplistNode`
memory layout as follows:

```
Before
                 +-------------+
         +-----> | element-sds |
         |       +-------------+
         |
 +------------------+-------+------------------+---------+-----+---------+
 | element--pointer | score | backward-pointer | level-0 | ... | level-N |
 +------------------+-------+------------------+---------+-----+---------+



After
 +-------+------------------+---------+-----+---------+-------------+
 + score | backward-pointer | level-0 | ... | level-N | element-sds |
 +-------+------------------+---------+-----+---------+-------------+
```

Before the embedded SDS representation, we include one byte representing
the size of the SDS header, i.e. the offset into the SDS representation
where that actual string starts.

The memory saving is therefore one pointer minus one byte = 7 bytes per
element, regardless of other factors such as element size or number of
elements.

### Benchmark step

I generated the test data using the following lua script && cli command.
And check memory usage using the `info` command.

**lua script**
```
local start_idx = tonumber(ARGV[1])
local end_idx = tonumber(ARGV[2])
local elem_count = tonumber(ARGV[3])

for i = start_idx, end_idx do
    local key = "zset:" .. string.format("%012d", i)
    local members = {}

    for j = 0, elem_count - 1 do
        table.insert(members, j)
        table.insert(members, "member:" .. j)
    end

    redis.call("ZADD", key, unpack(members))
end

return "OK: Created " .. (end_idx - start_idx + 1) .. " zsets"
```

**valkey-cli command**
`valkey-cli EVAL "$(catcreate_zsets.lua)" 0 0 100000
${ZSET_ELEMENT_NUM}`

### Benchmark result
|number of elements in a zset | memory usage before optimization |
memory usage after optimization | change |
|-------|-------|-------|-------|
| 129 | 1047MB | 943MB | -9.9% |
| 256 |  2010MB|  1803MB| -10.3%|
| 512 |  3904MB|3483MB| -10.8%|

---------

Signed-off-by: chzhoo <czawyx@163.com>
Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-11-18 14:27:15 +01:00
Roshan Khatri 616fccb4c5
Fix the failing warmup and duration are cumulative (#2854)
We need to verify total duration was at least 2 seconds, elapsed time
can be quite variable to check upper-bound

Fixes https://github.com/valkey-io/valkey/issues/2843

Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
2025-11-17 21:26:12 +01:00
Binbin aef56e52f5
Fix timing issue in dual channel replication COB test (#2847)
After #2829, valgrind report a test failure, it seems that the time is
not enough to generate a COB limit in valgrind.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-11-17 17:25:19 +08:00
Binbin a06cf15b20
Allow dual channel full sync in plain failover (#2659)
PSYNC_FULLRESYNC_DUAL_CHANNEL is also a full sync, as the comment says,
we need to allow it. While we have not yet identified the exact edge case
that leads to this line, but during a failover, there should be no difference
between different sync strategies.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-11-15 12:57:27 +08:00
Harkrishn Patro 86db609219
Print node name on a best effort basis if light weight message is received before link stabilization (#2825)
fixes: #2803

---------

Signed-off-by: Harkrishn Patro <harkrisp@amazon.com>
Signed-off-by: Harkrishn Patro <bunty.hari@gmail.com>
Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
Co-authored-by: Binbin <binloveplay1314@qq.com>
2025-11-14 14:33:16 -08:00
yzc-yzc b93cfcc332
Attempt to fix flaky SCAN consistency test (#2834)
Related test failures:

https://github.com/valkey-io/valkey/actions/runs/19282092345/job/55135193394

https://github.com/valkey-io/valkey/actions/runs/19200556305/job/54887767594

> *** [err]: scan family consistency with configured hash seed in
tests/integration/scan-family-consistency.tcl
> Expected '5 {k:1 k:25 z k:11 k:18 k:27 k:45 k:7 k:12 k:19 k:29 k:40
k:41 k:43}' to be equal to '5 {k:1 k:25 k:11 z k:18 k:27 k:45 k:7 k:12
k:19 k:29 k:40 k:41 k:43}' (context: type eval line 26 cmd {assert_equal
$primary_cursor_next $replica_cursor_next} proc ::start_server)

The reason is that the RDB part of the primary-replica synchronization
affects the resize policy of the hashtable.
See
b835463a73/src/server.c (L807-L818)

Signed-off-by: yzc-yzc <96833212+yzc-yzc@users.noreply.github.com>
2025-11-14 10:55:05 +01:00
Binbin 331a852821
Change DEFAULT_WAIT_BEFORE_RDB_CLIENT_FREE from 60s to 5s (#2829)
Consider this scenario:
1. Replica starts loading the RDB using the rdb connection
2. Replica finishes loading the RDB before the replica main connection has
   initiated the PSYNC request
3. Replica stops replicating after receiving replicaof no one
4. Primary can't know that the replica main connection will never ask for
   PSYNC, so it keeps the reference to the replica's replication buffer block
5. Primary has a shutdown-timeout configured and requires to wait for the rdb
   connection to close before it can shut down.

The current 60-second wait time (DEFAULT_WAIT_BEFORE_RDB_CLIENT_FREE) is excessive
and leads to prolonged resource retention in edge cases. Reducing this timeout to
5 seconds would provide adequate time for legitimate PSYNC requests while mitigating
the issue described above.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-11-14 11:32:29 +08:00
Ricardo Dias 8e0b375da4
Fix cluster slot stats for scripts with cross-slot keys (#2835)
This commit fixes the cluster slot stats for scripts executed by
scripting engines when the scripts access cross-slot keys.

This was not a bug in Lua scripting engine, but `VM_Call` was missing a
call to invalidate the script caller client slot to prevent the
accumulation of stats.

Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
2025-11-13 12:05:52 -08:00
Rain Valentine 01a7657b83
Add --warmup and --duration parameters to valkey-benchmark (#2581)
It's handy to be able to automatically do a warmup and/or test by
duration rather than request count. 🙂

I changed the real-time output a bit - not sure if that's wanted or not.
(Like, would it break people's weird scripts? It'll break my weird
scripts, but I know the price of writing weird fragile scripts.)

```
Prepended "Warming up " when in warmup phase:
Warming up SET: rps=69211.2 (overall: 69747.5) avg_msec=0.425 (overall: 0.393) 3.8 seconds
^^^^^^^^^^

Appended running request counter when based on -n:
SET: rps=70892.0 (overall: 69878.1) avg_msec=0.385 (overall: 0.398) 612482 requests
                                                                    ^^^^^^^^^^^^^^^

Appended running second counter when in warmup or based on --duration:
SET: rps=61508.0 (overall: 61764.2) avg_msec=0.430 (overall: 0.426) 4.8 seconds
                                                                    ^^^^^^^^^^^
```

To be clear, the report at the end remains unchanged.

---------

Signed-off-by: Rain Valentine <rsg000@gmail.com>
Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-11-13 12:57:46 +01:00
Sarthak Aggarwal b835463a73
Fixes test-freebsd workflow in daily (package lang/tclX) (#2832)
This PR fixes the freebsd daily job that has been failing consistently
for the last days with the error "pkg: No packages available to install
matching 'lang/tclx' have been found in the repositories".

The package name is corrected from `lang/tclx` to `lang/tclX`. The
lowercase version worked previously but appears to have stopped working
in an update of freebsd's pkg tool to 2.4.x.

Example of failed job:

https://github.com/valkey-io/valkey/actions/runs/19282092345/job/55135193499

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
2025-11-13 08:24:37 +01:00
Binbin 7ffe4dcec4
Remove the EXAT and PXAT from some HFE notifications tests (#2831)
As we can see, we expected to get hexpired, but we got hexpire instead,
this means tht the expiration time has expired during execution.
```
*** [err]: HGETEX EXAT keyspace notifications for active expiry in tests/unit/hashexpire.tcl
Expected 'pmessage __keyevent@* __keyevent@9__:hexpired myhash' to match 'pmessage __keyevent@* __keyevent@*:hexpire myhash'
```

We should remove the EXAT and PXAT from these fixtures. And we indeed
have
the dedicated tests that verify that we get 'expired' when EX,PX are set
to 0
or EXAT,PXAT are in the past.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-11-12 14:32:13 +02:00
eifrah-aws 1b0b5c0cfd
New module API to perform prefix‑aware ACL permission check (#2796)
## Description

This change introduces the ability for modules to check ACL permissions
against key prefix. The update adds a dedicated `prefixmatchlen` helper
and extends the core ACL selector logic to support a prefix‑matching
mode.

The new API `ValkeyModule_ACLCheckPrefixPermissions` is registered and
exposed to modules, and a corresponding implementation is added in
`module.c`. Existing internal callers that already perform prefix checks
(e.g., `VM_ACLCheckKeyPermissions`) are updated to use the new flag,
while all legacy paths remain unchanged.

The change also modifies the `aclcheck§ test module that exercises the
new prefix‑checking API, ensuring that read/write operations are
correctly allowed or denied based on the ACL configuration.

Key areas touched:

* ACL logic
* Module API
* Testing

# Motivation

The search module presently makes costly calls to verify index
permissions
(see https://github.com/valkey-io/valkey-search/blob/main/src/acl.cc#L295).
This PR introduces a more efficient approach for that.

---------

Signed-off-by: Eran Ifrah <eifrah@amazon.com>
Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
Signed-off-by: Ran Shidlansik <ranshid@amazon.com>
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
Co-authored-by: Ran Shidlansik <ranshid@amazon.com>
Co-authored-by: Binbin <binloveplay1314@qq.com>
2025-11-12 10:51:58 +02:00
Daniil Kashapov 3c378862c3
Cluster: Avoid usage of light weight messages to nodes with not ready bidirectional links (#2817)
After network failure nodes that come back to cluster do not always send
and/or receive messages from other nodes in shard, this fix avoids usage
of light weight messages to nodes with not ready bidirectional links.
When a light message comes before any normal message, freeing of cluster
link is happening because on the just established connection link->node
is not assigned yet. It is assigned in getNodeFromLinkAndMsg right after
the condition if (is_light).
So on a cluster with heavy pubsub load a long loop of disconnects is
possible, and we got this.
1. node A establishes cluster link to node B
2. node A propagates PUBLISH to node B
3. node B frees cluster link because of link->node == null as it has not
received non-light messages yet
4. go to 1.
During this loop subscribers of node B does not receive any messages
published to node A.

So here we want to make sure that PING was sent (and link->node was
initialized) on this connection before using lightweight messages.

---------

Signed-off-by: Daniil Kashapov <daniil.kashapov.ykt@gmail.com>
Co-authored-by: Harkrishn Patro <bunty.hari@gmail.com>
2025-11-11 20:03:24 -08:00
Jim Brunner 047080a622
shared zadd for geoadd (#2828)
GEOADD is allocating/destroying a string object for "ZADD"
each time it is called. Created a shared string instead.

Signed-off-by: Jim Brunner <brunnerj@amazon.com>
2025-11-11 15:26:53 -08:00
Roshan Khatri b7a3fc988a
Fix Test dual-channel: primary tracking replica backlog refcount (#2827)
This increases the times we check for the logs from 20 to 40.

I found that every `wait-for` check takes about 1.5 to 1.57 milliseconds
so when we were checking 2000 times after 1ms we were actually spending
(2000 * 1) + (2000 *1.75) = 5500ms time waiting.
this can be founds under: for 10 checks we took 35 ms more so thats
around 1.75 ms per check
```
Execution time: 2034 ms (failed)
[err]: 20 100 - Test dual-channel: primary tracking replica backlog refcount - start with empty backlog in tests/integration/dual-channel-replication-flaky.tcl
```

That is why increasing it to 40 100 will check for approx 4,070 ms which
is still less than the original 5500ms but should passes every single
time here:
https://github.com/roshkhatri/valkey/actions/runs/19279424967/job/55126976882

Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
2025-11-12 00:03:50 +01:00
Arthur Lee 2da21d9def
Allow partial sync after loading AOF with preamble (#2366)
The AOF preamble mechanism replaces the traditional AOF base file with
an RDB snapshot during rewrite operations, which reduces I/O overhead
and improves loading performance.
However, when valkey loads the RDB-formatted preamble from the base AOF
file, it does not process the replication ID (replid) information within
the RDB AUX fields. This omission has two limitations:

* On a primary, it prevents the primary from accepting PSYNC continue
  requests after restarting with a preamble-enabled AOF file.
* On a replica, it prevents the replica from successfully performing
  partial sync requests (avoiding full sync) after restarting with a
  preamble-enabled AOF file.

To resolve this, this commit aligns the AOF preamble handling with the
logic used for standalone RDB files, by storing the replication ID and
replication offset in the AOF preamble and restoring them when loading
the AOF file.

Resolves #2677

---------

Signed-off-by: arthur.lee <liziang.arthur@bytedance.com>
Signed-off-by: Arthur Lee <arthurkiller@users.noreply.github.com>
Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-11-11 12:41:27 +01:00
Ricardo Dias 7fbd4cb260
Expose SIMPLE_STRING and ARRAY_NULL reply type to the Module API (#2804)
This commit extends the Module API to expose the `SIMPLE_STRING` and
`ARRAY_NULL` reply types to modules, by passing the new flag `X` to
the `ValkeyModule_Call` function.

By only exposing the new reply types behind a flag we keep the
backward compatibility with existing module implementations and
allow new modules to working with these reply type, which are
required for scripts to process correctly the reply type of commands
called inside scripts.

Before this change, commands like `PING` or `SET`, which return `"OK"`
as a simple string reply, would be returned as string replies to
scripts.
To allow the support of the Lua engine as an external module, we need to
distinguish between simple string and string replies to keep backward
compatibility.

---------

Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
2025-11-10 15:05:26 +00:00
Ricardo Dias bb8989cfde
Adds new module context flag `VALKEYMODULE_CTX_SCRIPT_EXECUTION` (#2818)
The new module context flag `VALKEYMODULE_CTX_SCRIPT_EXECUTION` denotes
that the module API function is being called in the context of a
scripting engine execution.

Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
2025-11-10 10:29:40 +00:00
Vadym Khoptynets 65ab07dde7
Leverage zfree_with_size for client reply blocks (#2624)
clientReplyBlock stores the size of the actual allocation in it size
field (minus the header size). This can be used for more effective
deallocation with zfree_with_size.

Signed-off-by: Vadym Khoptynets <vadymkh@amazon.com>
2025-11-09 20:46:27 +02:00
Roshan Khatri 2288657a05
[DEFLAKE] Psync established after rdb load - beyond grace period (#2748)
Resolves: https://github.com/valkey-io/valkey/issues/2695

Increase the wait time for periodic log check for rdb load time. Also,
increases the delay of log check frequency.

---------

Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
Signed-off-by: Roshan Khatri <117414976+roshkhatri@users.noreply.github.com>
Co-authored-by: Harkrishn Patro <bunty.hari@gmail.com>
2025-11-07 15:11:37 -08:00
Harkrishn Patro 7f8c5b6f0c
[flaky-failure-fix] Increase the cluster-node-timeout to have longer delay between failover of each shard (#2793) 2025-11-06 16:14:45 -08:00
yzc-yzc 37d08d3866
Fix flaky DBSIZE test for atomic slot migration (#2805)
Related test failures: 

    *** [err]: Replica importing key containment (slot 0 from node 0 to 2) - DBSIZE command excludes importing keys in tests/unit/cluster/cluster-migrateslots.tcl
    Expected '1' to match '0' (context: type eval line 2 cmd {assert_match "0" [R $node_idx DBSIZE]} proc ::test)

The reason is that we don't wait for the primary-replica synchronization
to complete before starting the next testcase.

---------

Signed-off-by: yzc-yzc <96833212+yzc-yzc@users.noreply.github.com>
Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-11-06 18:02:27 +01:00
Ricardo Dias 7a1d989696
Add "script" context to ACL log entries (#2798)
In this commit we add a new context for the ACL log entries that is used
to log ACL failures that occur during scripts execution. To maintain
backward compatibility we still maintain the "lua" context for failures
that happen when running Lua scripts. For other scripting engines the
context description will be just "script".

---------

Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
2025-11-06 09:46:22 +00:00
hieu2102 cf7a628ada
Add instruction to build Valkey with fast_float (#2810)
The `README.md` file is currently missing a section to build Valkey with
`fast_float`, which was introduced in Valkey 8.1 as an optional
dependency (#1260)

Signed-off-by: hieu2102 <hieund2102@gmail.com>
2025-11-06 09:45:12 +00:00
Sarthak Aggarwal 32844b8b0a
Configurable DB hash seed for SCAN family commands consistency (#2608)
Introduce a new config `hash-seed` which can be set only at startup and
controls the hash seed for the server. This includes all hash tables.
This change makes it so that both primaries and replicas will return the
same results for SCAN/HSCAN/ZSCAN/SSCAN cursors. This is useful in order
to make sure SCAN behaves correctly after a failover.

Resolves #4

---------

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
Signed-off-by: Sarthak Aggarwal <sarthakaggarwal97@gmail.com>
Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-11-05 08:45:52 -08:00
xbasel c88c94e326
Reuse dbHasNoKeys() inside dbsHaveNoKeys() to remove duplicate logic (#2800)
Signed-off-by: xbasel <103044017+xbasel@users.noreply.github.com>
2025-11-04 11:28:39 -08:00
Sarthak Aggarwal a49d469f48
Reverts hashHashtableTypeValidate signature (#2799)
Fixes
https://github.com/valkey-io/valkey/actions/runs/19053371057/job/54418411647#step:6:202

Matched hashHashtableTypeValidate to the [generic hashtable callback
signature
](https://github.com/valkey-io/valkey/blob/unstable/src/hashtable.h#L62)and
performed the entry cast internally to preserve expiry checks.

---------

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
Signed-off-by: Ran Shidlansik <ranshid@amazon.com>
Co-authored-by: Ran Shidlansik <ranshid@amazon.com>
Co-authored-by: Jim Brunner <brunnerj@amazon.com>
2025-11-04 20:07:57 +02:00
Jim Brunner a99c636321
Improve header comment and strengthen type checking for entry (#2794)
In `entry.c`, the `entry` is a block of memory with variable contents.
The structure can be difficult to understand. A new header comment more
clearly documents the contents/layout of the `entry`.

Also, in `entry.h`, the `entry` was defined by `typedef void entry`.
This allows blind casting to the `entry` type. It defeats compiler type
checking.

Even though the `entry` has a variable definition, we can define entry
as a legitimate type which allows the compiler to perform type checking.
By performing `typedef struct _entry entry`, now the `entry` is
understood to be a pointer to some type of undefined structure. We can
pass a pointer and the compiler can typecheck the pointer. (Of course we
can't dereference it, because we haven't actually defined the struct.)

Signed-off-by: Jim Brunner <brunnerj@amazon.com>
2025-11-03 19:39:36 +02:00
Hanxi Zhang 4d78d36bff
HSETEX: Support NX/XX Flags (#2668)
### Summary
Addresses https://github.com/valkey-io/valkey/issues/2619.  
This PR extends the `HSETEX` command to support optional key-level `NX`
and `XX` flags, allowing operations conditional on the existence of the
hash key.

### Changes
- Updated `hsetex.json` and regenerated `commands.def`.
- Extended argument parsing for NX/XX.
- Added key-level `NX`/`XX` support in `HSETEX`.
- Added tests covering all four NX/XX scenarios.

---------

Signed-off-by: Hanxi Zhang <hanxizh@amazon.com>
Co-authored-by: Ran Shidlansik <ranshid@amazon.com>
2025-11-03 09:43:48 +02:00
Simon Baatz 6cbc1a31d7
Sentinel: fix regression requiring "+failover" ACL in failover path (#2780)
Since Valkey Sentinel 9.0, sentinel tries to abort an ongoing failover
when changing the role of a monitored instance. Since the result of the
command is ignored, the "FAILOVER ABORT" command is sent irrespective of
the actual failover status.

However, when using the documented pre 9.0 ACLs for a dedicated sentinel
user, the FAILOVER command is not allowed and _all_ failover cases fail.
(Additionally, the necessary ACL adaptation was not communicated well.)

Address this by:

- Updating the documentation in "sentinel.conf" to reflect the need for
an adapted ACL

- Only abort a failover when sentinel detected an ongoing (probably
stuck) failover. This means that standard failover and manual failover
continue to work with unchanged pre 9.0 ACLs. Only the new "SENTINEL
FAILOVER COORDINATED" requires to adapt the ACL on all Valkey nodes.

- Actually use a dedicated sentinel user and ACLs when testing standard
failover, manual failover, and manual coordinated failover.

Fixes #2779

Signed-off-by: Simon Baatz <gmbnomis@gmail.com>
2025-10-31 14:46:53 -04:00
harrylin98 189c69e315
Fix: ltrim should not call signalModifiedKey when no elements are removed (#2787)
There’s an issue with the LTRIM command. When LTRIM does not actually
modify the key — for example, with `LTRIM key 0 -1` — the server.dirty
counter is not updated because both ltrim and rtrim values are 0. As a
result, the command is not propagated. However, `signalModifiedKey` is
still called regardless of whether server.dirty changes. This behavior
is unexpected and can cause a mismatch between the source and target
during propagation, since the LTRIM command is not sent.

Signed-off-by: Harry Lin <harrylhl@amazon.com>
Co-authored-by: Harry Lin <harrylhl@amazon.com>
2025-10-31 14:46:36 -04:00
Jacob Murphy 43ee46da33
Authenticate slot migration client on source node to internal user (#2785)
Just setting the authenticated flag actually authenticates to the
default user in this case. The default user may be granted no permission
to use CLUSTER SYNCSLOTS.

Instaed, we now authenticate to the NULL/internal user, which grants
access to all commands. This is the same as what we do for replication:


864de555ce/src/replication.c (L4717)

Add a test for this case as well.

Closes #2783

Signed-off-by: Jacob Murphy <jkmurphy@google.com>
2025-10-31 10:57:05 -07:00
Ricardo Dias 84eb459cd4
Add ValkeyModule_ReplyWithCustomErrorFormat to module API (#2791)
Note: these changes are part of the effort to run Lua engine as an
external scripting engine module.

The new function `ValkeyModule_ReplyWithCustomErrorFormat` is being
added to the module API to allow scripting engines to return errors that
originated from running commands within the script code, without
counting twice in the error stats counters.

More details on why this is needed by scripting engines can be read in
an older commit aa856b39f2 messsage.

This PR also adds a new test to ensure the correctness of the newly
added function.

---------

Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
2025-10-31 16:02:27 +00:00
xbasel f54818cc60
Bug fix: reset io_last_written on c->buf resize to prevent stale pointers (#2786)
Fixes an assert crash in _writeToClient():

    serverAssert(c->io_last_written.data_len == 0 ||
                 c->io_last_written.buf == c->buf);

The issue occurs when clientsCronResizeOutputBuffer() grows or
reallocates c->buf while io_last_written still points to the old buffer
and data_len is non-zero. On the next write, both conditions in the
assertion become false.

Reset io_last_written when resizing the output buffer to prevent stale
pointers and keep state consistent.

fixes https://github.com/valkey-io/valkey/issues/2769

Signed-off-by: xbasel <103044017+xbasel@users.noreply.github.com>
2025-10-30 13:51:01 -07:00
Ricardo Dias 864de555ce
Make ValkeyModule_Call compatible with calling commands from scripting engines (#2782)
Note: these changes are another step towards being able to run Lua
engine as an external scripting engine module.

In this commit we improve the `ValkeyModule_Call` API function code to
match the validations and behavior of the `scriptCall` function,
currently used by the Lua engine to run commands using `server.call` Lua
Valkey API.

The changes made are backward compatible. The new behavior/validations
are only enabled when calling `ValkeyModule_Call` while running a script
using `EVAL` or `FCALL`.

To test these changes, we improved the `HELLO` dummy scripting engine
module to support calling commands, and compare the behavior with
calling the same command from a Lua script.

Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
2025-10-30 16:19:46 +00:00
Ricardo Dias ea103da5d6
New INFO section for scripting engines (#2738)
This commit adds a new `INFO` section called "Scripting Engines" that
shows the information of the current scripting engines available in the
server.

Here's an output example:

```
> info scriptingengines
# Scripting Engines
engines_count:2
engines_total_used_memory:68608
engines_total_memory_overhead:56
engine_0:name=LUA,module=built-in,abi_version=4,used_memory=68608,memory_overhead=16
engine_1:name=HELLO,module=helloengine,abi_version=4,used_memory=0,memory_overhead=40
```

---------

Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
2025-10-30 16:18:01 +00:00
Diego Ciciani e381182297
Add IPv6 availability check to skip tests when unavailable (#2674)
Skip IPv6 tests automatically when IPv6 is not available.

This fixes the problem that tests fail when IPv6 is not available on the
system, which can worry users when they run `make test`.

IPv6 availibility is detected by opening a dummy server socket and
trying to connect to it using a client socket over IPv6.

Fixes #2643

---------

Signed-off-by: diego-ciciani01 <diego.ciciani@gmail.com>
Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-10-30 11:20:56 +01:00
Sarthak Aggarwal 10281becaf
Adds a summary for tests (#2745)
```
Test Summary: 100 passed, 2 failed

!!! WARNING The following tests failed:
...
````

---------

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
2025-10-29 13:36:37 -07:00
Ken f3b2dee3b7
Add monotonic clock calibration handling if clock speed is not found (#2776)
Currently, monotonic clock initialization relies on the model name field
from /proc/cpuinfo to retrieve the clock speed. However, this is not
always present. In case it is not present, measure the clock tick and
use it instead.

Before fix:
```
monotonic: x86 linux, unable to determine clock rate
```

After fix:
```
21695:M 25 Oct 2025 20:16:23.168 * monotonic clock: X86 TSC @ 2649 ticks/us
```

Fixes #2774

---------

Signed-off-by: Ken Nam <otherscase@gmail.com>
Signed-off-by: Ran Shidlansik <ranshid@amazon.com>
Co-authored-by: Ran Shidlansik <ranshid@amazon.com>
2025-10-28 22:20:12 +02:00
Ritoban Dutta 909d082cd0
Reorder valkey.conf: move configs to correct sections (#2737)
- Moved `server-cpulist`, `bio-cpulist`, `aof-rewrite-cpulist`,
  `bgsave-cpulist` configurations to ADVANCED CONFIG.
- Moved `ignore-warnings` configuration to ADVANCED CONFIG.
- Moved `availability-zone` configuration to GENERAL.

These configs were incorrectly placed at the end of the file in the
ACTIVE DEFRAGMENTATION section.

Fixes #2736

---------

Signed-off-by: ritoban23 <ankudutt101@gmail.com>
2025-10-28 10:36:23 +01:00
Sarthak Aggarwal 2c92a6072d
Reverts rdb-key-save-delay value to fix dual channel replication test in macos (#2771)
Resolves #2696 

Set `rdb-key-save-delay` to 200 microseconds to reduce the overall RDB load time.

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
2025-10-27 13:08:40 -07:00
Zhijun Liao 861d0794b7
Sentinel: Skip IS-PRIMARY-DOWN-BY-ADDR requests when primary not SDOWN (#2763)
A super tiny change to optimize the function
`sentinelAskPrimaryStateToOtherSentinels` to early return when the
sentinel does not observe the primary as subjectively down.

Signed-off-by: Zhijun <dszhijun@gmail.com>
2025-10-24 16:15:54 -04:00
Zhijun Liao baf2d572f7
Ensure the server executable exists before running tests (#2762)
Previously, running ./runtest without src/valkey-server would hang, now it
throws an error.

Signed-off-by: Zhijun <dszhijun@gmail.com>
2025-10-23 19:29:50 +08:00
Ricardo Dias 7043c0f543
Scripting Engine Debugger Support (#1701)
This PR introduces the support for implementing remote debuggers in
scripting engines modules.

The module API is extended with scripting engines callbacks and new
functions that can be used by scripting engine modules to implement a
remote debugger.

Most of the code that was used to implement the Lua debugger, was
refactored and moved to the `scripting_engine.c` file, and only the code
specific to the Lua engine, remained in the `debug_lua.c` file.

The `SCRIPT DEBUG (YES|NO|SYNC)` command was extend with an optional
parameter that can be used to specify the engine name, where we want to
enable the debugger. If no engine name is specified, the Lua engine is
used to keep backwards compatibility.

In
[src/valkeymodule.h](https://github.com/valkey-io/valkey/pull/1701/files#diff-b91520205c29a3a5a940786e509b2f13a5e73a1ac2016be773e62ea64c7efb28)
we see the module API changes. And in the `helloscripting.c` file we can
see how to implement a simple debugger for the dummy HELLO scripting
engine.

---------

Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
2025-10-23 10:58:32 +01:00
Viktor Söderqvist f1270a851f
Adjust test runners to the number of tests to run (#2759)
This is fixing a minor annoyance when running single tests. With this
change, the runtest script doesn't start more runners than the total
number of tests to run. These are seen in the printouts like `[ready]:
68359`.

Screenshot before:

```
$ ./runtest --single tests/unit/limits.tcl 
Cleanup: may take some time... OK
Starting test server at port 21079
[ready]: 68359
Testing unit/limits
[ready]: 68355
[ready]: 68362
[ready]: 68356
[ready]: 68361
[ready]: 68358
[ready]: 68364
[ready]: 68366
[ready]: 68367
[ready]: 68368
[ready]: 68357
[ready]: 68360
[ready]: 68363
[ready]: 68365
[ready]: 68369
[ready]: 68370
[ok]: Check if maxclients works refusing connections (906 ms)
[1/1 done]: unit/limits (1 seconds)

                   The End

Execution time of different units:
  1 seconds - unit/limits

\o/ All tests passed without errors!

Cleanup: may take some time... OK
```

Screenshot after:

```
$ ./runtest --single tests/unit/limits.tcl 
Cleanup: may take some time... OK
Starting test server at port 21079
[ready]: 68439
Testing unit/limits
[ok]: Check if maxclients works refusing connections (906 ms)
[1/1 done]: unit/limits (1 seconds)

                   The End

Execution time of different units:
  1 seconds - unit/limits

\o/ All tests passed without errors!

Cleanup: may take some time... OK
```

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-10-23 09:48:08 +02:00
Zhijun Liao 746d9eca9e
Use the fetched TLS and TCP ports in clusterProcessGossipSection (#2761)
The TLS and TCP ports have been fetched and should be used.
This can improve code readability.

Signed-off-by: Zhijun <dszhijun@gmail.com>
2025-10-23 15:41:29 +08:00
Hanxi Zhang 7e0b3bbd52
Show RPS histogram in valkey-benchmark (#2471)
When benchmarking with the a target thoughput using the `--rps` flag,
display an RPS histogram in the benchmark report. This can help identify
if there is a bottleneck.

Related to #2219.

---------

Signed-off-by: Hanxi Zhang <hanxizh@amazon.com>
2025-10-22 11:35:53 +02:00
Jacob Murphy 0e5477fb9b
Fix atomic slot migration module notification comment (#2758)
Signed-off-by: Jacob Murphy <jkmurphy@google.com>
2025-10-21 20:28:55 -07:00
Madelyn Olson 35b4e2f1ab
Update module api generation and format module.c (#2757)
This change makes it so that the module API reference can be cleanly
generated from the module.c file. Most of this seems to be because of
the code formatting work we did. There are two parts:
1. Updated some of the odd corner cases in module.c so they can be
handled. For example, there was a method that had none of the method on
the first line, which was unhandled. None of these are functional and I
think format should be OK with them.
2. Better handle multi-line function prototypes in the ruby code. Before
we just relied on the function to be on a single line, now we handle it
on multiple lines. This feels pretty hacked in, but I don't really
understand the rest of the code but it does work.

Generated this PR:
https://github.com/valkey-io/valkey-doc/pull/374/files.

---------

Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
2025-10-21 20:28:40 -07:00
Binbin 9acea36841
Fix outdated comment around clusterLink->flags (#2752)
This clusterLink->flags was added in #2310, during the review, we at
the end chose to add a new CLUSTER_LINK_XXX flag instead of sharing the
old CLUSTER_NODE_XXX flag.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-10-21 10:18:41 +08:00
Jacob Murphy 1cf0df9fc3
Fix invalid memory address caused by hashtable shrinking during safe iteration (#2753)
Safe iterators pause rehashing, but don't pause auto shrinking. This
allows stale bucket references which then cause use after free (in this
case, via compactBucketChain on a deleted bucket).

This problem is easily reproducible via atomic slot migration, where we
call delKeysInSlot which relies on calling delete within a safe
iterator. After the fix, it no longer causes a crash.

Since all cases where rehashing is paused expect auto shrinking to also
be paused, I am making this happen automatically as part of pausing
reshashing.

---------

Signed-off-by: Jacob Murphy <jkmurphy@google.com>
2025-10-20 13:23:29 -07:00
Binbin 5d3cb3d04c
Initialize the lua attributes of the luaFunction script (#2750)
This was introduced in #1826. This create an `Uninitialised value was
created by a heap allocation` in the CI.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-10-20 11:17:05 +02:00
Jacob Murphy 2a914aa521
Fix incorrect kvstore size and BIT accounting after completed migration (#2749)
When working on #2635 I errorneously duplicated the
setSlotImportingStateInAllDbs call for successful imports. This resulted
in us doubling the key count in the kvstore. This results in DBSIZE
reporting an incorrect sum, and also causes BIT corruption that can
eventually result in a crash.

The solution is:

1. Only call setSlotImportingStateInAllDbs once (in our
finishSlotMigrationJob function)
2. Make setSlotImportingStateInAllDbs idempotent by checking if the
delete from the kvstore importing hashtable is a no-op

This also fixes a bug where the number of importing keys is not lowered
after the migration, but this is less critical since it is only used
when resizing the dictionary on RDB load. However, it could result in
un-loadable RDBs if the importing key count gets large enough.

Signed-off-by: Jacob Murphy <jkmurphy@google.com>
2025-10-18 10:20:18 -07:00
Binbin b4c93cc9c2
FUNCTION FLUSH re-create lua VM, fix flush not gc, fix flush async + load crash (#1826)
There will be two issues in this test:
```
test {FUNCTION - test function flush} {
    for {set i 0} {$i < 10000} {incr i} {
        r function load [get_function_code LUA test_$i test_$i {return 'hello'}]
    }
    set before_flush_memory [s used_memory_vm_functions]
    r function flush sync
    set after_flush_memory [s used_memory_vm_functions]
    puts "flush sync, before_flush_memory: $before_flush_memory, after_flush_memory: $after_flush_memory"

    for {set i 0} {$i < 10000} {incr i} {
        r function load [get_function_code LUA test_$i test_$i {return 'hello'}]
    }
    set before_flush_memory [s used_memory_vm_functions]
    r function flush async
    set after_flush_memory [s used_memory_vm_functions]
    puts "flush async, before_flush_memory: $before_flush_memory, after_flush_memory: $after_flush_memory"

    for {set i 0} {$i < 10000} {incr i} {
        r function load [get_function_code LUA test_$i test_$i {return 'hello'}]
    }
    puts "Test done"
}
```

The first one is the test output, we can see that after executing
FUNCTION FLUSH,
used_memory_vm_functions has not changed at all:
```
flush sync, before_flush_memory: 2962432, after_flush_memory: 2962432
flush async, before_flush_memory: 4504576, after_flush_memory: 4504576
```

The second one is there is a crash when loading the functions during the
async
flush:
```
=== VALKEY BUG REPORT START: Cut & paste starting from here ===
 # valkey 255.255.255 crashed by signal: 11, si_code: 2
 # Accessing address: 0xe0429b7100000a3c
 # Crashed running the instruction at: 0x102e0b09c

------ STACK TRACE ------
EIP:
0   valkey-server                       0x0000000102e0b09c luaH_getstr + 52

Backtrace:
0   libsystem_platform.dylib            0x000000018b066584 _sigtramp + 56
1   valkey-server                       0x0000000102e01054 luaD_precall + 96
2   valkey-server                       0x0000000102e01b10 luaD_call + 104
3   valkey-server                       0x0000000102e00d1c luaD_rawrunprotected + 76
4   valkey-server                       0x0000000102e01e3c luaD_pcall + 60
5   valkey-server                       0x0000000102dfc630 lua_pcall + 300
6   valkey-server                       0x0000000102f77770 luaEngineCompileCode + 708
7   valkey-server                       0x0000000102f71f50 scriptingEngineCallCompileCode + 104
8   valkey-server                       0x0000000102f700b0 functionsCreateWithLibraryCtx + 2088
9   valkey-server                       0x0000000102f70898 functionLoadCommand + 312
10  valkey-server                       0x0000000102e3978c call + 416
11  valkey-server                       0x0000000102e3b5b8 processCommand + 3340
12  valkey-server                       0x0000000102e563cc processInputBuffer + 520
13  valkey-server                       0x0000000102e55808 readQueryFromClient + 92
14  valkey-server                       0x0000000102f696e0 connSocketEventHandler + 180
15  valkey-server                       0x0000000102e20480 aeProcessEvents + 372
16  valkey-server                       0x0000000102e4aad0 main + 26412
17  dyld                                0x000000018acab154 start + 2476

------ STACK TRACE DONE ------
```

The reason is that, in the old implementation (introduced in 7.0),
FUNCTION FLUSH
use lua_unref to remove the script from lua VM. lua_unref does not
trigger the gc,
it causes us to not be able to effectively reclaim memory after the
FUNCTION FLUSH.

The other issue is that, since we don't re-create the lua VM in FUNCTION
FLUSH,
loading the functions during a FUNCTION FLUSH ASYNC will result a crash
because
lua engine state is not thread-safe.

The correct solution is to re-create a new Lua VM to use, just like
SCRIPT FLUSH.

---------

Signed-off-by: Binbin <binloveplay1314@qq.com>
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
Co-authored-by: Ricardo Dias <ricardo.dias@percona.com>
2025-10-17 15:25:17 +01:00
Harkrishn Patro 95154feaa1
Bump old engine version(s) for compatibility test (#2741)
Signed-off-by: Harkrishn Patro <harkrisp@amazon.com>
2025-10-16 22:43:19 -07:00
Roshan Khatri 898172bc9c
Deflake Psync established within grace period (#2743)
increased the wait time to a total of 10 seconds where we check the log
for `Done loading RDB` message

Fixes https://github.com/valkey-io/valkey/issues/2694

CI run (100 times):
https://github.com/roshkhatri/valkey/actions/runs/18576201712/job/52961907806

Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
2025-10-16 19:50:48 -07:00
Binbin af45feea12
Remove the outupdated unknown key/value pairs comment in CLUSTER SYNCSLOTS ESTABLISH (#2498)
We now have #2688 SYNCSLOTS CAPA, remove the outupdated comment.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-10-17 10:24:43 +08:00
Sarthak Aggarwal 981b8fe1bd
Deflakes Primary COB growth with inactive replica (#2715)
Resolves #2696 

The primary issue was that with sanitizer mode, the test needed more
time for primary’s replication buffers grow beyond `2 × backlog_size`.
Increasing the threshold of `repl-timeout` to 30s, ensures that the
inactive replica is not disconnected while the full sync is proceeding.
`rdb-key-save-delay` controls or throttles the data written to the
client output buffer, and in this case, we are deterministically able to
perform the fullsync within 10s (10000 keys * 0.001s).

Increasing the `wait_for_condition` gives it enough retries to verify
that `mem_total_replication_buffers` reaches the required `2 ×
backlog_size`.

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
2025-10-16 10:07:52 +08:00
Viktor Söderqvist 54da8344c1
Fix double MOVED reply on unblock at failover (#2734)
#2329 intoduced a bug that causes a blocked client in cluster mode to
receive two MOVED redirects instead of one. This was not seen in tests,
except in the reply schema validator.

The fix makes sure the client's pending command is cleared after sending
the MOVED redirect, to prevent if from being reprocessed.

Fixes #2676.

---------

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-10-14 17:44:17 +02:00
Binbin ccf6fca5f4
Add `Slot migration is ok when the replicas are down test` back (#2727)
The test was accidentally removed in PR #1671.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-10-14 12:56:36 +08:00
Jacob Murphy 28e5dcce2c
Fix crash that occurs sometimes when aborting a slot migration while child snapshot is active (#2721)
The race condition causes the client to be used and subsequently double
freed by the slot migration read pipe handler. The order of events is:

1. We kill the slot migration child process during CANCELSLOTMIGRATIONS
1. We then free the associated client to the target node
1. Although we kill the child process, it is not guaranteed that the
pipe will be empty from child to parent
1. If the pipe is not empty, we later will read that out in the
slotMigrationPipeReadHandler
1. In the pipe read handler, we attempt to write to the connection. If
writing to the connection fails, we will attempt to free the client
1. However, the client was already freed, so this a double free

Notably, the slot migration being aborted doesn't need to be triggered
by `CANCELSLOTMIGRATIONS`, it can be any failure.

To solve this, we simply:

1. Set the slot migration pipe connection to NULL whenever it is
unlinked
2. Bail out early in slot migration pipe read handler if the connection
is NULL

I also consolidate the killSlotMigrationChild call to one code path,
which is executed on client unlink. Before, there were two code paths
that would do this twice (once on slot migration job finish, and once on
client unlink). Sending the signal twice is fine, but inefficient.

Also, add a test to cancel during the slot migration snapshot to make
sure this case is covered (we only caught it during the module test).

---------

Signed-off-by: Jacob Murphy <jkmurphy@google.com>
2025-10-13 08:18:13 -07:00
Jacob Murphy 19474c867a
Deflake atomic slot migration client flag test (#2720)
This test was failing, and causing the next test to throw an exception.
It is failing since we never waited for the slot migration to connect
before proceeding.

Fixes #2692

Signed-off-by: Jacob Murphy <jkmurphy@google.com>
2025-10-13 08:17:59 -07:00
Jacob Murphy dbcf022480
Stop using DEBUG LOADAOF on replica in ASM tests (#2719)
DEBUG LOADAOF sometimes works but it results in -LOADING responses to
the primary so there are lots of race conditions. It isn't something we
should be doing anyways. To test, I just disconnect the replica before
loading the AOF, then reconnect it.

Fixes #2712

Signed-off-by: Jacob Murphy <jkmurphy@google.com>
2025-10-13 08:17:36 -07:00
Harkrishn Patro 181ca483b9
Bump CLUSTER SHARDS command update version to 9.1.0 (#2729) 2025-10-12 22:12:08 -07:00
Kyle Kim (kimkyle@) 0d500c2cc1
Update the misleading zdiff() comment on empty first key handling. (#747)
Currently, zdiff() becomes a no-op in the case that the first key is
empty.

The existing comment of "Skip everything if the smallest input is empty"
is misleading, as qsort is bypassed for the case of ZDIFF. There's no
guarantee that the first key holds the smallest cardinality.

The real reason behind such bypass is the following. ZDIFF computes the
difference between the first and all successive input sorted sets.
Meaning, if the first key is empty, we cannot reduce further from an
already empty collection, and thus zdiff() becomes a no-op.

Signed-off-by: Kyle Kim <kimkyle@amazon.com>
2025-10-10 18:54:58 -07:00
Viktor Söderqvist d29d580067
Tests: Don't dump logs when skipping test using 'skip' (#2718)
When using `skip` inside a test to skip a test, when running with
--dump-logs it causes the logs to be dumped. Introduced in #2342.

The reason is the "skipped" exception is caught in the start_server proc
in tests/support/server.tcl. This is where the $::dump_logs variable is
checked.

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-10-10 09:54:31 -07:00
Ran Shidlansik 8182f4a0b9
HSETEX with FXX should not create an object if it does not exist (#2716)
When the hash object does not exist FXX should simply fail the check
without creating the object while FNX should be trivial and succeed.

Note - also fix a potential compilation warning on some COMPILERS doing
constant folding of variable length array when size is const expression.

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>
2025-10-10 12:30:08 +03:00
Harkrishn Patro 18214be490
Add compatibility test with Valkey 7.2/8.0 (#2342)
* Add cross version compatibility test to run with Valkey 7.2 and 8.0
* Add mechanism in TCL test to skip tests dynamically - #2711

---------

Signed-off-by: Harkrishn Patro <harkrisp@amazon.com>
Signed-off-by: Harkrishn Patro <bunty.hari@gmail.com>
Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-10-09 17:00:10 -07:00
Sarthak Aggarwal f598d11a4b
Deflake replica selection test by relaxing cluster configurations (#2672)
We have relaxed the `cluster-ping-interval` and `cluster-node-timeout`
so that cluster has enough time to stabilize and propagate changes.

Fixes this test occasional failure when running with valgrind:

    [err]: Node #10 should eventually replicate node #5 in tests/unit/cluster/slave-selection.tcl
    #10 didn't became slave of #5

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
2025-10-10 01:03:53 +02:00
Harkrishn Patro f7c95277da
Add invalid RDB signature to log statement (#2710)
Signed-off-by: Harkrishn Patro <harkrisp@amazon.com>
2025-10-09 13:11:31 -07:00
Harkrishn Patro 155b0bb821
Fix memory leak with CLIENT LIST/KILL duplicate filters (#2362)
With #1401, we introduced additional filters to CLIENT LIST/KILL
subcommand. The intended behavior was to pick the last value of the
filter. However, we introduced memory leak for all the preceding
filters.

Before this change:
```
> CLIENT LIST IP 127.0.0.1 IP 127.0.0.1
id=4 addr=127.0.0.1:37866 laddr=127.0.0.1:6379 fd=10 name= age=0 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=21 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=16989 events=r cmd=client|list user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=49 tot-net-out=0 tot-cmds=0
```
Leak:
```
Direct leak of 11 byte(s) in 1 object(s) allocated from:
    #0 0x7f2901aa557d in malloc (/lib64/libasan.so.4+0xd857d)
    #1 0x76db76 in ztrymalloc_usable_internal /workplace/harkrisp/valkey/src/zmalloc.c:156
    #2 0x76db76 in zmalloc_usable /workplace/harkrisp/valkey/src/zmalloc.c:200
    #3 0x4c4121 in _sdsnewlen.constprop.230 /workplace/harkrisp/valkey/src/sds.c:113
    #4 0x4dc456 in parseClientFiltersOrReply.constprop.63 /workplace/harkrisp/valkey/src/networking.c:4264
    #5 0x4bb9f7 in clientListCommand /workplace/harkrisp/valkey/src/networking.c:4600
    #6 0x641159 in call /workplace/harkrisp/valkey/src/server.c:3772
    #7 0x6431a6 in processCommand /workplace/harkrisp/valkey/src/server.c:4434
    #8 0x4bfa9b in processCommandAndResetClient /workplace/harkrisp/valkey/src/networking.c:3571
    #9 0x4bfa9b in processInputBuffer /workplace/harkrisp/valkey/src/networking.c:3702
    #10 0x4bffa3 in readQueryFromClient /workplace/harkrisp/valkey/src/networking.c:3812
    #11 0x481015 in callHandler /workplace/harkrisp/valkey/src/connhelpers.h:79
    #12 0x481015 in connSocketEventHandler.lto_priv.394 /workplace/harkrisp/valkey/src/socket.c:301
    #13 0x7d3fb3 in aeProcessEvents /workplace/harkrisp/valkey/src/ae.c:486
    #14 0x7d4d44 in aeMain /workplace/harkrisp/valkey/src/ae.c:543
    #15 0x453925 in main /workplace/harkrisp/valkey/src/server.c:7319
    #16 0x7f2900cd7139 in __libc_start_main (/lib64/libc.so.6+0x21139)
```

Note: For filter ID / NOT-ID we group all the option and perform
filtering whereas for remaining filters we only pick the last filter
option.

---------

Signed-off-by: Harkrishn Patro <harkrisp@amazon.com>
2025-10-08 16:04:47 -07:00
Satheesha CH Gowda 5bbbc6bd9a
Add shard id field to CLUSTER SHARDS response (#2568)
This change exposes an existing, persistent (in nodes.conf) unique shard
identifier for each shard in the cluster as part of the `CLUSTER SHARDS` command
response.

```
1) 1) "slots"
   2) 1) (integer) 0
      2) (integer) 999
      3) (integer) 2001
      4) (integer) 3999
      5) (integer) 4501
      6) (integer) 5460
   3) "nodes"
   4) 1)  1) "id"
          2) "6e76043bed00e716e85035107866ea16e9a5f700"
          3) "port"
          4) (integer) 6385
          5) "ip"
          6) "127.0.0.1"
          7) "endpoint"
          8) "127.0.0.1"
          9) "role"
         10) "replica"
         11) "replication-offset"
         12) (integer) 8092
         13) "health"
         14) "online"
      2)  1) "id"
          2) "b2f8c841707b2246ec2a641c37f16e88fe0bb700"
          3) "port"
          4) (integer) 6380
          5) "ip"
          6) "127.0.0.1"
          7) "endpoint"
          8) "127.0.0.1"
          9) "role"
         10) "master"
         11) "replication-offset"
         12) (integer) 8092
         13) "health"
         14) "online"
   5) "id"
   6) "3f2a7bb7bbd5fc2a331fe9bf95f5e02bcca02430"
```

---------

Signed-off-by: Satheesha Chattenahalli Hanume Gowda <satheesha@apple.com>
Co-authored-by: Satheesha Chattenahalli Hanume Gowda <satheesha@apple.com>
2025-10-08 12:14:05 -07:00
Jacob Murphy 22247f6119
Reduce flakiness of atomic slot migration AOF test (#2705)
If we don't wait for the replica to resync, the migration may be
cancelled by the time the replica resyncs, resulting in a test failure
when we can't find the migration on the replica.

---------

Signed-off-by: Jacob Murphy <jkmurphy@google.com>
Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-10-08 12:52:36 +02:00
Viktor Söderqvist 3390b1e608
Allow TCL 9.0 for tests (#1673)
Makes our tests possible to run with TCL 9.

The latest Fedora now has TCL 9.0 and it's working now, including the
TCL TLS package. (This wasn't working earlier due to some packaging
errors for TCL packages in Fedora, which have been fixed now.)

This PR also removes the custom compilation of TCL 8 used in our Daily
jobs and uses the system default TCL version instead. The TCL version
depends on the OS. For the latest Fedora, you get 9.0, for macOS you get
8.5 and for most other OSes you get 8.6.

The checks for TCL 8.7 are removed, because 8.7 doesn't exist. It was
never released.

---------

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-10-08 11:37:15 +02:00
Jacob Murphy b3da88e94d
Use correct arguments in LOLWUT test (#2708)
Seeing test failures due to this on the 9.0.0 branch:

```
[exception]: Executing test client: ERR Syntax error. Use: LOLWUT [columns rows] [real imaginary].
ERR Syntax error. Use: LOLWUT [columns rows] [real imaginary]
```

It turns out we are just providing the version as an argument, instead
of specifying which version to run on

Signed-off-by: Jacob Murphy <jkmurphy@google.com>
2025-10-07 15:42:56 -07:00
Jacob Murphy a4317e9115
Introduce SYNCSLOTS CAPA for forwards compatibility (#2688)
For now, introduce this and have it do nothing. Eventually, we can use
this to negotiate supported capabilities on either end. Right now, there
is nothing to send or support, so it just accepts it and doesn't reply.

---------

Signed-off-by: Jacob Murphy <jkmurphy@google.com>
Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-10-07 10:19:36 -07:00
Jacob Murphy 383ffe4874
Prevent exposure of importing keys on replicas during atomic slot migration (#2635)
# Problem

In the current slot migration design, replicas are completely unaware of
the slot migration. Because of this, they do not know to hide importing
keys, which results in exposure of these keys to commands like KEYS,
SCAN, RANDOMKEY, and DBSIZE.

# Design

The main part of the design is that we will now listen for and process
the `SYNCSLOTS ESTABLISH` command on the replica. When a `SYNCSLOTS
ESTABLISH` command is received from the primary, we begin tracking a new
slot import in a special `SLOT_IMPORT_OCCURRING_ON_PRIMARY` state.
Replicas use this state to track the import, and await for a future
`SYNCSLOTS FINISH` message that tells them the import is
successful/failed.

## Success Case

```
     Source                                          Target                         Target Replica
       |                                                |                                 |
       |------------ SYNCSLOTS ESTABLISH -------------->|                                 |
       |                                                |----- SYNCSLOTS ESTABLISH ------>|
       |<-------------------- +OK ----------------------|                                 |
       |                                                |                                 |
       |~~~~~~~~~~~~~~ snapshot as AOF ~~~~~~~~~~~~~~~~>|                                 |
       |                                                |~~~~~~ forward snapshot ~~~~~~~~>|
       |----------- SYNCSLOTS SNAPSHOT-EOF ------------>|                                 |
       |                                                |                                 |
       |<----------- SYNCSLOTS REQUEST-PAUSE -----------|                                 |
       |                                                |                                 |
       |~~~~~~~~~~~~ incremental changes ~~~~~~~~~~~~~~>|                                 |
       |                                                |~~~~~~ forward changes ~~~~~~~~~>|
       |--------------- SYNCSLOTS PAUSED -------------->|                                 |
       |                                                |                                 |
       |<---------- SYNCSLOTS REQUEST-FAILOVER ---------|                                 |
       |                                                |                                 |
       |---------- SYNCSLOTS FAILOVER-GRANTED --------->|                                 |
       |                                                |                                 |
       |                                            (performs takeover &                  |
       |                                             propagates topology)                 |
       |                                                |                                 |
       |                                                |------- SYNCSLOTS FINISH ------->|
 (finds out about topology                              |                                 |
  change & marks migration done)                        |                                 |
       |                                                |                                 |
```

## Failure Case
```
     Source                                          Target                         Target Replica
       |                                                |                                 |
       |------------ SYNCSLOTS ESTABLISH -------------->|                                 |
       |                                                |----- SYNCSLOTS ESTABLISH ------>|
       |<-------------------- +OK ----------------------|                                 |
     ...                                              ...                               ...
       |                                                |                                 |
       |                                             <FAILURE>                            |
       |                                                |                                 |
       |                                      (performs cleanup)                          |
       |                                                | ~~~~~~ UNLINK <key> ... ~~~~~~~>|
       |                                                |                                 |
       |                                                | ------ SYNCSLOTS FINISH ------->|
       |                                                |                                 |
```

## Full Sync, Partial Sync, and RDB

In order to ensure replicas that resync during the import are still
aware of the import, the slot import is serialized to a new
`cluster-slot-imports` aux field. The encoding includes the job name,
the source node name, and the slot ranges being imported. Upon loading
an RDB with the `cluster-slot-imports` aux field, replicas will add a
new migration in the `SLOT_IMPORT_OCCURRING_ON_PRIMARY` state.

It's important to note that a previously saved RDB file can be used as
the basis for partial sync with a primary. Because of this, whenever we
load an RDB file with the `cluster-slot-imports` aux field, even from
disk, we will still add a new migration to track the import. If after
loading the RDB, the Valkey node is a primary, it will cancel the slot
migration. Having this tracking state loaded on primaries will ensure
that replicas partial syncing to a restarted primary still get their
`SYNCSLOTS FINISH` message in the replication stream.

## AOF

Since AOF cannot be used as the basis for a partial sync, we don't
necessarily need to persist the `SYNCSLOTS ESTABLISH` and `FINISH`
commands to the AOF.

However, considering there is work to change this (#59 #1901) this
design doesn't make any assumptions about this.

We will propagate the `ESTABLISH` and `FINISH` commands to the AOF, and
ensure that they can be properly replayed on AOF load to get to the
right state. Similar to RDB, if there are any pending "ESTABLISH"
commands that don't have a "FINISH" afterwards upon becoming primary, we
will make sure to fail those in `verifyClusterConfigWithData`.

Additionally, there was a bug in the existing slot migration where slot
import clients were not having their commands persisted to AOF. This has
been fixed by ensuring we still propagate to AOF even for slot import
clients.

## Promotion & Demotion

Since the primary is solely responsible for cleaning up unowned slots,
primaries that are demoted will not clean up previously active slot
imports. The promoted replica will be responsible for both cleaning up
the slot (`verifyClusterConifgWithData`) and sending a `SYNCSLOTS
FINISH`.

# Other Options Considered

I also considered tracking "dirty" slots rather than using the slot
import state machine. In this setup, primaries and replicas would simply
mark each slot's hashtable in the kvstore as dirty when something is
written to it and we do not currently own that slot.

This approach is simpler, but has a problem in that modules loaded on
the replica would still not get slot migration start/end notifications.
If the modules on the replica do not get such notifications, they will
not be able to properly contain these dirty keys during slot migration
events.

---------

Signed-off-by: Jacob Murphy <jkmurphy@google.com>
2025-10-07 09:17:07 -07:00
Madelyn Olson 064674945d
Fix format issues with CVE fix (#2679)
The CVE fixes had a formatting and external test issue that wasn't
caught because private branches don't run those CI steps.

Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
2025-10-03 10:26:28 -07:00
Madelyn Olson 6dd003e88f
Merge commit from fork
Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
2025-10-03 06:32:24 -07:00
Jacob Murphy d5bb986fd5
Add slot migration client flags and module context flags (#2639)
New client flags in reported by CLIENT INFO and CLIENT LIST:

* `i` for atomic slot migration importing client
* `E` for atomic slot migration exporting client

New flags in return value of `ValkeyModule_GetContextFlags`:

* `VALKEYMODULE_CTX_FLAGS_SLOT_IMPORT_CLIENT`: Indicate the that client
attached to this context is the slot import client.
* `VALKEYMODULE_CTX_FLAGS_SLOT_EXPORT_CLIENT`: Indicate the that client
attached to this context is the slot export client.

Users could use this to monitor the underlying client info of the slot
migration, and more clearly understand why they see extra clients during
the migration.

Modules can use these to detect keyspace notifications on import
clients. I am also adding export flags for symmetry, although there
should not be keyspace notifications. But they would potentially be
visible in command filters or in server events triggered by that client.

---------

Signed-off-by: Jacob Murphy <jkmurphy@google.com>
2025-10-02 21:12:34 +02:00
Alina Liu 307397904f
Defrag if slab 1/8 full to fix defrag didn't stop issue (#2656)
**Issue History:**
1. The flaky test issue "defrag didn't stop" was originally detected in
February 2025: https://github.com/valkey-io/valkey/issues/1746
Solution for 1746: https://github.com/valkey-io/valkey/pull/1762
2. Similar issue occurred recently:
https://github.com/valkey-io/valkey/actions/runs/16585350083/job/46909359496#step:5:7640

**Investigation:** 
1. First, the issue occurs specifically to Active Defrag stream in
cluster mode.
2. After investigating `test_stream` in `memefficiency.tcl`, I found the
root cause is in defrag logic rather than the test itself - There are
still failed tests with the same error even if I tried different
parameters for the test.
3. Then I looked at malloc-stats and identified potential defrag issues,
particularly in the 80B bin where utilization only reaches ~75% after
defrag instead of the expected near 100%, while other bins show proper
defrag behavior - 80B actually is the size of a new stream(confirmed in
`t_stream.c`) that we add during test.
4. For 80B, after adding 200000 streams and fragmenting, `curregs `=
100030, after a lot of defrag cycles, there are still 122
nonfull-slabs/511 slabs with the remaining 446 items not defragged
(average 4/nonfull-slab).

**Detailed malloc-stats:**
- Total slabs: 511
- Non-full slabs: 122
- Full slabs: 511-122=389
- Theoretical maximum per slab: 256 items
- Allocated items in non-full slabs: 100030-389*256=446
- Average items per non-full slab: 446/122=3.66

**Root Cause:** 
**There are some immovable items which  prevent complete defrag**

**Problems in old defrag logic:**
1. The previous condition (we don't defrag if slab utilization > 'avg
utilization' * 1.125), the 12.5% threshold doesn’t work well with low
utilizations.

- Let's imagine we have 446 items in 122 nonfull-slabs (avg 3.66
items/nonfull-slab), let's say, e.g. we have 81 slabs with 5 items each
+41 slabs with 1 item each)
- 12.5% threshold: 3.66*1.125=4.11
- If those 41 single items are immovable, they actually lower the
average, so the rest 81 slabs will be above the threshold (5>4.11) and
will not be defragged - defrag didn't stop.

2. Distribution of immovable items across slabs was causing inconsistent
defragmentation and flaky test outcome.

- If those 41 single items are movable, they will be moved and the avg
will be 5, then 12.5% threshold: 5*1.125=5.625, so the rest 81 slabs
will fall below the threshold (5<5.625) and will be defragged - defrag
success.
- This can explain why we got flaky defrag tests.


**Final solution :** 
1. Add one more condition before the old logic in `makeDefragDecision
`to trigger defragmentation when slab is less than 1/8 full (1/8
threshold (12.5%) chosen to align with existing utilization threshold
factor) - Ensures no low-utilization slabs left without defragged, and
stabilize the defrag behavior.
2. The reason why we have immovable items and how to handle them is
going to be investigate later.
3. Be sure to rebuild Valkey before testing it.


**Local test result:**
- Before fix: 
pass rate 80.8% (63/78)
- After fix:
Test only stream: pass rate 100% (200/200)
Test the whole memefficiency.tcl: pass rate 100% (100/100)

Resolves #2398 , the "defrag didn't stop" issue, with help from @JimB123
@madolson

---------

Signed-off-by: Alina Liu <liusalisa6363@gmail.com>
Signed-off-by: asagegeLiu <liusalisa6363@gmail.com>
Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
2025-10-01 20:37:15 -07:00
Madelyn Olson a9a65abc85
Implement a lolwut for version 9 (#2646)
As requested, here is a version of lolwut for 9 that visualizes a Julia
set with ASCII art.

Example:
```
127.0.0.1:6379> lolwut version 9
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                     .............                              
                                 ......................                         
                              ............................                      
                           ......:::--:::::::::::::::.......                    
                         .....:::=+*@@@=--------=+===--::....                   
                        ....:::-+@@@@&*+=====+%@@@@@@@@=-::....                 
                      .....:::-=+@@@@@%%*++*@@@@@@@@@&*=--::....                
                     .....::--=++#@@@@@@@@##@@@@@@@@@@@@@@=::....               
                    ......:-=@&#&@@@@@@@@@@@@@@@@@@@@@@@@@%-::...               
                   ......::-+@@@@@@@@@@@@@@@@@@&&@@@#%#&@@@-::....              
                  .......::-=+%@@@@@@@@@@@@@@@@#%%*+++++%@+-:.....              
                  .......::-=@&@@@@@@@@@@@@@@@@&*++=====---::.....              
                 .......:::--*@@@@@@@@@@@@@@@@@%++===----::::.....              
                ........::::-=+*%&@@@@@@@@@&&&%*+==----:::::......              
                ........::::--=+@@@@@@@@@@&##%*++==---:::::.......              
                .......:::::---=+#@@@@@@@@&&&#%*+==---:::::.......              
               ........:::::---=++*%%#&&@@@@@@@@@+=---::::........              
               .......:::::----=++*%##&@@@@@@@@@@%+=--::::.......               
               ......::::-----==++#@@@@@@@@@@@@@&%*+=-:::........               
               ......:::---====++*@@@@@@@@@@@@@@@@@@+-:::.......                
               .....::-=++==+++**%@@@@@@@@@@@@@@@@#*=--::.......                
                ....:-%@@%****%###&@@@@@@@@@@@@@@@@&+--:.......                 
                ....:-=@@@@@&@@@@@@@@@@@@@@@@@@@@@@@@=::......                  
                 ...::+@@@@@@@@@@@@@@@&&@@@@@@@@%**@+-::.....                   
                 ....::-=+%#@@@@@@@@@&%%%&@@@@@@*==-:::.....                    
                  ....::--+%@@@@@@@%++==++*#@@@@&=-:::....                      
                   ....:::-*@**@@+==----==*%@@@@+-:::....                       
                     .....:::---::::::::--=+@=--::.....                         
                       .........::::::::::::::.......                           
                         .........................                              
                             ..................                                 
                                    ...                                         
                                                                                
                                                                                
                                                                                
Ascii representation of Julia set with constant 0.41 + 0.29i
Don't forget to have fun! Valkey ver. 255.255.255
```

You can pass in arbitrary rows and colums (it's best when rows is 2x
number of columns) and an arbitrary julia constant so it is repeatable.
Worst case it takes about ~100us on my m2 macbook, which should be fine
to make sure it's not taking too many system resources.

---------

Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
2025-09-30 08:25:53 +02:00
Ran Shidlansik f39a809711
Fix module key memory usage accounting (#2661)
Make objectComputeSize account for the key size as well when the key is
a module datatype

fixes: https://github.com/valkey-io/valkey/issues/2660

---------

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>
2025-09-29 12:49:42 +02:00
Jacob Murphy e99380f75c
Fix atomic slot migration snapshot never proceeding with hz 1 (#2636)
The problem is that ACKs run on a set loop (once every second) and this
will happen every loop with hz 1.

Instead, we can do the ACK after running the main logic. We can also do
an additional optimization where we don't send an ACK from source to
target if we already sent some data this cron loop, since the target
will reset the ack timer on any data over the connection.

---------

Signed-off-by: Jacob Murphy <jkmurphy@google.com>
Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-09-29 12:42:25 +02:00
cjx-zar ae58b3d5be
Redirect blocked clients after failover (#2329)
In standalone mode, after a switch over, the command that was originally
blocking on primary returns -REDIRECT instead of -UNBLOCKED when the
client has the redirect capability.

Similarly, in cluster mode, after a switch over, the blocked commands
receive a -MOVED redirect instead of -UNBLOCKED.

After this fix, the handling of blocked connections during a switch over
between standalone and cluster is nearly identical. This can be
summarized as follows:

Standalone: 

1. Client that has the redirect capability blocked on the key on the
primary node will receive a -REDIRECT after the switch over completes
instead of -UNBLOCKED.
2. Readonly clients blocked on the primary or replica node will remain
blocked throughout the switch over.

Cluster:

1. Client blocked on the key served by the primary node will receive a
-MOVED instead of a probabilistic -UNBLOCKED error.
2. Readonly clients blocked on the key served by primary or replica node
will remain blocked throughout the switch over.

---------

Signed-off-by: cjx-zar <56825069+cjx-zar@users.noreply.github.com>
Co-authored-by: Simon Baatz <gmbnomis@gmail.com>
Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-09-29 11:13:06 +02:00
Binbin 80cec0a184
Minor fix for dual rdb channel connection conn error log (#2658)
This should be server.repl_rdb_transfer_s

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-09-28 19:50:34 +08:00
Jacob Murphy 6c1bb73a57
Add atomic slot migration test for unblock on migration complete (#2637)
This is already handled by `clusterRedirectBlockedClientIfNeeded`. With
the work we are doing in #2329, it makes sense to have an explicit test here
to prevent regression.

Signed-off-by: Jacob Murphy <jkmurphy@google.com>
Signed-off-by: Binbin <binloveplay1314@qq.com>
Co-authored-by: Binbin <binloveplay1314@qq.com>
2025-09-26 10:25:41 +08:00
Sarthak Aggarwal ffdf222694
Increasing retries to allow succcessful meet in Valgrind (#2644)
There is a daily test failure in valgrind, which looks like an issue related to
slowness in valgrind mode.

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
Signed-off-by: Binbin <binloveplay1314@qq.com>
Co-authored-by: Binbin <binloveplay1314@qq.com>
2025-09-25 10:23:12 +08:00
chzhoo d21d529851
Optimize skiplist random level generation logic (#2631)
Each insertion of a skiplist node requires generating a random level
(via the `zslRandomLevel` function), and some commands (such as
`zunionstore`) call the `zslRandomLevel` function multiple times.
Therefore, optimizing `zslRandomLevel` can significantly improve the
performance of these commands.

The main optimization approach is as follows:

1. Original logic: Each iteration called the `random` function, with a
0.25 probability of continuing to call `random` again. In the worst-case
scenario, it required up to 32 calls (though the probability of this
occurring is extremely low).
2. Optimized logic: We only need to call the `genrand64_int64` function
once. Each iteration uses only 2 bits of randomness, effectively
achieving the equivalent of 32 iterations in the original algorithm.
3. Additionally, the introduction of `__builtin_clzll` significantly
reduces CPU usage, which compiles into a single, highly efficient CPU
instruction (e.g., LZCNT on x86, CLZ on ARM) on supported hardware
platforms
4. Although I've explained a lot, the actual code changes are quite
minimal, so just look at the code directly.

---------

Signed-off-by: chzhoo <czawyx@163.com>
2025-09-22 14:11:49 +02:00
korjeek 298f7dae60
Adding unit tests for sha256 (#2632)
Adding comprehensive unit tests for SHA-256 implementation.
 
 These tests verify:
  1. Basic functionality with known test vectors (e.g., "abc")
  2. Handling of large input data (4KB repeated 1000 times)
3. Edge case with repeated single-byte input (1 million 'a' characters)
 
The tests ensure compatibility with standard SHA-256 implementations and
will help detect regressions during future code changes.
2025-09-21 18:34:30 +03:00
Sarthak Aggarwal 8d562d26df
Fix closing slot migration pipe read (#2630)
We probably should close the correct `slot_migration_pipe_read`. It
should resolve the valgrind errors.

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
2025-09-20 08:44:51 +08:00
Ricardo Dias 78060cb675
Fix test that checks `extended-redis-compatibility` config deprecation rules (#2629)
Following the decision in #2189, we need to fix this test because the
`extended-redis-compatibility` config option is not going to be
deprecated in 9.0.

This commit changes the test to postpone the deprecation of
`extended-redis-compatibility` until 10.0 release.

Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
2025-09-19 19:37:09 +01:00
Sarthak Aggarwal ff1d017958
Fix flaky cluster flush slot test (#2626)
The reason is that the replication stream may not have yet reached
the replica for execution. We could add a wait_for_condition. We decided
to replace those assert calls with assert_replication_stream to verify
the contents of the replication stream rather than the commandstats.
```
*** [err]: Flush slot command propagated to replica in tests/unit/cluster/cluster-flush-slot.tcl
```

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
Signed-off-by: Binbin <binloveplay1314@qq.com>
Co-authored-by: Binbin <binloveplay1314@qq.com>
2025-09-19 10:16:43 +08:00
Roshan Khatri e53d7de40e
Update automated benchmarking configs (#2625)
reduce the req and warmup time to finish in 6 hrs as the github workflow
times out after 6 hrs

---------

Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
2025-09-18 19:42:13 +02:00
Binbin f6a0f8cfc0
Separate RDB snapshotting from atomic slot migration (#2533)
When we adding atomic slot migration in #1949, we reused a lot of rdb save code,
it was an easier way to implement ASM in the first time, but it comes with some
side effect. Like we are using CHILD_TYPE_RDB to do the fork, we use rdb.c/rdb.h
function to save the snapshot, these mess up the logs (we will print some logs
saying we are doing RDB stuff) and mess up the info fields (we will say we are
rdb_bgsave_in_progress but actually we are doing slot migration).

In addition, it makes the code difficult to maintain. The rdb_save method uses
a lot of rdb_* variables, but we are actually doing slot migration. If we want
to support one fork with multiple target nodes, we need to rewrite these code
for a better cleanup.

Note that the changes to rdb.c/rdb.h are reverting previous changes from when
we was reusing this code for slot migration. The slot migration snapshot logic
is similar to the previous diskless replication. We use pipe to transfer the
snapshot data from the child process to the parent process.

Interface changes:
- New slot_migration_fork_in_progress info field.
- New cow_size field in CLUSTER GETSLOTMIGRATIONS command.
- Also add slot migration fork to the cluster class trace latency.

Signed-off-by: Binbin <binloveplay1314@qq.com>
Signed-off-by: Jacob Murphy <jkmurphy@google.com>
Co-authored-by: Jacob Murphy <jkmurphy@google.com>
2025-09-18 16:26:42 +08:00
uriyage 80bbbcf6fe
Fix memory leak in deferred reply buffer (#2615)
Set free method for deferred_reply list to properly clean up 
ClientReplyValue objects when the list is destroyed

Signed-off-by: Uri Yagelnik <uriy@amazon.com>
2025-09-17 11:14:44 +03:00
Roshan Khatri 73d5b0ed9b
Adds io-threads configs to PR-perf tests (#2598)
- Adds io-thread enabled perf-tests for pr
- changes the server and benchmark client cpu ranges so there are on
separate NUMA nodes of the metal machine.
- Also kill any servers that are active on the metal machine if anything
fails.
- Adds a benchmark wf to benchmark versions and publish on a issue id
provided:
<img width="340" height="449" alt="Screenshot 2025-09-11 at 12 14 28 PM"
src="https://github.com/user-attachments/assets/04f6a781-e163-4d6b-9b70-deedad15c9ef"
/>

- Comments on the issue with the full comparison like this:
 
<img width="936" height="1152" alt="Screenshot 2025-09-11 at 12 15
35 PM"
src="https://github.com/user-attachments/assets/e1584c8e-25dc-433f-a4d4-5b08d7548ddf"
/>

https://github.com/roshkhatri/valkey/pull/3#issuecomment-3282289440

---------

Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
2025-09-16 14:09:39 -07:00
Sarthak Aggarwal fab2a12c51
Increase wait time condition for New Master down consecutively test (#2612)
With #2604 merged, the `Node #10 should eventually replicate node #5`
started passing successfully with valgrind, but I guess we are seeing a
new daily failure from a `New Master down consecutively` test that runs
shortly after.

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
2025-09-16 10:55:27 +08:00
Sarthak Aggarwal 93d7ccab03
Fix accounting for dual channel RDB bytes in replication stats (#2602)
Resolves #2545 

Followed the steps to reproduce the issue, and was able to get non-zero
`total_net_repl_output_bytes`.

```
(base) ~/workspace/valkey git:[fix-bug-2545]
src/valkey-cli INFO | grep total_net_repl_output_bytes
total_net_repl_output_bytes:1788
```

---------

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
2025-09-15 17:08:59 -07:00
Vitali a47e8fa150
Expand wait condition time for slave selection test (#2604)
## Summary
- extend replication wait time in `slave-selection` test

```
*** [err]: Node #10 should eventually replicate node #5 in tests/unit/cluster/slave-selection.tcl
#10 didn't became slave of #5
```

## Testing
- `./runtest --single unit/cluster/slave-selection`
- `./runtest --single unit/cluster/slave-selection --valgrind`

Signed-off-by: Vitali Arbuzov <Vitali.Arbuzov@proton.me>
Signed-off-by: Binbin <binloveplay1314@qq.com>
Co-authored-by: Binbin <binloveplay1314@qq.com>
Co-authored-by: Harkrishn Patro <bunty.hari@gmail.com>
2025-09-13 15:53:07 +08:00
Jacob Murphy 2b7d5f7760
Make modules opt-in to atomic slot migration and add server events (#2593)
As discussed in https://github.com/valkey-io/valkey/issues/2579

Notably, I am exposing this feature as "Atomic Slot Migration" to
modules. If we want to call it something else, we should consider that
now (e.g.. "Replication-Based Slot Migration"?)

Also note: I am exposing both target and source node in the event. It is
not guaranteed that either target or source would be the node the event
fires on (e.g. replicas will fire the event after replica containment is
introduced). Even though it could be potentially inferred from CLUSTER
SLOTS, it should help modules parse it this way. Modules should be able
to infer whether it is occurring on primary/replica from `ctx` flags, so
not duplicating that here.

Closes #2579

---------

Signed-off-by: Jacob Murphy <jkmurphy@google.com>
2025-09-12 12:35:18 -07:00
Sarthak Aggarwal 9b11d3d9ed
Evict client only when limit is breached (#2596)
I believe we should evict the clients when the client eviction limit is
breached instead of _at_ the breach. I came across this function in the
failed [daily
test](https://github.com/valkey-io/valkey/actions/runs/17521272806/job/49765359298#step:6:7770),
which could possibly be related.

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
2025-09-11 17:19:05 +03:00
Ran Shidlansik 3213f48fd6
Increase frequency of time check during fields active expiration (#2595)
When we introduced the new Hash fields expiration functionality,
we decided to combine the current active expiration job between generic
keys and hash fields.
During that job we run a tight loop. In each loop iteration we scan over
maximum of 20 keys (with default expire effort) and try to "expire"
them.
For hash fields expiration job, the "expire" of a hash key, means
expiring maximum of 80 fields (with default expire effort).
The problem is that we might do much more work per each iteration of
hash fields expiration job.
The current code is shared between the 2 jobs, and currently we only
perform time check every 16 iterations.
as a result the CPU of fields active expiration can spike and consume
much higher CPU% than the current 25% bound allows.

Example:

Before this PR

| Scenario | AVG CPU | Time to expire all fields |

|----------------------------------------------------|---------|---------------------------|
| Expiring 10M volatile fields from a single hash | 20.18% | 26 seconds
|
| Expiring 10M volatile fields from 10K hash objects | 32.72% | 7
seconds |


After this PR
Scenario | AVG CPU | Time to expire all fields
| Scenario | AVG CPU | Time to expire all fields |

|----------------------------------------------------|---------|---------------------------|
| Expiring 10M volatile fields from a single hash | 20.23%. | 26 seconds
|
| Expiring 10M volatile fields from 10K hash objects | 20.76%. | 11
seconds |

*NOTE*
The change introduced here make the field job check the time every
iteration. We offer compile time option to use efficient time check
using TSC (X86) or VCR (ARM) on most modern CPU, so the impact is
expected to be low. Still, in order to avoid degradation for existing
workloads, the code change was made so it will not impact the existing
generic keys active expiration job.

---------

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-09-10 13:24:55 +03:00
Zhijun Liao cef8822f60
valkey-cli: Add word-jump navigation (Alt/Option/Ctrl + ←/→) (#2583)
Interactive use of valkey-cli often involves working with long keys
(e.g. MY:INCREDIBLY:LONG:keythattakesalongtimetotype). In shells like
bash, zsh, or psql, users can quickly move the cursor word by word with
**Alt/Option+Left/Right**, **Ctrl+Left/Right** or **Alt/Option+b/f**.
This makes editing long commands much more efficient.

Until now, valkey-cli (via linenoise) only supported single-character
cursor moves, which is painful for frequent key editing.

This patch adds such support, with simple code changes in linenoise. It
now supports both the Meta (Alt/Option) style and CSI (control sequence
introducer) style:

| | Meta style | CSI style (Ctrl) | CSI style (Alt) |
| --------------- | ---------- | ---------------- | --------- |
| move word left  | ESC b      | ESC [1;5D        | ESC [1;3D |
| move word right | ESC f      | ESC [1;5C        | ESC [1;3C |

Notice that I handle these two styles differently since people have
different preference on the definition of "what is a word".
Specifically, I define:
- "sub-word": just letters and digits. For example "my:namespace:key"
has 3 sub-words. This is handled by Meta style.
- "big-word": as any character that is not space. For example
"my:namespace:key" is just one single big-word. This is handled by CSI
style.


## How I verified

I'm using MacOS default terminal (`$TERM = xterm-256color`). I
customized the terminal keyboard setting to map option + left to `\033b`
, and ctrl + left to `\033[1;5D` so that I can produce both the Meta
style and CSI style. This code change should also work for
Linux/BSD/other terminal users.

Now the valkey-cli works like the following. `|` shows where the cursor
is currently at.

Press Alt + left (escape sequence `ESC b` ):

```
set cache:item itemid
                   |

set cache:item itemid
               |

set cache:item itemid
          |

set cache:item itemid
    |

set cache:item itemid
|
```

Press Ctrl + left (escape sequence `ESC [1;5D` ):

```
set cache:item itemid
                    |

set cache:item itemid
               |

set cache:item itemid
    |

set cache:item itemid
|
```

Press Alt + right (escape sequence `ESC f` ):

```
set cache:item itemid
|

set cache:item itemid
    |

set cache:item itemid
          |	

set cache:item itemid
               |

set cache:item itemid
                     |
```

Press Ctrl + right  (escape sequence `ESC [1;5C` ):

```
set cache:item itemid
|

set cache:item itemid
   |
    
set cache:item itemid
              |    

set cache:item itemid
                     |
```

---------

Signed-off-by: Zhijun <dszhijun@gmail.com>
2025-09-09 11:11:08 +02:00
Marvin Rösch 3b13a7cd13
Add cluster-announce-client-(port|tls-port) configs (#2429)
New config options:

 * cluster-announce-client-port
 * cluster-announce-client-tls-port

If enabled, clients will always get to see the configured port for a
node instead of the internally announced port(s), the same way that
`cluster-announce-client-ipv4` and `cluster-announce-client-ipv6` work.
Cluster-internal communication uses the non-client variant of these
options.

The configuration is propagated throughout the cluster using new ping
extensions.

Closes #2377

---------

Signed-off-by: Marvin Rösch <marvinroesch99@gmail.com>
Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-09-08 18:38:53 +02:00
Binbin 5a9ee5b944
Skip codeql-analysis ci on documentation changes as well (#2567)
Follow #2393.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-09-08 19:31:51 +08:00
Rain Valentine d9382bd9f3
ARM Neon SIMD optimization for hashtable findBucket() (#2573)
This PR resolves #2519. I worked with @ahmadbelb to get a pretty good
result. 😁 For `hashtableFind()` I measured 58% speed improvement on a
AWS t4g.2xlarge instance, and Ahmad measured 159% speed improvement on a
M1 Mac.

I'm still working on valkey-benchmark results for GET and SET
throughput.

I used Google Benchmark to make micro benchmarks that test 0% 50% and
100% hit rates for hashtableFind(). You can check out the test code in
this branch:
https://github.com/SoftlyRaining/valkey/tree/valkey-microbench

My AWS t4g.2xlarge micro benchmark results:
```
Scalar version:
Running ./valkey-microbench
Run on (8 X 243.75 MHz CPU s)
CPU Caches:
  L1 Data 64 KiB (x8)
  L1 Instruction 64 KiB (x8)
  L2 Unified 1024 KiB (x8)
  L3 Unified 32768 KiB (x1)
Load Average: 6.04, 2.10, 1.14
---------------------------------------------------------------------------------------------------
Benchmark                                                         Time             CPU   Iterations
---------------------------------------------------------------------------------------------------
BM_HashtableFind_0Miss/min_time:5.000/repeats:5                50.6 ns         50.6 ns    138342333
BM_HashtableFind_0Miss/min_time:5.000/repeats:5                50.4 ns         50.4 ns    138342333
BM_HashtableFind_0Miss/min_time:5.000/repeats:5                50.7 ns         50.7 ns    138342333
BM_HashtableFind_0Miss/min_time:5.000/repeats:5                50.4 ns         50.4 ns    138342333
BM_HashtableFind_0Miss/min_time:5.000/repeats:5                50.5 ns         50.4 ns    138342333
BM_HashtableFind_0Miss/min_time:5.000/repeats:5_mean           50.5 ns         50.5 ns            5
BM_HashtableFind_0Miss/min_time:5.000/repeats:5_median         50.5 ns         50.4 ns            5
BM_HashtableFind_0Miss/min_time:5.000/repeats:5_stddev        0.118 ns        0.118 ns            5
BM_HashtableFind_0Miss/min_time:5.000/repeats:5_cv             0.23 %          0.23 %             5
BM_HashtableFind_50Miss/min_time:5.000/repeats:5               56.6 ns         56.6 ns    123370046
BM_HashtableFind_50Miss/min_time:5.000/repeats:5               56.8 ns         56.8 ns    123370046
BM_HashtableFind_50Miss/min_time:5.000/repeats:5               55.8 ns         55.8 ns    123370046
BM_HashtableFind_50Miss/min_time:5.000/repeats:5               56.4 ns         56.4 ns    123370046
BM_HashtableFind_50Miss/min_time:5.000/repeats:5               56.3 ns         56.3 ns    123370046
BM_HashtableFind_50Miss/min_time:5.000/repeats:5_mean          56.4 ns         56.4 ns            5
BM_HashtableFind_50Miss/min_time:5.000/repeats:5_median        56.4 ns         56.4 ns            5
BM_HashtableFind_50Miss/min_time:5.000/repeats:5_stddev       0.363 ns        0.363 ns            5
BM_HashtableFind_50Miss/min_time:5.000/repeats:5_cv            0.64 %          0.64 %             5
BM_HashtableFind_100Miss/min_time:5.000/repeats:5              50.5 ns         50.5 ns    138420288
BM_HashtableFind_100Miss/min_time:5.000/repeats:5              50.4 ns         50.4 ns    138420288
BM_HashtableFind_100Miss/min_time:5.000/repeats:5              50.4 ns         50.4 ns    138420288
BM_HashtableFind_100Miss/min_time:5.000/repeats:5              50.5 ns         50.5 ns    138420288
BM_HashtableFind_100Miss/min_time:5.000/repeats:5              50.4 ns         50.4 ns    138420288
BM_HashtableFind_100Miss/min_time:5.000/repeats:5_mean         50.5 ns         50.4 ns            5
BM_HashtableFind_100Miss/min_time:5.000/repeats:5_median       50.4 ns         50.4 ns            5
BM_HashtableFind_100Miss/min_time:5.000/repeats:5_stddev      0.061 ns        0.062 ns            5
BM_HashtableFind_100Miss/min_time:5.000/repeats:5_cv           0.12 %          0.12 %             5

Neon version:
Running ./valkey-microbench
Run on (8 X 243.75 MHz CPU s)
CPU Caches:
  L1 Data 64 KiB (x8)
  L1 Instruction 64 KiB (x8)
  L2 Unified 1024 KiB (x8)
  L3 Unified 32768 KiB (x1)
Load Average: 0.35, 0.87, 0.72
---------------------------------------------------------------------------------------------------
Benchmark                                                         Time             CPU   Iterations
---------------------------------------------------------------------------------------------------
BM_HashtableFind_0Miss/min_time:5.000/repeats:5                30.7 ns         30.7 ns    226597253
BM_HashtableFind_0Miss/min_time:5.000/repeats:5                30.9 ns         30.9 ns    226597253
BM_HashtableFind_0Miss/min_time:5.000/repeats:5                30.7 ns         30.6 ns    226597253
BM_HashtableFind_0Miss/min_time:5.000/repeats:5                30.8 ns         30.8 ns    226597253
BM_HashtableFind_0Miss/min_time:5.000/repeats:5                30.6 ns         30.6 ns    226597253
BM_HashtableFind_0Miss/min_time:5.000/repeats:5_mean           30.7 ns         30.7 ns            5
BM_HashtableFind_0Miss/min_time:5.000/repeats:5_median         30.7 ns         30.7 ns            5
BM_HashtableFind_0Miss/min_time:5.000/repeats:5_stddev        0.118 ns        0.118 ns            5
BM_HashtableFind_0Miss/min_time:5.000/repeats:5_cv             0.38 %          0.38 %             5
BM_HashtableFind_50Miss/min_time:5.000/repeats:5               36.6 ns         36.6 ns    192568222
BM_HashtableFind_50Miss/min_time:5.000/repeats:5               35.2 ns         35.2 ns    192568222
BM_HashtableFind_50Miss/min_time:5.000/repeats:5               34.7 ns         34.7 ns    192568222
BM_HashtableFind_50Miss/min_time:5.000/repeats:5               36.2 ns         36.2 ns    192568222
BM_HashtableFind_50Miss/min_time:5.000/repeats:5               36.0 ns         36.0 ns    192568222
BM_HashtableFind_50Miss/min_time:5.000/repeats:5_mean          35.7 ns         35.7 ns            5
BM_HashtableFind_50Miss/min_time:5.000/repeats:5_median        36.0 ns         36.0 ns            5
BM_HashtableFind_50Miss/min_time:5.000/repeats:5_stddev       0.790 ns        0.789 ns            5
BM_HashtableFind_50Miss/min_time:5.000/repeats:5_cv            2.21 %          2.21 %             5
BM_HashtableFind_100Miss/min_time:5.000/repeats:5              30.5 ns         30.5 ns    229190934
BM_HashtableFind_100Miss/min_time:5.000/repeats:5              31.4 ns         31.4 ns    229190934
BM_HashtableFind_100Miss/min_time:5.000/repeats:5              30.7 ns         30.7 ns    229190934
BM_HashtableFind_100Miss/min_time:5.000/repeats:5              30.4 ns         30.4 ns    229190934
BM_HashtableFind_100Miss/min_time:5.000/repeats:5              31.1 ns         31.1 ns    229190934
BM_HashtableFind_100Miss/min_time:5.000/repeats:5_mean         30.8 ns         30.8 ns            5
BM_HashtableFind_100Miss/min_time:5.000/repeats:5_median       30.7 ns         30.7 ns            5
BM_HashtableFind_100Miss/min_time:5.000/repeats:5_stddev      0.415 ns        0.414 ns            5
BM_HashtableFind_100Miss/min_time:5.000/repeats:5_cv           1.35 %          1.34 %             5
```

Ahmad's M1 Mac micro benchmark results:
```
Scalar version:
Running ./src/valkey-microbench
Run on (8 X 24 MHz CPU s)
CPU Caches:
  L1 Data 64 KiB
  L1 Instruction 128 KiB
  L2 Unified 4096 KiB (x8)
Load Average: 4.84, 8.36, 8.53
---------------------------------------------------------------------------------------------------
Benchmark                                                         Time             CPU   Iterations
---------------------------------------------------------------------------------------------------
BM_HashtableFind_0Miss/min_time:5.000/repeats:5                29.8 ns         29.5 ns    238219371
BM_HashtableFind_0Miss/min_time:5.000/repeats:5                30.6 ns         29.9 ns    238219371
BM_HashtableFind_0Miss/min_time:5.000/repeats:5                29.9 ns         29.6 ns    238219371
BM_HashtableFind_0Miss/min_time:5.000/repeats:5                29.7 ns         29.4 ns    238219371
BM_HashtableFind_0Miss/min_time:5.000/repeats:5                29.3 ns         29.3 ns    238219371
BM_HashtableFind_0Miss/min_time:5.000/repeats:5_mean           29.8 ns         29.5 ns            5
BM_HashtableFind_0Miss/min_time:5.000/repeats:5_median         29.8 ns         29.5 ns            5
BM_HashtableFind_0Miss/min_time:5.000/repeats:5_stddev        0.468 ns        0.250 ns            5
BM_HashtableFind_0Miss/min_time:5.000/repeats:5_cv             1.57 %          0.85 %             5
BM_HashtableFind_50Miss/min_time:5.000/repeats:5               30.8 ns         30.7 ns    228068003
BM_HashtableFind_50Miss/min_time:5.000/repeats:5               31.3 ns         31.0 ns    228068003
BM_HashtableFind_50Miss/min_time:5.000/repeats:5               31.3 ns         31.0 ns    228068003
BM_HashtableFind_50Miss/min_time:5.000/repeats:5               30.9 ns         30.8 ns    228068003
BM_HashtableFind_50Miss/min_time:5.000/repeats:5               31.4 ns         31.1 ns    228068003
BM_HashtableFind_50Miss/min_time:5.000/repeats:5_mean          31.1 ns         30.9 ns            5
BM_HashtableFind_50Miss/min_time:5.000/repeats:5_median        31.3 ns         31.0 ns            5
BM_HashtableFind_50Miss/min_time:5.000/repeats:5_stddev       0.288 ns        0.134 ns            5
BM_HashtableFind_50Miss/min_time:5.000/repeats:5_cv            0.92 %          0.43 %             5
BM_HashtableFind_100Miss/min_time:5.000/repeats:5              29.3 ns         29.3 ns    237946966
BM_HashtableFind_100Miss/min_time:5.000/repeats:5              29.4 ns         29.4 ns    237946966
BM_HashtableFind_100Miss/min_time:5.000/repeats:5              29.4 ns         29.4 ns    237946966
BM_HashtableFind_100Miss/min_time:5.000/repeats:5              29.3 ns         29.3 ns    237946966
BM_HashtableFind_100Miss/min_time:5.000/repeats:5              29.4 ns         29.4 ns    237946966
BM_HashtableFind_100Miss/min_time:5.000/repeats:5_mean         29.4 ns         29.4 ns            5
BM_HashtableFind_100Miss/min_time:5.000/repeats:5_median       29.4 ns         29.4 ns            5
BM_HashtableFind_100Miss/min_time:5.000/repeats:5_stddev      0.058 ns        0.061 ns            5
BM_HashtableFind_100Miss/min_time:5.000/repeats:5_cv           0.20 %          0.21 %             5

NEON version:
Running ./src/valkey-microbench
Run on (8 X 24 MHz CPU s)
CPU Caches:
  L1 Data 64 KiB
  L1 Instruction 128 KiB
  L2 Unified 4096 KiB (x8)
Load Average: 4.56, 5.43, 7.09
---------------------------------------------------------------------------------------------------
Benchmark                                                         Time             CPU   Iterations
---------------------------------------------------------------------------------------------------
BM_HashtableFind_0Miss/min_time:5.000/repeats:5                11.6 ns         11.6 ns    596879005
BM_HashtableFind_0Miss/min_time:5.000/repeats:5                11.9 ns         11.7 ns    596879005
BM_HashtableFind_0Miss/min_time:5.000/repeats:5                11.9 ns         11.8 ns    596879005
BM_HashtableFind_0Miss/min_time:5.000/repeats:5                11.7 ns         11.7 ns    596879005
BM_HashtableFind_0Miss/min_time:5.000/repeats:5                11.7 ns         11.7 ns    596879005
BM_HashtableFind_0Miss/min_time:5.000/repeats:5_mean           11.8 ns         11.7 ns            5
BM_HashtableFind_0Miss/min_time:5.000/repeats:5_median         11.7 ns         11.7 ns            5
BM_HashtableFind_0Miss/min_time:5.000/repeats:5_stddev        0.119 ns        0.069 ns            5
BM_HashtableFind_0Miss/min_time:5.000/repeats:5_cv             1.01 %          0.59 %             5
BM_HashtableFind_50Miss/min_time:5.000/repeats:5               12.0 ns         11.9 ns    592642763
BM_HashtableFind_50Miss/min_time:5.000/repeats:5               11.9 ns         11.9 ns    592642763
BM_HashtableFind_50Miss/min_time:5.000/repeats:5               12.0 ns         12.0 ns    592642763
BM_HashtableFind_50Miss/min_time:5.000/repeats:5               12.2 ns         12.1 ns    592642763
BM_HashtableFind_50Miss/min_time:5.000/repeats:5               11.9 ns         11.9 ns    592642763
BM_HashtableFind_50Miss/min_time:5.000/repeats:5_mean          12.0 ns         12.0 ns            5
BM_HashtableFind_50Miss/min_time:5.000/repeats:5_median        12.0 ns         11.9 ns            5
BM_HashtableFind_50Miss/min_time:5.000/repeats:5_stddev       0.113 ns        0.069 ns            5
BM_HashtableFind_50Miss/min_time:5.000/repeats:5_cv            0.94 %          0.58 %             5
BM_HashtableFind_100Miss/min_time:5.000/repeats:5              11.9 ns         11.8 ns    590288406
BM_HashtableFind_100Miss/min_time:5.000/repeats:5              11.9 ns         11.9 ns    590288406
BM_HashtableFind_100Miss/min_time:5.000/repeats:5              12.0 ns         11.9 ns    590288406
BM_HashtableFind_100Miss/min_time:5.000/repeats:5              11.9 ns         11.8 ns    590288406
BM_HashtableFind_100Miss/min_time:5.000/repeats:5              12.6 ns         12.1 ns    590288406
BM_HashtableFind_100Miss/min_time:5.000/repeats:5_mean         12.1 ns         11.9 ns            5
BM_HashtableFind_100Miss/min_time:5.000/repeats:5_median       11.9 ns         11.9 ns            5
BM_HashtableFind_100Miss/min_time:5.000/repeats:5_stddev      0.303 ns        0.135 ns            5
BM_HashtableFind_100Miss/min_time:5.000/repeats:5_cv           2.51 %          1.14 %             5
```

---------

Signed-off-by: Rain Valentine <rsg000@gmail.com>
2025-09-07 09:52:21 +03:00
Viktor Söderqvist 9d10abfbde
Relaxed RDB check for foreign RDB formats (#2543)
In #1604, we attempt to read future Valkey RDB formats, but we rejected
foreign RDB formats, because of the risk that the opcodes and types
added by other projects collide with the new types and opcodes added in
recent Valkey versions.

This change accepts foreign RDB versions but limits the types and
opcodes to the ones that we can understand, to prevent misinterpretation
of types/opcodes which could lead to undefined behavior. If unsupported
RDB types or opcodes are seen, we error out.

Additional changes:

* Improve error reporting when encountering unknown RDB types in relaxed
version check mode.
* Tests for loading various RDB files.
* Improvement to valkey-check-rdb to accept future and foreign RDB
versions, including tests.

---------

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-09-04 17:19:52 +02:00
Kyle J. Davis 811a64407b
Un-deprecate commands (#2546)
As per #2459, this PR removes deprecation and `deprecated_since`
element and `DEPRECATED` doc flag from commands. Closes #2459.

---------

Signed-off-by: Kyle J. Davis <kyledvs@amazon.com>
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
2025-09-03 12:39:49 -07:00
Ted Lyngmo 971cdb7836
Don't use AVX2 instructions if the CPU don't support it (#2571)
`bitops.c`: `serverPopcount()` used `popcountAVX2()`, which as the name
implies requires AVX2 support, on AVX-only machines, causing an "illegal
instruction" error.

Added a `__builtin_cpu_supports("avx2")` check and falling back to the
platform agnostic version if AVX2 is not supported.

Fixes #2570

Signed-off-by: Ted Lyngmo <ted@lyncon.se>
2025-09-03 18:25:33 +02:00
Ran Shidlansik 47d2203c1b
Store number of keys with volatile items per slot in RDB aux field and pre-size hashtables on load (#2572)
In Valkey 9.0 we added HFE support which is currently using a per slot
hashtable in order to track keys (hash objects) which contains volatile
fields. in order to optimize the RDB load we should use the same method
for expires and generic keys kvstores.

---------

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>
Signed-off-by: Binbin <binloveplay1314@qq.com>
Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
Co-authored-by: Binbin <binloveplay1314@qq.com>
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
2025-09-03 10:03:49 +03:00
asagegeLiu 5ccedafe3f
Delete the previous comment explaining why changed latency from 5 to 40 (#2574)
Delete the out-of-date comment explaining why changed latency from 5 to
40 in #2421 , which is a leftover of #2553

Signed-off-by: Alina Liu <liusalisa6363@gmail.com>
2025-09-02 16:36:02 -07:00
Binbin eebed88429
Remove deny_blocking check in moduleBlockClient, cleanup code and doc (#2215)
This flag.deny_blocking check causes some code to becomde dead code.
Some of the code below checks islua or ismulti, but they are marked
with flag.deny_blocking and will return early.

In addition, cleanup some documents, some of them are inaccurate,
and restore the code of blocking_auth_cb in tests/modules/auth.c,
this reply should be returned from core. The reply used to from the
core and was changed to from the module in #1819, and now it is from
the core again.

Cleanup some dead code around #1819.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-09-02 09:08:14 +08:00
Ricardo Dias b44de37b7d
New module API event for tracking authentication attempts (#2237)
In this commit we introduce a new module API event called
`ValkeyModuleEvent_AuthenticationAttempt` to track successful/failed
authentication attempts.

This event will fill a struct, called `ValkeyModuleAuthenticationInfo`,
with the client ID of the connection, the username, the module name that
handle the authentication event, and the result of the authentication
attempt.

Fixes: #2211

---------

Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
2025-09-01 15:02:26 +01:00
Ricardo Dias c039b691c4
Fix module context object re-usage in scripting engines (#2358)
This commit refactors the scripting engine to support multiple cached
module contexts per engine, rather than relying on a single cached
`ValkeyModuleCtx` object.

Previously, having only one cached context object caused data races over
the state stored in the context object, because it's possible that a
script that is running for a long time to yield and the server event
loop may call the `scriptingEngineCallGetMemoryInfo` function to get the
scripting engine memory information, which re-uses the same cached
context object. Another possible data-race is caused by the asynchronous
scripts flush, which calls the `scriptingEngineCallFreeFunction`
function in an background thread, and also re-uses the cached context
object.

To address this, a cache array of module contexts was introduced in the
scripting engine structure, with each slot dedicated to a specific use
case—such as script execution, memory info queries, or function freeing.

---------

Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
2025-09-01 11:35:37 +01:00
Binbin e19fc4d1e6
Add jacob as a commiter (#2566)
Add @murphyjacob4 as one of the folks with write permissions on the
Valkey repo.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-08-30 18:06:12 +08:00
asagegeLiu d6e011f955
Reduce active defrag test latency by lowering hit threshold (#2553)
Changed the defrag hit threshold from 512 to 1 in the `defragLaterStep`
& `defragStageKvstoreHelper` function to reduce defrag latency. (idea
from Jim)

1. Previously, the defrag loop would continue processing up to 512
successful defragmentations before checking if the time limit was
exceeded. Now it checks after every single successful allocation move.
2. The trade-off is slightly more frequent time checks, but the time
check (~19ns) is negligible compared to the actual defragmentation work
(even a single 8-byte reallocation takes ~43ns and the
allocatorShouldDefrag function call takes ~49ns per block). This
overhead is minimal compared to the latency improvement gained from
better time management during active defragmentation.
3. Also revert the change from
https://github.com/valkey-io/valkey/pull/2421/files to test.
4. Solved compilation issue with unsigned by changing the type of the
local variable `prev_defragged `to match the type of
`server.stat_active_defrag_hits`

Closes #2444

---------

Signed-off-by: Alina Liu <alinalq@dev-dsk-alinalq-2b-2db84246.us-west-2.amazon.com>
Co-authored-by: Alina Liu <alinalq@dev-dsk-alinalq-2b-2db84246.us-west-2.amazon.com>
2025-08-29 21:32:33 +02:00
zhaozhao.zz 6774d173c4
Fix the issue of incorrect commandlog metrics in the script (#2565)
In script, client will be replaced with its caller, so commandlog needs
to use the metrics of the client that currently executing the command.

Signed-off-by: zhaozhao.zz <zhaozhao.zz@alibaba-inc.com>
2025-08-29 10:47:10 +08:00
withRiver 3db75f5551
Reset cluster related stats in CONFIG RESETSTATS (#2458)
Previously CONFIG RESETSTATS only resets the slot statistics in
cluster part, this PR makes it reset cluster bus messages at well.
Additionally, we also reset stat_cluster_links_buffer_limit_exceeded.

Now we will reset:
- cluster_stats_messages_sent*
- cluster_stats_messages_received*
- total_cluster_links_buffer_limit_exceeded

Closes #2439.

Signed-off-by: Hongrui <2086160503@qq.com>
2025-08-29 10:21:23 +08:00
Binbin 88cd2086ae
Split SLOT_EXPORT_AUTHENTICATING into SEND and READ to avoid synchronous reading of auth response (#2494)
The old SLOT_EXPORT_AUTHENTICATING added in #1949, when processed by the source node,
we will send the AUTH command and then reads the response. If the target node is blocked
during this process, the source node will also be blocked. We should use a read handler
to handle this. We split SLOT_EXPORT_AUTHENTICATING into SLOT_EXPORT_SEND_AUTH and
SLOT_EXPORT_READ_AUTH_RESPONSE to avoid this issue.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-08-29 10:20:50 +08:00
Marc Jakobi cf90acbc61
Correct path to gen-test-certs.sh in README.md (#2554)
Signed-off-by: Marc Jakobi <marc.jakobi@tiko.energy>
2025-08-29 10:19:46 +08:00
Binbin 39fade42d1
Do not migrate function in new atomic slot migration (#2547)
If all cluster nodes have functions, slot migration will fail
since the target will return the function already exists error
when doing the FUNCTION LOAD.

And in addition, the target's replica could panic when it executes
the FUNCTION LOAD propagated from the primary (see
propagation-error-behavior).

Introduced in #1949.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-08-29 10:18:45 +08:00
Binbin 682fc0b419
Remove the trailing chars of the --cluster reshard log message in valkey-cli (#2560)
It prints an extra ": " after the message, which is a bit weird, i
thought it was printing cluster-bus port information, but it was not.
```
Moving slot 5458 from 127.0.0.1:30001 to 127.0.0.1:30003:
Moving slot 5459 from 127.0.0.1:30001 to 127.0.0.1:30003:
Moving slot 5460 from 127.0.0.1:30001 to 127.0.0.1:30003:
```

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-08-29 10:17:10 +08:00
Viktor Söderqvist 06cefc181a
Attempt to fix sub-replica getting out of sync (#2548)
Try to fix the failures seen for `test "PSYNC2 #3899 regression: verify
consistency"`.

This change resets the query buffer parser state in
`replicationCachePrimary()` which is called when the connection to the
primary is lost. Before #2092, this was done by `resetClient()`.

The solution was inspired by the discussion about the regression
mentioned (discussion from 2017) and the related commits from that time:
6bc6bd4c38,
469d6e2b37,
c180bc7d98.

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-08-27 07:24:00 +02:00
Binbin 1ba622bad5
Update tests/xxx/tmp/.gitignore to ignore everything (#2542)
Ingore everything on its own directory.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-08-27 10:17:48 +08:00
Rain Valentine 40fe422899
deflake test: relax time requirement in hash ttl test (#2537)
tiny change. It failed once for me (a little time passed and it returned
2 seconds instead of 3), so I figured it's probably a little flaky for
others too

---------

Signed-off-by: Rain Valentine <rsg000@gmail.com>
2025-08-26 09:02:14 +02:00
Adam Fowler 3086e61d4b
Update reply schema for LMOVE and BLMOVE (#2541)
Both LMOVE and BLMOVE can return null values because the source key is
empty.
PR changes include 
- change the LMOVE reply_schema to include the possibility of a nil
return value
- Add comment to BLMOVE reply_schema to indicate it can return nil
because the source does not exist

This fixes #2532

---------

Signed-off-by: Adam Fowler <adamfowler71@gmail.com>
Co-authored-by: Ran Shidlansik <ranshid@amazon.com>
2025-08-26 09:01:12 +02:00
Ted Lyngmo 9d682bad0b
bio.c: Organize all worker data in a struct (#2530)
This gets rid of the need to use a void* as a carrier for the worker
number. Instead a pointer to the relevant worker data is passed to the
started thread.

Fixes #2529

---------

Signed-off-by: Ted Lyngmo <ted@lyncon.se>
2025-08-25 12:24:34 +02:00
Madelyn Olson 9aeab8ccba
Consistently use static_assert across code (#2538)
In C99, we had to use `#define static_assert(expr, lit) extern char
__static_assert_failure[(expr) ? 1 : -1]` for static assertions.
However, we now have native support for static_assert with
_Static_assert. Previously one of the correct #defines was getting
called first, setting it to _Static_assert, however after
65215e5378
the first import defining the symbol was "serverassert.h", which
included the old style.

This change removes all unnecessary imports and always defines
static_assert as _Static_assert.

Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
2025-08-24 14:59:13 +02:00
Binbin 8c55547999
Don't allow slot migration to myself node (#2497)
This may result in meaningless slot migration job, we should
return an error to user in advance to avoid operation error.
Also `by myself` is not correct English grammar and `myself`
is a internal code terminology, changed to `by this node`.

Was introduced in #1949.

---------

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-08-22 15:08:20 -04:00
Viktor Söderqvist 09fb436cf0
Optimize pipelining by parsing and prefetching multiple commands (#2092)
Instead of parsing only one command per client before executing it,
parse multiple commands from the query buffer and batch-prefetch the
keys accessed by the commands in the queue before executing them.

This is an optimization for pipelined commands, both with and without
I/O threads. The optimization is currently disabled for the replication
stream, due to failures (probably caused by how the replication offset
is calculated based on the query buffer offset).

* When parsing commands from the input buffer, multiple commands are
parsed and stored in a command queue per client.
* In single-threaded mode (I/O threads off) keys are batch-prefetched
before the commands in the queue are executed. Multi-key commands like
MGET, MSET and DEL benefit from this even if pipelining is not used.
* Prefetching when I/O threads are used does prefetching for multiple
clients in parallel. This code takes client command queues into account,
improving prefetching when pipelining is used. The batch size is
controlled by the existing config `prefetch-batch-max-size` (default
16), which so far only was used together with I/O threads. The config is
moved to a different section in `valkey.conf`.
* When I/O threads are used and the maximum number of keys are
prefetched, a client's command is executed, then the next one in the
queue, etc. If there are more commands in the queue for which the keys
have not been prefetched (say the client sends 16 pipelined MGET with 16
keys in each) keys for the next few commands in the queue are prefetched
before the commands is executed if prefetching has not been done for the
next command. (This utilizes the code path used in single-threaded
mode.)

Code improvements:

* Decoupling of command parser state and command execution state:
  * The variables reqtype, multibulklen and bulklen refer to the current
    position in the query buffer. These are no longer reset in resetClient
    (which runs after each command being executed). Instead, they are
    reset in the parser code after each completely parsed command.
  * The command parser code is partially decoupled from the client struct.
    The query buffer is still one per client, but the resulting argument
    vector is stored in caller-defined variables.

Fixes #2044

---------

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
2025-08-22 16:02:27 +02:00
Binbin ac515159a1
Remove debug logging from cluster-flush-slot.tcl (#2535)
I believe this was a debug log at the time, and its printing
was quite annoying locally. The test is quite simple so i think
we can just remove it.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-08-22 17:19:44 +08:00
Roshan Khatri a68040789e
Adds benchmark on demand workflow (#2442)
# Adds On-demand Benchmark Workflow

This PR introduces a new GitHub Actions workflow that enables on-demand
performance benchmarking for pull requests through label-based triggers.
This uses the new framework
[valkey-perf-benchmark](https://github.com/valkey-io/valkey-perf-benchmark)
developed for standard benchmarks and PR benchmarking.

## What is being added by this PR

- **Workflow File**: `.github/workflows/benchmark_on_label.yml`
- **Trigger**: Activated when specific labels are added to PRs
- **Supported Labels**:
  - `run-benchmark` - Runs standard benchmarks
  
## Features

### On-Demand Execution
- Benchmarks run only when explicitly requested via PR labels
- No automatic execution to avoid unnecessary resource usage

### Performance Comparison
- Compares PR performance against the `unstable` baseline
- Generates detailed comparison reports
- Posts results directly as PR comments for easy review

### Flexible Configuration
- Currently uses github runners by will use dedicated performance
runners (`ec2-perf-ubuntu-24`)
- Configurable benchmark suites via JSON config files

### Artifact Management
- Uploads benchmark results as workflow artifacts
- Preserves both baseline and PR metrics for analysis
- Includes comparison markdown for offline review

### Automatic Cleanup
- Removes trigger labels after execution (success or failure)
- Prevents accidental re-runs from stale labels

## Usage

To run benchmarks on a PR:

1. Add the `run-benchmark` label for standalone and cluster, non-tls
mode
2. Wait for the workflow to complete
3. Review results in the automated PR comment

This workflow enhances our CI/CD pipeline by providing easy access to
performance testing without impacting regular development workflows.

---------

Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
Signed-off-by: Roshan Khatri <117414976+roshkhatri@users.noreply.github.com>
Co-authored-by: Harkrishn Patro <bunty.hari@gmail.com>
2025-08-21 17:31:17 -07:00
Allen Samuels d2eee78a15
Module API: Add READONLY flag to ClientInfo.flags output structure (#2522)
* Add a flag `VALKEYMODULE_CLIENTINFO_FLAG_READONLY` to
  `ValkeyModuleClientInfo.flags` and set it in
  `ValkeyModule_GetClientInfoById()`.
* Add an optimization for accessing the current client by ID, to avoid
  looking it up in a radix tree.

Closes #2487

---------

Signed-off-by: Allen Samuels <allenss@amazon.com>
2025-08-21 22:09:22 +02:00
Hanxi Zhang 85a0a017d2
Wait for log message occurrence in module test on message received (#2517)
## Problem
Test `Cluster module receive message API -
VM_RegisterClusterMessageReceiver` fails sporadically in CI with
"Expected '2' to be equal to '1'". The test failed in Daily CI 4 days
ago but hasn't failed since, indicating flaky behavior that I cannot
reproduce locally.


## Hypothesis
The failing line `assert_equal 2 [count_log_message 0 "* <cluster> DONG
(type 2) RECEIVED*"]` counts DONG message entries in node 0's log file
and expects exactly 2. The failure suggests a possible race condition
where there's a timing gap between when cluster statistics are updated
and when the corresponding log entries become visible in the log file.


## Solution
Add `wait_for_condition` to ensure log messages are written before
checking count:

```tcl
wait_for_condition 50 100 {
    [count_log_message 0 "* <cluster> DONG (type 2) RECEIVED*"] eq 2
} else {
    fail "node 1 didn't log 2 DONG messages"
}

---------

Signed-off-by: Hanxi Zhang <hanxizh@amazon.com>
Co-authored-by: Harkrishn Patro <bunty.hari@gmail.com>
2025-08-21 12:34:27 -07:00
Sarthak Aggarwal b8357e5be8
Fix slot range lists overlap to rewind the nested list again (#2527)
In the current implementation, the second list was never rewound again
once it was iterated. So for the first element of `ranges1`, `ranges2`
was iterated fully. But when the second element of `ranges1` was
processed, the `ranges2` was not rewound again.

With this change, for every element of `ranges1`, we start from the
beginning for `ranges2`

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
2025-08-21 11:02:29 -07:00
Björn Svensson 2db4eeb1fc Remove temporary build correction for RDMA and libvalkey 0.1.0
Signed-off-by: Björn Svensson <bjorn.a.svensson@est.tech>
2025-08-21 12:35:10 +02:00
Björn Svensson 11780db513 Update deps/libvalkey to version 0.2.1
Squashed 'deps/libvalkey/' changes from abcd27fbf..b012f8e85

b012f8e85 Release 0.2.1
9e10acbf7 Fix duplicate Acks for RDMA events. (#229)
1eadedf48 Remove the unused valDup API from dict
a449f0ea1 Don't expose internal functions in shared libraries (#205)
178e350c7 Cluster code cleanup (#216)
4020396c8 Fix `unused-parameter` warning when building with `NDEBUG` (#212)
99aa158bc Use existing connections for blocking slotmap updates (#199)
969a8c546 Fix dependency issue with RDMA (#201)
...

git-subtree-dir: deps/libvalkey
git-subtree-split: b012f8e85a9cc2c68bc2be982a4f6545c15042f0

Signed-off-by: Björn Svensson <bjorn.a.svensson@est.tech>
2025-08-21 12:35:10 +02:00
asagegeLiu 84fd626419
Refactor scanLaterList to fix latency issue (#2514) 2025-08-20 17:18:44 -07:00
Ted Lyngmo c6c91d152f
Fix assumptions that pthread functions set errno (#2526)
pthread functions return the error instead of setting errno.

Fixes #2525

Signed-off-by: Ted Lyngmo <ted@lyncon.se>
2025-08-20 22:48:31 +02:00
Sarthak Aggarwal ab3ee2ca95
Fix total test count while running over loop (#2524)
Command: `./runtest --single unit/bitops --loops 3`

Unstable

```
[ignore]: large memory flag not provided
[-1/1 done]: unit/bitops (4 seconds)
[ignore]: large memory flag not provided
[0/1 done]: unit/bitops (4 seconds)
[ignore]: large memory flag not provided
[1/1 done]: unit/bitops (4 seconds)

                   The End

Execution time of different units:
  4 seconds - unit/bitops
  4 seconds - unit/bitops
  4 seconds - unit/bitops
```

After fix

```
[1/3 done]: unit/bitops (4 seconds)
[ignore]: large memory flag not provided
[2/3 done]: unit/bitops (4 seconds)
[ignore]: large memory flag not provided
[3/3 done]: unit/bitops (4 seconds)

                   The End

Execution time of different units:
  4 seconds - unit/bitops
  4 seconds - unit/bitops
  4 seconds - unit/bitops
```

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
2025-08-20 11:16:37 -07:00
Ran Shidlansik 23112fad87
fix hsetex handling of wrong number of fields (#2509)
currently hsetex is verifying the number of fields matches the provided
number of fields by using div. instead it can match to the
multiplication in order to prevent rounding the verified value down.

---------

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
2025-08-20 14:54:11 +03:00
Ran Shidlansik 39d5c6e6ee
Fix vset unittest compilation warning for bad signedness comparison (#2523)
Signed-off-by: Ran Shidlansik <ranshid@amazon.com>
2025-08-20 11:34:58 +02:00
Harkrishn Patro 3b6961814b
Update pong_received time via gossip only if the node is healthy from our view (#2512) 2025-08-19 21:38:46 -07:00
Harkrishn Patro e3bfc5f8c7
Atomically update cluster and replication layer while marking self node as primary (#2510)
Fixes: #2460

With this change we avoid divergence in cluster and replication layer
view. I've observed node can be marked as primary in cluster while it
can be marked as replica in replication layer view and have a active
replication link. Without this change, we used to end up in a invalid
replica chain in replication layer.

---------

Signed-off-by: Harkrishn Patro <harkrisp@amazon.com>
2025-08-19 17:28:51 -07:00
Seungmin Lee c8548f65d2
Skip failure reports for already failed nodes (#2434)
This change avoids additional failure report creation if the node is
already marked as failed. The failure report count has never been used
after a node has been marked as failed. So, there is no value addition
in maintaining it further. This reduces operation of both add and delete
failure report. Hence, the performance benefit.

We can observe an avg. of 10% reduction in p99 CPU utilization (in a 2000
nodes cluster (1000 primary/ 1000 replica) with 330 nodes in failed
state with this change.

---------

Signed-off-by: Seungmin Lee <sungming@amazon.com>
2025-08-19 15:24:26 -07:00
Hanxi Zhang eae23babe6
Add auto-author-assign workflow (#2410)
Fixes #2372

## Description

This PR adds an automated workflow that assigns PR authors to their own
pull requests when opened or reopened.

## Changes

- Added `.github/workflows/auto-author-assign.yml` workflow
- Uses `toshimaru/auto-author-assign@v2.1.1` action
- Triggers on `pull_request_target` events (opened, reopened)
- Requires `pull-requests: write` permission

## Benefits

- Improves PR tracking and organization
- Automatically assigns responsibility to PR authors
- Reduces manual assignment overhead for maintainers
- Helps contributors track their own PRs more easily

## Testing

 **Tested on my fork before submission:**

1. Merged the workflow into my fork's unstable branch
2. Created a test PR within my fork
3. Verified that I was automatically assigned as the assignee
4. Screenshot showing successful auto-assignment:
<img width="1278" height="684" alt="Screenshot 2025-08-01 at 3 39 05 PM"
src="https://github.com/user-attachments/assets/9ad643be-5eac-4ad6-bec7-184cf22e9cbd"
/>

The workflow executed successfully and assigned me to the test PR as
expected.

---------

Signed-off-by: Hanxi Zhang <hanxizh@amazon.com>
2025-08-19 13:49:25 -07:00
Binbin a97d584cc2
Fix slot-info expand not working, kvstoreHashtableExpand always creat… (#2466)
If we want to expand kvstoreHashtableExpand, we need to make sure the
hashtable exists. Currently, when processing RDB slot-info, our expand
has no effect because the hashtable does not exist (we initialize it only
when we need it).

We also update kvstoreExpand to use the kvstoreHashtableExpand to make
sure there is only one code path. Also see #1199 for more details.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-08-19 21:01:56 +08:00
Ran Shidlansik bb81f8d14f
fix hash ignore ttl management during active expiry (#2505)
several fixes:
1. fix not using bool input type for hashTypeIgnoreTTL - probably lost
during the 3 HFE PR merges
2. remove vset change hashtable encoding to single - The current code is
a bug. The entry is probably expired (or about to be expired soon) so we
can leave it as a hashtable till it does.
3. enable incremental rehashing for volatile item keys kvstore - This is
the center issue of this PR. without it the activeexpiry might not scan
the kvstore which is very fragmented with lots of empty buckets.

---------

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>
2025-08-19 11:31:37 +03:00
Binbin fb9503bfd1
CLUSTER SYNCSLOTS ESTABLISH added source node == myself check (#2500)
Minor cleanup.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-08-19 10:26:58 +08:00
Viktor Söderqvist 2ca0dd8781
Make cluster failover delay relative to node timeout (#2449)
In clusters with a very short node timeout such as 2-3 seconds, the
extra failover delay of 500-1000 milliseconds (500 + random value 0-500;
total 750 on average) before initiating a failover is a significant
extra downtime to the cluster. This PR makes this delay relative to node
timeout, using a shorter failover delay for a smaller configured node
timeout. The formula is `fixed_delay = min(500, node_timeout / 30)`.

| Node timeout  | Fixed failover delay |
|---------------|----------------------|
| 15000 or more | 500 (same as before) |
| 7500          | 250                  |
| 3000          | 100                  |
| 1500          | 50                   |

Additional change: Add an extra 500ms delay to new replicas that may not
yet know about the other replicas. This avoids the scenario where a new
replica with no data wins the failover. This change turned out to be
needed to for the stability of some test cases.

The purposes of the failover delay are

1. Allow FAIL to propagate to the voting primaries in the cluster
2. Allow replicas to exchange their offsets, so they will have a correct
view of their own rank.

A third (undocumented) purpose of this delay is to allow newly added
replicas to discover other replicas in the cluster via gossip and to
compute their rank, to realize it's are not the best replica. This case
is mitigated by adding another 500ms delay to new replicas, i.e. if it
has replication offset 0.

A low node timeout only makes sense in fast networks, so we can assume
that the above needs less time than in a cluster with a higher node
timeout.

These delays don't affect the correctness of the algorithm. They are
just there to increase the probability that a failover will succeed by
making sure that the FAIL message has enough time to propagate in the
cluster and to the random part is to reduce the probability that two
replicas initiates the failover at the same time.

The typical use case is when data consistency matters and writes can't
be skipped. For example, in some application, we buffer writes in the
application during node failures to be able to apply them when the
failover is completed. The application can't buffer them for a very long
time, so we need the cluster to be up again within e.g. 5 seconds from
the time a node starts to fail.

I hope this PR can be considered safer than #2227, although the two
changes are orthogonal.

Part of issue #2023.

---------

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-08-18 15:34:51 +02:00
Binbin 7a5c0d0ebf
Fix memory leak in saveSnapshotToConnectionSockets (#2503)
We now pass in rdbSnapshotOptions options in this function, and
options.conns
is now malloc'ed in the caller side, so we need to zfree it when
returning early
due to an error. Previously, conns was malloc'ed after the error
handling, so we
don't have this.

Introduced in #1949.

---------

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-08-18 12:59:40 +02:00
yzc-yzc 390e0c9555
CONFIG GET command return sorted output (#2493)
Previously, the config names and values were stored in a temporary dict.
This ensures that no duplicates are returned, but it also makes the
order random.

In this PR, the config names and values still stored in the temporary
dict, but then they are copied to an array, which is sorted, before the
reply is sent.

Resolves #2042

---------

Signed-off-by: yzc-yzc <96833212+yzc-yzc@users.noreply.github.com>
2025-08-18 11:00:13 +02:00
Binbin 4a12021707
Fix memory leak in rdbLoadObject when loading a wrong HFE (#2502)
There are memory leaks when we return NULL.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-08-18 07:18:32 +03:00
Sarthak Aggarwal 410976d509
Add test failure template to contributing guide (#2491)
We recently introduced a new template to create `test failures` issues
from a template. This change makes this template visible in the
`CONTRIBUTING.md` file. Also, added a tip to paste the stack trace since
outputs of CI links can expire.

---------

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
2025-08-17 11:58:32 +02:00
Ran Shidlansik e364c57e3d
simplify COPY Preserves TTLs hashexpire test (#2495)
Simplifies a test case seen to be flaky.

fixes: https://github.com/valkey-io/valkey/issues/2482

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>
2025-08-17 11:53:41 +02:00
Hanxi Zhang f36cc20836
Fix cluster test module to pass null terminated node-id to SendClusterMessage (#2484)
Fix https://github.com/valkey-io/valkey/issues/2438

Modified `DingReceiver` function in `tests/modules/cluster.c` by adding
null-termination logic for cross-version compatibility

---------

Signed-off-by: Hanxi Zhang <hanxizh@amazon.com>
2025-08-15 20:40:34 +02:00
yzc-yzc e7bb2354da
Don't call SSL_write() with num=0 (#2490)
aefed3d363/src/networking.c (L2279-L2293)
From above code, we can see that `c->repl_data->ref_block_pos` could be
equal to `o->used`.
When `o->used == o->size`, we may call SSL_write() with num=0 which does
not comply with the openSSL specification.
(ref: https://docs.openssl.org/master/man3/SSL_write/#warnings)

What's worse is that it's still the case after the reconnection. See
aefed3d363/src/replication.c (L756-L769).
So in this case the replica will keep reconnecting again and again until
it doesn't meet the requirements for partial synchronization.

Resolves #2119

---------

Signed-off-by: yzc-yzc <96833212+yzc-yzc@users.noreply.github.com>
2025-08-15 20:37:15 +02:00
Viktor Söderqvist 87d2330c22
Fix timeout in defrag tests (#2483)
* Use pipelines of length 1000 instead of up to 200000.
* Use CLIENT REPLY OFF instead of reading and discarding the replies.

Fixes #2205

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2025-08-15 20:28:14 +02:00
Binbin aefed3d363
Don't allow resize hashtable if rehashing is ongoing (#2465)
Similar to dicts, we disallow resizing while the hashtable is
rehashing. In the previous code, if a resize was triggered during
rehashing, like if the rehashing wasn't fast enough, we would do
a while loop until the rehashing was complete, which could be a
potential issue when doing resize.

---------

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-08-15 11:55:57 +02:00
Sarthak Aggarwal dcbaecddce
Ensures presence of slots on the node before test is run (#2486)
The change will ensure that the slot is present on the node before the
slot is populated. This will avoid the errors during populating the
slot.

Resolves #2480

---------

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
2025-08-15 11:23:34 +02:00
Sarthak Aggarwal de0e581432
Add bug / test-failure / enhancement label to issue template (#2273)
Automatically attach respective label to newly filled issues.
---------

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
Signed-off-by: Sarthak Aggarwal <sarthakaggarwal97@gmail.com>
Co-authored-by: Harkrishn Patro <bunty.hari@gmail.com>
2025-08-14 19:47:38 -07:00
Sarthak Aggarwal 287fbcce33
Fixing Slot Migration Test Failure (#2485)
Make sure slot migration has finished before moving on to the next test.

Resolves #2479

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
2025-08-14 15:20:13 -07:00
Ping Xie 0b4352c98c
Consolidate slot migration logs by grouping consecutive slot migrations into a single log entry (#2481)
Previously, each slot migration was logged individually, which could
lead to log spam in scenarios where many slots are migrated at once.

This commit enhances the logging mechanism to group consecutive slot
migrations into a single log entry, improving log readability and
reducing noise.

Log snippets

```
1661951:S 13 Aug 2025 15:47:10.132 * Slot range [16383, 16383] is migrated from node c3926da75f7c3a0a1bcd07e088b0bde09d48024c () in shard 7746b693330c0814178b90b757e2711ebb8c6609 to node 2465c29c8afb9231525e281e5825684d0bb79f7b () in shard 39342c039d2a6c7ef0ff96314b230dfd7737d646.
1661951:S 13 Aug 2025 15:47:10.289 * Slot range [10924, 16383] is migrated from node 2465c29c8afb9231525e281e5825684d0bb79f7b () in shard 39342c039d2a6c7ef0ff96314b230dfd7737d646 to node c3926da75f7c3a0a1bcd07e088b0bde09d48024c () in shard 7746b693330c0814178b90b757e2711ebb8c6609.
1661951:S 13 Aug 2025 15:47:10.524 * Slot range [10924, 16383] is migrated from node c3926da75f7c3a0a1bcd07e088b0bde09d48024c () in shard 7746b693330c0814178b90b757e2711ebb8c6609 to node 2465c29c8afb9231525e281e5825684d0bb79f7b () in shard 39342c039d2a6c7ef0ff96314b230dfd7737d646.
```

---------

Signed-off-by: Ping Xie <pingxie@google.com>
2025-08-14 13:15:52 -07:00
Binbin 3d1ff2a38c
Remove if condition and disable the new failover test (#2477)
In #2431 we changed the assert to a if condition, and the test cause
some trouble, now we just remove the assert (if condition) and disable
the test for now due to #2441.

Signed-off-by: Binbin <binloveplay1314@qq.com>
2025-08-13 17:01:00 +08:00
Ran Shidlansik 15355124d4
HSETEX 'hset' notification should only be generated if not expired (#2475)
Currently HSETEX always generate `hset` notification. In order to align
with generic `set` command, it should only generate `hset` if the
provided time-to-live is a valid future time.

---------

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>
2025-08-13 10:22:44 +03:00
Ran Shidlansik c93b2971e6
Increment expired_fields stat when assigned TTL is in the past (#2474)
fixes: https://github.com/valkey-io/valkey/issues/2461

---------

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>
2025-08-13 09:37:54 +03:00
ruihong123 58f5562d22
Fix duplicate Acks for RDMA events and fix extremely large max latency for RDMA benchmark. (#2430)
(1) The old logic may result in the RDMA event being acknowledged
unexpectly in the following two scenarios.
* ibv_get_cq_event get an EAGAIN error. 
* ibv_get_cq_event get one event but it may ack multiple times in the
pollcq loop.

(2) In the benchmark result of valkey over RDMA, the tail latency is as
high as 177 milliseconds(almost 80x of TCP). This results from incorrect
benchmark client setup which includes the connection setup time into the
benchmark latency recording. This patch fixes this crazy tail latency
issue by modifying the valkey-benchmark.c. This change only affects
benchmark over RDMA as updates are regulated under Macro USE_RDMA.

There are following updates on valkey RDMA but I am willing to create
separated pull requests.

---------

Signed-off-by: Ruihong Wang <ruihong@google.com>
2025-08-13 09:28:44 +03:00
719 changed files with 15075 additions and 5152 deletions

View File

@ -2,7 +2,7 @@
name: Bug report
about: Help us improve by reporting a bug
title: '[BUG]'
labels: ''
labels: ['bug']
assignees: ''
---

View File

@ -1,6 +1,7 @@
name: Crash report
description: Submit a crash report
title: '[CRASH] <short description>'
labels: ['bug']
body:
- type: markdown
attributes:

View File

@ -2,7 +2,7 @@
name: Feature request
about: Suggest a feature
title: '[NEW]'
labels: ''
labels: ['enhancement']
assignees: ''
---

24
.github/ISSUE_TEMPLATE/test-failure.md vendored Normal file
View File

@ -0,0 +1,24 @@
---
name: Test failure
about: Report a failing or a flaky test
title: '[TEST-FAILURE] <short description>'
labels: ['test-failure']
assignees: ''
---
<!--Before submitting a test failure report, please check open issues to ensure this failure has not already been reported. -->
**Summary**
A short description of the failure.
**Failing test(s)**
- Test name:
- CI link(s):
**Error stack trace**
A relevant stack trace for the test failure.
> **Tip:** Copy and paste the full stack trace from the CI output into your issue, as CI links may expire over time.

View File

@ -1,31 +0,0 @@
name: 'Install TCL8'
description: 'Installs tcl8 and tcltls from source on Fedora-based or uses the system package on others.'
inputs:
matrix_name:
description: 'The name of the matrix to check for fedora'
required: true
runs:
using: "composite"
steps:
# Fedora 42 comes with Tcl 9 by default, but tcltls is currently incompatible with Tcl 9.
# As a workaround, we install Tcl 8 and manually build tcltls 1.7.22 from source.
# Once tcltls adds support for Tcl 9, this logic can be removed and system packages used instead.
- run: |
if [[ "${{ inputs.matrix_name }}" =~ "fedora" ]]; then
dnf -y install tcl8 tcl8-devel gcc make awk openssl openssl-devel
ln -s /usr/bin/tclsh8.6 /usr/bin/tclsh
curl -LO https://core.tcl-lang.org/tcltls/uv/tcltls-1.7.22.tar.gz
tar -xzf tcltls-1.7.22.tar.gz
pushd tcltls-1.7.22
./configure --with-tcl=/usr/lib64/tcl8.6
make
mkdir -p /usr/lib64/tcl8.6/tls1.7.22
cp tcltls.so pkgIndex.tcl /usr/lib64/tcl8.6/tls1.7.22/
popd
else
dnf -y install tcl tcltls
fi
./utils/gen-test-certs.sh
shell: bash

View File

@ -0,0 +1,20 @@
[
{
"duration": 180,
"keyspacelen": [3000000],
"data_sizes": [16,96],
"pipelines": [1, 10],
"clients": [1600],
"commands": [
"SET",
"GET"
],
"cluster_mode": false,
"tls_mode": false,
"warmup": 30,
"io-threads": [1,9],
"benchmark-threads": 90,
"server_cpu_range": "0-8",
"client_cpu_range": "96-191"
}
]

View File

@ -0,0 +1,20 @@
[
{
"duration": 180,
"keyspacelen": [3000000],
"data_sizes": [16,96],
"pipelines": [1, 10],
"clients": [1600],
"commands": [
"SET",
"GET"
],
"cluster_mode": false,
"tls_mode": false,
"warmup": 30,
"io-threads": [1,9],
"benchmark-threads": 90,
"server_cpu_range": "0-8",
"client_cpu_range": "144-191,48-95"
}
]

View File

@ -0,0 +1,14 @@
name: Auto Author Assign
on:
pull_request_target:
types: [opened, reopened]
permissions:
pull-requests: write
jobs:
assign-author:
runs-on: ubuntu-latest
steps:
- uses: toshimaru/auto-author-assign@16f0022cf3d7970c106d8d1105f75a1165edb516 # v2.1.1

173
.github/workflows/benchmark-on-label.yml vendored Normal file
View File

@ -0,0 +1,173 @@
name: On-Demand Labeled Benchmark
on:
pull_request_target:
types: [labeled]
concurrency:
group: ec2-al-2023-pr-benchmarking-arm64
cancel-in-progress: false
defaults:
run:
shell: "bash -Eeuo pipefail -x {0}"
permissions:
contents: read
pull-requests: write
issues: write
jobs:
benchmark:
if: |
github.event.action == 'labeled' && github.event.label.name == 'run-benchmark' &&
github.repository == 'valkey-io/valkey'
runs-on: ["self-hosted", "ec2-al-2023-pr-benchmarking-arm64"]
timeout-minutes: 7200
steps:
- name: Checkout valkey
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
path: valkey
fetch-depth: 0
ref: ${{ github.event.pull_request.merge_commit_sha }}
persist-credentials: false
- name: Checkout valkey-perf-benchmark
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
repository: ${{ github.repository_owner }}/valkey-perf-benchmark
path: valkey-perf-benchmark
fetch-depth: 1
persist-credentials: false
- name: Checkout valkey for latest benchmark.
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
repository: valkey-io/valkey
ref: "unstable"
path: valkey_latest
fetch-depth: 0
persist-credentials: false
- name: Set up Python
uses: kishaningithub/setup-python-amazon-linux@63dc7bd642c6a564d0a93dd4b1cbe3e448ce3621 # v1.0.18
with:
python-version: "3.10"
cache: "pip"
- name: Install dependencies
working-directory: valkey-perf-benchmark
run: |
sudo dnf groupinstall "Development Tools" -y
sudo dnf install -y gcc gcc-c++ make \
python3-devel \
openssl-devel \
bzip2-devel \
libffi-devel
pip install -r requirements.txt
- name: Build latest valkey_latest
working-directory: valkey_latest
run: |
echo "Building latest valkey-benchmark for latest benchmark executable..."
make distclean || true
make -j
if [[ -f "src/valkey-benchmark" ]]; then
echo "Successfully built latest valkey-benchmark"
ls -la src/valkey-benchmark
./src/valkey-benchmark --version || echo "Version check completed"
else
echo "Failed to build valkey-benchmark"
exit 1
fi
VALKEY_BENCHMARK_PATH="$(pwd)/src/valkey-benchmark"
echo "VALKEY_BENCHMARK_PATH=$VALKEY_BENCHMARK_PATH" >> $GITHUB_ENV
echo "Latest valkey-benchmark path: $VALKEY_BENCHMARK_PATH"
- name: Run benchmarks
working-directory: valkey-perf-benchmark
run: |
CONFIG_FILE="../valkey/.github/benchmark_configs/benchmark-config-arm.json"
# Base benchmark arguments
BENCHMARK_ARGS=(
--config "$CONFIG_FILE"
--commits "${{ github.event.pull_request.merge_commit_sha }}"
--baseline "${{ github.event.pull_request.base.ref }}"
--valkey-benchmark-path "$VALKEY_BENCHMARK_PATH"
--target-ip ${{ secrets.EC2_ARM64_IP }}
--valkey-path "../valkey"
--results-dir "results"
--runs 5
)
# Run benchmark
python ./benchmark.py "${BENCHMARK_ARGS[@]}"
- name: Compare results
working-directory: valkey-perf-benchmark
run: |
python ./utils/compare_benchmark_results.py \
--baseline ./results/${{ github.event.pull_request.base.ref }}/metrics.json \
--new ./results/${{ github.event.pull_request.merge_commit_sha }}/metrics.json \
--output ../comparison.md \
--metrics rps
- name: Upload artifacts
if: always()
continue-on-error: true
uses: actions/upload-artifact@6f51ac03b9356f520e9adb1b1b7802705f340c2b # v4.5.0
with:
name: benchmark-results
path: |
./valkey-perf-benchmark/results/${{ github.event.pull_request.merge_commit_sha }}/metrics.json
./valkey-perf-benchmark/results/${{ github.event.pull_request.base.ref }}/metrics.json
comparison.md
- name: Comment PR with results
uses: actions/github-script@5c56fde4671bc2d3592fb0f2c5b5bab9ddae03b1 # v7
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const fs = require('fs');
const body = fs.readFileSync('comparison.md', 'utf8');
const {owner, repo} = context.repo;
const sha = '${{ github.event.pull_request.head.sha }}';
const short = sha.slice(0,7);
const link = `[\`${short}\`](https://github.com/${owner}/${repo}/commit/${sha})`
await github.rest.issues.createComment({
issue_number: context.issue.number,
owner,
repo,
body: `**Benchmark ran on this commit:** ${link}\n\n${body}`
});
- name: Cleanup any running valkey processes
if: always()
continue-on-error: true
run: |
rm -rf comparison.md valkey*
pkill -f valkey
exit_code=$?
if [ $exit_code -eq 0 ]; then
echo "Killed running valkey processes"
elif [ $exit_code -eq 1 ]; then
echo "No valkey processes found to kill"
else
echo "Warning: pkill failed with exit code $exit_code"
fi
- name: Remove ${{ github.event.label.name }} label
if: always() && github.event.label.name == 'run-benchmark'
uses: actions/github-script@5c56fde4671bc2d3592fb0f2c5b5bab9ddae03b1 # v7
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
await github.rest.issues.removeLabel({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
name: '${{ github.event.label.name }}'
}).catch(err => console.log('label not present', err.message));

288
.github/workflows/benchmark-release.yml vendored Normal file
View File

@ -0,0 +1,288 @@
name: Compare Valkey Versions
on:
workflow_dispatch:
inputs:
version1:
description: "First version to compare (commit SHA, branch, or tag)"
required: true
type: string
version2:
description: "Second version to compare (commit SHA, branch, or tag)"
required: true
type: string
issue_id:
description: "Issue ID to comment results on"
required: true
type: string
runs:
description: "Number of benchmark runs per configuration"
required: false
type: number
default: 1
defaults:
run:
shell: "bash -Eeuo pipefail -x {0}"
permissions:
contents: read
pull-requests: write
issues: write
jobs:
benchmark:
if: github.repository == 'valkey-io/valkey'
strategy:
matrix:
include:
- arch: x86
machine: ec2-al-2023-pr-benchmarking-x86
config: benchmark-config-x86.json
- arch: arm64
machine: ec2-al-2023-pr-benchmarking-arm64
config: benchmark-config-arm.json
concurrency:
group: ${{ matrix.machine }}
cancel-in-progress: false
runs-on: ["self-hosted", "${{ matrix.machine }}"]
timeout-minutes: 7200
steps:
- name: Validate inputs
run: |
echo "Version 1: ${{ github.event.inputs.version1 }}"
echo "Version 2: ${{ github.event.inputs.version2 }}"
echo "Issue ID: ${{ github.event.inputs.issue_id }}"
echo "Architecture: ${{ matrix.arch }}"
echo "Config: ${{ matrix.config }}"
echo "Runs: ${{ github.event.inputs.runs }}"
# Validate issue ID is numeric
if ! [[ "${{ github.event.inputs.issue_id }}" =~ ^[0-9]+$ ]]; then
echo "Error: Issue ID must be a number"
exit 1
fi
# Validate runs is a positive number
if ! [[ "${{ github.event.inputs.runs }}" =~ ^[1-9][0-9]*$ ]]; then
echo "Error: Runs must be a positive number"
exit 1
fi
- name: Checkout valkey for latest benchmark.
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
repository: valkey-io/valkey
ref: "unstable"
path: valkey_latest
fetch-depth: 0
persist-credentials: false
- name: Checkout valkey
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
path: valkey
- name: Checkout valkey-perf-benchmark
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
repository: ${{ github.repository_owner }}/valkey-perf-benchmark
path: valkey-perf-benchmark
fetch-depth: 1
persist-credentials: false
- name: Set up Python
uses: kishaningithub/setup-python-amazon-linux@63dc7bd642c6a564d0a93dd4b1cbe3e448ce3621 # v1.0.18
with:
python-version: "3.10"
cache: "pip"
- name: Install dependencies
working-directory: valkey-perf-benchmark
run: |
sudo dnf groupinstall "Development Tools" -y
sudo dnf install -y gcc gcc-c++ make \
python3-devel \
openssl-devel \
bzip2-devel \
libffi-devel
pip install -r requirements.txt
- name: Build latest valkey_latest
working-directory: valkey_latest
run: |
echo "Building latest valkey-benchmark for latest benchmark executable..."
# Clean any previous builds
make distclean || true
# Build valkey-benchmark with latest code
make -j$(nproc)
# Verify the binary was created
if [[ -f "src/valkey-benchmark" ]]; then
echo "✓ Successfully built latest valkey-benchmark"
ls -la src/valkey-benchmark
# Test the binary
echo "Testing valkey-benchmark binary..."
./src/valkey-benchmark --version || echo "Version check completed"
else
echo "Failed to build valkey-benchmark"
exit 1
fi
# Store the absolute path for later use
VALKEY_BENCHMARK_PATH="$(pwd)/src/valkey-benchmark"
echo "VALKEY_BENCHMARK_PATH=$VALKEY_BENCHMARK_PATH" >> $GITHUB_ENV
echo "Latest valkey-benchmark path: $VALKEY_BENCHMARK_PATH"
- name: Run benchmarks
working-directory: valkey-perf-benchmark
env:
EC2_X86_IP: ${{ secrets.EC2_X86_IP }}
EC2_ARM64_IP: ${{ secrets.EC2_ARM64_IP }}
run: |
# Set the target IP based on the matrix architecture
if [[ "${{ matrix.arch }}" == "x86" ]]; then
TARGET_IP=$EC2_X86_IP
echo "Using x86 machine IP"
elif [[ "${{ matrix.arch }}" == "arm64" ]]; then
TARGET_IP=$EC2_ARM64_IP
echo "Using ARM64 machine IP"
else
echo "Error: Unknown architecture: ${{ matrix.arch }}"
exit 1
fi
CONFIG_FILE="../valkey/.github/benchmark_configs/${{ matrix.config }}"
# Verify our custom valkey-benchmark exists
if [[ ! -f "$VALKEY_BENCHMARK_PATH" ]]; then
echo "Custom valkey-benchmark not found at: $VALKEY_BENCHMARK_PATH"
exit 1
fi
echo "Using custom valkey-benchmark from: $VALKEY_BENCHMARK_PATH"
# Base benchmark arguments with custom valkey-benchmark path
BENCHMARK_ARGS=(
--config "$CONFIG_FILE"
--commits ${{ github.event.inputs.version1 }} ${{ github.event.inputs.version2 }}
--valkey-benchmark-path "$VALKEY_BENCHMARK_PATH"
--target-ip $TARGET_IP
--results-dir "results"
--runs ${{ github.event.inputs.runs }}
)
echo "Running benchmarks with the following setup:"
echo "- Version 1: ${{ github.event.inputs.version1 }}"
echo "- Version 2: ${{ github.event.inputs.version2 }}"
echo "- Architecture: ${{ matrix.arch }}"
echo "- Config: ${{ matrix.config }}"
echo "- Using latest valkey-benchmark: $VALKEY_BENCHMARK_PATH"
echo "- This ensures both versions use the same (latest) benchmark tool for consistent results"
# Run benchmark with custom valkey-benchmark executable
python ./benchmark.py "${BENCHMARK_ARGS[@]}"
- name: Compare results
working-directory: valkey-perf-benchmark
run: |
# Find the actual result directories (they might have different names than input)
VERSION1_DIR=$(find ./results -maxdepth 1 -type d -name "*${{ github.event.inputs.version1 }}*" | head -1)
VERSION2_DIR=$(find ./results -maxdepth 1 -type d -name "*${{ github.event.inputs.version2 }}*" | head -1)
if [[ -z "$VERSION1_DIR" ]]; then
echo "Could not find results for version1: ${{ github.event.inputs.version1 }}"
echo "Available result directories:"
ls -la ./results/
exit 1
fi
if [[ -z "$VERSION2_DIR" ]]; then
echo "Could not find results for version2: ${{ github.event.inputs.version2 }}"
echo "Available result directories:"
ls -la ./results/
exit 1
fi
echo "Comparing results:"
echo "Version 1 (${{ github.event.inputs.version1 }}): $VERSION1_DIR"
echo "Version 2 (${{ github.event.inputs.version2 }}): $VERSION2_DIR"
# Generate RPS-focused comparison and graphs for GitHub comments
python utils/compare_benchmark_results.py \
--baseline "$VERSION1_DIR/metrics.json" \
--new "$VERSION2_DIR/metrics.json" \
--output ../comparison.md \
--metrics rps
- name: Upload artifacts
if: always()
continue-on-error: true
uses: actions/upload-artifact@6f51ac03b9356f520e9adb1b1b7802705f340c2b # v4.5.0
with:
name: benchmark-results-${{ matrix.arch }}-${{ github.event.inputs.issue_id }}
path: |
./valkey-perf-benchmark/results/*
comparison.md
- name: Cleanup any running valkey processes and files
if: always()
continue-on-error: true
run: |
pkill -f valkey || echo "No valkey processes found to kill"
rm -rf *
combine-results:
needs: benchmark
runs-on: ubuntu-latest
steps:
- name: Download all artifacts
uses: actions/download-artifact@v4
with:
path: artifacts
- name: Combine results and create comprehensive report
run: |
echo "# Multi-Architecture Benchmark Comparison: ${{ github.event.inputs.version1 }} vs ${{ github.event.inputs.version2 }}" > combined_report.md
echo "" >> combined_report.md
echo "**Versions Compared:**" >> combined_report.md
echo "- Version 1: \`${{ github.event.inputs.version1 }}\`" >> combined_report.md
echo "- Version 2: \`${{ github.event.inputs.version2 }}\`" >> combined_report.md
echo "" >> combined_report.md
echo "**Runs:** ${{ github.event.inputs.runs }} per configuration" >> combined_report.md
echo "**Workflow Run:** [View Details](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})" >> combined_report.md
echo "" >> combined_report.md
echo "---" >> combined_report.md
echo "" >> combined_report.md
# Process each architecture's results
for arch in x86 arm64; do
artifact_dir="artifacts/benchmark-results-${arch}-${{ github.event.inputs.issue_id }}"
if [[ -d "$artifact_dir" && -f "$artifact_dir/comparison.md" ]]; then
echo "## ${arch^^} Architecture Results" >> combined_report.md
echo "" >> combined_report.md
cat "$artifact_dir/comparison.md" >> combined_report.md
echo "" >> combined_report.md
echo "---" >> combined_report.md
echo "" >> combined_report.md
fi
done
- name: Comment on issue with combined results
uses: actions/github-script@5c56fde4671bc2d3592fb0f2c5b5bab9ddae03b1 # v7
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const fs = require('fs');
// Read the combined report
const body = fs.readFileSync('combined_report.md', 'utf8');
await github.rest.issues.createComment({
issue_number: ${{ github.event.inputs.issue_id }},
owner: context.repo.owner,
repo: context.repo.repo,
body: body
});

View File

@ -28,17 +28,12 @@ jobs:
# Fail build if there are warnings
# build with TLS just for compilation coverage
run: make -j4 all-with-unit-tests SERVER_CFLAGS='-Werror' BUILD_TLS=yes USE_FAST_FLOAT=yes
- name: install old server for compatibility testing
run: |
cd tests/tmp
wget https://download.valkey.io/releases/valkey-8.1.3-noble-x86_64.tar.gz
tar -xvf valkey-8.1.3-noble-x86_64.tar.gz
- name: test
run: |
sudo apt-get install tcl8.6 tclx
./runtest --verbose --tags -slow --dump-logs --other-server-path tests/tmp/valkey-8.1.3-noble-x86_64/bin/valkey-server
./runtest --verbose --tags -slow --dump-logs
- name: module api test
run: CFLAGS='-Werror' ./runtest-moduleapi --verbose --dump-logs --other-server-path tests/tmp/valkey-8.1.3-noble-x86_64/bin/valkey-server
run: CFLAGS='-Werror' ./runtest-moduleapi --verbose --dump-logs
- name: validate commands.def up to date
run: |
touch src/commands/ping.json
@ -49,6 +44,40 @@ jobs:
run: |
./src/valkey-unit-tests
test-ubuntu-latest-compatibility:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
server:
- {version: "7.2.11", file: "valkey-7.2.11-noble-x86_64.tar.gz"}
- {version: "8.0.6", file: "valkey-8.0.6-noble-x86_64.tar.gz"}
- {version: "8.1.4", file: "valkey-8.1.4-noble-x86_64.tar.gz"}
steps:
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
- name: make
# Fail build if there are warnings
# build with TLS just for compilation coverage
run: make -j4 all-with-unit-tests SERVER_CFLAGS='-Werror' BUILD_TLS=yes USE_FAST_FLOAT=yes
- name: Install old server (${{ matrix.server.version }}) for compatibility testing
run: |
mkdir -p tests/tmp
cd tests/tmp
wget https://download.valkey.io/releases/${{ matrix.server.file }}
tar -xvf ${{ matrix.server.file }}
- name: Run compatibility tests against ${{ matrix.server.version }}
run: |
sudo apt-get install -y tcl8.6 tclx
./runtest --verbose --tags "-slow needs:other-server" --dump-logs \
--other-server-path tests/tmp/valkey-${{ matrix.server.version }}-noble-x86_64/bin/valkey-server
- name: Module API tests against ${{ matrix.server.version }}
run: |
CFLAGS='-Werror' ./runtest-moduleapi --tags needs:other-server --verbose --dump-logs \
--other-server-path tests/tmp/valkey-${{ matrix.server.version }}-noble-x86_64/bin/valkey-server
test-ubuntu-latest-cmake:
runs-on: ubuntu-latest
steps:

View File

@ -1,7 +1,16 @@
name: "CodeQL"
on:
push:
paths-ignore:
- '**/*.md'
- '**/00-RELEASENOTES'
- '**/COPYING'
pull_request:
paths-ignore:
- '**/*.md'
- '**/00-RELEASENOTES'
- '**/COPYING'
schedule:
# run weekly new vulnerability was added to the database
- cron: '0 3 * * 0'

View File

@ -947,12 +947,10 @@ jobs:
run: dnf -y install epel-release
- name: make
run: |
dnf -y install gcc make procps-ng openssl-devel openssl which /usr/bin/kill /usr/bin/awk
dnf -y install gcc make procps-ng which /usr/bin/kill /usr/bin/awk
make -j SERVER_CFLAGS='-Werror'
- name: testprep
uses: ./.github/actions/rpm-distros-tcl8
with:
matrix_name: ${{ matrix.name }}
run: dnf -y install tcl tcltls
- name: test
if: true && !contains(github.event.inputs.skiptests, 'valkey')
run: ./runtest ${{ github.event_name != 'pull_request' && '--accurate' || '' }} --verbose --dump-logs ${{github.event.inputs.test_args}}
@ -1018,9 +1016,9 @@ jobs:
dnf -y install make gcc openssl-devel openssl procps-ng which /usr/bin/kill /usr/bin/awk
make -j BUILD_TLS=module SERVER_CFLAGS='-Werror'
- name: testprep
uses: ./.github/actions/rpm-distros-tcl8
with:
matrix_name: ${{ matrix.name }}
run: |
dnf -y install tcl tcltls
./utils/gen-test-certs.sh
- name: test
if: true && !contains(github.event.inputs.skiptests, 'valkey')
run: |
@ -1090,9 +1088,9 @@ jobs:
dnf -y install make gcc openssl-devel openssl procps-ng which /usr/bin/kill /usr/bin/awk
make -j BUILD_TLS=module SERVER_CFLAGS='-Werror'
- name: testprep
uses: ./.github/actions/rpm-distros-tcl8
with:
matrix_name: ${{ matrix.name }}
run: |
dnf -y install tcl tcltls
./utils/gen-test-certs.sh
- name: test
if: true && !contains(github.event.inputs.skiptests, 'valkey')
run: |
@ -1201,7 +1199,7 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [macos-13, macos-14]
os: [macos-14]
runs-on: ${{ matrix.os }}
if: |
(github.event_name == 'workflow_dispatch' ||
@ -1230,7 +1228,7 @@ jobs:
run: make SERVER_CFLAGS='-Werror'
test-freebsd:
runs-on: macos-13
runs-on: ubuntu-latest
if: |
(github.event_name == 'workflow_dispatch' ||
(github.event_name == 'schedule' && github.repository == 'valkey-io/valkey') ||
@ -1248,14 +1246,14 @@ jobs:
repository: ${{ env.GITHUB_REPOSITORY }}
ref: ${{ env.GITHUB_HEAD_REF }}
- name: test
uses: cross-platform-actions/action@5800fa0060a22edf69992a779adac3d2bb3a6f8a # v0.22.0
uses: cross-platform-actions/action@46e8d7fb25520a8d6c64fd2b7a1192611da98eda # v0.30.0
with:
operating_system: freebsd
environment_variables: MAKE
version: 13.2
shell: bash
run: |
sudo pkg install -y bash gmake lang/tcl86 lang/tclx
sudo pkg install -y bash gmake lang/tcl86 lang/tclX
gmake
./runtest --single unit/keyspace --single unit/auth --single unit/networking --single unit/protocol

View File

@ -43,3 +43,34 @@ unset(BUILD_TEST_MODULES CACHE)
unset(BUILD_EXAMPLE_MODULES CACHE)
unset(USE_TLS CACHE)
unset(DEBUG_FORCE_DEFRAG CACHE)
# Helper to copy runtest scripts to allow running tests with CMake built binaries
function(copy_runtest_script script_name)
set(src "${CMAKE_SOURCE_DIR}/${script_name}")
set(dst "${CMAKE_BINARY_DIR}/${script_name}")
file(READ "${src}" contents)
# Split at the first newline (after shebang #!/bin/sh)
string(FIND "${contents}" "\n" index_of_first_newline_char)
math(EXPR first_index_script_body "${index_of_first_newline_char} + 1")
string(SUBSTRING "${contents}" 0 ${first_index_script_body} shebang_line)
string(SUBSTRING "${contents}" ${first_index_script_body} -1 script_body)
# Insert our environment variable lines
set(insert_content
"# Most tests assume running from the project root
cd ${CMAKE_SOURCE_DIR}
export VALKEY_BIN_DIR=\"${CMAKE_BINARY_DIR}/bin\"
")
# Reconstruct the full script
set(new_contents "${shebang_line}${insert_content}${script_body}")
file(WRITE "${dst}" "${new_contents}")
file(CHMOD "${dst}" PERMISSIONS OWNER_READ OWNER_WRITE OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE)
endfunction()
copy_runtest_script("runtest")
copy_runtest_script("runtest-cluster")
copy_runtest_script("runtest-moduleapi")
copy_runtest_script("runtest-rdma")
copy_runtest_script("runtest-sentinel")

View File

@ -16,6 +16,7 @@ The Valkey project is led by a Technical Steering Committee, whose responsibilit
* Found a bug? [Report it here](https://github.com/valkey-io/valkey/issues/new?template=bug_report.md&title=%5BBUG%5D)
* Valkey crashed? [Submit a crash report here](https://github.com/valkey-io/valkey/issues/new?template=crash_report.md&title=%5BCRASH%5D+%3Cshort+description%3E)
* Suggest a new feature? [Post your detailed feature request here](https://github.com/valkey-io/valkey/issues/new?template=feature_request.md&title=%5BNEW%5D)
* Report a test failure? [Report it here](https://github.com/valkey-io/valkey/issues/new?template=test-failure.md)
* Want to help with documentation? [Move on to valkey-doc](https://github.com/valkey-io/valkey-doc)
* Report a vulnerability? See [SECURITY.md](SECURITY.md)

View File

@ -4,7 +4,7 @@ The Valkey project is managed by a Technical Steering Committee (TSC) composed o
The Valkey project includes all of the current and future repositories under the Valkey-io organization.
Committers are defined as individuals with write access to the code within a repository.
Maintainers are defined as individuals with full access to a repository and own its governance.
Both maintainers and committers should be clearly listed in the MAINTAINERS.md file in a given projects repository.
Both maintainers and committers shall be clearly listed in the MAINTAINERS.md file in a given project's repository.
Maintainers of other repositories within the Valkey project are not members of the TSC unless explicitly added.
## Technical Steering Committee
@ -12,10 +12,15 @@ Maintainers of other repositories within the Valkey project are not members of t
The TSC is responsible for oversight of all technical, project, approval, and policy matters for Valkey.
The TSC members are listed in the [MAINTAINERS.md](MAINTAINERS.md) file in the Valkey repository.
Maintainers (and accordingly, TSC members) may be added or removed by no less than 2/3 affirmative vote of the current TSC.
At any time, no more than one third (1/3) of the TSC members may be employees, contractors, or representatives of the same organization or affiliated organizations.
For the purposes of this document, “organization” includes companies, corporations, universities, research institutes, non-profits, governmental institutions, and any of their subsidiaries or affiliates.
If, at any time, the 1/3 organization limit is exceeded (for example, due to changes in employment, company acquisitions, or organizational affiliations), the TSC shall be notified as soon as possible.
The TSC must promptly take action to restore compliance, which may include removing or reassigning members in accordance with the procedures outlined in the [Termination of Membership](#termination-of-membership) section.
The TSC shall strive to resolve the situation within 30 days of notification, and document the steps taken to restore compliance.
The TSC shall appoint a Chair responsible for organizing TSC meetings.
If the TSC Chair is removed from the TSC (or the Chair steps down from that role), it is the responsibility of the TSC to appoint a new Chair.
The TSC can amend this governance document by no less than a 2/3 affirmative vote.
The TSC may, at its discretion, add or remove members who are not maintainers of the main Valkey repository.
The TSC may, at its discretion, add or remove maintainers from other repositories within the Valkey project.
@ -29,16 +34,33 @@ The TSC shall document evidence of consensus in accordance with these requiremen
If consensus cannot be reached, the TSC shall make the decision by a vote.
A vote shall also be called when an issue or pull request is marked as a major decision, which are decisions that have a significant impact on the Valkey architecture or design.
Examples of major decisions:
* Fundamental changes to the Valkey core datastructures
* Adding a new data structure or API
* Changes that affect backward compatibility
* New user visible fields that need to be maintained
* Modifications to the TSC or other governance documents
* Adding members to other roles within the Valkey project
* Delegation of maintainership for projects to other groups or individuals
* Adding or removing a new external library such as a client
or module to the project.
### Technical Major Decisions
Technical major decisions include:
* Fundamental changes to the Valkey core datastructures
* Adding a new data structure or API
* Changes that affect backward compatibility
* New user visible fields that need to be maintained
* Adding or removing a new external library such as a client or module to the project when it affects runtime behavior
Technical major decisions shall be approved by a simple majority vote whenever one can be obtained.
If a simple majority cannot be reached within a two-week voting period, and no TSC member has voted against, the decision may instead be approved through explicit “+2” support from at least two TSC members, recorded on the relevant issue or pull request.
If the pull request author or issue proposer is a TSC member, their +1 counts toward the +2.
If any TSC member casts a negative vote, the decision must follow the simple majority voting process and cannot be approved through +2.
Once a technical major decision has been approved through the +2 mechanism, any subsequent concerns shall be raised through a new major decision process; +2 approvals are not retracted directly.
### Governance Major Decisions
Governance major decisions include:
* Adding TSC members or involuntary removal of TSC members
* Modifying this governance document
* Delegation of maintainership for projects or governance authority
* Creating, modifying, or removing roles within the Valkey project
* Any change that alters voting rules, TSC responsibilities, or project oversight
* Structural changes to the TSC, including composition limits
Governance major decisions shall require approval by a super-majority vote of at least two thirds (2/3) of the entire TSC.
Any member of the TSC can call a vote with reasonable notice to the TSC, setting out a discussion period and a separate voting period.
Any discussion may be conducted in person or electronically by text, voice, or video.
@ -46,23 +68,24 @@ The discussion shall be open to the public, with the notable exception of discus
In any vote, each voting TSC member will have one vote.
The TSC shall give at least two weeks for all members to submit their vote.
Except as specifically noted elsewhere in this document, decisions by vote require a simple majority vote of all voting members.
It is the responsibility of the TSC chair to help facilitate the voting process as needed to make sure it completes within the voting period.
If a vote results in a tie, the status quo is preserved.
It is the responsibility of the TSC Chair to help facilitate the voting process as needed to make sure it completes within the voting period.
## Termination of Membership
A maintainer's access (and accordingly, their position on the TSC) will be removed if any of the following occur:
* Involuntary Removal: Removal via the [Governance Major Decision](#governance-major-decisions) voting process.
* Resignation: Written notice of resignation to the TSC.
* TSC Vote: 2/3 affirmative vote of the TSC to remove a member
* Unreachable Member: If a member is unresponsive for more than six months, the remaining active members of the TSC may vote to remove the unreachable member by a simple majority.
## Technical direction for other Valkey projects
The TSC may delegate decision making for other projects within the Valkey organization to the maintainers responsible for those projects.
Delegation of decision making for a project is considered a major decision, and shall happen with an explicit vote.
Delegation of decision making for a project is considered a [Governance Major Decision](#governance-major-decisions).
Projects within the Valkey organization must indicate the individuals with commit permissions by updating the MAINTAINERS.md within their repositories.
The TSC may, at its discretion, overrule the decisions made by other projects within the Valkey organization, although they should show restraint in doing so.
The TSC may, at its discretion, overrule the decisions made by other projects within the Valkey organization, although they shall show restraint in doing so.
## License of this document

View File

@ -3,30 +3,36 @@
This document contains a list of maintainers in this repo.
See [GOVERNANCE.md](GOVERNANCE.md) that explains the function of this file.
## Committee Chair
- **Madelyn Olson**
Term: March 28, 2024 Present
## Current Maintainers
Maintainers listed in alphabetical order by their github ID.
| Maintainer | GitHub ID | Affiliation |
| ------------------- | ----------------------------------------------- | ----------- |
| Zhu Binbin | [enjoy-binbin](https://github.com/enjoy-binbin) | Tencent |
| Wen Hui | [hwware](https://github.com/hwware) | Huawei |
| Madelyn Olson | [madolson](https://github.com/madolson) | Amazon |
| Ping Xie | [pingxie](https://github.com/pingxie) | Google |
| Zhao Zhao | [soloestoy](https://github.com/soloestoy) | Alibaba |
| Viktor Söderqvist | [zuiderkwast](https://github.com/zuiderkwast) | Ericsson |
| ------------------- | ------------- | ----------- |
| Binbin Zhu | @enjoy-binbin | Tencent |
| Harkrishn Patro | @hpatro | Amazon |
| Lucas Yang | @lucasyonge | - |
| Madelyn Olson | @madolson | Amazon |
| Jacob Murphy | @murphyjacob4 | Google |
| Ping Xie | @pingxie | Oracle |
| Ran Shidlansik | @ranshid | Amazon |
| Zhao Zhao | @soloestoy | Alibaba |
| Viktor Söderqvist | @zuiderkwast | Ericsson |
## Current Committers
Committers listed in alphabetical order by their github ID.
| Committer | GitHub ID | Affiliation |
| ------------------- | ----------------------------------------------- | ----------- |
| Harkrishn Patro | [hpatro](https://github.com/hpatro) | Amazon |
| Ran Shidlansik | [ranshid](https://github.com/ranshid) | Amazon |
| Ricardo Dias | [rjd15372](https://github.com/rjd15372) | Percona |
| ------------------- | ------------- | ----------- |
| Ricardo Dias | @rjd15372 | Percona |
### Former Maintainers and Committers
## Former Maintainers and Committers
| Maintainer | GitHub ID | Affiliation |
| ------------------- | ----------------------------------------------- | ----------- |
| Name | GitHub ID | Role | Term | Affiliation |
| ------------------- | ------------- | ---- | ---- | ----------- |

View File

@ -52,6 +52,13 @@ as libsystemd-dev on Debian/Ubuntu or systemd-devel on CentOS) and run:
% make USE_SYSTEMD=yes
Since Valkey version 8.1, `fast_float` has been introduced as an optional
dependency, which can speed up sorted sets and other commands that use
the double datatype. To build with `fast_float` support, you'll need a
C++ compiler and run:
% make USE_FAST_FLOAT=yes
To append a suffix to Valkey program names, use:
% make PROG_SUFFIX="-alt"
@ -167,7 +174,8 @@ line, with exactly the same name.
## Running manually
To manually run a Valkey server with TLS mode (assuming `./gen-test-certs.sh` was invoked so sample certificates/keys are available):
To manually run a Valkey server with TLS mode (assuming `./utils/gen-test-certs.sh`
was invoked so sample certificates/keys are available):
* TLS built-in mode:
```

View File

@ -72,6 +72,7 @@ set(VALKEY_SERVER_SRCS
${CMAKE_SOURCE_DIR}/src/geo.c
${CMAKE_SOURCE_DIR}/src/lazyfree.c
${CMAKE_SOURCE_DIR}/src/module.c
${CMAKE_SOURCE_DIR}/src/lrulfu.c
${CMAKE_SOURCE_DIR}/src/evict.c
${CMAKE_SOURCE_DIR}/src/expire.c
${CMAKE_SOURCE_DIR}/src/geohash.c
@ -87,6 +88,7 @@ set(VALKEY_SERVER_SRCS
${CMAKE_SOURCE_DIR}/src/lolwut.c
${CMAKE_SOURCE_DIR}/src/lolwut5.c
${CMAKE_SOURCE_DIR}/src/lolwut6.c
${CMAKE_SOURCE_DIR}/src/lolwut9.c
${CMAKE_SOURCE_DIR}/src/acl.c
${CMAKE_SOURCE_DIR}/src/tracking.c
${CMAKE_SOURCE_DIR}/src/socket.c

View File

@ -50,13 +50,13 @@ endif ()
# Helper function for creating symbolic link so that: link -> source
macro (valkey_create_symlink source link)
install(
CODE "execute_process( \
COMMAND /bin/bash ${CMAKE_BINARY_DIR}/CreateSymlink.sh \
${source} \
${link} \
)"
COMPONENT "valkey")
add_custom_command(
TARGET ${source} POST_BUILD
COMMAND ${CMAKE_COMMAND} -E create_symlink
"$<TARGET_FILE_NAME:${source}>"
"$<TARGET_FILE_DIR:${source}>/${link}"
VERBATIM
)
endmacro ()
# Install a binary
@ -91,11 +91,7 @@ macro (valkey_build_and_install_bin target sources ld_flags libs link_name)
if (USE_RDMA)
# Add required libraries needed for RDMA
# Bug in libvalkey 0.1.0: The order is important and we need to link
# valkey::valkey after valkey::valkey_rdma. This will be fixed upstream.
# TODO: Next time we lift libvalkey, remove valkey::valkey from
# the next line.
target_link_libraries(${target} valkey::valkey_rdma valkey::valkey)
target_link_libraries(${target} valkey::valkey_rdma)
endif ()
if (IS_FREEBSD)

View File

@ -7,7 +7,7 @@ antirez
api
APIs
ASYNC
asyncronous
asynchronous
atomicity
AUTOFREE
autoload

View File

@ -8,7 +8,7 @@ jobs:
runs-on: ubuntu-latest
container: almalinux:8
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Install dependencies
run: |
@ -35,7 +35,7 @@ jobs:
runs-on: ubuntu-latest
container: rockylinux:8
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Install dependencies
run: |
@ -64,10 +64,10 @@ jobs:
runs-on: ubuntu-latest
name: FreeBSD
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Build in FreeBSD
uses: vmactions/freebsd-vm@c3ae29a132c8ef1924775414107a97cac042aad5 # v1.2.0
uses: vmactions/freebsd-vm@05856381fab64eeee9b038a0818f6cec649ca17a # v1.2.3
with:
prepare: pkg install -y gmake cmake
run: |
@ -94,9 +94,9 @@ jobs:
- {target: mipsel, host: mipsel-linux-gnu, qemu: mipsel, gccver: 10 }
- {target: mips64el, host: mips64el-linux-gnuabi64, qemu: mips64el, gccver: 10 }
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Prepare
uses: awalsh128/cache-apt-pkgs-action@7ca5f46d061ad9aa95863cd9b214dd48edef361d # v1.5.0
uses: awalsh128/cache-apt-pkgs-action@2c09a5e66da6c8016428a2172bd76e5e4f14bb17 # v1.5.3
with:
packages: gcc-${{ matrix.config.gccver }}-${{ matrix.config.host }}
version: ${{ matrix.config.target }}-1.0

View File

@ -10,51 +10,53 @@ jobs:
name: Run static checkers
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Run clang-format style check (.c and .h)
uses: jidicula/clang-format-action@4726374d1aa3c6aecf132e5197e498979588ebc8 # v4.15.0
with:
clang-format-version: '18'
ubuntu-cmake:
name: Build with CMake [${{ matrix.cmake-build-type }}, ${{ matrix.compiler }}, cmake-${{ matrix.cmake-version }}, sanitizer="${{ matrix.sanitizer }}"]
name: Build with CMake ${{ matrix.sanitizer && format('and {0}-sanitizer', matrix.sanitizer ) }} [${{ matrix.compiler }}, cmake-${{ matrix.cmake-version }}, ${{ matrix.cmake-build-type }}]
runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
include:
# New compilers
- compiler: 'clang-20'
cmake-version: '4.0'
cmake-build-type: Release
- compiler: 'gcc-14'
cmake-version: '4.0'
cmake-build-type: Release
# Old compilers
- compiler: 'clang-14'
cmake-version: '3.13'
cmake-build-type: Release
- compiler: 'gcc-9'
cmake-version: '3.13'
cmake-build-type: Release
# Sanitizers enabled
compiler: ['gcc-14', 'clang-20']
cmake-version: ['4.0']
cmake-build-type: [Release]
sanitizer: ["", thread, undefined, leak, address]
include:
- compiler: clang-18
cmake-version: 3.29
cmake-build-type: Debug
- compiler: gcc-9
cmake-version: 3.13
cmake-build-type: Release
- compiler: clang-14
cmake-version: 3.13
cmake-build-type: Release
cmake-build-type: [RelWithDebInfo]
sanitizer: [thread, undefined, leak, address]
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Prepare
uses: awalsh128/cache-apt-pkgs-action@7ca5f46d061ad9aa95863cd9b214dd48edef361d # v1.5.0
uses: awalsh128/cache-apt-pkgs-action@2c09a5e66da6c8016428a2172bd76e5e4f14bb17 # v1.5.3
with:
packages: libevent-dev libuv1-dev libev-dev libglib2.0-dev
packages: libevent-dev libuv1-dev libev-dev libglib2.0-dev valkey-server
version: 1.0
- name: Setup compiler
uses: aminya/setup-cpp@6370aaa0252a93c71dcc4cf49397f46810eeda56 # v1.5.3
uses: aminya/setup-cpp@a276e6e3d1db9160db5edc458e99a30d3b109949 # v1.7.1
with:
compiler: ${{ matrix.compiler }}
- name: Setup CMake
uses: jwlawson/actions-setup-cmake@802fa1a2c4e212495c05bf94dba2704a92a472be # v2.0.2
with:
cmake-version: ${{ matrix.cmake-version }}
- name: Install Valkey for non-cluster tests
run: |
sudo apt update
sudo apt install valkey
- name: Generate makefiles
run: |
if [ -n "${{ matrix.sanitizer }}" ]; then
@ -82,16 +84,12 @@ jobs:
name: Build with make
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Prepare
uses: awalsh128/cache-apt-pkgs-action@7ca5f46d061ad9aa95863cd9b214dd48edef361d # v1.5.0
uses: awalsh128/cache-apt-pkgs-action@2c09a5e66da6c8016428a2172bd76e5e4f14bb17 # v1.5.3
with:
packages: libevent-dev valgrind
packages: libevent-dev valgrind valkey-server
version: 1.0
- name: Install Valkey
run: |
sudo apt update
sudo apt install valkey
- name: Build
run: USE_TLS=1 TEST_ASYNC=1 make
- name: Run tests
@ -112,16 +110,12 @@ jobs:
name: Build for 32-bit
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Prepare
uses: awalsh128/cache-apt-pkgs-action@7ca5f46d061ad9aa95863cd9b214dd48edef361d # v1.5.0
uses: awalsh128/cache-apt-pkgs-action@2c09a5e66da6c8016428a2172bd76e5e4f14bb17 # v1.5.3
with:
packages: gcc-multilib
packages: gcc-multilib valkey-server
version: 1.0
- name: Install Valkey
run: |
sudo apt update
sudo apt install valkey
- name: Build
run: |
make 32bit
@ -134,43 +128,55 @@ jobs:
name: Installation tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Prepare
uses: awalsh128/cache-apt-pkgs-action@7ca5f46d061ad9aa95863cd9b214dd48edef361d # v1.5.0
uses: awalsh128/cache-apt-pkgs-action@2c09a5e66da6c8016428a2172bd76e5e4f14bb17 # v1.5.3
with:
packages: libevent-dev libuv1-dev libev-dev libglib2.0-dev
version: 1.0
- name: Install libvalkey using Makefile
- name: Install static and dynamic libraries using Makefile
run: |
make USE_TLS=1 DESTDIR=${{ github.workspace }}/make-install install
- name: Install libvalkey using CMake
- name: Install dynamic libraries using CMake
run: |
mkdir build && cd build
mkdir build-dyn && cd build-dyn
cmake -DDISABLE_TESTS=ON -DENABLE_TLS=ON ..
make DESTDIR=${{ github.workspace }}/cmake-install install
make DESTDIR=${{ github.workspace }}/dynamic-cmake-install install
- name: Install static libraries using CMake
run: |
mkdir build-static && cd build-static
cmake -DDISABLE_TESTS=ON -DENABLE_TLS=ON -DBUILD_SHARED_LIBS=OFF ..
make DESTDIR=${{ github.workspace }}/static-cmake-install install
- name: Build examples with Makefile using a Makefile-installed libvalkey
run: |
make CFLAGS="-I${{ github.workspace }}/make-install/usr/local/include" \
STLIBNAME="${{ github.workspace }}/make-install/usr/local/lib/libvalkey.a" \
make STLIBNAME="${{ github.workspace }}/make-install/usr/local/lib/libvalkey.a" \
TLS_STLIBNAME="${{ github.workspace }}/make-install/usr/local/lib/libvalkey_tls.a" \
INCLUDE_DIR="${{ github.workspace }}/make-install/usr/local/include" \
USE_TLS=1 -C examples
- name: Build examples with Makefile using a CMake-installed libvalkey
run: |
make CFLAGS="-I${{ github.workspace }}/cmake-install/usr/local/include" \
STLIBNAME="${{ github.workspace }}/cmake-install/usr/local/lib/libvalkey.a" \
make STLIBNAME="${{ github.workspace }}/static-cmake-install/usr/local/lib/libvalkey.a" \
TLS_STLIBNAME="${{ github.workspace }}/static-cmake-install/usr/local/lib/libvalkey_tls.a" \
INCLUDE_DIR="${{ github.workspace }}/static-cmake-install/usr/local/include" \
USE_TLS=1 -C examples
- name: Build examples with CMake using a CMake-installed libvalkey
- name: Build examples with CMake using CMake-installed dynamic libraries
run: |
cd examples && mkdir build && cd build
cmake -DCMAKE_PREFIX_PATH=${{ github.workspace }}/cmake-install/usr/local -DENABLE_TLS=ON ..
cd examples && mkdir build-dyn && cd build-dyn
cmake -DCMAKE_PREFIX_PATH=${{ github.workspace }}/dynamic-cmake-install/usr/local -DENABLE_TLS=ON ..
make
- name: Build examples with CMake using CMake-installed static libraries
run: |
cd examples && mkdir build-static && cd build-static
cmake -DCMAKE_PREFIX_PATH=${{ github.workspace }}/static-cmake-install/usr/local -DENABLE_TLS=ON ..
make
rdma:
name: RDMA support enabled
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Prepare
uses: awalsh128/cache-apt-pkgs-action@7ca5f46d061ad9aa95863cd9b214dd48edef361d # v1.5.0
uses: awalsh128/cache-apt-pkgs-action@2c09a5e66da6c8016428a2172bd76e5e4f14bb17 # v1.5.3
with:
packages: librdmacm-dev libibverbs-dev
version: 1.0
@ -192,7 +198,7 @@ jobs:
name: CMake 3.7.0 (min. required)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Setup CMake
uses: jwlawson/actions-setup-cmake@802fa1a2c4e212495c05bf94dba2704a92a472be # v2.0.2
with:
@ -207,16 +213,16 @@ jobs:
name: macOS
runs-on: macos-latest
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Install dependencies
run: |
brew update
brew install valkey
- name: Build using CMake
- name: Build and install using CMake
run: |
mkdir build && cd build
cmake .. -G Ninja -DCMAKE_BUILD_TYPE=Release -DENABLE_TLS=ON
ninja -v
sudo ninja -v install
- name: Build using Makefile
run: USE_TLS=1 make
- name: Run tests
@ -229,21 +235,25 @@ jobs:
name: Windows
runs-on: windows-latest
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- uses: ilammy/msvc-dev-cmd@0b201ec74fa43914dc39ae48a89fd1d8cb592756 # v1.13.0
- name: Install dependencies
- name: Remove installed OpenSSL 1.1.1 (has reached End of Life)
shell: bash
run: rm -rf 'C:/Program Files/OpenSSL/'
- name: Install dependencies (choco)
run: |
choco install -y memurai-developer
# Workaround for libevent that specify minimum CMake version 3.1 (incompatible with CMake >= 4.0).
sed -i '2i if(CMAKE_VERSION VERSION_GREATER_EQUAL "4.0")' ${env:VCPKG_INSTALLATION_ROOT}/scripts/ports.cmake
sed -i '3i set(ENV{CMAKE_POLICY_VERSION_MINIMUM} 3.5)' ${env:VCPKG_INSTALLATION_ROOT}/scripts/ports.cmake
sed -i '4i endif()' ${env:VCPKG_INSTALLATION_ROOT}/scripts/ports.cmake
vcpkg install --triplet x64-windows pkgconf libevent
- name: Build
- name: Install dependencies (vcpkg)
uses: johnwason/vcpkg-action@caa1c94fbb94d8b023a0cc93edf10cd3791349a7 # v7.0.1
with:
pkgs: pkgconf libevent openssl
triplet: x64-windows
token: ${{ github.token }}
- name: Build and install
run: |
mkdir build && cd build
cmake .. -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=C:\vcpkg\scripts\buildsystems\vcpkg.cmake
ninja -v
cmake .. -G Ninja -DENABLE_TLS=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=${{ github.workspace }}\vcpkg\scripts\buildsystems\vcpkg.cmake
ninja -v install
- name: Run tests
working-directory: build
run: .\tests\client_test.exe
@ -256,9 +266,9 @@ jobs:
run: |
git config --global core.autocrlf input
choco install -y memurai-developer
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Install Cygwin
uses: cygwin/cygwin-install-action@f61179d72284ceddc397ed07ddb444d82bf9e559 # v5
uses: cygwin/cygwin-install-action@f2009323764960f80959895c7bc3bb30210afe4d # v6
with:
packages: make gcc-core cmake libssl-devel
- name: Build with CMake using Cygwin
@ -274,9 +284,9 @@ jobs:
name: Windows (MinGW64)
runs-on: windows-latest
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Set up MinGW
uses: msys2/setup-msys2@61f9e5e925871ba6c9e3e8da24ede83ea27fa91f # v2.27.0
uses: msys2/setup-msys2@40677d36a502eb2cf0fb808cc9dec31bf6152638 # v2.28.0
with:
msystem: mingw64
install: |
@ -284,9 +294,9 @@ jobs:
mingw-w64-x86_64-cmake
mingw-w64-x86_64-ninja
mingw-w64-x86_64-libevent
- name: Build
- name: Build and install
shell: msys2 {0}
run: |
mkdir build && cd build
cmake .. -G Ninja
cmake --build .
ninja -v install

View File

@ -12,9 +12,9 @@ jobs:
if: github.repository == 'valkey-io/libvalkey'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Install dependencies
uses: awalsh128/cache-apt-pkgs-action@7ca5f46d061ad9aa95863cd9b214dd48edef361d # v1.5.0
uses: awalsh128/cache-apt-pkgs-action@2c09a5e66da6c8016428a2172bd76e5e4f14bb17 # v1.5.3
with:
packages: libevent-dev
version: 1.0

View File

@ -18,7 +18,7 @@ jobs:
- valkey-version: '7.2.8'
steps:
- name: Prepare
uses: awalsh128/cache-apt-pkgs-action@7ca5f46d061ad9aa95863cd9b214dd48edef361d # v1.5.0
uses: awalsh128/cache-apt-pkgs-action@2c09a5e66da6c8016428a2172bd76e5e4f14bb17 # v1.5.3
with:
packages: libevent-dev
version: 1.0
@ -26,7 +26,7 @@ jobs:
run: |
git clone --depth 1 --branch ${{ matrix.valkey-version }} https://github.com/valkey-io/valkey.git
cd valkey && BUILD_TLS=yes make install
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Create build folder
run: cmake -E make_directory build
- name: Generate makefiles
@ -66,7 +66,7 @@ jobs:
- redis-version: '6.2.14'
steps:
- name: Prepare
uses: awalsh128/cache-apt-pkgs-action@7ca5f46d061ad9aa95863cd9b214dd48edef361d # v1.5.0
uses: awalsh128/cache-apt-pkgs-action@2c09a5e66da6c8016428a2172bd76e5e4f14bb17 # v1.5.3
with:
packages: libevent-dev
version: 1.0
@ -74,7 +74,7 @@ jobs:
run: |
git clone --depth 1 --branch ${{ matrix.redis-version }} https://github.com/redis/redis.git
cd redis && BUILD_TLS=yes make install
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Create build folder
run: cmake -E make_directory build

View File

@ -10,18 +10,18 @@ jobs:
spellcheck:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Run spellcheck
uses: rojopolis/spellcheck-github-actions@23dc186319866e1de224f94fe1d31b72797aeec7 # 0.48.0
uses: rojopolis/spellcheck-github-actions@35a02bae020e6999c5c37fabaf447f2eb8822ca7 # 0.51.0
with:
config_path: .github/spellcheck-settings.yml
task_name: Markdown
typos:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Install typos
uses: taiki-e/install-action@ab3728c7ba6948b9b429627f4d55a68842b27f18 # v2.50.3
uses: taiki-e/install-action@ad95d4e02e061d4390c4b66ef5ed56c7fee3d2ce # v2.58.17
with:
tool: typos
- name: Run typos

View File

@ -64,13 +64,12 @@ endif()
ADD_LIBRARY(valkey ${valkey_sources})
ADD_LIBRARY(valkey::valkey ALIAS valkey)
set(valkey_export_name valkey CACHE STRING "Name of the exported target")
set_target_properties(valkey PROPERTIES EXPORT_NAME ${valkey_export_name})
SET_TARGET_PROPERTIES(valkey
PROPERTIES WINDOWS_EXPORT_ALL_SYMBOLS TRUE
set_target_properties(valkey PROPERTIES
C_VISIBILITY_PRESET hidden
WINDOWS_EXPORT_ALL_SYMBOLS TRUE
SOVERSION "${VERSION_MAJOR}"
VERSION "${VERSION}")
IF(MSVC)
SET_TARGET_PROPERTIES(valkey
PROPERTIES COMPILE_FLAGS /Z7)
@ -132,8 +131,11 @@ if (MSVC AND BUILD_SHARED_LIBS)
CONFIGURATIONS Debug RelWithDebInfo)
endif()
# Install public headers
install(DIRECTORY include/valkey
DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})
DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}
PATTERN "tls.h" EXCLUDE
PATTERN "rdma.h" EXCLUDE)
INSTALL(FILES ${CMAKE_CURRENT_BINARY_DIR}/valkey.pc
DESTINATION ${CMAKE_INSTALL_LIBDIR}/pkgconfig)
@ -189,8 +191,8 @@ IF(ENABLE_TLS)
SET_PROPERTY(TARGET valkey_tls PROPERTY LINK_FLAGS "-Wl,-undefined -Wl,dynamic_lookup")
ENDIF()
SET_TARGET_PROPERTIES(valkey_tls
PROPERTIES
set_target_properties(valkey_tls PROPERTIES
C_VISIBILITY_PRESET hidden
WINDOWS_EXPORT_ALL_SYMBOLS TRUE
SOVERSION "${VERSION_MAJOR}"
VERSION "${VERSION}")
@ -210,6 +212,10 @@ IF(ENABLE_TLS)
LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR})
# Install public header
install(FILES include/valkey/tls.h
DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/valkey)
if (MSVC AND BUILD_SHARED_LIBS)
INSTALL(FILES $<TARGET_PDB_FILE:valkey_tls>
DESTINATION ${CMAKE_INSTALL_BINDIR}
@ -257,8 +263,8 @@ if(ENABLE_RDMA)
$<BUILD_INTERFACE:${SDS_INCLUDE_DIR}>
)
set_target_properties(valkey_rdma
PROPERTIES
set_target_properties(valkey_rdma PROPERTIES
C_VISIBILITY_PRESET hidden
WINDOWS_EXPORT_ALL_SYMBOLS TRUE
SOVERSION "${VERSION_MAJOR}"
VERSION "${VERSION}")
@ -270,6 +276,10 @@ if(ENABLE_RDMA)
LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR})
# Install public header
install(FILES include/valkey/rdma.h
DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/valkey)
install(FILES ${CMAKE_CURRENT_BINARY_DIR}/valkey_rdma.pc
DESTINATION ${CMAKE_INSTALL_LIBDIR}/pkgconfig)
@ -293,6 +303,24 @@ endif()
# Add tests
if(NOT DISABLE_TESTS)
if(BUILD_SHARED_LIBS)
# Test using a static library since symbols are not hidden then.
# Use same source, include dirs and dependencies as the shared library.
add_library(valkey_unittest STATIC ${valkey_sources})
get_target_property(include_directories valkey::valkey INCLUDE_DIRECTORIES)
target_include_directories(valkey_unittest PUBLIC ${include_directories})
get_target_property(link_libraries valkey::valkey LINK_LIBRARIES)
if(link_libraries)
target_link_libraries(valkey_unittest PUBLIC ${link_libraries})
endif()
# Create libvalkey_unittest.a in the tests directory.
set_target_properties(valkey_unittest PROPERTIES
ARCHIVE_OUTPUT_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/tests")
else()
# Target is an alias for the static library.
add_library(valkey_unittest ALIAS valkey)
endif()
# Make sure ctest prints the output when a test fails.
# Must be set before including CTest.
set(CMAKE_CTEST_ARGUMENTS "--output-on-failure")

View File

@ -76,7 +76,7 @@ ifeq ($(USE_WERROR),1)
WARNINGS+=-Werror
endif
DEBUG_FLAGS?= -g -ggdb
REAL_CFLAGS=$(OPTIMIZATION) -fPIC $(CPPFLAGS) $(CFLAGS) $(WARNINGS) $(DEBUG_FLAGS) $(PLATFORM_FLAGS)
REAL_CFLAGS=$(OPTIMIZATION) -fPIC -fvisibility=hidden $(CPPFLAGS) $(CFLAGS) $(WARNINGS) $(DEBUG_FLAGS) $(PLATFORM_FLAGS)
REAL_LDFLAGS=$(LDFLAGS)
DYLIBSUFFIX=so

View File

@ -12,88 +12,85 @@ else()
include_directories(${CMAKE_SOURCE_DIR}/include)
endif()
# Check for GLib
find_package(PkgConfig QUIET)
if(PkgConfig_FOUND)
pkg_check_modules(GLIB2 glib-2.0)
if(GLIB2_FOUND)
add_executable(example-async-glib async-glib.c)
target_include_directories(example-async-glib PUBLIC ${GLIB2_INCLUDE_DIRS})
target_link_libraries(example-async-glib valkey::valkey ${GLIB2_LIBRARIES})
endif()
endif()
find_path(LIBEV ev.h
HINTS /usr/local /usr/opt/local
ENV LIBEV_INCLUDE_DIR)
if(LIBEV)
add_executable(example-async-libev async-libev.c)
target_link_libraries(example-async-libev valkey::valkey ev)
endif()
find_path(LIBEVENT event.h)
if(LIBEVENT)
add_executable(example-async-libevent async-libevent.c)
target_link_libraries(example-async-libevent valkey::valkey event)
if(ENABLE_TLS)
add_executable(example-async-libevent-tls async-libevent-tls.c)
target_link_libraries(example-async-libevent-tls valkey::valkey valkey::valkey_tls event)
endif()
endif()
find_path(LIBHV hv/hv.h)
if(LIBHV)
add_executable(example-async-libhv async-libhv.c)
target_link_libraries(example-async-libhv valkey::valkey hv)
endif()
find_path(LIBUV uv.h)
if(LIBUV)
add_executable(example-async-libuv async-libuv.c)
target_link_libraries(example-async-libuv valkey::valkey uv)
endif()
find_path(LIBSDEVENT systemd/sd-event.h)
if(LIBSDEVENT)
add_executable(example-async-libsdevent async-libsdevent.c)
target_link_libraries(example-async-libsdevent valkey::valkey systemd)
endif()
if(APPLE)
find_library(CF CoreFoundation)
add_executable(example-async-macosx async-macosx.c)
target_link_libraries(example-async-macosx valkey::valkey ${CF})
endif()
if(ENABLE_TLS)
add_executable(example-blocking-tls blocking-tls.c)
target_link_libraries(example-blocking-tls valkey::valkey valkey::valkey_tls)
endif()
add_executable(example-blocking blocking.c)
target_link_libraries(example-blocking valkey::valkey)
add_executable(example-blocking-push blocking-push.c)
target_link_libraries(example-blocking-push valkey::valkey)
if(LIBEVENT)
add_executable(example-cluster-async cluster-async.c)
target_link_libraries(example-cluster-async valkey::valkey event)
if(ENABLE_TLS)
add_executable(example-cluster-async-tls cluster-async-tls.c)
target_link_libraries(example-cluster-async-tls valkey::valkey valkey::valkey_tls event)
endif()
add_executable(example-cluster-clientside-caching-async cluster-clientside-caching-async.c)
target_link_libraries(example-cluster-clientside-caching-async valkey::valkey event)
endif()
add_executable(example-cluster-simple cluster-simple.c)
target_link_libraries(example-cluster-simple valkey::valkey)
if(ENABLE_TLS)
add_executable(example-blocking-tls blocking-tls.c)
target_link_libraries(example-blocking-tls valkey::valkey valkey::valkey_tls)
add_executable(example-cluster-tls cluster-tls.c)
target_link_libraries(example-cluster-tls valkey::valkey valkey::valkey_tls)
endif()
# Examples using GLib
find_package(PkgConfig QUIET)
if(PkgConfig_FOUND)
pkg_check_modules(GLIB2 IMPORTED_TARGET glib-2.0)
if(GLIB2_FOUND)
add_executable(example-async-glib async-glib.c)
target_link_libraries(example-async-glib valkey::valkey PkgConfig::GLIB2)
endif()
endif()
# Examples using libev
find_path(LIBEV ev.h)
if(LIBEV)
add_executable(example-async-libev async-libev.c)
target_link_libraries(example-async-libev valkey::valkey ev)
endif()
# Examples using libevent
find_path(LIBEVENT event.h)
if(LIBEVENT)
add_executable(example-async-libevent async-libevent.c)
target_link_libraries(example-async-libevent valkey::valkey event)
add_executable(example-cluster-async cluster-async.c)
target_link_libraries(example-cluster-async valkey::valkey event)
add_executable(example-cluster-clientside-caching-async cluster-clientside-caching-async.c)
target_link_libraries(example-cluster-clientside-caching-async valkey::valkey event)
if(ENABLE_TLS)
add_executable(example-async-libevent-tls async-libevent-tls.c)
target_link_libraries(example-async-libevent-tls valkey::valkey valkey::valkey_tls event)
add_executable(example-cluster-async-tls cluster-async-tls.c)
target_link_libraries(example-cluster-async-tls valkey::valkey valkey::valkey_tls event)
endif()
endif()
# Examples using libhv
find_path(LIBHV hv/hv.h)
if(LIBHV)
add_executable(example-async-libhv async-libhv.c)
target_link_libraries(example-async-libhv valkey::valkey hv)
endif()
# Examples using libuv
find_path(LIBUV uv.h)
if(LIBUV)
add_executable(example-async-libuv async-libuv.c)
target_link_libraries(example-async-libuv valkey::valkey uv)
endif()
# Examples using libsystemd
find_path(LIBSDEVENT systemd/sd-event.h)
if(LIBSDEVENT)
add_executable(example-async-libsdevent async-libsdevent.c)
target_link_libraries(example-async-libsdevent valkey::valkey systemd)
endif()
# Examples using the RunLoop in Apple's CoreFoundation
if(APPLE)
find_library(CF CoreFoundation)
add_executable(example-async-macosx async-macosx.c)
target_link_libraries(example-async-macosx valkey::valkey ${CF})
endif()

View File

@ -1,9 +1,12 @@
CC?=gcc
CXX?=g++
WARNINGS=-Wall -Wextra
CFLAGS?=-fPIC -g -O2 $(WARNINGS) -I../include
STLIBNAME?=../lib/libvalkey.a
TLS_STLIBNAME?=../lib/libvalkey_tls.a
INCLUDE_DIR?=../include
WARNINGS=-Wall -Wextra
CFLAGS?=-fPIC -g -O2 $(WARNINGS) -I $(INCLUDE_DIR)
USE_WERROR?=1
ifeq ($(USE_WERROR),1)
@ -19,7 +22,6 @@ EXAMPLES=example-blocking example-blocking-push example-async-libevent \
ifeq ($(USE_TLS),1)
EXAMPLES+=example-blocking-tls example-async-libevent-tls \
example-cluster-async-tls example-cluster-tls
TLS_STLIBNAME=../lib/libvalkey_tls.a
TLS_LDFLAGS=-lssl -lcrypto
endif

View File

@ -51,10 +51,7 @@ void disconnectCallback(const valkeyAsyncContext *ac, int status) {
printf("Disconnected from %s:%d\n", ac->c.tcp.host, ac->c.tcp.port);
}
int main(int argc, char **argv) {
UNUSED(argc);
UNUSED(argv);
int main(void) {
valkeyTLSContext *tls;
valkeyTLSContextError tls_error;

View File

@ -46,9 +46,7 @@ void disconnectCallback(const valkeyAsyncContext *ac, int status) {
printf("Disconnected from %s:%d\n", ac->c.tcp.host, ac->c.tcp.port);
}
int main(int argc, char **argv) {
(void)argc;
(void)argv;
int main(void) {
struct event_base *base = event_base_new();
valkeyClusterOptions options = {0};

View File

@ -139,9 +139,7 @@ void modifyKey(const char *key, const char *value) {
valkeyClusterFree(cc);
}
int main(int argc, char **argv) {
(void)argc;
(void)argv;
int main(void) {
struct event_base *base = event_base_new();
valkeyClusterOptions options = {0};

View File

@ -3,9 +3,7 @@
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char **argv) {
UNUSED(argc);
UNUSED(argv);
int main(void) {
struct timeval timeout = {1, 500000}; // 1.5s
valkeyClusterOptions options = {0};

View File

@ -6,10 +6,7 @@
#define CLUSTER_NODE_TLS "127.0.0.1:7301"
int main(int argc, char **argv) {
UNUSED(argc);
UNUSED(argv);
int main(void) {
valkeyTLSContext *tls;
valkeyTLSContextError tls_error;

View File

@ -142,7 +142,7 @@ static void valkeyLibuvTimeout(uv_timer_t *timer) {
static void valkeyLibuvSetTimeout(void *privdata, struct timeval tv) {
valkeyLibuvEvents *p = (valkeyLibuvEvents *)privdata;
uint64_t millsec = tv.tv_sec * 1000 + tv.tv_usec / 1000.0;
uint64_t millisec = tv.tv_sec * 1000 + tv.tv_usec / 1000.0;
if (!p->timer.data) {
// timer is uninitialized
if (uv_timer_init(p->handle.loop, &p->timer) != 0) {
@ -152,7 +152,7 @@ static void valkeyLibuvSetTimeout(void *privdata, struct timeval tv) {
}
// updates the timeout if the timer has already started
// or start the timer
uv_timer_start(&p->timer, valkeyLibuvTimeout, millsec, 0);
uv_timer_start(&p->timer, valkeyLibuvTimeout, millisec, 0);
}
static void valkeyLibuvCleanup(void *privdata) {

View File

@ -100,7 +100,7 @@ static int valkeyPollTick(valkeyAsyncContext *ac, double timeout) {
* handle it the same as writable. */
if (writing && (pfd.revents & (POLLOUT | POLLERR))) {
/* context Read callback may have caused context to be deleted, e.g.
by doing an valkeyAsyncDisconnect() */
by doing a valkeyAsyncDisconnect() */
if (!e->deleted) {
valkeyAsyncHandleWrite(ac);
handled |= VALKEY_POLL_HANDLED_WRITE;

View File

@ -30,6 +30,7 @@
#ifndef VALKEY_ALLOC_H
#define VALKEY_ALLOC_H
#include "visibility.h"
#include <stddef.h> /* for size_t */
#include <stdint.h>
@ -47,13 +48,13 @@ typedef struct valkeyAllocFuncs {
void (*freeFn)(void *);
} valkeyAllocFuncs;
valkeyAllocFuncs valkeySetAllocators(valkeyAllocFuncs *fns);
void valkeyResetAllocators(void);
LIBVALKEY_API valkeyAllocFuncs valkeySetAllocators(valkeyAllocFuncs *fns);
LIBVALKEY_API void valkeyResetAllocators(void);
#ifndef _WIN32
/* valkey' configured allocator function pointer struct */
extern valkeyAllocFuncs valkeyAllocFns;
LIBVALKEY_API extern valkeyAllocFuncs valkeyAllocFns;
static inline void *vk_malloc(size_t size) {
return valkeyAllocFns.mallocFn(size);
@ -81,11 +82,11 @@ static inline void vk_free(void *ptr) {
#else
void *vk_malloc(size_t size);
void *vk_calloc(size_t nmemb, size_t size);
void *vk_realloc(void *ptr, size_t size);
char *vk_strdup(const char *str);
void vk_free(void *ptr);
LIBVALKEY_API void *vk_malloc(size_t size);
LIBVALKEY_API void *vk_calloc(size_t nmemb, size_t size);
LIBVALKEY_API void *vk_realloc(void *ptr, size_t size);
LIBVALKEY_API char *vk_strdup(const char *str);
LIBVALKEY_API void vk_free(void *ptr);
#endif

View File

@ -32,6 +32,7 @@
#ifndef VALKEY_ASYNC_H
#define VALKEY_ASYNC_H
#include "valkey.h"
#include "visibility.h"
#ifdef __cplusplus
extern "C" {
@ -122,34 +123,33 @@ typedef struct valkeyAsyncContext {
valkeyAsyncPushFn *push_cb;
} valkeyAsyncContext;
/* Functions that proxy to libvalkey */
valkeyAsyncContext *valkeyAsyncConnectWithOptions(const valkeyOptions *options);
valkeyAsyncContext *valkeyAsyncConnect(const char *ip, int port);
valkeyAsyncContext *valkeyAsyncConnectBind(const char *ip, int port, const char *source_addr);
valkeyAsyncContext *valkeyAsyncConnectBindWithReuse(const char *ip, int port,
LIBVALKEY_API valkeyAsyncContext *valkeyAsyncConnectWithOptions(const valkeyOptions *options);
LIBVALKEY_API valkeyAsyncContext *valkeyAsyncConnect(const char *ip, int port);
LIBVALKEY_API valkeyAsyncContext *valkeyAsyncConnectBind(const char *ip, int port, const char *source_addr);
LIBVALKEY_API valkeyAsyncContext *valkeyAsyncConnectBindWithReuse(const char *ip, int port,
const char *source_addr);
valkeyAsyncContext *valkeyAsyncConnectUnix(const char *path);
int valkeyAsyncSetConnectCallback(valkeyAsyncContext *ac, valkeyConnectCallback *fn);
int valkeyAsyncSetDisconnectCallback(valkeyAsyncContext *ac, valkeyDisconnectCallback *fn);
LIBVALKEY_API valkeyAsyncContext *valkeyAsyncConnectUnix(const char *path);
LIBVALKEY_API int valkeyAsyncSetConnectCallback(valkeyAsyncContext *ac, valkeyConnectCallback *fn);
LIBVALKEY_API int valkeyAsyncSetDisconnectCallback(valkeyAsyncContext *ac, valkeyDisconnectCallback *fn);
valkeyAsyncPushFn *valkeyAsyncSetPushCallback(valkeyAsyncContext *ac, valkeyAsyncPushFn *fn);
int valkeyAsyncSetTimeout(valkeyAsyncContext *ac, struct timeval tv);
void valkeyAsyncDisconnect(valkeyAsyncContext *ac);
void valkeyAsyncFree(valkeyAsyncContext *ac);
LIBVALKEY_API valkeyAsyncPushFn *valkeyAsyncSetPushCallback(valkeyAsyncContext *ac, valkeyAsyncPushFn *fn);
LIBVALKEY_API int valkeyAsyncSetTimeout(valkeyAsyncContext *ac, struct timeval tv);
LIBVALKEY_API void valkeyAsyncDisconnect(valkeyAsyncContext *ac);
LIBVALKEY_API void valkeyAsyncFree(valkeyAsyncContext *ac);
/* Handle read/write events */
void valkeyAsyncHandleRead(valkeyAsyncContext *ac);
void valkeyAsyncHandleWrite(valkeyAsyncContext *ac);
void valkeyAsyncHandleTimeout(valkeyAsyncContext *ac);
void valkeyAsyncRead(valkeyAsyncContext *ac);
void valkeyAsyncWrite(valkeyAsyncContext *ac);
LIBVALKEY_API void valkeyAsyncHandleRead(valkeyAsyncContext *ac);
LIBVALKEY_API void valkeyAsyncHandleWrite(valkeyAsyncContext *ac);
LIBVALKEY_API void valkeyAsyncHandleTimeout(valkeyAsyncContext *ac);
LIBVALKEY_API void valkeyAsyncRead(valkeyAsyncContext *ac);
LIBVALKEY_API void valkeyAsyncWrite(valkeyAsyncContext *ac);
/* Command functions for an async context. Write the command to the
* output buffer and register the provided callback. */
int valkeyvAsyncCommand(valkeyAsyncContext *ac, valkeyCallbackFn *fn, void *privdata, const char *format, va_list ap);
int valkeyAsyncCommand(valkeyAsyncContext *ac, valkeyCallbackFn *fn, void *privdata, const char *format, ...);
int valkeyAsyncCommandArgv(valkeyAsyncContext *ac, valkeyCallbackFn *fn, void *privdata, int argc, const char **argv, const size_t *argvlen);
int valkeyAsyncFormattedCommand(valkeyAsyncContext *ac, valkeyCallbackFn *fn, void *privdata, const char *cmd, size_t len);
LIBVALKEY_API int valkeyvAsyncCommand(valkeyAsyncContext *ac, valkeyCallbackFn *fn, void *privdata, const char *format, va_list ap);
LIBVALKEY_API int valkeyAsyncCommand(valkeyAsyncContext *ac, valkeyCallbackFn *fn, void *privdata, const char *format, ...);
LIBVALKEY_API int valkeyAsyncCommandArgv(valkeyAsyncContext *ac, valkeyCallbackFn *fn, void *privdata, int argc, const char **argv, const size_t *argvlen);
LIBVALKEY_API int valkeyAsyncFormattedCommand(valkeyAsyncContext *ac, valkeyCallbackFn *fn, void *privdata, const char *cmd, size_t len);
#ifdef __cplusplus
}

View File

@ -35,8 +35,7 @@
#include "async.h"
#include "valkey.h"
#define UNUSED(x) (void)(x)
#include "visibility.h"
#define VALKEYCLUSTER_SLOTS 16384
@ -214,13 +213,13 @@ typedef struct {
/* --- Synchronous API --- */
valkeyClusterContext *valkeyClusterConnectWithOptions(const valkeyClusterOptions *options);
valkeyClusterContext *valkeyClusterConnect(const char *addrs);
valkeyClusterContext *valkeyClusterConnectWithTimeout(const char *addrs, const struct timeval tv);
void valkeyClusterFree(valkeyClusterContext *cc);
LIBVALKEY_API valkeyClusterContext *valkeyClusterConnectWithOptions(const valkeyClusterOptions *options);
LIBVALKEY_API valkeyClusterContext *valkeyClusterConnect(const char *addrs);
LIBVALKEY_API valkeyClusterContext *valkeyClusterConnectWithTimeout(const char *addrs, const struct timeval tv);
LIBVALKEY_API void valkeyClusterFree(valkeyClusterContext *cc);
/* Options configurable in runtime. */
int valkeyClusterSetOptionTimeout(valkeyClusterContext *cc, const struct timeval tv);
LIBVALKEY_API int valkeyClusterSetOptionTimeout(valkeyClusterContext *cc, const struct timeval tv);
/* Blocking
* The following functions will block for a reply, or return NULL if there was
@ -228,21 +227,21 @@ int valkeyClusterSetOptionTimeout(valkeyClusterContext *cc, const struct timeval
*/
/* Variadic commands (like printf) */
void *valkeyClusterCommand(valkeyClusterContext *cc, const char *format, ...);
void *valkeyClusterCommandToNode(valkeyClusterContext *cc,
LIBVALKEY_API void *valkeyClusterCommand(valkeyClusterContext *cc, const char *format, ...);
LIBVALKEY_API void *valkeyClusterCommandToNode(valkeyClusterContext *cc,
valkeyClusterNode *node, const char *format,
...);
/* Variadic using va_list */
void *valkeyClustervCommand(valkeyClusterContext *cc, const char *format,
LIBVALKEY_API void *valkeyClustervCommand(valkeyClusterContext *cc, const char *format,
va_list ap);
void *valkeyClustervCommandToNode(valkeyClusterContext *cc,
LIBVALKEY_API void *valkeyClustervCommandToNode(valkeyClusterContext *cc,
valkeyClusterNode *node, const char *format,
va_list ap);
/* Using argc and argv */
void *valkeyClusterCommandArgv(valkeyClusterContext *cc, int argc,
LIBVALKEY_API void *valkeyClusterCommandArgv(valkeyClusterContext *cc, int argc,
const char **argv, const size_t *argvlen);
/* Send a Valkey protocol encoded string */
void *valkeyClusterFormattedCommand(valkeyClusterContext *cc, char *cmd,
LIBVALKEY_API void *valkeyClusterFormattedCommand(valkeyClusterContext *cc, char *cmd,
int len);
/* Pipelining
@ -252,59 +251,59 @@ void *valkeyClusterFormattedCommand(valkeyClusterContext *cc, char *cmd,
*/
/* Variadic commands (like printf) */
int valkeyClusterAppendCommand(valkeyClusterContext *cc, const char *format,
LIBVALKEY_API int valkeyClusterAppendCommand(valkeyClusterContext *cc, const char *format,
...);
int valkeyClusterAppendCommandToNode(valkeyClusterContext *cc,
LIBVALKEY_API int valkeyClusterAppendCommandToNode(valkeyClusterContext *cc,
valkeyClusterNode *node,
const char *format, ...);
/* Variadic using va_list */
int valkeyClustervAppendCommand(valkeyClusterContext *cc, const char *format,
LIBVALKEY_API int valkeyClustervAppendCommand(valkeyClusterContext *cc, const char *format,
va_list ap);
int valkeyClustervAppendCommandToNode(valkeyClusterContext *cc,
LIBVALKEY_API int valkeyClustervAppendCommandToNode(valkeyClusterContext *cc,
valkeyClusterNode *node,
const char *format, va_list ap);
/* Using argc and argv */
int valkeyClusterAppendCommandArgv(valkeyClusterContext *cc, int argc,
LIBVALKEY_API int valkeyClusterAppendCommandArgv(valkeyClusterContext *cc, int argc,
const char **argv, const size_t *argvlen);
/* Use a Valkey protocol encoded string as command */
int valkeyClusterAppendFormattedCommand(valkeyClusterContext *cc, char *cmd,
LIBVALKEY_API int valkeyClusterAppendFormattedCommand(valkeyClusterContext *cc, char *cmd,
int len);
/* Flush output buffer and return first reply */
int valkeyClusterGetReply(valkeyClusterContext *cc, void **reply);
LIBVALKEY_API int valkeyClusterGetReply(valkeyClusterContext *cc, void **reply);
/* Reset context after a performed pipelining */
void valkeyClusterReset(valkeyClusterContext *cc);
LIBVALKEY_API void valkeyClusterReset(valkeyClusterContext *cc);
/* Update the slotmap by querying any node. */
int valkeyClusterUpdateSlotmap(valkeyClusterContext *cc);
LIBVALKEY_API int valkeyClusterUpdateSlotmap(valkeyClusterContext *cc);
/* Get the valkeyContext used for communication with a given node.
* Connects or reconnects to the node if necessary. */
valkeyContext *valkeyClusterGetValkeyContext(valkeyClusterContext *cc,
LIBVALKEY_API valkeyContext *valkeyClusterGetValkeyContext(valkeyClusterContext *cc,
valkeyClusterNode *node);
/* --- Asynchronous API --- */
valkeyClusterAsyncContext *valkeyClusterAsyncConnectWithOptions(const valkeyClusterOptions *options);
void valkeyClusterAsyncDisconnect(valkeyClusterAsyncContext *acc);
void valkeyClusterAsyncFree(valkeyClusterAsyncContext *acc);
LIBVALKEY_API valkeyClusterAsyncContext *valkeyClusterAsyncConnectWithOptions(const valkeyClusterOptions *options);
LIBVALKEY_API void valkeyClusterAsyncDisconnect(valkeyClusterAsyncContext *acc);
LIBVALKEY_API void valkeyClusterAsyncFree(valkeyClusterAsyncContext *acc);
/* Commands */
int valkeyClusterAsyncCommand(valkeyClusterAsyncContext *acc,
LIBVALKEY_API int valkeyClusterAsyncCommand(valkeyClusterAsyncContext *acc,
valkeyClusterCallbackFn *fn, void *privdata,
const char *format, ...);
int valkeyClusterAsyncCommandToNode(valkeyClusterAsyncContext *acc,
LIBVALKEY_API int valkeyClusterAsyncCommandToNode(valkeyClusterAsyncContext *acc,
valkeyClusterNode *node,
valkeyClusterCallbackFn *fn, void *privdata,
const char *format, ...);
int valkeyClustervAsyncCommand(valkeyClusterAsyncContext *acc,
LIBVALKEY_API int valkeyClustervAsyncCommand(valkeyClusterAsyncContext *acc,
valkeyClusterCallbackFn *fn, void *privdata,
const char *format, va_list ap);
int valkeyClusterAsyncCommandArgv(valkeyClusterAsyncContext *acc,
LIBVALKEY_API int valkeyClusterAsyncCommandArgv(valkeyClusterAsyncContext *acc,
valkeyClusterCallbackFn *fn, void *privdata,
int argc, const char **argv,
const size_t *argvlen);
int valkeyClusterAsyncCommandArgvToNode(valkeyClusterAsyncContext *acc,
LIBVALKEY_API int valkeyClusterAsyncCommandArgvToNode(valkeyClusterAsyncContext *acc,
valkeyClusterNode *node,
valkeyClusterCallbackFn *fn,
void *privdata, int argc,
@ -312,10 +311,10 @@ int valkeyClusterAsyncCommandArgvToNode(valkeyClusterAsyncContext *acc,
const size_t *argvlen);
/* Use a Valkey protocol encoded string as command */
int valkeyClusterAsyncFormattedCommand(valkeyClusterAsyncContext *acc,
LIBVALKEY_API int valkeyClusterAsyncFormattedCommand(valkeyClusterAsyncContext *acc,
valkeyClusterCallbackFn *fn,
void *privdata, char *cmd, int len);
int valkeyClusterAsyncFormattedCommandToNode(valkeyClusterAsyncContext *acc,
LIBVALKEY_API int valkeyClusterAsyncFormattedCommandToNode(valkeyClusterAsyncContext *acc,
valkeyClusterNode *node,
valkeyClusterCallbackFn *fn,
void *privdata, char *cmd,
@ -323,17 +322,17 @@ int valkeyClusterAsyncFormattedCommandToNode(valkeyClusterAsyncContext *acc,
/* Get the valkeyAsyncContext used for communication with a given node.
* Connects or reconnects to the node if necessary. */
valkeyAsyncContext *valkeyClusterGetValkeyAsyncContext(valkeyClusterAsyncContext *acc,
LIBVALKEY_API valkeyAsyncContext *valkeyClusterGetValkeyAsyncContext(valkeyClusterAsyncContext *acc,
valkeyClusterNode *node);
/* Cluster node iterator functions */
void valkeyClusterInitNodeIterator(valkeyClusterNodeIterator *iter,
LIBVALKEY_API void valkeyClusterInitNodeIterator(valkeyClusterNodeIterator *iter,
valkeyClusterContext *cc);
valkeyClusterNode *valkeyClusterNodeNext(valkeyClusterNodeIterator *iter);
LIBVALKEY_API valkeyClusterNode *valkeyClusterNodeNext(valkeyClusterNodeIterator *iter);
/* Helper functions */
unsigned int valkeyClusterGetSlotByKey(char *key);
valkeyClusterNode *valkeyClusterGetNodeByKey(valkeyClusterContext *cc,
LIBVALKEY_API unsigned int valkeyClusterGetSlotByKey(char *key);
LIBVALKEY_API valkeyClusterNode *valkeyClusterGetNodeByKey(valkeyClusterContext *cc,
char *key);
#ifdef __cplusplus

View File

@ -36,17 +36,18 @@
#define VALKEY_NET_H
#include "valkey.h"
#include "visibility.h"
void valkeyNetClose(valkeyContext *c);
LIBVALKEY_API void valkeyNetClose(valkeyContext *c);
int valkeyHasMptcp(void);
int valkeyCheckSocketError(valkeyContext *c);
int valkeyTcpSetTimeout(valkeyContext *c, const struct timeval tv);
int valkeyContextConnectTcp(valkeyContext *c, const valkeyOptions *options);
int valkeyKeepAlive(valkeyContext *c, int interval);
int valkeyCheckConnectDone(valkeyContext *c, int *completed);
LIBVALKEY_API int valkeyHasMptcp(void);
LIBVALKEY_API int valkeyCheckSocketError(valkeyContext *c);
LIBVALKEY_API int valkeyTcpSetTimeout(valkeyContext *c, const struct timeval tv);
LIBVALKEY_API int valkeyContextConnectTcp(valkeyContext *c, const valkeyOptions *options);
LIBVALKEY_API int valkeyKeepAlive(valkeyContext *c, int interval);
LIBVALKEY_API int valkeyCheckConnectDone(valkeyContext *c, int *completed);
int valkeySetTcpNoDelay(valkeyContext *c);
int valkeyContextSetTcpUserTimeout(valkeyContext *c, unsigned int timeout);
LIBVALKEY_API int valkeySetTcpNoDelay(valkeyContext *c);
LIBVALKEY_API int valkeyContextSetTcpUserTimeout(valkeyContext *c, unsigned int timeout);
#endif /* VALKEY_NET_H */

View File

@ -35,6 +35,8 @@
#ifndef VALKEY_RDMA_H
#define VALKEY_RDMA_H
#include "visibility.h"
#ifdef __cplusplus
extern "C" {
#endif
@ -58,7 +60,7 @@ extern "C" {
(opts)->endpoint.tcp.source_addr = source_addr_; \
} while (0)
int valkeyInitiateRdma(void);
LIBVALKEY_API int valkeyInitiateRdma(void);
#ifdef __cplusplus
}

View File

@ -31,6 +31,8 @@
#ifndef VALKEY_READ_H
#define VALKEY_READ_H
#include "visibility.h"
#include <stdio.h> /* for size_t */
#define VALKEY_ERR -1
@ -112,10 +114,10 @@ typedef struct valkeyReader {
} valkeyReader;
/* Public API for the protocol parser. */
valkeyReader *valkeyReaderCreateWithFunctions(valkeyReplyObjectFunctions *fn);
void valkeyReaderFree(valkeyReader *r);
int valkeyReaderFeed(valkeyReader *r, const char *buf, size_t len);
int valkeyReaderGetReply(valkeyReader *r, void **reply);
LIBVALKEY_API valkeyReader *valkeyReaderCreateWithFunctions(valkeyReplyObjectFunctions *fn);
LIBVALKEY_API void valkeyReaderFree(valkeyReader *r);
LIBVALKEY_API int valkeyReaderFeed(valkeyReader *r, const char *buf, size_t len);
LIBVALKEY_API int valkeyReaderGetReply(valkeyReader *r, void **reply);
#define valkeyReaderSetPrivdata(_r, _p) (int)(((valkeyReader *)(_r))->privdata = (_p))
#define valkeyReaderGetObject(_r) (((valkeyReader *)(_r))->reply)

View File

@ -31,6 +31,8 @@
#ifndef VALKEY_TLS_H
#define VALKEY_TLS_H
#include "visibility.h"
#ifdef __cplusplus
extern "C" {
#endif
@ -87,7 +89,7 @@ typedef struct {
/**
* Return the error message corresponding with the specified error code.
*/
const char *valkeyTLSContextGetError(valkeyTLSContextError error);
LIBVALKEY_API const char *valkeyTLSContextGetError(valkeyTLSContextError error);
/**
* Helper function to initialize the OpenSSL library.
@ -96,7 +98,7 @@ const char *valkeyTLSContextGetError(valkeyTLSContextError error);
* call this function only once, and only if OpenSSL is not directly initialized
* elsewhere.
*/
int valkeyInitOpenSSL(void);
LIBVALKEY_API int valkeyInitOpenSSL(void);
/**
* Helper function to initialize an OpenSSL context that can be used
@ -118,7 +120,7 @@ int valkeyInitOpenSSL(void);
* If error is non-null, it will be populated in case the context creation fails
* (returning a NULL).
*/
valkeyTLSContext *valkeyCreateTLSContext(const char *cacert_filename, const char *capath,
LIBVALKEY_API valkeyTLSContext *valkeyCreateTLSContext(const char *cacert_filename, const char *capath,
const char *cert_filename, const char *private_key_filename,
const char *server_name, valkeyTLSContextError *error);
@ -131,13 +133,13 @@ valkeyTLSContext *valkeyCreateTLSContext(const char *cacert_filename, const char
* If error is non-null, it will be populated in case the context creation fails
* (returning a NULL).
*/
valkeyTLSContext *valkeyCreateTLSContextWithOptions(valkeyTLSOptions *options,
LIBVALKEY_API valkeyTLSContext *valkeyCreateTLSContextWithOptions(valkeyTLSOptions *options,
valkeyTLSContextError *error);
/**
* Free a previously created OpenSSL context.
*/
void valkeyFreeTLSContext(valkeyTLSContext *valkey_tls_ctx);
LIBVALKEY_API void valkeyFreeTLSContext(valkeyTLSContext *valkey_tls_ctx);
/**
* Initiate TLS on an existing valkeyContext.
@ -146,12 +148,12 @@ void valkeyFreeTLSContext(valkeyTLSContext *valkey_tls_ctx);
* to directly interact with OpenSSL, and instead uses a valkeyTLSContext
* previously created using valkeyCreateTLSContext().
*/
int valkeyInitiateTLSWithContext(struct valkeyContext *c, valkeyTLSContext *valkey_tls_ctx);
LIBVALKEY_API int valkeyInitiateTLSWithContext(struct valkeyContext *c, valkeyTLSContext *valkey_tls_ctx);
/**
* Initiate TLS negotiation on a provided OpenSSL SSL object.
*/
int valkeyInitiateTLS(struct valkeyContext *c, struct ssl_st *ssl);
LIBVALKEY_API int valkeyInitiateTLS(struct valkeyContext *c, struct ssl_st *ssl);
#ifdef __cplusplus
}

View File

@ -34,6 +34,7 @@
#ifndef VALKEY_VALKEY_H
#define VALKEY_VALKEY_H
#include "read.h"
#include "visibility.h"
#include <stdarg.h> /* for va_list */
#ifndef _MSC_VER
@ -49,8 +50,8 @@ typedef SSIZE_T ssize_t;
#include <stdint.h> /* uintXX_t, etc */
#define LIBVALKEY_VERSION_MAJOR 0
#define LIBVALKEY_VERSION_MINOR 1
#define LIBVALKEY_VERSION_PATCH 0
#define LIBVALKEY_VERSION_MINOR 2
#define LIBVALKEY_VERSION_PATCH 1
/* Connection type can be blocking or non-blocking and is set in the
* least significant bit of the flags field in valkeyContext. */
@ -138,16 +139,16 @@ typedef struct valkeyReply {
struct valkeyReply **element; /* elements vector for VALKEY_REPLY_ARRAY */
} valkeyReply;
valkeyReader *valkeyReaderCreate(void);
LIBVALKEY_API valkeyReader *valkeyReaderCreate(void);
/* Function to free the reply objects hivalkey returns by default. */
void freeReplyObject(void *reply);
LIBVALKEY_API void freeReplyObject(void *reply);
/* Functions to format a command according to the protocol. */
int valkeyvFormatCommand(char **target, const char *format, va_list ap);
int valkeyFormatCommand(char **target, const char *format, ...);
long long valkeyFormatCommandArgv(char **target, int argc, const char **argv, const size_t *argvlen);
void valkeyFreeCommand(char *cmd);
LIBVALKEY_API int valkeyvFormatCommand(char **target, const char *format, va_list ap);
LIBVALKEY_API int valkeyFormatCommand(char **target, const char *format, ...);
LIBVALKEY_API long long valkeyFormatCommandArgv(char **target, int argc, const char **argv, const size_t *argvlen);
LIBVALKEY_API void valkeyFreeCommand(char *cmd);
enum valkeyConnectionType {
VALKEY_CONN_TCP,
@ -315,18 +316,18 @@ typedef struct valkeyContext {
valkeyPushFn *push_cb;
} valkeyContext;
valkeyContext *valkeyConnectWithOptions(const valkeyOptions *options);
valkeyContext *valkeyConnect(const char *ip, int port);
valkeyContext *valkeyConnectWithTimeout(const char *ip, int port, const struct timeval tv);
valkeyContext *valkeyConnectNonBlock(const char *ip, int port);
valkeyContext *valkeyConnectBindNonBlock(const char *ip, int port,
LIBVALKEY_API valkeyContext *valkeyConnectWithOptions(const valkeyOptions *options);
LIBVALKEY_API valkeyContext *valkeyConnect(const char *ip, int port);
LIBVALKEY_API valkeyContext *valkeyConnectWithTimeout(const char *ip, int port, const struct timeval tv);
LIBVALKEY_API valkeyContext *valkeyConnectNonBlock(const char *ip, int port);
LIBVALKEY_API valkeyContext *valkeyConnectBindNonBlock(const char *ip, int port,
const char *source_addr);
valkeyContext *valkeyConnectBindNonBlockWithReuse(const char *ip, int port,
LIBVALKEY_API valkeyContext *valkeyConnectBindNonBlockWithReuse(const char *ip, int port,
const char *source_addr);
valkeyContext *valkeyConnectUnix(const char *path);
valkeyContext *valkeyConnectUnixWithTimeout(const char *path, const struct timeval tv);
valkeyContext *valkeyConnectUnixNonBlock(const char *path);
valkeyContext *valkeyConnectFd(valkeyFD fd);
LIBVALKEY_API valkeyContext *valkeyConnectUnix(const char *path);
LIBVALKEY_API valkeyContext *valkeyConnectUnixWithTimeout(const char *path, const struct timeval tv);
LIBVALKEY_API valkeyContext *valkeyConnectUnixNonBlock(const char *path);
LIBVALKEY_API valkeyContext *valkeyConnectFd(valkeyFD fd);
/**
* Reconnect the given context using the saved information.
@ -337,47 +338,47 @@ valkeyContext *valkeyConnectFd(valkeyFD fd);
*
* Returns VALKEY_OK on successful connect or VALKEY_ERR otherwise.
*/
int valkeyReconnect(valkeyContext *c);
LIBVALKEY_API int valkeyReconnect(valkeyContext *c);
valkeyPushFn *valkeySetPushCallback(valkeyContext *c, valkeyPushFn *fn);
int valkeySetTimeout(valkeyContext *c, const struct timeval tv);
LIBVALKEY_API valkeyPushFn *valkeySetPushCallback(valkeyContext *c, valkeyPushFn *fn);
LIBVALKEY_API int valkeySetTimeout(valkeyContext *c, const struct timeval tv);
/* Configurations using socket options. Applied directly to the underlying
* socket and not automatically applied after a reconnect. */
int valkeyEnableKeepAlive(valkeyContext *c);
int valkeyEnableKeepAliveWithInterval(valkeyContext *c, int interval);
int valkeySetTcpUserTimeout(valkeyContext *c, unsigned int timeout);
LIBVALKEY_API int valkeyEnableKeepAlive(valkeyContext *c);
LIBVALKEY_API int valkeyEnableKeepAliveWithInterval(valkeyContext *c, int interval);
LIBVALKEY_API int valkeySetTcpUserTimeout(valkeyContext *c, unsigned int timeout);
void valkeyFree(valkeyContext *c);
valkeyFD valkeyFreeKeepFd(valkeyContext *c);
int valkeyBufferRead(valkeyContext *c);
int valkeyBufferWrite(valkeyContext *c, int *done);
LIBVALKEY_API void valkeyFree(valkeyContext *c);
LIBVALKEY_API valkeyFD valkeyFreeKeepFd(valkeyContext *c);
LIBVALKEY_API int valkeyBufferRead(valkeyContext *c);
LIBVALKEY_API int valkeyBufferWrite(valkeyContext *c, int *done);
/* In a blocking context, this function first checks if there are unconsumed
* replies to return and returns one if so. Otherwise, it flushes the output
* buffer to the socket and reads until it has a reply. In a non-blocking
* context, it will return unconsumed replies until there are no more. */
int valkeyGetReply(valkeyContext *c, void **reply);
int valkeyGetReplyFromReader(valkeyContext *c, void **reply);
LIBVALKEY_API int valkeyGetReply(valkeyContext *c, void **reply);
LIBVALKEY_API int valkeyGetReplyFromReader(valkeyContext *c, void **reply);
/* Write a formatted command to the output buffer. Use these functions in blocking mode
* to get a pipeline of commands. */
int valkeyAppendFormattedCommand(valkeyContext *c, const char *cmd, size_t len);
LIBVALKEY_API int valkeyAppendFormattedCommand(valkeyContext *c, const char *cmd, size_t len);
/* Write a command to the output buffer. Use these functions in blocking mode
* to get a pipeline of commands. */
int valkeyvAppendCommand(valkeyContext *c, const char *format, va_list ap);
int valkeyAppendCommand(valkeyContext *c, const char *format, ...);
int valkeyAppendCommandArgv(valkeyContext *c, int argc, const char **argv, const size_t *argvlen);
LIBVALKEY_API int valkeyvAppendCommand(valkeyContext *c, const char *format, va_list ap);
LIBVALKEY_API int valkeyAppendCommand(valkeyContext *c, const char *format, ...);
LIBVALKEY_API int valkeyAppendCommandArgv(valkeyContext *c, int argc, const char **argv, const size_t *argvlen);
/* Issue a command to Valkey. In a blocking context, it is identical to calling
* valkeyAppendCommand, followed by valkeyGetReply. The function will return
* NULL if there was an error in performing the request, otherwise it will
* NULL if there was an error in performing the request; otherwise, it will
* return the reply. In a non-blocking context, it is identical to calling
* only valkeyAppendCommand and will always return NULL. */
void *valkeyvCommand(valkeyContext *c, const char *format, va_list ap);
void *valkeyCommand(valkeyContext *c, const char *format, ...);
void *valkeyCommandArgv(valkeyContext *c, int argc, const char **argv, const size_t *argvlen);
LIBVALKEY_API void *valkeyvCommand(valkeyContext *c, const char *format, va_list ap);
LIBVALKEY_API void *valkeyCommand(valkeyContext *c, const char *format, ...);
LIBVALKEY_API void *valkeyCommandArgv(valkeyContext *c, int argc, const char **argv, const size_t *argvlen);
#ifdef __cplusplus
}

View File

@ -0,0 +1,49 @@
/*
* Copyright (c) 2025-present, libvalkey contributors
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of the copyright holder nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef VALKEY_VISIBILITY_H
#define VALKEY_VISIBILITY_H
#ifdef __cplusplus
extern "C" {
#endif
#if defined __GNUC__
#define LIBVALKEY_API __attribute__((visibility("default")))
#elif defined(_MSC_VER)
#define LIBVALKEY_API __declspec(dllexport)
#else
#define LIBVALKEY_API /* Unknown compiler */
#endif
#ifdef __cplusplus
}
#endif
#endif /* VALKEY_VISIBILITY_H */

View File

@ -57,7 +57,7 @@ def any_argument_is_key(arguments):
# keys (example EVAL)
def firstkey(props):
if not "key_specs" in props:
# Key specs missing. Best-effort fallback to "arguments".
# Key specs missing. Best-effort fall back to "arguments".
if "arguments" in props:
args = props["arguments"]
for i in range(1, len(args)):
@ -128,7 +128,7 @@ def extract_command_info(name, props):
name = container.upper()
else:
# Ad-hoc handling of command and subcommand in the same string,
# sepatated by a space. This form is used in e.g. RediSearch's JSON file
# separated by a space. This form is used in e.g. RediSearch's JSON file
# in commands like "FT.CONFIG GET".
tokens = name.split(maxsplit=1)
if len(tokens) > 1:

View File

@ -38,7 +38,7 @@
* AlFreeList(), but private value of every node need to be freed
* by the user before to call AlFreeList().
*
* On error, NULL is returned. Otherwise the pointer to the new list. */
* On error, NULL is returned. Otherwise, the pointer to the new list. */
hilist *listCreate(void) {
struct hilist *list;
@ -233,7 +233,7 @@ listNode *listNext(listIter *iter) {
* On success a copy of the original list is returned.
*
* The 'Dup' method set with listSetDupMethod() function is used
* to copy the node value. Otherwise the same pointer value of
* to copy the node value. Otherwise, the same pointer value of
* the original node is used as value of the copied node.
*
* The original list both on success or error is never modified. */

View File

@ -493,7 +493,7 @@ static int valkeyGetSubscribeCallback(valkeyAsyncContext *ac, valkeyReply *reply
}
}
/* If this is an subscribe reply decrease pending counter. */
/* If this is a subscribe reply decrease pending counter. */
if (strcasecmp(stype + pvariant + svariant, "subscribe") == 0) {
assert(cb != NULL);
cb->pending_subs -= 1;

View File

@ -31,6 +31,7 @@
#ifndef VALKEY_ASYNC_PRIVATE_H
#define VALKEY_ASYNC_PRIVATE_H
#include "visibility.h"
#define _EL_ADD_READ(ctx) \
do { \
@ -77,7 +78,8 @@ static inline void refreshTimeout(valkeyAsyncContext *ctx) {
}
}
void valkeyAsyncDisconnectInternal(valkeyAsyncContext *ac);
void valkeyProcessCallbacks(valkeyAsyncContext *ac);
/* Visible although private since required by libvalkey_tls.so */
LIBVALKEY_API void valkeyAsyncDisconnectInternal(valkeyAsyncContext *ac);
LIBVALKEY_API void valkeyProcessCallbacks(valkeyAsyncContext *ac);
#endif /* VALKEY_ASYNC_PRIVATE_H */

View File

@ -63,29 +63,13 @@ vk_static_assert(VALKEY_OPT_USE_CLUSTER_NODES > VALKEY_OPT_LAST_SA_OPTION);
// standard Valkey errors
#define VALKEY_ERR_CLUSTER_TOO_MANY_RETRIES 100
#define VALKEY_ERROR_MOVED "MOVED"
#define VALKEY_ERROR_ASK "ASK"
#define VALKEY_ERROR_TRYAGAIN "TRYAGAIN"
#define VALKEY_ERROR_CLUSTERDOWN "CLUSTERDOWN"
#define VALKEY_STATUS_OK "OK"
#define VALKEY_COMMAND_CLUSTER_NODES "CLUSTER NODES"
#define VALKEY_COMMAND_CLUSTER_SLOTS "CLUSTER SLOTS"
#define VALKEY_COMMAND_ASKING "ASKING"
#define IP_PORT_SEPARATOR ':'
#define PORT_CPORT_SEPARATOR '@'
#define CLUSTER_ADDRESS_SEPARATOR ","
#define CLUSTER_DEFAULT_MAX_RETRY_COUNT 5
#define NO_RETRY -1
#define CRLF "\x0d\x0a"
#define CRLF_LEN (sizeof("\x0d\x0a") - 1)
#define SLOTMAP_UPDATE_THROTTLE_USEC 1000000
#define SLOTMAP_UPDATE_ONGOING INT64_MAX
@ -97,14 +81,14 @@ typedef struct cluster_async_data {
void *privdata;
} cluster_async_data;
typedef enum CLUSTER_ERR_TYPE {
CLUSTER_NOT_ERR = 0,
typedef enum {
CLUSTER_NO_ERROR = 0,
CLUSTER_ERR_MOVED,
CLUSTER_ERR_ASK,
CLUSTER_ERR_TRYAGAIN,
CLUSTER_ERR_CLUSTERDOWN,
CLUSTER_ERR_SENTINEL
} CLUSTER_ERR_TYPE;
CLUSTER_ERR_OTHER
} replyErrorType;
static void freeValkeyClusterNode(valkeyClusterNode *node);
static void cluster_slot_destroy(cluster_slot *slot);
@ -223,35 +207,20 @@ static inline void valkeyClusterClearError(valkeyClusterContext *cc) {
cc->errstr[0] = '\0';
}
static int cluster_reply_error_type(valkeyReply *reply) {
static replyErrorType getReplyErrorType(valkeyReply *reply) {
assert(reply);
if (reply == NULL) {
return VALKEY_ERR;
}
if (reply->type == VALKEY_REPLY_ERROR) {
if ((int)strlen(VALKEY_ERROR_MOVED) < reply->len &&
memcmp(reply->str, VALKEY_ERROR_MOVED,
strlen(VALKEY_ERROR_MOVED)) == 0) {
if (reply->type != VALKEY_REPLY_ERROR)
return CLUSTER_NO_ERROR;
if (memcmp(reply->str, "MOVED", 5) == 0)
return CLUSTER_ERR_MOVED;
} else if ((int)strlen(VALKEY_ERROR_ASK) < reply->len &&
memcmp(reply->str, VALKEY_ERROR_ASK,
strlen(VALKEY_ERROR_ASK)) == 0) {
if (memcmp(reply->str, "ASK", 3) == 0)
return CLUSTER_ERR_ASK;
} else if ((int)strlen(VALKEY_ERROR_TRYAGAIN) < reply->len &&
memcmp(reply->str, VALKEY_ERROR_TRYAGAIN,
strlen(VALKEY_ERROR_TRYAGAIN)) == 0) {
if (memcmp(reply->str, "TRYAGAIN", 8) == 0)
return CLUSTER_ERR_TRYAGAIN;
} else if ((int)strlen(VALKEY_ERROR_CLUSTERDOWN) < reply->len &&
memcmp(reply->str, VALKEY_ERROR_CLUSTERDOWN,
strlen(VALKEY_ERROR_CLUSTERDOWN)) == 0) {
if (memcmp(reply->str, "CLUSTERDOWN", 11) == 0)
return CLUSTER_ERR_CLUSTERDOWN;
} else {
return CLUSTER_ERR_SENTINEL;
}
}
return CLUSTER_NOT_ERR;
return CLUSTER_ERR_OTHER;
}
/* Create and initiate the cluster node structure */
@ -561,7 +530,7 @@ static dict *parse_cluster_slots(valkeyClusterContext *cc, valkeyContext *c,
valkeyClusterSetError(
cc, VALKEY_ERR_OTHER,
"Command(cluster slots) reply error: "
"nodes sub_reply is not an correct array.");
"nodes sub_reply is not a correct array.");
goto error;
}
@ -816,12 +785,12 @@ static int parse_cluster_nodes_line(valkeyClusterContext *cc, valkeyContext *c,
/* Parse the address field: <ip:port@cport[,hostname]>
* Remove @cport.. to get <ip>:<port> which is our dict key. */
if ((p = strchr(addr, PORT_CPORT_SEPARATOR)) != NULL) {
if ((p = strchr(addr, '@')) != NULL) {
*p = '\0';
}
/* Find the required port separator. */
if ((p = strrchr(addr, IP_PORT_SEPARATOR)) == NULL) {
if ((p = strrchr(addr, ':')) == NULL) {
valkeyClusterSetError(cc, VALKEY_ERR_OTHER, "Invalid node address");
freeValkeyClusterNode(node);
return VALKEY_ERR;
@ -1059,68 +1028,6 @@ static int clusterUpdateRouteHandleReply(valkeyClusterContext *cc,
return updateNodesAndSlotmap(cc, nodes);
}
/**
* Update route with the "cluster nodes" or "cluster slots" command reply.
*/
static int cluster_update_route_by_addr(valkeyClusterContext *cc,
const char *ip, int port) {
valkeyContext *c = NULL;
if (cc == NULL) {
return VALKEY_ERR;
}
if (ip == NULL || port <= 0) {
valkeyClusterSetError(cc, VALKEY_ERR_OTHER, "Ip or port error!");
goto error;
}
valkeyOptions options = {0};
VALKEY_OPTIONS_SET_TCP(&options, ip, port);
options.connect_timeout = cc->connect_timeout;
options.command_timeout = cc->command_timeout;
options.options = cc->options;
c = valkeyConnectWithOptions(&options);
if (c == NULL) {
valkeyClusterSetError(cc, VALKEY_ERR_OOM, "Out of memory");
return VALKEY_ERR;
}
if (cc->on_connect) {
cc->on_connect(c, c->err ? VALKEY_ERR : VALKEY_OK);
}
if (c->err) {
valkeyClusterSetError(cc, c->err, c->errstr);
goto error;
}
if (cc->tls && cc->tls_init_fn(c, cc->tls) != VALKEY_OK) {
valkeyClusterSetError(cc, c->err, c->errstr);
goto error;
}
if (authenticate(cc, c) != VALKEY_OK) {
goto error;
}
if (clusterUpdateRouteSendCommand(cc, c) != VALKEY_OK) {
goto error;
}
if (clusterUpdateRouteHandleReply(cc, c) != VALKEY_OK) {
goto error;
}
valkeyFree(c);
return VALKEY_OK;
error:
valkeyFree(c);
return VALKEY_ERR;
}
/* Update known cluster nodes with a new collection of valkeyClusterNodes.
* Will also update the slot-to-node lookup table for the new nodes. */
static int updateNodesAndSlotmap(valkeyClusterContext *cc, dict *nodes) {
@ -1214,37 +1121,33 @@ error:
}
int valkeyClusterUpdateSlotmap(valkeyClusterContext *cc) {
int ret;
int flag_err_not_set = 1;
valkeyClusterNode *node;
dictEntry *de;
if (cc == NULL) {
return VALKEY_ERR;
}
valkeyClusterNode *node;
dictEntry *de;
dictIterator di;
dictInitIterator(&di, cc->nodes);
while ((de = dictNext(&di)) != NULL) {
node = dictGetVal(de);
if (node == NULL || node->host == NULL) {
/* Use existing connection or (re)connect to the node. */
valkeyContext *c = valkeyClusterGetValkeyContext(cc, node);
if (c == NULL)
continue;
if (clusterUpdateRouteSendCommand(cc, c) != VALKEY_OK ||
clusterUpdateRouteHandleReply(cc, c) != VALKEY_OK) {
valkeyFree(node->con);
node->con = NULL;
continue;
}
ret = cluster_update_route_by_addr(cc, node->host, node->port);
if (ret == VALKEY_OK) {
valkeyClusterClearError(cc);
return VALKEY_OK;
}
flag_err_not_set = 0;
}
if (flag_err_not_set) {
valkeyClusterSetError(cc, VALKEY_ERR_OTHER, "no valid server address");
}
return VALKEY_ERR;
}
@ -1389,11 +1292,12 @@ static int valkeyClusterSetOptionAddNode(valkeyClusterContext *cc, const char *a
node_entry = dictFind(cc->nodes, addr_sds);
sdsfree(addr_sds);
if (node_entry == NULL) {
char *p;
if ((p = strrchr(addr, IP_PORT_SEPARATOR)) == NULL) {
valkeyClusterSetError(
cc, VALKEY_ERR_OTHER,
/* Find the last occurrence of the port separator since
* IPv6 addresses can contain ':' */
if ((p = strrchr(addr, ':')) == NULL) {
valkeyClusterSetError(cc, VALKEY_ERR_OTHER,
"server address is incorrect, port separator missing.");
return VALKEY_ERR;
}
@ -1476,8 +1380,8 @@ static int valkeyClusterSetOptionAddNodes(valkeyClusterContext *cc,
return VALKEY_ERR;
}
address = sdssplitlen(addrs, strlen(addrs), CLUSTER_ADDRESS_SEPARATOR,
strlen(CLUSTER_ADDRESS_SEPARATOR), &address_count);
/* Split into individual addresses. */
address = sdssplitlen(addrs, strlen(addrs), ",", strlen(","), &address_count);
if (address == NULL) {
valkeyClusterSetError(cc, VALKEY_ERR_OOM, "Out of memory");
return VALKEY_ERR;
@ -1801,7 +1705,7 @@ static int valkeyClusterGetReplyFromNode(valkeyClusterContext *cc,
return VALKEY_ERR;
}
if (cluster_reply_error_type(*reply) == CLUSTER_ERR_MOVED)
if (getReplyErrorType(*reply) == CLUSTER_ERR_MOVED)
cc->need_update_route = 1;
return VALKEY_OK;
@ -1845,7 +1749,7 @@ static valkeyClusterNode *getNodeFromRedirectReply(valkeyClusterContext *cc,
}
/* Find the last occurrence of the port separator since
* IPv6 addresses can contain ':' */
if ((p = strrchr(addr, IP_PORT_SEPARATOR)) == NULL) {
if ((p = strrchr(addr, ':')) == NULL) {
valkeyClusterSetError(cc, VALKEY_ERR_OTHER, "Invalid address in redirect");
return NULL;
}
@ -1934,7 +1838,6 @@ static void *valkey_cluster_command_execute(valkeyClusterContext *cc,
void *reply = NULL;
valkeyClusterNode *node;
valkeyContext *c = NULL;
int error_type;
valkeyContext *c_updating_route = NULL;
retry:
@ -1999,8 +1902,8 @@ ask_retry:
goto error;
}
error_type = cluster_reply_error_type(reply);
if (error_type > CLUSTER_NOT_ERR && error_type < CLUSTER_ERR_SENTINEL) {
replyErrorType error_type = getReplyErrorType(reply);
if (error_type > CLUSTER_NO_ERROR && error_type < CLUSTER_ERR_OTHER) {
cc->retry_count++;
if (cc->retry_count > cc->max_retry_count) {
valkeyClusterSetError(cc, VALKEY_ERR_CLUSTER_TOO_MANY_RETRIES,
@ -2829,6 +2732,19 @@ static int valkeyClusterAsyncConnect(valkeyClusterAsyncContext *acc) {
valkeyClusterAsyncSetError(acc, acc->cc.err, acc->cc.errstr);
return VALKEY_ERR;
}
/* Disconnect any non-async context used for the initial update. */
dictIterator di;
dictInitIterator(&di, acc->cc.nodes);
dictEntry *de;
while ((de = dictNext(&di)) != NULL) {
valkeyClusterNode *node = dictGetVal(de);
if (node->con) {
valkeyFree(node->con);
node->con = NULL;
}
}
return VALKEY_OK;
}
/* Use non-blocking initial slotmap update. */
@ -2985,7 +2901,6 @@ static void valkeyClusterAsyncCallback(valkeyAsyncContext *ac, void *r,
valkeyClusterAsyncContext *acc;
valkeyClusterContext *cc;
valkeyAsyncContext *ac_retry = NULL;
int error_type;
valkeyClusterNode *node;
struct cmd *command;
@ -3025,9 +2940,8 @@ static void valkeyClusterAsyncCallback(valkeyAsyncContext *ac, void *r,
if (cad->retry_count == NO_RETRY || cc->flags & VALKEY_FLAG_DISCONNECTING)
goto done;
error_type = cluster_reply_error_type(reply);
if (error_type > CLUSTER_NOT_ERR && error_type < CLUSTER_ERR_SENTINEL) {
replyErrorType error_type = getReplyErrorType(reply);
if (error_type > CLUSTER_NO_ERROR && error_type < CLUSTER_ERR_OTHER) {
cad->retry_count++;
if (cad->retry_count > cc->max_retry_count) {
cad->retry_count = 0;

View File

@ -75,7 +75,7 @@ struct cmd {
/* Command destination */
int slot_num; /* Command should be sent to slot.
* Set to -1 if command is sent to a given node,
* or if a slot can not be found or calculated. */
* or if a slot cannot be found or calculated. */
char *node_addr; /* Command sent to this node address */
};

View File

@ -43,6 +43,8 @@
#include <limits.h>
#include <stdlib.h>
#define UNUSED(x) (void)(x)
/* -------------------------- types ----------------------------------------- */
struct dictEntry {
void *key;
@ -71,7 +73,7 @@ uint64_t dictGenHashFunction(const unsigned char *buf, int len) {
/* ----------------------------- API implementation ------------------------- */
/* Reset an hashtable already initialized with ht_init().
/* Reset a hashtable already initialized with ht_init().
* NOTE: This function should only called by ht_destroy(). */
static void _dictReset(dict *ht) {
ht->table = NULL;
@ -116,7 +118,7 @@ int dictExpand(dict *ht, unsigned long size) {
/* Copy all the elements from the old to the new table:
* note that if the old hash table is empty ht->size is zero,
* so dictExpand just creates an hash table. */
* so dictExpand just creates a hash table. */
n.used = ht->used;
for (i = 0; i < ht->size && ht->used > 0; i++) {
dictEntry *he, *nextHe;
@ -356,7 +358,7 @@ static unsigned long _dictNextPower(unsigned long size) {
}
/* Returns the index of a free slot that can be populated with
* an hash entry for the given 'key'.
* a hash entry for the given 'key'.
* If the key already exists, -1 is returned. */
static int _dictKeyIndex(dict *ht, const void *key) {
unsigned int h;

View File

@ -397,6 +397,7 @@ int valkeyHasMptcp(void) {
static int valkeyTcpGetProtocol(int is_mptcp_enabled) {
assert(!is_mptcp_enabled);
(void)is_mptcp_enabled; /* Suppress unused warning when NDEBUG is defined. */
return IPPROTO_TCP;
}
#endif /* IPPROTO_MPTCP */

View File

@ -390,7 +390,12 @@ static int connRdmaHandleCq(valkeyContext *c) {
valkeySetError(c, VALKEY_ERR_OTHER, "RDMA: get cq event failed");
return VALKEY_ERR;
}
} else if (ibv_req_notify_cq(ev_cq, 0)) {
return VALKEY_OK;
}
ibv_ack_cq_events(ctx->cq, 1);
if (ibv_req_notify_cq(ev_cq, 0)) {
valkeySetError(c, VALKEY_ERR_OTHER, "RDMA: notify cq failed");
return VALKEY_ERR;
}
@ -404,8 +409,6 @@ pollcq:
return VALKEY_OK;
}
ibv_ack_cq_events(ctx->cq, 1);
if (wc.status != IBV_WC_SUCCESS) {
valkeySetError(c, VALKEY_ERR_OTHER, "RDMA: send/recv failed");
return VALKEY_ERR;
@ -614,7 +617,7 @@ waitcq:
}
}
/* RDMA has no POLLOUT event supported, so it could't work well with valkey async mechanism */
/* RDMA has no POLLOUT event supported, so it couldn't work well with valkey async mechanism */
static void valkeyRdmaAsyncRead(VALKEY_UNUSED valkeyAsyncContext *ac) {
assert("valkey async mechanism can't work with RDMA" == NULL);
}

View File

@ -198,7 +198,7 @@ static int string2ll(const char *s, size_t slen, long long *value) {
return VALKEY_ERR;
}
/* First digit should be 1-9, otherwise the string should just be 0. */
/* First digit should be 1-9; otherwise, the string should just be 0. */
if (p[0] >= '1' && p[0] <= '9') {
v = p[0] - '0';
p++;

View File

@ -444,81 +444,125 @@ sds sdscpy(sds s, const char *t) {
return sdscpylen(s, t, strlen(t));
}
/* Helper for sdscatlonglong() doing the actual number -> string
* conversion. 's' must point to a string with room for at least
* SDS_LLSTR_SIZE bytes.
*
* The function returns the length of the null-terminated string
* representation stored at 's'. */
#define SDS_LLSTR_SIZE 21
int sdsll2str(char *s, long long value) {
char *p, aux;
unsigned long long v;
size_t l;
/* Generate the string representation, this method produces
* an reversed string. */
v = (value < 0) ? -value : value;
p = s;
do {
*p++ = '0' + (v % 10);
v /= 10;
} while (v);
if (value < 0)
*p++ = '-';
/* Compute length and add null term. */
l = p - s;
*p = '\0';
/* Reverse the string. */
p--;
while (s < p) {
aux = *s;
*s = *p;
*p = aux;
s++;
p--;
/* Return the number of digits of 'v' when converted to string in radix 10.
* See ll2string() for more information. */
static uint32_t digits10(uint64_t v) {
if (v < 10)
return 1;
if (v < 100)
return 2;
if (v < 1000)
return 3;
if (v < 1000000000000UL) {
if (v < 100000000UL) {
if (v < 1000000) {
if (v < 10000)
return 4;
return 5 + (v >= 100000);
}
return l;
return 7 + (v >= 10000000UL);
}
if (v < 10000000000UL) {
return 9 + (v >= 1000000000UL);
}
return 11 + (v >= 100000000000UL);
}
return 12 + digits10(v / 1000000000000UL);
}
/* Identical sdsll2str(), but for unsigned long long type. */
int sdsull2str(char *s, unsigned long long v) {
char *p, aux;
size_t l;
/* Convert a unsigned long long into a string. Returns the number of
* characters needed to represent the number.
* If the buffer is not big enough to store the string, 0 is returned.
*
* Based on the following article (that apparently does not provide a
* novel approach but only publicizes an already used technique):
*
* https://www.facebook.com/notes/facebook-engineering/three-optimization-tips-for-c/10151361643253920 */
static int ull2string(char *dst, size_t dstlen, unsigned long long value) {
static const char digits[201] = "0001020304050607080910111213141516171819"
"2021222324252627282930313233343536373839"
"4041424344454647484950515253545556575859"
"6061626364656667686970717273747576777879"
"8081828384858687888990919293949596979899";
/* Generate the string representation, this method produces
* an reversed string. */
p = s;
do {
*p++ = '0' + (v % 10);
v /= 10;
} while (v);
/* Check length. */
uint32_t length = digits10(value);
if (length >= dstlen)
goto err;
/* Compute length and add null term. */
l = p - s;
*p = '\0';
/* Reverse the string. */
p--;
while (s < p) {
aux = *s;
*s = *p;
*p = aux;
s++;
p--;
/* Null term. */
uint32_t next = length - 1;
dst[next + 1] = '\0';
while (value >= 100) {
int const i = (value % 100) * 2;
value /= 100;
dst[next] = digits[i + 1];
dst[next - 1] = digits[i];
next -= 2;
}
return l;
/* Handle last 1-2 digits. */
if (value < 10) {
dst[next] = '0' + (uint32_t)value;
} else {
int i = (uint32_t)value * 2;
dst[next] = digits[i + 1];
dst[next - 1] = digits[i];
}
return length;
err:
/* force add Null termination */
if (dstlen > 0)
dst[0] = '\0';
return 0;
}
/* Convert a long long into a string. Returns the number of
* characters needed to represent the number.
* If the buffer is not big enough to store the string, 0 is returned. */
static int ll2string(char *dst, size_t dstlen, long long svalue) {
unsigned long long value;
int negative = 0;
/* The ull2string function with 64bit unsigned integers for simplicity, so
* we convert the number here and remember if it is negative. */
if (svalue < 0) {
if (svalue != LLONG_MIN) {
value = -svalue;
} else {
value = ((unsigned long long)LLONG_MAX) + 1;
}
if (dstlen < 2)
goto err;
negative = 1;
dst[0] = '-';
dst++;
dstlen--;
} else {
value = svalue;
}
/* Converts the unsigned long long value to string */
int length = ull2string(dst, dstlen, value);
if (length == 0)
return 0;
return length + negative;
err:
/* force add Null termination */
if (dstlen > 0)
dst[0] = '\0';
return 0;
}
/* Create an sds string from a long long value. It is much faster than:
*
* sdscatprintf(sdsempty(),"%lld\n", value);
*/
#define SDS_LLSTR_SIZE 21
sds sdsfromlonglong(long long value) {
char buf[SDS_LLSTR_SIZE];
int len = sdsll2str(buf, value);
int len = ll2string(buf, sizeof(buf), value);
return sdsnewlen(buf, len);
}
@ -652,7 +696,7 @@ sds sdscatfmt(sds s, char const *fmt, ...) {
num = va_arg(ap, long long);
{
char buf[SDS_LLSTR_SIZE];
l = sdsll2str(buf, num);
l = ll2string(buf, sizeof(buf), num);
if (sdsavail(s) < l) {
s = sdsMakeRoomFor(s, l);
if (s == NULL)
@ -671,7 +715,7 @@ sds sdscatfmt(sds s, char const *fmt, ...) {
unum = va_arg(ap, unsigned long long);
{
char buf[SDS_LLSTR_SIZE];
l = sdsull2str(buf, unum);
l = ull2string(buf, sizeof(buf), unum);
if (sdsavail(s) < l) {
s = sdsMakeRoomFor(s, l);
if (s == NULL)
@ -707,7 +751,7 @@ fmt_error:
}
/* Remove the part of the string from left and from right composed just of
* contiguous characters found in 'cset', that is a null terminted C string.
* contiguous characters found in 'cset', that is a null terminated C string.
*
* After the call, the modified sds string is no longer valid and all the
* references must be substituted with the new pointer returned by the call.
@ -1220,7 +1264,6 @@ void sds_free(void *ptr) { s_free(ptr); }
#include <stdio.h>
#define UNUSED(x) (void)(x)
int sdsTest(void) {
{
sds x = sdsnew("foo"), y;

View File

@ -38,8 +38,11 @@
#include <errno.h>
#include <string.h>
#ifdef _WIN32
#include <wincrypt.h>
/* clang-format off */
// wincrypt.h need macro defined in windows.h
#include <windows.h>
#include <wincrypt.h>
/* clang-format on */
#ifdef OPENSSL_IS_BORINGSSL
#undef X509_NAME
#undef X509_EXTENSIONS

View File

@ -218,7 +218,7 @@ static void *createIntegerObject(const valkeyReadTask *task, long long value) {
static void *createDoubleObject(const valkeyReadTask *task, double value, char *str, size_t len) {
valkeyReply *r, *parent;
if (len == SIZE_MAX) // Prevents vk_malloc(0) if len equals to SIZE_MAX
if (len == SIZE_MAX) // Prevents vk_malloc(0) if len equals SIZE_MAX
return NULL;
r = createReplyObject(VALKEY_REPLY_DOUBLE);
@ -235,7 +235,7 @@ static void *createDoubleObject(const valkeyReadTask *task, double value, char *
/* The double reply also has the original protocol string representing a
* double as a null terminated string. This way the caller does not need
* to format back for string conversion, especially since Valkey does efforts
* to make the string more human readable avoiding the calssical double
* to make the string more human readable avoiding the classical double
* decimal string conversion artifacts. */
memcpy(r->str, str, len);
r->str[len] = '\0';

View File

@ -34,13 +34,14 @@
#include "win32.h"
#include "valkey.h"
#include "visibility.h"
#include <sds.h>
#include <limits.h>
#include <string.h>
void valkeySetError(valkeyContext *c, int type, const char *str);
LIBVALKEY_API void valkeySetError(valkeyContext *c, int type, const char *str);
/* Helper function. Convert struct timeval to millisecond. */
static inline int valkeyContextTimeoutMsec(const struct timeval *timeout, long *result) {

View File

@ -28,16 +28,7 @@
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include <stdio.h>
#include <stdlib.h>
#ifndef _WIN32
#include <sys/time.h>
#endif
#include "win32.h"
#include "alloc.h"
#include "vkutil.h"
int _vk_atoi(uint8_t *line, size_t n) {
@ -61,32 +52,3 @@ int _vk_atoi(uint8_t *line, size_t n) {
return value;
}
/*
* Return the current time in microseconds since Epoch
*/
int64_t vk_usec_now(void) {
int64_t usec;
#ifdef _WIN32
LARGE_INTEGER counter, frequency;
if (!QueryPerformanceCounter(&counter) ||
!QueryPerformanceFrequency(&frequency)) {
return -1;
}
usec = counter.QuadPart * 1000000 / frequency.QuadPart;
#else
struct timeval now;
int status;
status = gettimeofday(&now, NULL);
if (status < 0) {
return -1;
}
usec = (int64_t)now.tv_sec * 1000000LL + (int64_t)now.tv_usec;
#endif
return usec;
}

View File

@ -33,8 +33,14 @@
#ifndef VALKEY_VKUTIL_H
#define VALKEY_VKUTIL_H
#include "win32.h"
#include <stddef.h>
#include <stdint.h>
#include <sys/types.h>
#ifndef _WIN32
#include <sys/time.h>
#endif
/* Static assert macro for C99. */
#define vk_static_assert(cond) extern char vk_static_assert[sizeof(char[(cond) ? 1 : -1])]
@ -47,7 +53,25 @@
int _vk_atoi(uint8_t *line, size_t n);
int64_t vk_usec_now(void);
/* Return the current time in microseconds since Epoch */
static inline int64_t vk_usec_now(void) {
int64_t usec;
#ifdef _WIN32
LARGE_INTEGER counter, frequency;
if (!QueryPerformanceCounter(&counter) ||
!QueryPerformanceFrequency(&frequency)) {
return -1;
}
usec = counter.QuadPart * 1000000 / frequency.QuadPart;
#else
struct timeval now;
if (gettimeofday(&now, NULL) < 0) {
return -1;
}
usec = (int64_t)now.tv_sec * 1000000LL + (int64_t)now.tv_usec;
#endif
return usec;
}
static inline int64_t vk_msec_now(void) {
return vk_usec_now() / 1000;

View File

@ -58,8 +58,9 @@ endif()
if(MSVC OR MINGW)
find_library(LIBEVENT_LIBRARY Libevent)
else()
# Use the Debug configuration when building tests (no -DNDEBUG)
set(CMAKE_BUILD_TYPE "Debug" CACHE STRING "" FORCE)
# Undefine any -DNDEBUG to make sure that tests can assert.
# CMake defines NDEBUG in Release and RelWithDebInfo.
add_compile_options("-UNDEBUG")
endif()
# Make sure ctest gives the output when tests fail
@ -68,6 +69,7 @@ list(APPEND CMAKE_CTEST_ARGUMENTS "--output-on-failure")
# Add non-cluster tests
add_executable(client_test client_test.c)
target_include_directories(client_test PRIVATE "${PROJECT_SOURCE_DIR}/src")
target_link_libraries(client_test valkey_unittest)
if(TLS_LIBRARY)
target_compile_definitions(client_test PUBLIC VALKEY_TEST_TLS=1)
target_link_libraries(client_test ${TLS_LIBRARY})
@ -80,7 +82,6 @@ if(LIBEVENT_LIBRARY)
target_compile_definitions(client_test PUBLIC VALKEY_TEST_ASYNC=1)
target_link_libraries(client_test ${LIBEVENT_LIBRARY})
endif()
target_link_libraries(client_test valkey)
add_test(NAME client_test COMMAND "${CMAKE_CURRENT_SOURCE_DIR}/test.sh")
if(TEST_WITH_REDIS_VERSION)
@ -99,12 +100,12 @@ endif()
# Unit tests
add_executable(ut_parse_cmd ut_parse_cmd.c test_utils.c)
target_include_directories(ut_parse_cmd PRIVATE "${PROJECT_SOURCE_DIR}/src")
target_link_libraries(ut_parse_cmd valkey)
target_link_libraries(ut_parse_cmd valkey_unittest)
add_test(NAME ut_parse_cmd COMMAND "$<TARGET_FILE:ut_parse_cmd>")
add_executable(ut_slotmap_update ut_slotmap_update.c)
target_include_directories(ut_slotmap_update PRIVATE "${PROJECT_SOURCE_DIR}/src")
target_link_libraries(ut_slotmap_update valkey)
target_link_libraries(ut_slotmap_update valkey_unittest)
add_test(NAME ut_slotmap_update COMMAND "$<TARGET_FILE:ut_slotmap_update>")
# Cluster tests

View File

@ -210,7 +210,7 @@ static valkeyContext *select_database(valkeyContext *c) {
/* Awesome, DB 9 is empty and we can continue. */
freeReplyObject(reply);
} else {
printf("Database #9 is not empty, test can not continue\n");
printf("Database #9 is not empty, test cannot continue\n");
exit(1);
}
@ -738,7 +738,7 @@ static void test_reply_reader(void) {
freeReplyObject(reply);
valkeyReaderFree(reader);
/* RESP3 push messages (Github issue #815) */
/* RESP3 push messages (GitHub issue #815) */
test("Can parse RESP3 push messages: ");
reader = valkeyReaderCreate();
valkeyReaderFeed(reader, (char *)">2\r\n$6\r\nLOLWUT\r\n:42\r\n", 21);
@ -1034,7 +1034,7 @@ static void test_allocator_injection(void) {
valkeyResetAllocators();
}
#define VALKEY_BAD_DOMAIN "idontexist-noreally.com"
#define VALKEY_BAD_DOMAIN "nonexistent.example.com"
static void test_blocking_connection_errors(void) {
struct addrinfo hints = {.ai_family = AF_INET};
struct addrinfo *ai_tmp = NULL;
@ -1091,7 +1091,7 @@ static void test_blocking_connection_errors(void) {
valkeyFree(c);
test("Returns error when the unix_sock socket path doesn't accept connections: ");
c = valkeyConnectUnix((char *)"/tmp/idontexist.sock");
c = valkeyConnectUnix((char *)"/tmp/nonexistent.sock");
test_cond(c->err == VALKEY_ERR_IO); /* Don't care about the message... */
valkeyFree(c);
#endif
@ -1787,7 +1787,7 @@ void ssubscribe_crossslot_error_cb(valkeyAsyncContext *ac, void *r, void *privda
}
/* Subscribe callback for test_sharded_pubsub_crossslot_handling:
* - a published message triggers another ssubscribe to first channel and other slot chahhel
* - a published message triggers another ssubscribe to first channel and other slot channel
* - after receiving CROSSLOT error send smessage to first channel
* - a command is sent before the unsubscribe response is received. */
void ssubscribe_crossslot_cb(valkeyAsyncContext *ac, void *r, void *privdata) {

View File

@ -57,7 +57,7 @@ int num_running = 0;
int resend_failed_cmd = 0;
int send_to_all = 0;
int show_events = 0;
int async_initial_update = 0;
int blocking_initial_update = 0;
void sendNextCommand(evutil_socket_t, short, void *);
@ -80,7 +80,7 @@ void printReply(const valkeyReply *reply) {
void replyCallback(valkeyClusterAsyncContext *acc, void *r, void *privdata) {
valkeyReply *reply = (valkeyReply *)r;
intptr_t cmd_id = (intptr_t)privdata; /* Id to corresponding cmd */
intptr_t cmd_id = (intptr_t)privdata; /* ID for corresponding cmd */
if (reply == NULL) {
if (acc->err) {
@ -251,8 +251,8 @@ int main(int argc, char **argv) {
show_events = 1;
} else if (strcmp(argv[optind], "--connection-events") == 0) {
show_connection_events = 1;
} else if (strcmp(argv[optind], "--async-initial-update") == 0) {
async_initial_update = 1;
} else if (strcmp(argv[optind], "--blocking-initial-update") == 0) {
blocking_initial_update = 1;
} else {
fprintf(stderr, "Unknown argument: '%s'\n", argv[optind]);
}
@ -273,7 +273,7 @@ int main(int argc, char **argv) {
options.command_timeout = &timeout;
options.event_callback = eventCallback;
options.max_retry = 1;
if (!async_initial_update) {
if (blocking_initial_update) {
options.options = VALKEY_OPT_BLOCKING_INITIAL_UPDATE;
}
if (use_cluster_nodes) {

View File

@ -52,7 +52,8 @@ void test_password_ok(void) {
ASSERT_MSG(cc && cc->err == 0, cc ? cc->errstr : "OOM");
assert(connect_success_counter == 1); // for CLUSTER SLOTS
load_valkey_version(cc);
assert(connect_success_counter == 2); // for checking valkey version
// Check that the initial slotmap update connection is reused.
assert(connect_success_counter == 1);
// Test connection
valkeyReply *reply;
@ -62,7 +63,7 @@ void test_password_ok(void) {
valkeyClusterFree(cc);
// Check counters incremented by connect callback
assert(connect_success_counter == 3); // for SET (to a different node)
assert(connect_success_counter == 2); // for SET (to a different node)
assert(connect_failure_counter == 0);
reset_counters();
}

View File

@ -318,7 +318,7 @@ void test_alloc_failure_handling(void) {
freeReplyObject(reply);
/* Test ASK reply handling with OOM */
for (int i = 0; i < 30; ++i) {
for (int i = 0; i < 45; ++i) {
prepare_allocation_test(cc, i);
reply = valkeyClusterCommand(cc, "GET foo");
assert(reply == NULL);
@ -326,7 +326,7 @@ void test_alloc_failure_handling(void) {
}
/* Test ASK reply handling without OOM */
prepare_allocation_test(cc, 30);
prepare_allocation_test(cc, 45);
reply = valkeyClusterCommand(cc, "GET foo");
CHECK_REPLY_STR(cc, reply, "one");
freeReplyObject(reply);
@ -347,7 +347,7 @@ void test_alloc_failure_handling(void) {
freeReplyObject(reply);
/* Test MOVED reply handling with OOM */
for (int i = 0; i < 31; ++i) {
for (int i = 0; i < 32; ++i) {
prepare_allocation_test(cc, i);
reply = valkeyClusterCommand(cc, "GET foo");
assert(reply == NULL);
@ -355,7 +355,7 @@ void test_alloc_failure_handling(void) {
}
/* Test MOVED reply handling without OOM */
prepare_allocation_test(cc, 31);
prepare_allocation_test(cc, 32);
reply = valkeyClusterCommand(cc, "GET foo");
CHECK_REPLY_STR(cc, reply, "one");
freeReplyObject(reply);

View File

@ -20,9 +20,6 @@ timeout 5s ./simulated-valkey.pl -p 7401 -d --sigcont $syncpid1 <<'EOF' &
EXPECT CONNECT
EXPECT ["CLUSTER", "SLOTS"]
SEND [[0, 16383, ["127.0.0.1", 7401, "nodeid123"]]]
EXPECT CLOSE
EXPECT CONNECT
EXPECT ["SET", "foo", "initial"]
SEND +OK
@ -50,6 +47,7 @@ wait $syncpid1;
# Run client
timeout 4s "$clientprog" 127.0.0.1:7401 > "$testname.out" <<'EOF'
SET foo initial
!sleep
!async
SET foo timeout1

View File

@ -16,8 +16,6 @@ timeout 5s ./simulated-valkey.pl -p 7401 -d --sigcont $syncpid1 <<'EOF' &
EXPECT CONNECT
EXPECT ["CLUSTER", "SLOTS"]
SEND [[0, 16383, ["127.0.0.1", 7401, "nodeid123"]]]
EXPECT CLOSE
EXPECT CONNECT
EXPECT ["GET", "foo"]
SEND -ASK 12182 127.0.0.1:7402
# A second redirect is needed to test reuse of the new cluster node

View File

@ -48,7 +48,7 @@ server2=$!
wait $syncpid1 $syncpid2;
# Run client
timeout 3s "$clientprog" --use-cluster-nodes 127.0.0.1:7400 > "$testname.out" <<'EOF'
timeout 3s "$clientprog" --blocking-initial-update --use-cluster-nodes 127.0.0.1:7400 > "$testname.out" <<'EOF'
GET foo
GET foo
EOF

View File

@ -35,7 +35,7 @@ server1=$!
wait $syncpid1;
# Run client
timeout 4s "$clientprog" --connection-events 127.0.0.1:7401 > "$testname.out" <<'EOF'
timeout 4s "$clientprog" --blocking-initial-update --connection-events 127.0.0.1:7401 > "$testname.out" <<'EOF'
SET foo initial
# Send a command that is expected to be redirected just before

View File

@ -55,7 +55,7 @@ server2=$!
wait $syncpid1 $syncpid2;
# Run client
timeout 4s "$clientprog" --connection-events 127.0.0.1:7401 > "$testname.out" <<'EOF'
timeout 4s "$clientprog" --blocking-initial-update --connection-events 127.0.0.1:7401 > "$testname.out" <<'EOF'
SET foo initial
SET bar initial

View File

@ -42,7 +42,7 @@ server2=$!
wait $syncpid1 $syncpid2;
# Run client
timeout 4s "$clientprog" 127.0.0.1:7401 > "$testname.out" <<'EOF'
timeout 4s "$clientprog" --blocking-initial-update 127.0.0.1:7401 > "$testname.out" <<'EOF'
!async
SET foo initial
SET bar initial

View File

@ -36,22 +36,17 @@ timeout 5s ./simulated-valkey.pl -p 7401 -d --sigcont $syncpid1 <<'EOF' &
EXPECT CONNECT
EXPECT ["CLUSTER", "SLOTS"]
SEND [[0, 5000, ["127.0.0.1", 7401, "nodeid1"]],[5001, 10000, ["127.0.0.1", 7402, "nodeid2"]],[10001, 16383, ["127.0.0.1", 7403, "nodeid3"]]]
EXPECT CLOSE
# The command "GET {foo}1" is first sent to the slot owner nodeid3, but since this
# node does not exist the failed connection attempt will trigger a slotmap update.
EXPECT CONNECT
EXPECT ["CLUSTER", "SLOTS"]
SEND [[0, 8000, ["127.0.0.1", 7401, "nodeid1"]],[8001, 16383, ["127.0.0.1", 7402, "nodeid2"]]]
EXPECT CLOSE
# The send failure of "GET {foo}2" schedules a slotmap update, which is
# performed when (and just before) the next command "GET {foo}3" is sent.
EXPECT CONNECT
EXPECT ["CLUSTER", "SLOTS"]
SEND [[0, 16383, ["127.0.0.1", 7401, "nodeid1"]]]
EXPECT CLOSE
EXPECT CONNECT
EXPECT ["GET", "{foo}3"]
SEND "bar3"
EXPECT CLOSE

View File

@ -70,7 +70,7 @@ server3=$!
wait $syncpid1 $syncpid2 $syncpid3;
# Run client which resends failed commands in the reply callback
timeout 4s "$clientprog" 127.0.0.1:7401 > "$testname.out" <<'EOF'
timeout 4s "$clientprog" --blocking-initial-update 127.0.0.1:7401 > "$testname.out" <<'EOF'
!resend
!async
GET foo

View File

@ -4,7 +4,7 @@
#
# The first attempt to get the slotmap will receive a reply without any
# slot information and this should result in a retry.
# The following slotmap updates tests the handling of an nil/empty IP address.
# The following slotmap updates tests the handling of a nil/empty IP address.
#
# The client is configured to use the CLUSTER SLOTS command.
#
@ -44,7 +44,7 @@ server=$!
wait $syncpid;
# Run client which will fetch the initial slotmap asynchronously.
timeout 3s "$clientprog" --events --async-initial-update 127.0.0.1:7400 > "$testname.out" <<'EOF'
timeout 3s "$clientprog" --events 127.0.0.1:7400 > "$testname.out" <<'EOF'
# Slot not yet handled, will trigger a slotmap update which will be throttled.
SET foo bar1

View File

@ -37,7 +37,7 @@ server=$!
wait $syncpid;
# Run client which will fetch the initial slotmap asynchronously using CLUSTER NODES.
timeout 3s "$clientprog" --events --use-cluster-nodes --async-initial-update 127.0.0.1:7400 > "$testname.out" <<'EOF'
timeout 3s "$clientprog" --events --use-cluster-nodes 127.0.0.1:7400 > "$testname.out" <<'EOF'
SET foo bar
EOF
clientexit=$?

View File

@ -44,7 +44,7 @@ server1=$!
wait $syncpid1
# Run client
timeout 4s "$clientprog" 127.0.0.1:7401 > "$testname.out" <<'EOF'
timeout 4s "$clientprog" --blocking-initial-update 127.0.0.1:7401 > "$testname.out" <<'EOF'
SET bar initial
# Send commands aimed for nodeid2

View File

@ -53,7 +53,7 @@ server2=$!
wait $syncpid1 $syncpid2;
# Run client
timeout 5s "$clientprog" 127.0.0.1:7401 > "$testname.out" <<'EOF'
timeout 5s "$clientprog" --blocking-initial-update 127.0.0.1:7401 > "$testname.out" <<'EOF'
!all
DBSIZE
DBSIZE

View File

@ -25,8 +25,6 @@ timeout 5s ./simulated-valkey.pl -p 7401 -d --sigcont $syncpid1 <<'EOF' &
EXPECT CONNECT
EXPECT ["CLUSTER", "SLOTS"]
SEND [[0, 8383, ["127.0.0.1", 7401, "nodeid7401"]], [8384, 16383, ["127.0.0.1", 7402, "nodeid7402"]]]
EXPECT CLOSE
EXPECT CONNECT
EXPECT ["DBSIZE"]
SEND 10
EXPECT ["DBSIZE"]

View File

@ -16,8 +16,6 @@ timeout 5s ./simulated-valkey.pl -p 7403 -d --sigcont $syncpid1 <<'EOF' &
EXPECT CONNECT
EXPECT ["CLUSTER", "SLOTS"]
SEND [[0, 8383, ["127.0.0.1", 7403, "nodeid7403"]], [8384, 16383, ["127.0.0.1", 7404, "nodeid7404"]]]
EXPECT CLOSE
EXPECT CONNECT
EXPECT ["DBSIZE"]
SEND 11
EXPECT CLOSE

View File

@ -24,10 +24,10 @@ timeout 5s ./simulated-valkey.pl -p 7403 -d --sigcont $syncpid1 <<'EOF' &
EXPECT CONNECT
EXPECT ["CLUSTER", "SLOTS"]
SEND [[0, 16383, ["127.0.0.1", 7403, "nodeid7403"]]]
EXPECT CLOSE
EXPECT ["GET", "foo"]
SEND "bar"
# Test 1: Handle MOVED redirect.
EXPECT CONNECT
EXPECT ["GET", "foo"]
SEND -MOVED 12182 127.0.0.1:7404
EXPECT ["CLUSTER", "SLOTS"]
@ -54,6 +54,7 @@ EXPECT ["GET", "foo"]
SEND -MOVED 9718 :7403
EXPECT ["CLUSTER", "SLOTS"]
SEND [[0, 16383, ["127.0.0.1", 7403, "nodeid7403"]]]
EXPECT CLOSE
EOF
server2=$!
@ -66,6 +67,8 @@ timeout 3s "$clientprog" --events 127.0.0.1:7403 > "$testname.out" <<'EOF'
GET foo
!sleep
GET foo
!sleep
GET foo
EOF
clientexit=$?
@ -90,6 +93,7 @@ fi
# Check the output from clusterclient
expected="Event: slotmap-updated
Event: ready
bar
Event: slotmap-updated
bar
Event: slotmap-updated

View File

@ -42,7 +42,7 @@ server2=$!
wait $syncpid1 $syncpid2;
# Run client
timeout 3s "$clientprog" --use-cluster-nodes 127.0.0.1:7400 > "$testname.out" <<'EOF'
timeout 3s "$clientprog" --blocking-initial-update --use-cluster-nodes 127.0.0.1:7400 > "$testname.out" <<'EOF'
GET foo
EOF
clientexit=$?

View File

@ -28,10 +28,10 @@ timeout 5s ./simulated-valkey.pl -p 7401 -d --sigcont $syncpid1 <<'EOF' &
EXPECT CONNECT
EXPECT ["CLUSTER", "SLOTS"]
SEND [[0, 16383, ["localhost", 7401, "nodeid1", ["ip", "192.168.254.254"]]]]
EXPECT CLOSE
EXPECT ["GET", "foo"]
SEND "bar"
# Verify ASK redirect
EXPECT CONNECT
EXPECT ["GET", "foo"]
SEND -ASK 12182 localhost:7402
@ -64,7 +64,12 @@ wait $syncpid1 $syncpid2;
# Run client
timeout 3s "$clientprog" localhost:7401 > "$testname.out" <<'EOF'
# Trigger initial slotmap update
GET foo
!sleep
# Verify ASK redirect
GET foo
# Verify MOVED redirect
GET foo
EOF
clientexit=$?
@ -88,7 +93,7 @@ if [ $clientexit -ne 0 ]; then
fi
# Check the output from clusterclient
printf 'bar\nbar\n' | cmp "$testname.out" - || exit 99
printf 'bar\nbar\nbar\n' | cmp "$testname.out" - || exit 99
# Clean up
rm "$testname.out"

View File

@ -19,10 +19,10 @@ timeout 5s ./simulated-valkey.pl -p 7401 --ipv6 -d --sigcont $syncpid1 <<'EOF' &
EXPECT CONNECT
EXPECT ["CLUSTER", "SLOTS"]
SEND [[0, 16383, ["::1", 7401, "nodeid1"]]]
EXPECT CLOSE
EXPECT ["GET", "foo"]
SEND "bar"
# Verify ASK redirect
EXPECT CONNECT
EXPECT ["GET", "foo"]
SEND -ASK 12182 ::1:7402
@ -55,7 +55,12 @@ wait $syncpid1 $syncpid2;
# Run client
timeout 3s "$clientprog" ::1:7401 > "$testname.out" <<'EOF'
# Trigger initial slotmap update
GET foo
!sleep
# Verify ASK redirect
GET foo
# Verify MOVED redirect
GET foo
EOF
clientexit=$?
@ -79,7 +84,7 @@ if [ $clientexit -ne 0 ]; then
fi
# Check the output from clusterclient
printf 'bar\nbar\n' | cmp "$testname.out" - || exit 99
printf 'bar\nbar\nbar\n' | cmp "$testname.out" - || exit 99
# Clean up
rm "$testname.out"

View File

@ -14,8 +14,6 @@ timeout 5s ./simulated-valkey.pl -p 7400 -d --sigcont $syncpid <<'EOF' &
EXPECT CONNECT
EXPECT ["CLUSTER", "SLOTS"]
SEND [[0, 16383, ["127.0.0.1", 7400, "nodeid123"]]]
EXPECT CLOSE
EXPECT CONNECT
EXPECT ["SET", "foo", "bar"]
SEND +OK
EXPECT ["GET", "foo"]

View File

@ -32,7 +32,7 @@ server1=$!
wait $syncpid1;
# Run client
timeout 3s "$clientprog" --events 127.0.0.1:7401 > "$testname.out" <<'EOF'
timeout 3s "$clientprog" --blocking-initial-update --events 127.0.0.1:7401 > "$testname.out" <<'EOF'
GET foo
# Allow slotmap update to finish.
!sleep

View File

@ -15,23 +15,17 @@ timeout 5s ./simulated-valkey.pl -p 7401 -d --sigcont $syncpid1 <<'EOF' &
EXPECT CONNECT
EXPECT ["CLUSTER", "SLOTS"]
SEND [[0, 1, ["127.0.0.1", 7401, "nodeid7401"]]]
EXPECT CLOSE
# Slotmap update due to the slot for `foo1` is not served.
# The reply is still missing slots.
EXPECT CONNECT
EXPECT ["CLUSTER", "SLOTS"]
SEND [[0, 1, ["127.0.0.1", 7401, "nodeid7401"]]]
EXPECT CLOSE
# Slotmap update due to the slot for `foo2` is not served.
# The reply now has full slot coverage.
EXPECT CONNECT
EXPECT ["CLUSTER", "SLOTS"]
SEND [[0, 16383, ["127.0.0.1", 7401, "nodeid7401"]]]
EXPECT CLOSE
EXPECT CONNECT
EXPECT ["GET", "foo2"]
SEND "bar2"
EXPECT CLOSE

View File

@ -59,7 +59,7 @@ server2=$!
wait $syncpid1 $syncpid2;
# Run client
timeout 4s "$clientprog" 127.0.0.1:7401 > "$testname.out" <<'EOF'
timeout 4s "$clientprog" --blocking-initial-update 127.0.0.1:7401 > "$testname.out" <<'EOF'
SET foo initial
SET bar initial

View File

@ -5,6 +5,8 @@
#include <winsock2.h> /* For struct timeval */
#endif
#define UNUSED(x) (void)(x)
#define ASSERT_MSG(_x, _msg) \
if (!(_x)) { \
fprintf(stderr, "ERROR: %s (%s)\n", _msg, #_x); \

View File

@ -60,7 +60,7 @@ int main(int argc, char **argv) {
/* Do something with the string. */
if (line[0] != '\0' && line[0] != '/') {
printf("echo: '%s'\n", line);
linenoiseHistoryAdd(line); /* Add to the history. */
linenoiseHistoryAdd(line, 0); /* Add to the history. */
linenoiseHistorySave("history.txt"); /* Save the history on disk. */
} else if (!strncmp(line,"/historylen",11)) {
/* The "/historylen" command will change the history len. */

View File

@ -708,6 +708,49 @@ void linenoiseEditMoveRight(struct linenoiseState *l) {
}
}
/* People have different preferences on what is considered a word
* when editing commandline. For example, in command
* `set business:department:product value`,
* some might treat the entire long key as a word, while others
* treat it as 3 subwords and prefer granular word jumping navigation
* to make fixing a typo (e.g. "business:deaprtment:product") easier.
* In order to accommodate this, we treat escape sequences
* ESC b/f (usually Alt + / on MacOS, but you might get ESC[1;3D/C) and
* ESC[1;5D/C (usually Ctrl + /) differently.
*
* Notice that the exact behavior of key mapping depends on your
* terminal/keyboard setup. To see how your environment maps keys
* to escape sequences, you can run command `cat -v` and then press
* arbitrary keys to see how they work.
*/
static int isSubWordDelimiter(const char c) {
return !isalnum(c);
}
static int isBigWordDelimiter(const char c) {
return isspace(c);
}
typedef int (*isDelimiterFunc)(char c);
static void linenoiseEditMoveWordLeft(struct linenoiseState *l, const isDelimiterFunc isDelimiter) {
if (l->pos == 0) return;
/* Move cursor to the left over any delimiters */
while (l->pos > 0 && isDelimiter(l->buf[l->pos - 1])) l->pos--;
/* Then continue moving over a word */
while (l->pos > 0 && !isDelimiter(l->buf[l->pos - 1])) l->pos--;
refreshLine(l);
}
static void linenoiseEditMoveWordRight(struct linenoiseState *l, const isDelimiterFunc isDelimiter) {
if (l->pos == l->len) return;
/* Move cursor to the right over any delimiters */
while (l->pos < l->len && isDelimiter(l->buf[l->pos])) l->pos++;
/* Then continue moving over a word */
while (l->pos < l->len && !isDelimiter(l->buf[l->pos])) l->pos++;
refreshLine(l);
}
/* Move cursor to the start of the line. */
void linenoiseEditMoveHome(struct linenoiseState *l) {
if (l->pos != 0) {
@ -901,20 +944,57 @@ static int linenoiseEdit(int stdin_fd, int stdout_fd, char *buf, size_t buflen,
* Use two calls to handle slow terminals returning the two
* chars at different times. */
if (read(l.ifd,seq,1) == -1) break;
/* Handle Meta-b / Meta-f directly.
* Usually generated by Alt + / (but not always)
*/
if (seq[0] == 'b') { /* ESC b → word left */
linenoiseEditMoveWordLeft(&l, isSubWordDelimiter);
break;
}
if (seq[0] == 'f') { /* ESC f → word right */
linenoiseEditMoveWordRight(&l, isSubWordDelimiter);
break;
}
if (read(l.ifd,seq+1,1) == -1) break;
/* ESC [ sequences. */
if (seq[0] == '[') {
if (seq[1] >= '0' && seq[1] <= '9') {
/* Extended escape, read additional byte. */
if (read(l.ifd,seq+2,1) == -1) break;
if (seq[2] == '~') {
switch(seq[1]) {
case '3': /* Delete key. */
linenoiseEditDelete(&l);
/* Extended escape, read additional bytes.
* Examples: ESC [1;5C ESC [3~ */
const int seqBufferMaxLength = 8;
char seqBuffer[seqBufferMaxLength];
int i = 0;
seqBuffer[i++] = seq[1];
/* If first param is digit or ';', read more until we see a final in @~ */
char additionalChar;
while (i < seqBufferMaxLength-1 && read(l.ifd, &additionalChar, 1) != -1) {
seqBuffer[i++] = additionalChar;
if (additionalChar >= '@' && additionalChar <= '~') { /* CSI final byte */
seqBuffer[i] = '\0';
break;
}
}
/* The exact key mapping behavior depends on your keyboard/terminal setup */
/* Usually Alt/Ctrl + ← */
if (!strcmp(seqBuffer, "1;5D") || !strcmp(seqBuffer, "1;3D")) {
linenoiseEditMoveWordLeft(&l, isBigWordDelimiter);
break;
}
/* Usually Alt/Ctrl + → */
if (!strcmp(seqBuffer, "1;5C") || !strcmp(seqBuffer, "1;3C")) {
linenoiseEditMoveWordRight(&l, isBigWordDelimiter);
break;
}
/* Usually the `delete` key */
if (!strcmp(seqBuffer, "3~")) {
linenoiseEditDelete(&l);
break;
}
} else {
switch(seq[1]) {
case 'A': /* Up */

View File

@ -340,13 +340,14 @@ static int luaB_assert (lua_State *L) {
static int luaB_unpack (lua_State *L) {
int i, e, n;
int i, e;
unsigned int n;
luaL_checktype(L, 1, LUA_TTABLE);
i = luaL_optint(L, 2, 1);
e = luaL_opt(L, luaL_checkint, 3, luaL_getn(L, 1));
if (i > e) return 0; /* empty range */
n = e - i + 1; /* number of elements */
if (n <= 0 || !lua_checkstack(L, n)) /* n <= 0 means arith. overflow */
n = (unsigned int)e - (unsigned int)i; /* number of elements minus 1 */
if (n >= INT_MAX || !lua_checkstack(L, ++n))
return luaL_error(L, "too many results to unpack");
lua_rawgeti(L, 1, i); /* push arg[i] (avoiding overflow problems) */
while (i++ < e) /* push arg[i + 1...e] */

33
deps/lua/src/llex.c vendored
View File

@ -138,6 +138,7 @@ static void inclinenumber (LexState *ls) {
void luaX_setinput (lua_State *L, LexState *ls, ZIO *z, TString *source) {
ls->t.token = 0;
ls->decpoint = '.';
ls->L = L;
ls->lookahead.token = TK_EOS; /* no look-ahead token */
@ -207,8 +208,13 @@ static void read_numeral (LexState *ls, SemInfo *seminfo) {
}
static int skip_sep (LexState *ls) {
int count = 0;
/*
** reads a sequence '[=*[' or ']=*]', leaving the last bracket.
** If a sequence is well-formed, return its number of '='s + 2; otherwise,
** return 1 if there is no '='s or 0 otherwise (an unfinished '[==...').
*/
static size_t skip_sep (LexState *ls) {
size_t count = 0;
int s = ls->current;
lua_assert(s == '[' || s == ']');
save_and_next(ls);
@ -216,11 +222,13 @@ static int skip_sep (LexState *ls) {
save_and_next(ls);
count++;
}
return (ls->current == s) ? count : (-count) - 1;
return (ls->current == s) ? count + 2
: (count == 0) ? 1
: 0;
}
static void read_long_string (LexState *ls, SemInfo *seminfo, int sep) {
static void read_long_string (LexState *ls, SemInfo *seminfo, size_t sep) {
int cont = 0;
(void)(cont); /* avoid warnings when `cont' is not used */
save_and_next(ls); /* skip 2nd `[' */
@ -270,8 +278,8 @@ static void read_long_string (LexState *ls, SemInfo *seminfo, int sep) {
}
} endloop:
if (seminfo)
seminfo->ts = luaX_newstring(ls, luaZ_buffer(ls->buff) + (2 + sep),
luaZ_bufflen(ls->buff) - 2*(2 + sep));
seminfo->ts = luaX_newstring(ls, luaZ_buffer(ls->buff) + sep,
luaZ_bufflen(ls->buff) - 2 * sep);
}
@ -346,9 +354,9 @@ static int llex (LexState *ls, SemInfo *seminfo) {
/* else is a comment */
next(ls);
if (ls->current == '[') {
int sep = skip_sep(ls);
size_t sep = skip_sep(ls);
luaZ_resetbuffer(ls->buff); /* `skip_sep' may dirty the buffer */
if (sep >= 0) {
if (sep >= 2) {
read_long_string(ls, NULL, sep); /* long comment */
luaZ_resetbuffer(ls->buff);
continue;
@ -360,13 +368,14 @@ static int llex (LexState *ls, SemInfo *seminfo) {
continue;
}
case '[': {
int sep = skip_sep(ls);
if (sep >= 0) {
size_t sep = skip_sep(ls);
if (sep >= 2) {
read_long_string(ls, seminfo, sep);
return TK_STRING;
}
else if (sep == -1) return '[';
else luaX_lexerror(ls, "invalid long string delimiter", TK_STRING);
else if (sep == 0) /* '[=...' missing second bracket */
luaX_lexerror(ls, "invalid long string delimiter", TK_STRING);
return '[';
}
case '=': {
next(ls);

View File

@ -384,13 +384,17 @@ Proto *luaY_parser (lua_State *L, ZIO *z, Mbuffer *buff, const char *name) {
struct LexState lexstate;
struct FuncState funcstate;
lexstate.buff = buff;
luaX_setinput(L, &lexstate, z, luaS_new(L, name));
TString *tname = luaS_new(L, name);
setsvalue2s(L, L->top, tname);
incr_top(L);
luaX_setinput(L, &lexstate, z, tname);
open_func(&lexstate, &funcstate);
funcstate.f->is_vararg = VARARG_ISVARARG; /* main func. is always vararg */
luaX_next(&lexstate); /* read first token */
chunk(&lexstate);
check(&lexstate, TK_EOS);
close_func(&lexstate);
--L->top;
lua_assert(funcstate.prev == NULL);
lua_assert(funcstate.f->nups == 0);
lua_assert(lexstate.fs == NULL);

View File

@ -434,8 +434,7 @@ static TValue *newkey (lua_State *L, Table *t, const TValue *key) {
** search function for integers
*/
const TValue *luaH_getnum (Table *t, int key) {
/* (1 <= key && key <= t->sizearray) */
if (cast(unsigned int, key-1) < cast(unsigned int, t->sizearray))
if (1 <= key && key <= t->sizearray)
return &t->array[key-1];
else {
lua_Number nk = cast_num(key);

View File

@ -1,5 +1,5 @@
#!/bin/sh
TCL_VERSIONS="8.5 8.6 8.7"
TCL_VERSIONS="8.5 8.6 9.0"
TCLSH=""
for VERSION in $TCL_VERSIONS; do

View File

@ -1,5 +1,5 @@
#!/bin/sh
TCL_VERSIONS="8.5 8.6 8.7"
TCL_VERSIONS="8.5 8.6 9.0"
TCLSH=""
for VERSION in $TCL_VERSIONS; do

View File

@ -1,5 +1,5 @@
#!/bin/sh
TCL_VERSIONS="8.5 8.6 8.7"
TCL_VERSIONS="8.5 8.6 9.0"
TCLSH=""
[ -z "$MAKE" ] && MAKE=make

View File

@ -1,5 +1,5 @@
#!/bin/sh
TCL_VERSIONS="8.5 8.6 8.7"
TCL_VERSIONS="8.5 8.6 9.0"
TCLSH=""
for VERSION in $TCL_VERSIONS; do

Some files were not shown because too many files have changed in this diff Show More