mirror
/
valkey
mirror of https://github.com/valkey-io/valkey
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Introduce HASH items expiration 

Closes https://github.com/valkey-io/valkey/issues/640

This PR introduces support for **field-level expiration in Valkey hash types**, making it possible for individual fields inside a hash to expire independently — creating what we call **volatile fields**.
This is just the first out of 3 PRs. The content of this PR focus on enabling the basic ability to set and modify hash fields expiration as well as persistency (AOF+RDB) and defrag.
[The second PR](https://github.com/ranshid/valkey/pull/5) introduces the new algorithm (volatile-set) to track volatile hash fields is in the last stages of review. The current implementation in this PR (in volatile-set.h/c) is just s tub implementation and will be replaced by [The second PR](https://github.com/ranshid/valkey/pull/5)
[The third PR](https://github.com/ranshid/valkey/pull/4/) which introduces the active expiration and defragmentation jobs.

For more highlevel design details you can track the RFC PR: https://github.com/valkey-io/valkey-rfc/pull/22.

---

Some highlevel major decisions which are taken as part of this work:
1. We decided to copy the existing Redis API in order to maintain compatibility with existing clients.
2. We decided to avoid introducing lazy-expiration at this point, in order to reduce complexity and rely only on active-expiration for memory reclamation. This will require us to continue to work on improving the active expiration job and potentially consider introduce lazy-expiration support later on.
3. Although different commands which are adding expiration on hash fields are influencing the memory utilization (by allocating more memory for expiration time and metadata) we decided to avoid adding the DENYOOM for these commands (an exception is HSETEX) in order to be better aligned with highlevel keys commands like `expire`
4. Some hash type commands will produce unexpected results:
 - HLEN - will still reflect the number of fields which exists in the hash object (either actually expired or not).
 - HRANDFIELD - in some cases we will not be able to randomly select a field which was not already expired. this case happen in 2 cases: 1/ when we are asked to provide a non-uniq fields (i.e negative count) 2/ when the size of the hash is much bigger than the count and we need to provide uniq results. In both cases it is possible that an empty response will be returned to the caller, even in case there are fields in the hash which are either persistent or not expired.
5. For the case were a field is provided with a zero (0) expiration time or expiration time in the past, it is immediately deleted. We decided that, in order to be aligned with how high level keys are handled, we will emit hexpired keyspace event for that case (instead of hdel). For example:
for the case:
6. We will ALWAYS load hash fields during rdb load. This means that when primary is rebooting with an old snapshot, it will take time to reclaim all the expired fields. However this simplifies the current logic and avoid major refactoring that I suspect will be needed.
```
HSET myhash f1 v1
> 0
HGETEX myhash EX 0 FIELDS 1 f1
> "v1"
HTTL myhash FIELDS 1 f1
>  -2
```

The reported events are:
```
1) "psubscribe"
2) "__keyevent@0__*"
3) (integer) 1
1) "pmessage"
2) "__keyevent@0__*"
3) "__keyevent@0__:hset"
4) "myhash"
1) "pmessage"
2) "__keyevent@0__*"
3) "__keyevent@0__:hexpired" <---------------- note this
4) "myhash"
1) "pmessage"
2) "__keyevent@0__*"
3) "__keyevent@0__:del"
4) "myhash"
```
---

This PR also **modularizes and exposes the internal `hashTypeEntry` logic** as a new standalone `entry.c/h` module. This new abstraction handles all aspects of **field–value–expiry encoding** using multiple memory layouts optimized for performance and memory efficiency.

An `entry` is an abstraction that represents a single **field–value pair with optional expiration**. Internally, Valkey uses different memory layouts for compactness and efficiency, chosen dynamically based on size and encoding constraints.

The entry pointer is the field sds. Which make us use an entry just like any sds. We encode the entry layout type
in the field SDS header. Field type SDS_TYPE_5 doesn't have any spare bits to
encode this so we use it only for the first layout type.

Entry with embedded value, used for small sizes. The value is stored as
SDS_TYPE_8. The field can use any SDS type.

Entry can also have expiration timestamp, which is the UNIX timestamp for it to be expired.
For aligned fast access, we keep the expiry timestamp prior to the start of the sds header.

     +----------------+--------------+---------------+
     | Expiration     | field        | value         |
     | 1234567890LL   | hdr "foo" \0 | hdr8 "bar" \0 |
     +-----------------------^-------+---------------+
                             |
                             |
                            entry pointer (points to field sds content)

Entry with value pointer, used for larger fields and values. The field is SDS
type 8 or higher.

     +--------------+-------+--------------+
     | Expiration   | value | field        |
     | 1234567890LL | ptr   | hdr "foo" \0 |
     +--------------+--^----+------^-------+
                       |           |
                       |           |
                       |         entry pointer (points to field sds content)
                       |
                      value pointer = value sds

The `entry.c/h` API provides methods to:
- Create, read, and write and Update field/value/expiration
- Set or clear expiration
- Check expiration state
- Clone or delete an entry

---

This PR introduces **new commands** and extends existing ones to support field expiration:

The proposed API is very much identical to the Redis provided API (Redis 7.4 + 8.0). This is intentionally proposed in order to avoid breaking client applications already opted to use hash items TTL.

**Synopsis**

```
HSETEX key [NX | XX] [FNX | FXX] [EX seconds | PX milliseconds |
  EXAT unix-time-seconds | PXAT unix-time-milliseconds | KEEPTTL]
  FIELDS numfields field value [field value ...]
```

Set the value of one or more fields of a given hash key, and optionally set their expiration time or time-to-live (TTL).

The HSETEX command supports the following set of options:

* `NX` — Only set the fields if the hash object does NOT exist.
* `XX` — Only set the fields if if the hash object doesx exist.
* `FNX` — Only set the fields if none of them already exist.
* `FXX` — Only set the fields if all of them already exist.
* `EX seconds` — Set the specified expiration time in seconds.
* `PX milliseconds` — Set the specified expiration time in milliseconds.
* `EXAT unix-time-seconds` — Set the specified Unix time in seconds at which the fields will expire.
* `PXAT unix-time-milliseconds` — Set the specified Unix time in milliseconds at which the fields will expire.
* `KEEPTTL` — Retain the TTL associated with the fields.

The `EX`, `PX`, `EXAT`, `PXAT`, and `KEEPTTL` options are mutually exclusive.

**Synopsis**

```
HGETEX key [EX seconds | PX milliseconds | EXAT unix-time-seconds |
  PXAT unix-time-milliseconds | PERSIST] FIELDS numfields field
  [field ...]
```

Get the value of one or more fields of a given hash key and optionally set their expiration time or time-to-live (TTL).

The `HGETEX` command supports a set of options:

* `EX seconds` — Set the specified expiration time, in seconds.
* `PX milliseconds` — Set the specified expiration time, in milliseconds.
* `EXAT unix-time-seconds` — Set the specified Unix time at which the fields will expire, in seconds.
* `PXAT unix-time-milliseconds` — Set the specified Unix time at which the fields will expire, in milliseconds.
* `PERSIST` — Remove the TTL associated with the fields.

The `EX`, `PX`, `EXAT`, `PXAT`, and `PERSIST` options are mutually exclusive.

**Synopsis**

```
HEXPIRE key seconds [NX | XX | GT | LT] FIELDS numfields
  field [field ...]
```

Set an expiration (TTL or time to live) on one or more fields of a given hash key. You must specify at least one field. Field(s) will automatically be deleted from the hash key when their TTLs expire.
Field expirations will only be cleared by commands that delete or overwrite the contents of the hash fields, including `HDEL` and `HSET` commands. This means that all the operations that conceptually *alter* the value stored at a hash key's field without replacing it with a new one will leave the TTL untouched.
You can clear the TTL of a specific field by specifying 0 for the ‘seconds’ argument.
Note that calling `HEXPIRE`/`HPEXPIRE` with a time in the past will result in the hash field being deleted immediately.

The `HEXPIRE` command supports a set of options:

* `NX` — For each specified field, set expiration only when the field has no expiration.
* `XX` — For each specified field, set expiration only when the field has an existing expiration.
* `GT` — For each specified field, set expiration only when the new expiration is greater than current one.
* `LT` — For each specified field, set expiration only when the new expiration is less than current one.

**Synopsis**

```
HEXPIREAT key unix-time-seconds [NX | XX | GT | LT] FIELDS numfields
  field [field ...]
```

`HEXPIREAT` has the same effect and semantics as `HEXPIRE`, but instead of specifying the number of seconds for the TTL (time to live), it takes an absolute Unix timestamp in seconds since Unix epoch. A timestamp in the past will delete the field immediately.

The `HEXPIREAT` command supports a set of options:

* `NX` — For each specified field, set expiration only when the field has no expiration.
* `XX` — For each specified field, set expiration only when the field has an existing expiration.
* `GT` — For each specified field, set expiration only when the new expiration is greater than current one.
* `LT` — For each specified field, set expiration only when the new expiration is less than current one.

**Synopsis**

```
HPEXPIRE key milliseconds [NX | XX | GT | LT] FIELDS numfields
  field [field ...]
```

This command works like `HEXPIRE`, but the expiration of a field is specified in milliseconds instead of seconds.

The `HPEXPIRE` command supports a set of options:

* `NX` — For each specified field, set expiration only when the field has no expiration.
* `XX` — For each specified field, set expiration only when the field has an existing expiration.
* `GT` — For each specified field, set expiration only when the new expiration is greater than current one.
* `LT` — For each specified field, set expiration only when the new expiration is less than current one.

**Synopsis**

```
HPEXPIREAT key unix-time-milliseconds [NX | XX | GT | LT]
  FIELDS numfields field [field ...]
```

`HPEXPIREAT` has the same effect and semantics as `HEXPIREAT``,` but the Unix time at which the field will expire is specified in milliseconds since Unix epoch instead of seconds.

**Synopsis**

```
HPERSIST key FIELDS numfields field [field ...]
```

Remove the existing expiration on a hash key's field(s), turning the field(s) from *volatile* (a field with expiration set) to *persistent* (a field that will never expire as no TTL (time to live) is associated).

**Synopsis**

```
HSETEX key [NX] seconds field value [field value ...]
```

Similar to `HSET` but adds one or more hash fields that expire after specified number of seconds. By default, this command overwrites the values and expirations of specified fields that exist in the hash. If `NX` option is specified, the field data will not be overwritten. If `key` doesn't exist, a new Hash key is created.

The HSETEX command supports a set of options:

* `NX` — For each specified field, set expiration only when the field has no expiration.

**Synopsis**

```
HTTL key FIELDS numfields field [field ...]
```

Returns the **remaining** TTL (time to live) of a hash key's field(s) that have a set expiration. This introspection capability allows you to check how many seconds a given hash field will continue to be part of the hash key.

```
HPTTL key FIELDS numfields field [field ...]
```

Like `HTTL`, this command returns the remaining TTL (time to live) of a field that has an expiration set, but in milliseconds instead of seconds.

**Synopsis**

```
HEXPIRETIME key FIELDS numfields field [field ...]
```

Returns the absolute Unix timestamp in seconds since Unix epoch at which the given key's field(s) will expire.

**Synopsis**

```
HPEXPIRETIME key FIELDS numfields field [field ...]
```

`HPEXPIRETIME` has the same semantics as `HEXPIRETIME`, but returns the absolute Unix expiration timestamp in milliseconds since Unix epoch instead of seconds.

This PR introduces new notification events to support field-level expiration:

| Event       | Trigger                                  |
|-------------|-------------------------------------------|
| `hexpire`   | Field expiration was set                  |
| `hexpired`  | Field was deleted due to expiration       |
| `hpersist`  | Expiration was removed from a field       |
| `del`       | Key was deleted after all fields expired  |

Note that we diverge from Redis in the cases we emit hexpired event.
For example:
given the following usecase:
```
HSET myhash f1 v1
(integer) 0
HGETEX myhash EX 0 FIELDS 1 f1
1) "v1"
 HTTL myhash FIELDS 1 f1
1) (integer) -2
```
regarding the keyspace-notifications:
Redis reports:
```
1) "psubscribe"
2) "__keyevent@0__:*"
3) (integer) 1
1) "pmessage"
2) "__keyevent@0__:*"
3) "__keyevent@0__:hset"
4) "myhash2"
1) "pmessage"
2) "__keyevent@0__:*"
3) "__keyevent@0__:hdel" <---------------- note this
4) "myhash2"
1) "pmessage"
2) "__keyevent@0__:*"
3) "__keyevent@0__:del"
4) "myhash2"
```

However In our current suggestion, Valkey will emit:
```
1) "psubscribe"
2) "__keyevent@0__*"
3) (integer) 1
1) "pmessage"
2) "__keyevent@0__*"
3) "__keyevent@0__:hset"
4) "myhash"
1) "pmessage"
2) "__keyevent@0__*"
3) "__keyevent@0__:hexpired" <---------------- note this
4) "myhash"
1) "pmessage"
2) "__keyevent@0__*"
3) "__keyevent@0__:del"
4) "myhash"
```
---

- Expiration-aware commands (`HSETEX`, `HGETEX`, etc.) are **not propagated as-is**.
- Instead, Valkey rewrites them into equivalent commands like:
  - `HDEL` (for expired fields)
  - `HPEXPIREAT` (for setting absolute expiration)
  - `HPERSIST` (for removing expiration)

This ensures compatibility with replication and AOF while maintaining consistent field-level expiry behavior.

---

| Command Name | QPS Standard | QPS HFE | QPS Diff % | Latency Standard (ms) | Latency HFE (ms) | Latency Diff % |
|--------------|-------------|---------|------------|----------------------|------------------|----------------|
| **One Large Hash Table** |
| HGET | 137988.12 | 138484.97 | +0.36% | 0.951 | 0.949 | -0.21% |
| HSET | 138561.73 | 137343.77 | -0.87% | 0.948 | 0.956 | +0.84% |
| HEXISTS | 139431.12 | 138677.02 | -0.54% | 0.942 | 0.946 | +0.42% |
| HDEL | 140114.89 | 138966.09 | -0.81% | 0.938 | 0.945 | +0.74% |
| **Many Hash Tables (100 fields)** |
| HGET | 136798.91 | 137419.27 | +0.45% | 0.959 | 0.956 | -0.31% |
| HEXISTS | 138946.78 | 139645.31 | +0.50% | 0.946 | 0.941 | -0.52% |
| HGETALL | 42194.09 | 42016.80 | -0.42% | 0.621 | 0.625 | +0.64% |
| HSET | 137230.69 | 137249.53 | +0.01% | 0.959 | 0.958 | -0.10% |
| HDEL | 138985.41 | 138619.34 | -0.26% | 0.948 | 0.949 | +0.10% |
| **Many Hash Tables (1000 fields)** |
| HGET | 135795.77 | 139256.36 | +2.54% | 0.965 | 0.943 | -2.27% |
| HEXISTS | 138121.55 | 137950.06 | -0.12% | 0.951 | 0.952 | +0.10% |
| HGETALL | 5885.81 | 5633.80 | **-4.28%** | 2.690 | 2.841 | **+5.61%** |
| HSET | 137005.08 | 137400.39 | +0.28% | 0.959 | 0.955 | -0.41% |
| HDEL | 138293.45 | 137381.52 | -0.65% | 0.948 | 0.955 | +0.73% |

[ ] Consider extending HSETEX with extra arguments: NX/XX so that it is possible to prevent adding/setting/mutating fields of a non-existent hash
[ ] Avoid loading expired fields when non-preamble RDB is being loaded on primary. This is an optimization in order to reduce loading unnecessary fields (which are expired). This would also require us to propagate the HDEL to the replicas in case of RDBFLAGS_FEED_REPL. Note that it might have to require some refactoring:
1/ propagate the rdbflags and current time to rdbLoadObject. 2/ consider the case of restore and check_rdb etc...
For this reason I would like to avoid this optimizationfor the first drop.

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>
This commit is contained in:
Ran Shidlansik 2025-08-05 11:12:27 +03:00
parent 3c738f08a4
commit 65215e5378
42 changed files with 6943 additions and 673 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
cmake/Modules/SourceFiles.cmake
View File

@ -117,7 +117,10 @@ set(VALKEY_SERVER_SRCS
${CMAKE_SOURCE_DIR}/src/connection.c
${CMAKE_SOURCE_DIR}/src/unix.c
${CMAKE_SOURCE_DIR}/src/server.c
${CMAKE_SOURCE_DIR}/src/logreqres.c)
${CMAKE_SOURCE_DIR}/src/logreqres.c
${CMAKE_SOURCE_DIR}/src/entry.c
${CMAKE_SOURCE_DIR}/src/volatile_set.c)
# valkey-cli
set(VALKEY_CLI_SRCS

2
src/Makefile
View File

@ -423,7 +423,7 @@ ENGINE_NAME=valkey
SERVER_NAME=$(ENGINE_NAME)-server$(PROG_SUFFIX)
ENGINE_SENTINEL_NAME=$(ENGINE_NAME)-sentinel$(PROG_SUFFIX)
ENGINE_TRACE_OBJ=trace/trace.o trace/trace_commands.o trace/trace_db.o trace/trace_cluster.o trace/trace_server.o trace/trace_rdb.o trace/trace_aof.o
ENGINE_SERVER_OBJ=threads_mngr.o adlist.o vector.o quicklist.o ae.o anet.o dict.o hashtable.o kvstore.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o memory_prefetch.o io_threads.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o cluster_legacy.o cluster_slot_stats.o crc16.o endianconv.o commandlog.o eval.o bio.o rio.o rand.o memtest.o syscheck.o crcspeed.o crccombine.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o valkey-check-rdb.o valkey-check-aof.o geo.o lazyfree.o module.o evict.o expire.o geohash.o geohash_helper.o childinfo.o allocator_defrag.o defrag.o siphash.o rax.o t_stream.o listpack.o localtime.o lolwut.o lolwut5.o lolwut6.o acl.o tracking.o socket.o tls.o sha256.o timeout.o setcpuaffinity.o monotonic.o mt19937-64.o resp_parser.o call_reply.o script.o functions.o commands.o strl.o connection.o unix.o logreqres.o rdma.o scripting_engine.o lua/script_lua.o lua/function_lua.o lua/engine_lua.o lua/debug_lua.o
ENGINE_SERVER_OBJ=threads_mngr.o adlist.o vector.o quicklist.o ae.o anet.o dict.o hashtable.o kvstore.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o memory_prefetch.o io_threads.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o cluster_legacy.o cluster_slot_stats.o crc16.o endianconv.o commandlog.o eval.o bio.o rio.o rand.o memtest.o syscheck.o crcspeed.o crccombine.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o valkey-check-rdb.o valkey-check-aof.o geo.o lazyfree.o module.o evict.o expire.o geohash.o geohash_helper.o childinfo.o allocator_defrag.o defrag.o siphash.o rax.o t_stream.o listpack.o localtime.o lolwut.o lolwut5.o lolwut6.o acl.o tracking.o socket.o tls.o sha256.o timeout.o setcpuaffinity.o monotonic.o mt19937-64.o resp_parser.o call_reply.o script.o functions.o commands.o strl.o connection.o unix.o logreqres.o rdma.o scripting_engine.o entry.o volatile_set.o lua/script_lua.o lua/function_lua.o lua/engine_lua.o lua/debug_lua.o
ENGINE_SERVER_OBJ+=$(ENGINE_TRACE_OBJ)
ENGINE_CLI_NAME=$(ENGINE_NAME)-cli$(PROG_SUFFIX)
ENGINE_CLI_OBJ=anet.o adlist.o dict.o valkey-cli.o zmalloc.o release.o ae.o serverassert.o crcspeed.o crccombine.o crc64.o siphash.o crc16.o monotonic.o cli_common.o mt19937-64.o strl.o cli_commands.o sds.o util.o sha256.o

2
src/anet.c
View File

@ -52,8 +52,6 @@
#include "util.h"
#include "serverassert.h"
#define UNUSED(x) (void)(x)
static void anetSetError(char *err, const char *fmt, ...) {
va_list ap;

32
src/aof.c
View File

@ -1955,12 +1955,32 @@ static int rioWriteHashIteratorCursor(rio *r, hashTypeIterator *hi, int what) {
* The function returns 0 on error, 1 on success. */
int rewriteHashObject(rio *r, robj *key, robj *o) {
hashTypeIterator hi;
long long count = 0, items = hashTypeLength(o);
long long count = 0, volatile_items = 0, non_volatile_items;
/* First serialize volatile items if exist */
if (hashTypeHasVolatileElements(o)) {
hashTypeInitVolatileIterator(o, &hi);
while (hashTypeNext(&hi) != C_ERR) {
long long expiry = entryGetExpiry(hi.next);
sds field = entryGetField(hi.next);
sds value = entryGetValue(hi.next);
if (rioWriteBulkCount(r, '*', 8) == 0) return 0;
if (rioWriteBulkString(r, "HSETEX", 6) == 0) return 0;
if (rioWriteBulkObject(r, key) == 0) return 0;
if (rioWriteBulkString(r, "PXAT", 4) == 0) return 0;
if (rioWriteBulkLongLong(r, expiry) == 0) return 0;
if (rioWriteBulkString(r, "FIELDS", 6) == 0) return 0;
if (rioWriteBulkLongLong(r, 1) == 0) return 0;
if (rioWriteBulkString(r, field, sdslen(field)) == 0) return 0;
if (rioWriteBulkString(r, value, sdslen(value)) == 0) return 0;
volatile_items++;
}
hashTypeResetIterator(&hi);
}
non_volatile_items = hashTypeLength(o) - volatile_items;
hashTypeInitIterator(o, &hi);
while (hashTypeNext(&hi) != C_ERR) {
if (count == 0) {
int cmd_items = (items > AOF_REWRITE_ITEMS_PER_CMD) ? AOF_REWRITE_ITEMS_PER_CMD : items;
int cmd_items = (non_volatile_items > AOF_REWRITE_ITEMS_PER_CMD) ? AOF_REWRITE_ITEMS_PER_CMD : non_volatile_items;
if (!rioWriteBulkCount(r, '*', 2 + cmd_items * 2) || !rioWriteBulkString(r, "HMSET", 5) ||
!rioWriteBulkObject(r, key)) {
@ -1969,16 +1989,18 @@ int rewriteHashObject(rio *r, robj *key, robj *o) {
}
}
if (volatile_items > 0 && entryHasExpiry(hi.next))
continue;
if (!rioWriteHashIteratorCursor(r, &hi, OBJ_HASH_FIELD) || !rioWriteHashIteratorCursor(r, &hi, OBJ_HASH_VALUE)) {
hashTypeResetIterator(&hi);
return 0;
}
if (++count == AOF_REWRITE_ITEMS_PER_CMD) count = 0;
items--;
non_volatile_items--;
}
hashTypeResetIterator(&hi);
return 1;
}

425
src/commands.def
View File

@ -3564,6 +3564,119 @@ struct COMMAND_ARG HEXISTS_Args[] = {
{MAKE_ARG("field",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
};
/********** HEXPIRE ********************/
#ifndef SKIP_CMD_HISTORY_TABLE
/* HEXPIRE history */
#define HEXPIRE_History NULL
#endif
#ifndef SKIP_CMD_TIPS_TABLE
/* HEXPIRE tips */
#define HEXPIRE_Tips NULL
#endif
#ifndef SKIP_CMD_KEY_SPECS_TABLE
/* HEXPIRE key specs */
keySpec HEXPIRE_Keyspecs[1] = {
{NULL,CMD_KEY_RW|CMD_KEY_UPDATE,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={0,1,0}}
};
#endif
/* HEXPIRE condition argument table */
struct COMMAND_ARG HEXPIRE_condition_Subargs[] = {
{MAKE_ARG("nx",ARG_TYPE_PURE_TOKEN,-1,"NX",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("xx",ARG_TYPE_PURE_TOKEN,-1,"XX",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("gt",ARG_TYPE_PURE_TOKEN,-1,"GT",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("lt",ARG_TYPE_PURE_TOKEN,-1,"LT",NULL,NULL,CMD_ARG_NONE,0,NULL)},
};
/* HEXPIRE fields argument table */
struct COMMAND_ARG HEXPIRE_fields_Subargs[] = {
{MAKE_ARG("numfields",ARG_TYPE_INTEGER,-1,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("field",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_MULTIPLE,0,NULL)},
};
/* HEXPIRE argument table */
struct COMMAND_ARG HEXPIRE_Args[] = {
{MAKE_ARG("key",ARG_TYPE_KEY,0,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("seconds",ARG_TYPE_INTEGER,-1,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("condition",ARG_TYPE_ONEOF,-1,NULL,NULL,NULL,CMD_ARG_OPTIONAL,4,NULL),.subargs=HEXPIRE_condition_Subargs},
{MAKE_ARG("fields",ARG_TYPE_BLOCK,-1,"FIELDS",NULL,NULL,CMD_ARG_NONE,2,NULL),.subargs=HEXPIRE_fields_Subargs},
};
/********** HEXPIREAT ********************/
#ifndef SKIP_CMD_HISTORY_TABLE
/* HEXPIREAT history */
#define HEXPIREAT_History NULL
#endif
#ifndef SKIP_CMD_TIPS_TABLE
/* HEXPIREAT tips */
#define HEXPIREAT_Tips NULL
#endif
#ifndef SKIP_CMD_KEY_SPECS_TABLE
/* HEXPIREAT key specs */
keySpec HEXPIREAT_Keyspecs[1] = {
{NULL,CMD_KEY_RW|CMD_KEY_UPDATE,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={0,1,0}}
};
#endif
/* HEXPIREAT condition argument table */
struct COMMAND_ARG HEXPIREAT_condition_Subargs[] = {
{MAKE_ARG("nx",ARG_TYPE_PURE_TOKEN,-1,"NX",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("xx",ARG_TYPE_PURE_TOKEN,-1,"XX",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("gt",ARG_TYPE_PURE_TOKEN,-1,"GT",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("lt",ARG_TYPE_PURE_TOKEN,-1,"LT",NULL,NULL,CMD_ARG_NONE,0,NULL)},
};
/* HEXPIREAT fields argument table */
struct COMMAND_ARG HEXPIREAT_fields_Subargs[] = {
{MAKE_ARG("numfields",ARG_TYPE_INTEGER,0,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("field",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_MULTIPLE,0,NULL)},
};
/* HEXPIREAT argument table */
struct COMMAND_ARG HEXPIREAT_Args[] = {
{MAKE_ARG("key",ARG_TYPE_KEY,0,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("unix-time-seconds",ARG_TYPE_INTEGER,-1,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("condition",ARG_TYPE_ONEOF,-1,NULL,NULL,"9.0.0",CMD_ARG_OPTIONAL,4,NULL),.subargs=HEXPIREAT_condition_Subargs},
{MAKE_ARG("fields",ARG_TYPE_BLOCK,-1,"FIELDS",NULL,NULL,CMD_ARG_NONE,2,NULL),.subargs=HEXPIREAT_fields_Subargs},
};
/********** HEXPIRETIME ********************/
#ifndef SKIP_CMD_HISTORY_TABLE
/* HEXPIRETIME history */
#define HEXPIRETIME_History NULL
#endif
#ifndef SKIP_CMD_TIPS_TABLE
/* HEXPIRETIME tips */
#define HEXPIRETIME_Tips NULL
#endif
#ifndef SKIP_CMD_KEY_SPECS_TABLE
/* HEXPIRETIME key specs */
keySpec HEXPIRETIME_Keyspecs[1] = {
{NULL,CMD_KEY_RO|CMD_KEY_ACCESS,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={0,1,0}}
};
#endif
/* HEXPIRETIME fields argument table */
struct COMMAND_ARG HEXPIRETIME_fields_Subargs[] = {
{MAKE_ARG("numfields",ARG_TYPE_INTEGER,0,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("field",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_MULTIPLE,0,NULL)},
};
/* HEXPIRETIME argument table */
struct COMMAND_ARG HEXPIRETIME_Args[] = {
{MAKE_ARG("key",ARG_TYPE_KEY,0,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("fields",ARG_TYPE_BLOCK,-1,"FIELDS",NULL,NULL,CMD_ARG_NONE,2,NULL),.subargs=HEXPIRETIME_fields_Subargs},
};
/********** HGET ********************/
#ifndef SKIP_CMD_HISTORY_TABLE
@ -3615,6 +3728,47 @@ struct COMMAND_ARG HGETALL_Args[] = {
{MAKE_ARG("key",ARG_TYPE_KEY,0,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
};
/********** HGETEX ********************/
#ifndef SKIP_CMD_HISTORY_TABLE
/* HGETEX history */
#define HGETEX_History NULL
#endif
#ifndef SKIP_CMD_TIPS_TABLE
/* HGETEX tips */
#define HGETEX_Tips NULL
#endif
#ifndef SKIP_CMD_KEY_SPECS_TABLE
/* HGETEX key specs */
keySpec HGETEX_Keyspecs[1] = {
{NULL,CMD_KEY_RW|CMD_KEY_ACCESS,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={0,1,0}}
};
#endif
/* HGETEX expiration argument table */
struct COMMAND_ARG HGETEX_expiration_Subargs[] = {
{MAKE_ARG("seconds",ARG_TYPE_INTEGER,-1,"EX",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("milliseconds",ARG_TYPE_INTEGER,-1,"PX",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("unix-time-seconds",ARG_TYPE_UNIX_TIME,-1,"EXAT",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("unix-time-milliseconds",ARG_TYPE_UNIX_TIME,-1,"PXAT",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("persist",ARG_TYPE_PURE_TOKEN,-1,"PERSIST",NULL,NULL,CMD_ARG_NONE,0,NULL)},
};
/* HGETEX fields argument table */
struct COMMAND_ARG HGETEX_fields_Subargs[] = {
{MAKE_ARG("numfields",ARG_TYPE_INTEGER,0,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("field",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_MULTIPLE,0,NULL)},
};
/* HGETEX argument table */
struct COMMAND_ARG HGETEX_Args[] = {
{MAKE_ARG("key",ARG_TYPE_KEY,0,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("expiration",ARG_TYPE_ONEOF,-1,NULL,NULL,NULL,CMD_ARG_OPTIONAL,5,NULL),.subargs=HGETEX_expiration_Subargs},
{MAKE_ARG("fields",ARG_TYPE_BLOCK,-1,"FIELDS",NULL,NULL,CMD_ARG_NONE,2,NULL),.subargs=HGETEX_fields_Subargs},
};
/********** HINCRBY ********************/
#ifndef SKIP_CMD_HISTORY_TABLE
@ -3773,6 +3927,181 @@ struct COMMAND_ARG HMSET_Args[] = {
{MAKE_ARG("data",ARG_TYPE_BLOCK,-1,NULL,NULL,NULL,CMD_ARG_MULTIPLE,2,NULL),.subargs=HMSET_data_Subargs},
};
/********** HPERSIST ********************/
#ifndef SKIP_CMD_HISTORY_TABLE
/* HPERSIST history */
#define HPERSIST_History NULL
#endif
#ifndef SKIP_CMD_TIPS_TABLE
/* HPERSIST tips */
#define HPERSIST_Tips NULL
#endif
#ifndef SKIP_CMD_KEY_SPECS_TABLE
/* HPERSIST key specs */
keySpec HPERSIST_Keyspecs[1] = {
{NULL,CMD_KEY_RW|CMD_KEY_UPDATE,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={0,1,0}}
};
#endif
/* HPERSIST fields argument table */
struct COMMAND_ARG HPERSIST_fields_Subargs[] = {
{MAKE_ARG("numfields",ARG_TYPE_INTEGER,0,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("field",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_MULTIPLE,0,NULL)},
};
/* HPERSIST argument table */
struct COMMAND_ARG HPERSIST_Args[] = {
{MAKE_ARG("key",ARG_TYPE_KEY,0,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("fields",ARG_TYPE_BLOCK,-1,"FIELDS",NULL,NULL,CMD_ARG_NONE,2,NULL),.subargs=HPERSIST_fields_Subargs},
};
/********** HPEXPIRE ********************/
#ifndef SKIP_CMD_HISTORY_TABLE
/* HPEXPIRE history */
#define HPEXPIRE_History NULL
#endif
#ifndef SKIP_CMD_TIPS_TABLE
/* HPEXPIRE tips */
#define HPEXPIRE_Tips NULL
#endif
#ifndef SKIP_CMD_KEY_SPECS_TABLE
/* HPEXPIRE key specs */
keySpec HPEXPIRE_Keyspecs[1] = {
{NULL,CMD_KEY_RW|CMD_KEY_UPDATE,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={0,1,0}}
};
#endif
/* HPEXPIRE condition argument table */
struct COMMAND_ARG HPEXPIRE_condition_Subargs[] = {
{MAKE_ARG("nx",ARG_TYPE_PURE_TOKEN,-1,"NX",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("xx",ARG_TYPE_PURE_TOKEN,-1,"XX",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("gt",ARG_TYPE_PURE_TOKEN,-1,"GT",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("lt",ARG_TYPE_PURE_TOKEN,-1,"LT",NULL,NULL,CMD_ARG_NONE,0,NULL)},
};
/* HPEXPIRE fields argument table */
struct COMMAND_ARG HPEXPIRE_fields_Subargs[] = {
{MAKE_ARG("numfields",ARG_TYPE_INTEGER,0,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("field",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_MULTIPLE,0,NULL)},
};
/* HPEXPIRE argument table */
struct COMMAND_ARG HPEXPIRE_Args[] = {
{MAKE_ARG("key",ARG_TYPE_KEY,0,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("milliseconds",ARG_TYPE_INTEGER,-1,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("condition",ARG_TYPE_ONEOF,-1,NULL,NULL,"9.0.0",CMD_ARG_OPTIONAL,4,NULL),.subargs=HPEXPIRE_condition_Subargs},
{MAKE_ARG("fields",ARG_TYPE_BLOCK,-1,"FIELDS",NULL,NULL,CMD_ARG_NONE,2,NULL),.subargs=HPEXPIRE_fields_Subargs},
};
/********** HPEXPIREAT ********************/
#ifndef SKIP_CMD_HISTORY_TABLE
/* HPEXPIREAT history */
#define HPEXPIREAT_History NULL
#endif
#ifndef SKIP_CMD_TIPS_TABLE
/* HPEXPIREAT tips */
#define HPEXPIREAT_Tips NULL
#endif
#ifndef SKIP_CMD_KEY_SPECS_TABLE
/* HPEXPIREAT key specs */
keySpec HPEXPIREAT_Keyspecs[1] = {
{NULL,CMD_KEY_RW|CMD_KEY_UPDATE,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={0,1,0}}
};
#endif
/* HPEXPIREAT condition argument table */
struct COMMAND_ARG HPEXPIREAT_condition_Subargs[] = {
{MAKE_ARG("nx",ARG_TYPE_PURE_TOKEN,-1,"NX",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("xx",ARG_TYPE_PURE_TOKEN,-1,"XX",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("gt",ARG_TYPE_PURE_TOKEN,-1,"GT",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("lt",ARG_TYPE_PURE_TOKEN,-1,"LT",NULL,NULL,CMD_ARG_NONE,0,NULL)},
};
/* HPEXPIREAT fields argument table */
struct COMMAND_ARG HPEXPIREAT_fields_Subargs[] = {
{MAKE_ARG("numfields",ARG_TYPE_INTEGER,0,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("field",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_MULTIPLE,0,NULL)},
};
/* HPEXPIREAT argument table */
struct COMMAND_ARG HPEXPIREAT_Args[] = {
{MAKE_ARG("key",ARG_TYPE_KEY,0,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("unix-time-milliseconds",ARG_TYPE_INTEGER,-1,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("condition",ARG_TYPE_ONEOF,-1,NULL,NULL,"9.0.0",CMD_ARG_OPTIONAL,4,NULL),.subargs=HPEXPIREAT_condition_Subargs},
{MAKE_ARG("fields",ARG_TYPE_BLOCK,-1,"FIELDS",NULL,NULL,CMD_ARG_NONE,2,NULL),.subargs=HPEXPIREAT_fields_Subargs},
};
/********** HPEXPIRETIME ********************/
#ifndef SKIP_CMD_HISTORY_TABLE
/* HPEXPIRETIME history */
#define HPEXPIRETIME_History NULL
#endif
#ifndef SKIP_CMD_TIPS_TABLE
/* HPEXPIRETIME tips */
#define HPEXPIRETIME_Tips NULL
#endif
#ifndef SKIP_CMD_KEY_SPECS_TABLE
/* HPEXPIRETIME key specs */
keySpec HPEXPIRETIME_Keyspecs[1] = {
{NULL,CMD_KEY_RO|CMD_KEY_ACCESS,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={0,1,0}}
};
#endif
/* HPEXPIRETIME fields argument table */
struct COMMAND_ARG HPEXPIRETIME_fields_Subargs[] = {
{MAKE_ARG("numfields",ARG_TYPE_INTEGER,0,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("field",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_MULTIPLE,0,NULL)},
};
/* HPEXPIRETIME argument table */
struct COMMAND_ARG HPEXPIRETIME_Args[] = {
{MAKE_ARG("key",ARG_TYPE_KEY,0,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("fields",ARG_TYPE_BLOCK,-1,"FIELDS",NULL,NULL,CMD_ARG_NONE,2,NULL),.subargs=HPEXPIRETIME_fields_Subargs},
};
/********** HPTTL ********************/
#ifndef SKIP_CMD_HISTORY_TABLE
/* HPTTL history */
#define HPTTL_History NULL
#endif
#ifndef SKIP_CMD_TIPS_TABLE
/* HPTTL tips */
#define HPTTL_Tips NULL
#endif
#ifndef SKIP_CMD_KEY_SPECS_TABLE
/* HPTTL key specs */
keySpec HPTTL_Keyspecs[1] = {
{NULL,CMD_KEY_RO|CMD_KEY_ACCESS,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={0,1,0}}
};
#endif
/* HPTTL fields argument table */
struct COMMAND_ARG HPTTL_fields_Subargs[] = {
{MAKE_ARG("numfields",ARG_TYPE_INTEGER,0,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("field",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_MULTIPLE,0,NULL)},
};
/* HPTTL argument table */
struct COMMAND_ARG HPTTL_Args[] = {
{MAKE_ARG("key",ARG_TYPE_KEY,0,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("fields",ARG_TYPE_BLOCK,-1,"FIELDS",NULL,NULL,CMD_ARG_NONE,2,NULL),.subargs=HPTTL_fields_Subargs},
};
/********** HRANDFIELD ********************/
#ifndef SKIP_CMD_HISTORY_TABLE
@ -3869,6 +4198,60 @@ struct COMMAND_ARG HSET_Args[] = {
{MAKE_ARG("data",ARG_TYPE_BLOCK,-1,NULL,NULL,NULL,CMD_ARG_MULTIPLE,2,NULL),.subargs=HSET_data_Subargs},
};
/********** HSETEX ********************/
#ifndef SKIP_CMD_HISTORY_TABLE
/* HSETEX history */
#define HSETEX_History NULL
#endif
#ifndef SKIP_CMD_TIPS_TABLE
/* HSETEX tips */
#define HSETEX_Tips NULL
#endif
#ifndef SKIP_CMD_KEY_SPECS_TABLE
/* HSETEX key specs */
keySpec HSETEX_Keyspecs[1] = {
{NULL,CMD_KEY_RW|CMD_KEY_INSERT,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={0,1,0}}
};
#endif
/* HSETEX fields_condition argument table */
struct COMMAND_ARG HSETEX_fields_condition_Subargs[] = {
{MAKE_ARG("fnx",ARG_TYPE_PURE_TOKEN,-1,"FNX",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("fxx",ARG_TYPE_PURE_TOKEN,-1,"FXX",NULL,NULL,CMD_ARG_NONE,0,NULL)},
};
/* HSETEX expiration argument table */
struct COMMAND_ARG HSETEX_expiration_Subargs[] = {
{MAKE_ARG("seconds",ARG_TYPE_INTEGER,-1,"EX",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("milliseconds",ARG_TYPE_INTEGER,-1,"PX",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("unix-time-seconds",ARG_TYPE_UNIX_TIME,-1,"EXAT",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("unix-time-milliseconds",ARG_TYPE_UNIX_TIME,-1,"PXAT",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("keepttl",ARG_TYPE_PURE_TOKEN,-1,"KEEPTTL",NULL,NULL,CMD_ARG_NONE,0,NULL)},
};
/* HSETEX fields data argument table */
struct COMMAND_ARG HSETEX_fields_data_Subargs[] = {
{MAKE_ARG("field",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("value",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
};
/* HSETEX fields argument table */
struct COMMAND_ARG HSETEX_fields_Subargs[] = {
{MAKE_ARG("numfields",ARG_TYPE_INTEGER,0,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("data",ARG_TYPE_BLOCK,-1,NULL,NULL,NULL,CMD_ARG_MULTIPLE,2,NULL),.subargs=HSETEX_fields_data_Subargs},
};
/* HSETEX argument table */
struct COMMAND_ARG HSETEX_Args[] = {
{MAKE_ARG("key",ARG_TYPE_KEY,0,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("fields-condition",ARG_TYPE_ONEOF,-1,NULL,NULL,NULL,CMD_ARG_OPTIONAL,2,NULL),.subargs=HSETEX_fields_condition_Subargs},
{MAKE_ARG("expiration",ARG_TYPE_ONEOF,-1,NULL,NULL,NULL,CMD_ARG_OPTIONAL,5,NULL),.subargs=HSETEX_expiration_Subargs},
{MAKE_ARG("fields",ARG_TYPE_BLOCK,-1,"FIELDS",NULL,NULL,CMD_ARG_NONE,2,NULL),.subargs=HSETEX_fields_Subargs},
};
/********** HSETNX ********************/
#ifndef SKIP_CMD_HISTORY_TABLE
@ -3920,6 +4303,37 @@ struct COMMAND_ARG HSTRLEN_Args[] = {
{MAKE_ARG("field",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
};
/********** HTTL ********************/
#ifndef SKIP_CMD_HISTORY_TABLE
/* HTTL history */
#define HTTL_History NULL
#endif
#ifndef SKIP_CMD_TIPS_TABLE
/* HTTL tips */
#define HTTL_Tips NULL
#endif
#ifndef SKIP_CMD_KEY_SPECS_TABLE
/* HTTL key specs */
keySpec HTTL_Keyspecs[1] = {
{NULL,CMD_KEY_RO|CMD_KEY_ACCESS,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={0,1,0}}
};
#endif
/* HTTL fields argument table */
struct COMMAND_ARG HTTL_fields_Subargs[] = {
{MAKE_ARG("numfields",ARG_TYPE_INTEGER,0,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("field",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_MULTIPLE,0,NULL)},
};
/* HTTL argument table */
struct COMMAND_ARG HTTL_Args[] = {
{MAKE_ARG("key",ARG_TYPE_KEY,0,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("fields",ARG_TYPE_BLOCK,-1,"FIELDS",NULL,NULL,CMD_ARG_NONE,2,NULL),.subargs=HTTL_fields_Subargs},
};
/********** HVALS ********************/
#ifndef SKIP_CMD_HISTORY_TABLE
@ -11278,19 +11692,30 @@ struct COMMAND_STRUCT serverCommandTable[] = {
/* hash */
{MAKE_CMD("hdel","Deletes one or more fields and their values from a hash. Deletes the hash if no fields remain.","O(N) where N is the number of fields to be removed.","2.0.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HDEL_History,1,HDEL_Tips,0,hdelCommand,-3,CMD_WRITE|CMD_FAST,ACL_CATEGORY_HASH,HDEL_Keyspecs,1,NULL,2),.args=HDEL_Args},
{MAKE_CMD("hexists","Determines whether a field exists in a hash.","O(1)","2.0.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HEXISTS_History,0,HEXISTS_Tips,0,hexistsCommand,3,CMD_READONLY|CMD_FAST,ACL_CATEGORY_HASH,HEXISTS_Keyspecs,1,NULL,2),.args=HEXISTS_Args},
{MAKE_CMD("hexpire","Set expiry time on hash fields.","O(N) where N is the number of specified fields.","9.0.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HEXPIRE_History,0,HEXPIRE_Tips,0,hexpireCommand,-6,CMD_WRITE|CMD_FAST,ACL_CATEGORY_HASH,HEXPIRE_Keyspecs,1,NULL,4),.args=HEXPIRE_Args},
{MAKE_CMD("hexpireat","Set expiry time on hash fields.","O(N) where N is the number of specified fields.","9.0.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HEXPIREAT_History,0,HEXPIREAT_Tips,0,hexpireatCommand,-6,CMD_WRITE|CMD_FAST,ACL_CATEGORY_HASH,HEXPIREAT_Keyspecs,1,NULL,4),.args=HEXPIREAT_Args},
{MAKE_CMD("hexpiretime","Returns Unix timestamps in seconds since the epoch at which the given key's field(s) will expire","O(1) for each field, so O(N) for N items when the command is called with multiple fields.","9.0.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HEXPIRETIME_History,0,HEXPIRETIME_Tips,0,hexpiretimeCommand,-5,CMD_READONLY|CMD_FAST,ACL_CATEGORY_HASH,HEXPIRETIME_Keyspecs,1,NULL,2),.args=HEXPIRETIME_Args},
{MAKE_CMD("hget","Returns the value of a field in a hash.","O(1)","2.0.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HGET_History,0,HGET_Tips,0,hgetCommand,3,CMD_READONLY|CMD_FAST,ACL_CATEGORY_HASH,HGET_Keyspecs,1,NULL,2),.args=HGET_Args},
{MAKE_CMD("hgetall","Returns all fields and values in a hash.","O(N) where N is the size of the hash.","2.0.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HGETALL_History,0,HGETALL_Tips,1,hgetallCommand,2,CMD_READONLY,ACL_CATEGORY_HASH,HGETALL_Keyspecs,1,NULL,1),.args=HGETALL_Args},
{MAKE_CMD("hgetex","Get the value of one or more fields of a given hash key, and optionally set their expiration time or time-to-live (TTL).","O(1)","9.0.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HGETEX_History,0,HGETEX_Tips,0,hgetexCommand,-5,CMD_WRITE|CMD_FAST,ACL_CATEGORY_HASH,HGETEX_Keyspecs,1,NULL,3),.args=HGETEX_Args},
{MAKE_CMD("hincrby","Increments the integer value of a field in a hash by a number. Uses 0 as initial value if the field doesn't exist.","O(1)","2.0.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HINCRBY_History,0,HINCRBY_Tips,0,hincrbyCommand,4,CMD_WRITE|CMD_DENYOOM|CMD_FAST,ACL_CATEGORY_HASH,HINCRBY_Keyspecs,1,NULL,3),.args=HINCRBY_Args},
{MAKE_CMD("hincrbyfloat","Increments the floating point value of a field by a number. Uses 0 as initial value if the field doesn't exist.","O(1)","2.6.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HINCRBYFLOAT_History,0,HINCRBYFLOAT_Tips,0,hincrbyfloatCommand,4,CMD_WRITE|CMD_DENYOOM|CMD_FAST,ACL_CATEGORY_HASH,HINCRBYFLOAT_Keyspecs,1,NULL,3),.args=HINCRBYFLOAT_Args},
{MAKE_CMD("hkeys","Returns all fields in a hash.","O(N) where N is the size of the hash.","2.0.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HKEYS_History,0,HKEYS_Tips,1,hkeysCommand,2,CMD_READONLY,ACL_CATEGORY_HASH,HKEYS_Keyspecs,1,NULL,1),.args=HKEYS_Args},
{MAKE_CMD("hlen","Returns the number of fields in a hash.","O(1)","2.0.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HLEN_History,0,HLEN_Tips,0,hlenCommand,2,CMD_READONLY|CMD_FAST,ACL_CATEGORY_HASH,HLEN_Keyspecs,1,NULL,1),.args=HLEN_Args},
{MAKE_CMD("hmget","Returns the values of all fields in a hash.","O(N) where N is the number of fields being requested.","2.0.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HMGET_History,0,HMGET_Tips,0,hmgetCommand,-3,CMD_READONLY|CMD_FAST,ACL_CATEGORY_HASH,HMGET_Keyspecs,1,NULL,2),.args=HMGET_Args},
{MAKE_CMD("hmset","Sets the values of multiple fields.","O(N) where N is the number of fields being set.","2.0.0",CMD_DOC_DEPRECATED,"`HSET` with multiple field-value pairs","4.0.0","hash",COMMAND_GROUP_HASH,HMSET_History,0,HMSET_Tips,0,hsetCommand,-4,CMD_WRITE|CMD_DENYOOM|CMD_FAST,ACL_CATEGORY_HASH,HMSET_Keyspecs,1,NULL,2),.args=HMSET_Args},
{MAKE_CMD("hpersist","Remove the existing expiration on a hash key's field(s).","O(1) for each field assigned with TTL, so O(N) to persist N items when the command is called with multiple fields.","9.0.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HPERSIST_History,0,HPERSIST_Tips,0,hpersistCommand,-5,CMD_WRITE|CMD_FAST,ACL_CATEGORY_HASH,HPERSIST_Keyspecs,1,NULL,2),.args=HPERSIST_Args},
{MAKE_CMD("hpexpire","Set expiry time on hash object.","O(N) where N is the number of specified fields.","9.0.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HPEXPIRE_History,0,HPEXPIRE_Tips,0,hpexpireCommand,-6,CMD_WRITE|CMD_FAST,ACL_CATEGORY_HASH,HPEXPIRE_Keyspecs,1,NULL,4),.args=HPEXPIRE_Args},
{MAKE_CMD("hpexpireat","Set expiration time on hash field.","O(N) where N is the number of specified fields.","9.0.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HPEXPIREAT_History,0,HPEXPIREAT_Tips,0,hpexpireatCommand,-6,CMD_WRITE|CMD_FAST,ACL_CATEGORY_HASH,HPEXPIREAT_Keyspecs,1,NULL,4),.args=HPEXPIREAT_Args},
{MAKE_CMD("hpexpiretime","Returns the Unix timestamp in milliseconds since Unix epoch at which the given key's field(s) will expire","O(1) for each field, so O(N) for N items when the command is called with multiple fields.","9.0.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HPEXPIRETIME_History,0,HPEXPIRETIME_Tips,0,hpexpiretimeCommand,-5,CMD_READONLY|CMD_FAST,ACL_CATEGORY_HASH,HPEXPIRETIME_Keyspecs,1,NULL,2),.args=HPEXPIRETIME_Args},
{MAKE_CMD("hpttl","Returns the remaining time to live in milliseconds of a hash key's field(s) that have an associated expiration.","O(1) for each field assigned with TTL, so O(N) for N items when the command is called with multiple fields.","9.0.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HPTTL_History,0,HPTTL_Tips,0,hpttlCommand,-5,CMD_READONLY|CMD_FAST,ACL_CATEGORY_HASH,HPTTL_Keyspecs,1,NULL,2),.args=HPTTL_Args},
{MAKE_CMD("hrandfield","Returns one or more random fields from a hash.","O(N) where N is the number of fields returned","6.2.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HRANDFIELD_History,0,HRANDFIELD_Tips,1,hrandfieldCommand,-2,CMD_READONLY,ACL_CATEGORY_HASH,HRANDFIELD_Keyspecs,1,NULL,2),.args=HRANDFIELD_Args},
{MAKE_CMD("hscan","Iterates over fields and values of a hash.","O(1) for every call. O(N) for a complete iteration, including enough command calls for the cursor to return back to 0. N is the number of elements inside the collection.","2.8.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HSCAN_History,0,HSCAN_Tips,1,hscanCommand,-3,CMD_READONLY,ACL_CATEGORY_HASH,HSCAN_Keyspecs,1,NULL,5),.args=HSCAN_Args},
{MAKE_CMD("hset","Creates or modifies the value of a field in a hash.","O(1) for each field/value pair added, so O(N) to add N field/value pairs when the command is called with multiple field/value pairs.","2.0.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HSET_History,1,HSET_Tips,0,hsetCommand,-4,CMD_WRITE|CMD_DENYOOM|CMD_FAST,ACL_CATEGORY_HASH,HSET_Keyspecs,1,NULL,2),.args=HSET_Args},
{MAKE_CMD("hsetex","Set the value of one or more fields of a given hash key, and optionally set their expiration time.","O(1)","9.0.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HSETEX_History,0,HSETEX_Tips,0,hsetexCommand,-6,CMD_WRITE|CMD_DENYOOM|CMD_FAST,ACL_CATEGORY_HASH,HSETEX_Keyspecs,1,NULL,4),.args=HSETEX_Args},
{MAKE_CMD("hsetnx","Sets the value of a field in a hash only when the field doesn't exist.","O(1)","2.0.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HSETNX_History,0,HSETNX_Tips,0,hsetnxCommand,4,CMD_WRITE|CMD_DENYOOM|CMD_FAST,ACL_CATEGORY_HASH,HSETNX_Keyspecs,1,NULL,3),.args=HSETNX_Args},
{MAKE_CMD("hstrlen","Returns the length of the value of a field.","O(1)","3.2.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HSTRLEN_History,0,HSTRLEN_Tips,0,hstrlenCommand,3,CMD_READONLY|CMD_FAST,ACL_CATEGORY_HASH,HSTRLEN_Keyspecs,1,NULL,2),.args=HSTRLEN_Args},
{MAKE_CMD("httl","Returns the remaining time to live (in seconds) of a hash key's field(s) that have an associated expiration.","O(1) for each field, so O(N) for N items when the command is called with multiple fields.","9.0.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HTTL_History,0,HTTL_Tips,0,httlCommand,-5,CMD_READONLY|CMD_FAST,ACL_CATEGORY_HASH,HTTL_Keyspecs,1,NULL,2),.args=HTTL_Args},
{MAKE_CMD("hvals","Returns all values in a hash.","O(N) where N is the size of the hash.","2.0.0",CMD_DOC_NONE,NULL,NULL,"hash",COMMAND_GROUP_HASH,HVALS_History,0,HVALS_Tips,1,hvalsCommand,2,CMD_READONLY,ACL_CATEGORY_HASH,HVALS_Keyspecs,1,NULL,1),.args=HVALS_Args},
/* hyperloglog */
{MAKE_CMD("pfadd","Adds elements to a HyperLogLog key. Creates the key if it doesn't exist.","O(1) to add every element.","2.8.9",CMD_DOC_NONE,NULL,NULL,"hyperloglog",COMMAND_GROUP_HYPERLOGLOG,PFADD_History,0,PFADD_Tips,0,pfaddCommand,-2,CMD_WRITE|CMD_DENYOOM|CMD_FAST,ACL_CATEGORY_HYPERLOGLOG,PFADD_Keyspecs,1,NULL,2),.args=PFADD_Args},

118
src/commands/hexpire.json Normal file
View File

@ -0,0 +1,118 @@
{
"HEXPIRE": {
"summary": "Set expiry time on hash fields.",
"complexity": "O(N) where N is the number of specified fields.",
"group": "hash",
"since": "9.0.0",
"arity": -6,
"function": "hexpireCommand",
"command_flags": [
"WRITE",
"FAST"
],
"acl_categories": [
"HASH"
],
"key_specs": [
{
"flags": [
"RW",
"UPDATE"
],
"begin_search": {
"index": {
"pos": 1
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"reply_schema": {
"description": "List of integer codes indicating the result of setting expiry on each specified field, in the same order as the fields are requested.",
"type": "array",
"minItems": 1,
"items": {
"oneOf": [
{
"description": "Field does not exist in the HASH, or key does not exist.",
"const": -2
},
{
"description": "The specified NX | XX | GT | LT condition has not been met.",
"const": 0
},
{
"description": "The expiration time was applied.",
"const": 1
},
{
"description": "When called with a 0 second",
"const": 2
}
]
}
},
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "seconds",
"type": "integer"
},
{
"name": "condition",
"type": "oneof",
"optional": true,
"arguments": [
{
"name": "nx",
"type": "pure-token",
"token": "NX"
},
{
"name": "xx",
"type": "pure-token",
"token": "XX"
},
{
"name": "gt",
"type": "pure-token",
"token": "GT"
},
{
"name": "lt",
"type": "pure-token",
"token": "LT"
}
]
},
{
"name": "fields",
"token": "FIELDS",
"type": "block",
"arguments": [
{
"name": "numfields",
"type": "integer",
"multiple": false,
"minimum": 1
},
{
"name": "field",
"type": "string",
"multiple": true
}
]
}
]
}
}

120
src/commands/hexpireat.json Normal file
View File

@ -0,0 +1,120 @@
{
"HEXPIREAT": {
"summary": "Set expiry time on hash fields.",
"complexity": "O(N) where N is the number of specified fields.",
"group": "hash",
"since": "9.0.0",
"arity": -6,
"function": "hexpireatCommand",
"command_flags": [
"WRITE",
"FAST"
],
"acl_categories": [
"HASH"
],
"key_specs": [
{
"flags": [
"RW",
"UPDATE"
],
"begin_search": {
"index": {
"pos": 1
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"reply_schema": {
"description": "List of integer codes indicating the result of setting expiry on each specified field, in the same order as the fields are requested.",
"type": "array",
"minItems": 1,
"items": {
"oneOf": [
{
"description": "Field does not exist in the HASH, or HASH is empty.",
"const": -2
},
{
"description": "The specified NX | XX | GT | LT condition has not been met.",
"const": 0
},
{
"description": "The expiration time was applied.",
"const": 1
},
{
"description": "When called with a 0 second or is called with a past Unix time in seconds.",
"const": 2
}
]
}
},
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "unix-time-seconds",
"type": "integer"
},
{
"name": "condition",
"type": "oneof",
"optional": true,
"since": "9.0.0",
"arguments": [
{
"name": "nx",
"type": "pure-token",
"token": "NX"
},
{
"name": "xx",
"type": "pure-token",
"token": "XX"
},
{
"name": "gt",
"type": "pure-token",
"token": "GT"
},
{
"name": "lt",
"type": "pure-token",
"token": "LT"
}
]
},
{
"name": "fields",
"token": "FIELDS",
"type": "block",
"arguments": [
{
"name": "numfields",
"type": "integer",
"key_spec_index": 0,
"multiple": false,
"minimum": 1
},
{
"name": "field",
"type": "string",
"multiple": true
}
]
}
]
}
}

85
src/commands/hexpiretime.json Normal file
View File

@ -0,0 +1,85 @@
{
"HEXPIRETIME": {
"summary": "Returns Unix timestamps in seconds since the epoch at which the given key's field(s) will expire",
"complexity": "O(1) for each field, so O(N) for N items when the command is called with multiple fields.",
"group": "hash",
"since": "9.0.0",
"arity": -5,
"function": "hexpiretimeCommand",
"command_flags": [
"READONLY",
"FAST"
],
"acl_categories": [
"HASH"
],
"key_specs": [
{
"flags": [
"RO",
"ACCESS"
],
"begin_search": {
"index": {
"pos": 1
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"reply_schema": {
"description": "List of values associated with the result of getting the absolute expiry timestamp of the specific fields, in the same order as they are requested.",
"type": "array",
"minItems": 1,
"items": {
"oneOf": [
{
"description": "Field does not exist in the provided hash key, or the hash key is empty.",
"const": -2
},
{
"description": "Field exists in the provided hash key, but has no expiration associated with it.",
"const": -1
},
{
"description": "The expiration time associated with the hash key field, in seconds.",
"type": "integer",
"minimum": 0
}
]
}
},
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "fields",
"token": "FIELDS",
"type": "block",
"arguments": [
{
"name": "numfields",
"type": "integer",
"key_spec_index": 0,
"multiple": false,
"minimum": 1
},
{
"name": "field",
"type": "string",
"multiple": true
}
]
}
]
}
}

118
src/commands/hgetex.json Normal file
View File

@ -0,0 +1,118 @@
{
"HGETEX": {
"summary": "Get the value of one or more fields of a given hash key, and optionally set their expiration time or time-to-live (TTL).",
"complexity": "O(1)",
"group": "hash",
"since": "9.0.0",
"arity": -5,
"function": "hgetexCommand",
"command_flags": [
"WRITE",
"FAST"
],
"acl_categories": [
"HASH"
],
"key_specs": [
{
"flags": [
"RW",
"ACCESS"
],
"begin_search": {
"index": {
"pos": 1
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"reply_schema": {
"oneOf": [
{
"description": "List of values associated with the given fields, in the same order as they are requested.",
"type": "array",
"minItems": 1,
"items": {
"oneOf": [
{
"type": "string"
},
{
"type": "null"
}
]
}
},
{
"description": "Key does not exist.",
"type": "null"
}
]
},
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "expiration",
"type": "oneof",
"optional": true,
"arguments": [
{
"name": "seconds",
"type": "integer",
"token": "EX"
},
{
"name": "milliseconds",
"type": "integer",
"token": "PX"
},
{
"name": "unix-time-seconds",
"type": "unix-time",
"token": "EXAT"
},
{
"name": "unix-time-milliseconds",
"type": "unix-time",
"token": "PXAT"
},
{
"name": "persist",
"type": "pure-token",
"token": "PERSIST"
}
]
},
{
"name": "fields",
"token": "FIELDS",
"type": "block",
"arguments": [
{
"name": "numfields",
"type": "integer",
"key_spec_index": 0,
"multiple": false,
"minimum": 1
},
{
"name": "field",
"type": "string",
"multiple": true
}
]
}
]
}
}

84
src/commands/hpersist.json Normal file
View File

@ -0,0 +1,84 @@
{
"HPERSIST": {
"summary": "Remove the existing expiration on a hash key's field(s).",
"complexity": "O(1) for each field assigned with TTL, so O(N) to persist N items when the command is called with multiple fields.",
"group": "hash",
"since": "9.0.0",
"arity": -5,
"function": "hpersistCommand",
"command_flags": [
"WRITE",
"FAST"
],
"acl_categories": [
"HASH"
],
"key_specs": [
{
"flags": [
"RW",
"UPDATE"
],
"begin_search": {
"index": {
"pos": 1
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"reply_schema": {
"description": "List of integer codes indicating the result of setting expiry on each specified field, in the same order as the fields are requested.",
"type": "array",
"minItems": 1,
"items": {
"oneOf": [
{
"description": "Field does not exist in the provided hash key, or the hash key does not exist.",
"const": -2
},
{
"description": "Field exists in the provided hash key, but has no expiration associated with it.",
"const": -1
},
{
"description": "The expiration time was removed from the hash key field.",
"const": 1
}
]
}
},
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "fields",
"token": "FIELDS",
"type": "block",
"arguments": [
{
"name": "numfields",
"type": "integer",
"key_spec_index": 0,
"multiple": false,
"minimum": 1
},
{
"name": "field",
"type": "string",
"multiple": true
}
]
}
]
}
}

120
src/commands/hpexpire.json Normal file
View File

@ -0,0 +1,120 @@
{
"HPEXPIRE": {
"summary": "Set expiry time on hash object.",
"complexity": "O(N) where N is the number of specified fields.",
"group": "hash",
"since": "9.0.0",
"arity": -6,
"function": "hpexpireCommand",
"command_flags": [
"WRITE",
"FAST"
],
"acl_categories": [
"HASH"
],
"key_specs": [
{
"flags": [
"RW",
"UPDATE"
],
"begin_search": {
"index": {
"pos": 1
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"reply_schema": {
"description": "List of integer codes indicating the result of setting expiry on each specified field, in the same order as the fields are requested.",
"type": "array",
"minItems": 1,
"items": {
"oneOf": [
{
"description": "Field does not exist in the HASH, or HASH is empty.",
"const": -2
},
{
"description": "The specified NX | XX | GT | LT condition has not been met.",
"const": 0
},
{
"description": "The expiration time was applied.",
"const": 1
},
{
"description": "When called with a 0 millisecond",
"const": 2
}
]
}
},
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "milliseconds",
"type": "integer"
},
{
"name": "condition",
"type": "oneof",
"optional": true,
"since": "9.0.0",
"arguments": [
{
"name": "nx",
"type": "pure-token",
"token": "NX"
},
{
"name": "xx",
"type": "pure-token",
"token": "XX"
},
{
"name": "gt",
"type": "pure-token",
"token": "GT"
},
{
"name": "lt",
"type": "pure-token",
"token": "LT"
}
]
},
{
"name": "fields",
"token": "FIELDS",
"type": "block",
"arguments": [
{
"name": "numfields",
"type": "integer",
"key_spec_index": 0,
"multiple": false,
"minimum": 1
},
{
"name": "field",
"type": "string",
"multiple": true
}
]
}
]
}
}

120
src/commands/hpexpireat.json Normal file
View File

@ -0,0 +1,120 @@
{
"HPEXPIREAT": {
"summary": "Set expiration time on hash field.",
"complexity": "O(N) where N is the number of specified fields.",
"group": "hash",
"since": "9.0.0",
"arity": -6,
"function": "hpexpireatCommand",
"command_flags": [
"WRITE",
"FAST"
],
"acl_categories": [
"HASH"
],
"key_specs": [
{
"flags": [
"RW",
"UPDATE"
],
"begin_search": {
"index": {
"pos": 1
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"reply_schema": {
"description": "List of integer codes indicating the result of setting expiry on each specified field, in the same order as the fields are requested.",
"type": "array",
"minItems": 1,
"items": {
"oneOf": [
{
"description": "Field does not exist in the HASH, or HASH is empty.",
"const": -2
},
{
"description": "The specified NX | XX | GT | LT condition has not been met.",
"const": 0
},
{
"description": "The expiration time was applied.",
"const": 1
},
{
"description": "When called with a 0 second or is called with a past Unix time in milliseconds.",
"const": 2
}
]
}
},
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "unix-time-milliseconds",
"type": "integer"
},
{
"name": "condition",
"type": "oneof",
"optional": true,
"since": "9.0.0",
"arguments": [
{
"name": "nx",
"type": "pure-token",
"token": "NX"
},
{
"name": "xx",
"type": "pure-token",
"token": "XX"
},
{
"name": "gt",
"type": "pure-token",
"token": "GT"
},
{
"name": "lt",
"type": "pure-token",
"token": "LT"
}
]
},
{
"name": "fields",
"token": "FIELDS",
"type": "block",
"arguments": [
{
"name": "numfields",
"type": "integer",
"key_spec_index": 0,
"multiple": false,
"minimum": 1
},
{
"name": "field",
"type": "string",
"multiple": true
}
]
}
]
}
}

85
src/commands/hpexpiretime.json Normal file
View File

@ -0,0 +1,85 @@
{
"HPEXPIRETIME": {
"summary": "Returns the Unix timestamp in milliseconds since Unix epoch at which the given key's field(s) will expire",
"complexity": "O(1) for each field, so O(N) for N items when the command is called with multiple fields.",
"group": "hash",
"since": "9.0.0",
"arity": -5,
"function": "hpexpiretimeCommand",
"command_flags": [
"READONLY",
"FAST"
],
"acl_categories": [
"HASH"
],
"key_specs": [
{
"flags": [
"RO",
"ACCESS"
],
"begin_search": {
"index": {
"pos": 1
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"reply_schema": {
"description": "List of values associated with the result of getting the absolute expiry timestamp of the specific fields, in the same order as they are requested.",
"type": "array",
"minItems": 1,
"items": {
"oneOf": [
{
"description": "Field does not exist in the provided hash key, or the hash key is empty.",
"const": -2
},
{
"description": "Field exists in the provided hash key, but has no expiration associated with it.",
"const": -1
},
{
"description": "The expiration time associated with the hash key field, in milliseconds.",
"type": "integer",
"minimum": 0
}
]
}
},
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "fields",
"token": "FIELDS",
"type": "block",
"arguments": [
{
"name": "numfields",
"type": "integer",
"key_spec_index": 0,
"multiple": false,
"minimum": 1
},
{
"name": "field",
"type": "string",
"multiple": true
}
]
}
]
}
}

85
src/commands/hpttl.json Normal file
View File

@ -0,0 +1,85 @@
{
"HPTTL": {
"summary": "Returns the remaining time to live in milliseconds of a hash key's field(s) that have an associated expiration.",
"complexity": "O(1) for each field assigned with TTL, so O(N) for N items when the command is called with multiple fields.",
"group": "hash",
"since": "9.0.0",
"arity": -5,
"function": "hpttlCommand",
"command_flags": [
"READONLY",
"FAST"
],
"acl_categories": [
"HASH"
],
"key_specs": [
{
"flags": [
"RO",
"ACCESS"
],
"begin_search": {
"index": {
"pos": 1
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"reply_schema": {
"description": "List of values associated with the result of getting the remaining time-to-live of the specific fields, in the same order as they are requested.",
"type": "array",
"minItems": 1,
"items": {
"oneOf": [
{
"description": "Field does not exist in the provided hash key, or the hash key is empty",
"const": -2
},
{
"description": "Field exists in the provided hash key, but has no expiration associated with it.",
"const": -1
},
{
"description": "The expiration time associated with the hash key field, in milliseconds.",
"type": "integer",
"minimum": 0
}
]
}
},
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "fields",
"token": "FIELDS",
"type": "block",
"arguments": [
{
"name": "numfields",
"type": "integer",
"key_spec_index": 0,
"multiple": false,
"minimum": 1
},
{
"name": "field",
"type": "string",
"multiple": true
}
]
}
]
}
}

135
src/commands/hsetex.json Normal file
View File

@ -0,0 +1,135 @@
{
"HSETEX": {
"summary": "Set the value of one or more fields of a given hash key, and optionally set their expiration time.",
"complexity": "O(1)",
"group": "hash",
"since": "9.0.0",
"arity": -6,
"function": "hsetexCommand",
"command_flags": [
"WRITE",
"DENYOOM",
"FAST"
],
"acl_categories": [
"HASH"
],
"key_specs": [
{
"flags": [
"RW",
"INSERT"
],
"begin_search": {
"index": {
"pos": 1
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"reply_schema": {
"oneOf": [
{
"description": "None of the provided fields value and or expiration time was set.",
"const": 0
},
{
"description": "All the fields value and or expiration time was set.",
"const": 1
}
]
},
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "fields-condition",
"type": "oneof",
"optional": true,
"arguments": [
{
"name": "fnx",
"type": "pure-token",
"token": "FNX"
},
{
"name": "fxx",
"type": "pure-token",
"token": "FXX"
}
]
},
{
"name": "expiration",
"type": "oneof",
"optional": true,
"arguments": [
{
"name": "seconds",
"type": "integer",
"token": "EX"
},
{
"name": "milliseconds",
"type": "integer",
"token": "PX"
},
{
"name": "unix-time-seconds",
"type": "unix-time",
"token": "EXAT"
},
{
"name": "unix-time-milliseconds",
"type": "unix-time",
"token": "PXAT"
},
{
"name": "keepttl",
"type": "pure-token",
"token": "KEEPTTL"
}
]
},
{
"name": "fields",
"token": "FIELDS",
"type": "block",
"arguments": [
{
"name": "numfields",
"type": "integer",
"key_spec_index": 0,
"multiple": false,
"minimum": 1
},
{
"name": "data",
"type": "block",
"multiple": true,
"arguments": [
{
"name": "field",
"type": "string"
},
{
"name": "value",
"type": "string"
}
]
}
]
}
]
}
}

85
src/commands/httl.json Normal file
View File

@ -0,0 +1,85 @@
{
"HTTL": {
"summary": "Returns the remaining time to live (in seconds) of a hash key's field(s) that have an associated expiration.",
"complexity": "O(1) for each field, so O(N) for N items when the command is called with multiple fields.",
"group": "hash",
"since": "9.0.0",
"arity": -5,
"function": "httlCommand",
"command_flags": [
"READONLY",
"FAST"
],
"acl_categories": [
"HASH"
],
"key_specs": [
{
"flags": [
"RO",
"ACCESS"
],
"begin_search": {
"index": {
"pos": 1
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"reply_schema": {
"description": "List of values associated with the result of getting the remaining time-to-live of the specific fields, in the same order as they are requested.",
"type": "array",
"minItems": 1,
"items": {
"oneOf": [
{
"description": "Field does not exist in the provided hash key, or the hash key is empty",
"const": -2
},
{
"description": "Field exists in the provided hash key, but has no expiration associated with it.",
"const": -1
},
{
"description": "The expiration time associated with the hash key field, in seconds.",
"type": "integer",
"minimum": 0
}
]
}
},
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "fields",
"token": "FIELDS",
"type": "block",
"arguments": [
{
"name": "numfields",
"type": "integer",
"key_spec_index": 0,
"multiple": false,
"minimum": 1
},
{
"name": "field",
"type": "string",
"multiple": true
}
]
}
]
}
}

81
src/db.c
View File

@ -35,6 +35,7 @@
#include "io_threads.h"
#include "module.h"
#include "vector.h"
#include "expire.h"
#include <signal.h>
#include <ctype.h>
@ -43,17 +44,6 @@
* C-level DB API
*----------------------------------------------------------------------------*/
/* Flags for expireIfNeeded */
#define EXPIRE_FORCE_DELETE_EXPIRED 1
#define EXPIRE_AVOID_DELETE_EXPIRED 2
/* Return values for expireIfNeeded */
typedef enum {
KEY_VALID = 0, /* Could be volatile and not yet expired, non-volatile, or even non-existing key. */
KEY_EXPIRED, /* Logically expired but not yet deleted. */
KEY_DELETED /* The key was deleted now. */
} keyStatus;
static keyStatus expireIfNeededWithDictIndex(serverDb *db, robj *key, robj *val, int flags, int dict_index);
static keyStatus expireIfNeeded(serverDb *db, robj *key, robj *val, int flags);
static int keyIsExpiredWithDictIndex(serverDb *db, robj *key, int dict_index);
@ -125,7 +115,7 @@ robj *lookupKey(serverDb *db, robj *key, int flags) {
/* Update the access time for the ageing algorithm.
* Don't do it if we have a saving child, as this will trigger
* a copy on write madness. */
if (server.current_client && server.current_client->flag.no_touch &&
if (server.current_client && server.current_client->flag.no_touch && server.executing_client &&
server.executing_client->cmd->proc != touchCommand)
flags |= LOOKUP_NOTOUCH;
if (!hasActiveChildProcess() && !(flags & LOOKUP_NOTOUCH)) {
@ -1004,9 +994,9 @@ void hashtableScanCallback(void *privdata, void *entry) {
key = node->ele;
/* zset data is copied after filtering by key */
} else if (o->type == OBJ_HASH) {
key = hashTypeEntryGetField(entry);
key = entryGetField(entry);
if (!data->only_keys) {
val = hashTypeEntryGetValue(entry);
val = entryGetValue(entry);
}
} else {
serverPanic("Type not handled in hashtable SCAN callback.");
@ -1900,16 +1890,6 @@ void propagateDeletion(serverDb *db, robj *key, int lazy) {
server.replication_allowed = prev_replication_allowed;
}
/* Returns 1 if the expire value is expired, 0 otherwise. */
static int timestampIsExpired(mstime_t when) {
if (when < 0) return 0; /* no expire */
mstime_t now = commandTimeSnapshot();
/* The key expired if the current (virtual or real) time is greater
* than the expire time of the key. */
return now > when;
}
/* Use this instead of keyIsExpired if you already have the value object. */
static int objectIsExpired(robj *val) {
/* Don't expire anything while loading. It will be done later. */
@ -1925,7 +1905,7 @@ static int keyIsExpiredWithDictIndexImpl(serverDb *db, robj *key, int dict_index
/* Don't expire anything while loading. It will be done later. */
if (server.loading) return 0;
mstime_t when = getExpireWithDictIndex(db, key, dict_index);
return timestampIsExpired(when);
return timestampIsExpired(when) ? 1 : 0;
}
/* Check if the key is expired. */
@ -1953,52 +1933,11 @@ static keyStatus expireIfNeededWithDictIndex(serverDb *db, robj *key, robj *val,
} else {
if (!keyIsExpiredWithDictIndexImpl(db, key, dict_index)) return KEY_VALID;
}
/* If we are running in the context of a replica, instead of
* evicting the expired key from the database, we return ASAP:
* the replica key expiration is controlled by the primary that will
* send us synthesized DEL operations for expired keys. The
* exception is when write operations are performed on writable
* replicas.
*
* Still we try to return the right information to the caller,
* that is, KEY_VALID if we think the key should still be valid,
* KEY_EXPIRED if we think the key is expired but don't want to delete it at this time.
*
* When replicating commands from the primary, keys are never considered
* expired. */
if (server.primary_host != NULL) {
if (server.current_client && (server.current_client->flag.primary)) return KEY_VALID;
if (!(flags & EXPIRE_FORCE_DELETE_EXPIRED)) return KEY_EXPIRED;
} else if (server.import_mode) {
/* If we are running in the import mode on a primary, instead of
* evicting the expired key from the database, we return ASAP:
* the key expiration is controlled by the import source that will
* send us synthesized DEL operations for expired keys. The
* exception is when write operations are performed on this server
* because it's a primary.
*
* Notice: other clients, apart from the import source, should not access
* the data imported by import source.
*
* Still we try to return the right information to the caller,
* that is, KEY_VALID if we think the key should still be valid,
* KEY_EXPIRED if we think the key is expired but don't want to delete it at this time.
*
* When receiving commands from the import source, keys are never considered
* expired. */
if (server.current_client && (server.current_client->flag.import_source)) return KEY_VALID;
if (!(flags & EXPIRE_FORCE_DELETE_EXPIRED)) return KEY_EXPIRED;
}
/* In some cases we're explicitly instructed to return an indication of a
* missing key without actually deleting it, even on primaries. */
if (flags & EXPIRE_AVOID_DELETE_EXPIRED) return KEY_EXPIRED;
/* If 'expire' action is paused, for whatever reason, then don't expire any key.
* Typically, at the end of the pause we will properly expire the key OR we
* will have failed over and the new primary will send us the expire. */
if (isPausedActionsWithUpdate(PAUSE_ACTION_EXPIRE)) return KEY_EXPIRED;
expirationPolicy policy = getExpirationPolicyWithFlags(flags);
if (policy == POLICY_IGNORE_EXPIRE) /* Ignore keys expiration. treat all keys as valid. */
return KEY_VALID;
else if (policy == POLICY_KEEP_EXPIRED) /* Treat expired keys as invalid, but do not delete them. */
return KEY_EXPIRED;
/* The key needs to be converted from static to heap before deleted */
int static_key = key->refcount == OBJ_STATIC_REFCOUNT;

24
src/defrag.c
View File

@ -39,6 +39,7 @@
*/
#include "server.h"
#include "entry.h"
#include "hashtable.h"
#include "eval.h"
#include "script.h"
@ -442,18 +443,27 @@ static void scanLaterSet(robj *ob, unsigned long *cursor) {
}
/* Hashtable scan callback for hash datatype */
static void activeDefragHashTypeEntry(void *privdata, void *element_ref) {
UNUSED(privdata);
hashTypeEntry **entry_ref = (hashTypeEntry **)element_ref;
static void activeDefragEntry(void *privdata, void *element_ref) {
entry **entry_ref = (entry **)element_ref;
entry *old_entry = *entry_ref, *new_entry = NULL;
long long old_expiry = entryGetExpiry(old_entry);
hashTypeEntry *new_entry = hashTypeEntryDefrag(*entry_ref, activeDefragAlloc, activeDefragSds);
if (new_entry) *entry_ref = new_entry;
new_entry = entryDefrag(*entry_ref, activeDefragAlloc, activeDefragSds);
if (new_entry) {
/* In case the entry is tracked we need to update it in the volatile set */
if (entryHasExpiry(new_entry)) {
robj *obj = (robj *)privdata;
serverAssert(obj);
hashTypeTrackUpdateEntry(obj, old_entry, new_entry, old_expiry, entryGetExpiry(new_entry));
}
*entry_ref = new_entry;
}
}
static void scanLaterHash(robj *ob, unsigned long *cursor) {
serverAssert(ob->type == OBJ_HASH && ob->encoding == OBJ_ENCODING_HASHTABLE);
hashtable *ht = ob->ptr;
*cursor = hashtableScanDefrag(ht, *cursor, activeDefragHashTypeEntry, NULL, activeDefragAlloc, HASHTABLE_SCAN_EMIT_REF);
*cursor = hashtableScanDefrag(ht, *cursor, activeDefragEntry, ob, activeDefragAlloc, HASHTABLE_SCAN_EMIT_REF);
}
static void defragQuicklist(robj *ob) {
@ -498,7 +508,7 @@ static void defragHash(robj *ob) {
} else {
unsigned long cursor = 0;
do {
cursor = hashtableScanDefrag(ht, cursor, activeDefragHashTypeEntry, NULL, activeDefragAlloc, HASHTABLE_SCAN_EMIT_REF);
cursor = hashtableScanDefrag(ht, cursor, activeDefragEntry, ob, activeDefragAlloc, HASHTABLE_SCAN_EMIT_REF);
} while (cursor != 0);
}
/* defrag the hashtable struct and tables */

410
src/entry.c Normal file
View File

@ -0,0 +1,410 @@
#include <stdbool.h>
#include "server.h"
#include "serverassert.h"
#include "entry.h"
#include <stdbool.h>
/*-----------------------------------------------------------------------------
* Entry API
*----------------------------------------------------------------------------*/
/* The entry pointer is the field sds. We encode the entry layout type
* in the field SDS header. Field type SDS_TYPE_5 doesn't have any spare bits to
* encode this so we use it only for the first layout type.
*
* Entry with embedded value, used for small sizes. The value is stored as
* SDS_TYPE_8. The field can use any SDS type.
*
* Entry can also have expiration timestamp, which is the UNIX timestamp for it to be expired.
* For aligned fast access, we keep the expiry timestamp prior to the start of the sds header.
*
* +--------------+--------------+---------------+
* | Expiration | field | value |
* | 1234567890LL | hdr "foo" \0 | hdr8 "bar" \0 |
* +--------------+--------------+---------------+
*
* Entry with value pointer, used for larger fields and values. The field is SDS
* type 8 or higher.
*
* +--------------+-------+--------------+
* | Expiration | value | field |
* | 1234567890LL | ptr | hdr "foo" \0 |
* +--------------+---^---+--------------+
* |
* |
* value pointer = value sds
*/
enum {
/* SDS aux flag. If set, it indicates that the entry has TTL metadata set. */
FIELD_SDS_AUX_BIT_ENTRY_HAS_EXPIRY = 0,
/* SDS aux flag. If set, it indicates that the entry has an embedded value
* pointer located in memory before the embedded field. If unset, the entry
* instead has an embedded value located after the embedded field. */
FIELD_SDS_AUX_BIT_ENTRY_HAS_VALUE_PTR = 1,
FIELD_SDS_AUX_BIT_MAX
};
static_assert(FIELD_SDS_AUX_BIT_MAX < sizeof(char) - SDS_TYPE_BITS, "too many sds bits are used for entry metadata");
/* Returns true in case the entry's value is not embedded in the entry.
* Returns false otherwise. */
static inline bool entryHasValuePtr(const entry *entry) {
return sdsGetAuxBit(entry, FIELD_SDS_AUX_BIT_ENTRY_HAS_VALUE_PTR);
}
/* Returns true in case the entry's value is embedded in the entry.
* Returns false otherwise. */
bool entryHasEmbeddedValue(entry *entry) {
return (!entryHasValuePtr(entry));
}
/* Returns true in case the entry has expiration timestamp.
* Returns false otherwise. */
bool entryHasExpiry(const entry *entry) {
return sdsGetAuxBit(entry, FIELD_SDS_AUX_BIT_ENTRY_HAS_EXPIRY);
}
/* The entry pointer is the field sds, but that's an implementation detail. */
sds entryGetField(const entry *entry) {
return (sds)entry;
}
/* Returns the location of a pointer to a separately allocated value. Only for
* an entry without an embedded value. */
static sds *entryGetValueRef(const entry *entry) {
serverAssert(entryHasValuePtr(entry));
char *field_data = sdsAllocPtr(entry);
field_data -= sizeof(sds);
return (sds *)field_data;
}
/* Returns the sds of the entry's value. */
sds entryGetValue(const entry *entry) {
if (entryHasValuePtr(entry)) {
return *entryGetValueRef(entry);
} else {
/* Skip field content, field null terminator and value sds8 hdr. */
size_t offset = sdslen(entry) + 1 + sdsHdrSize(SDS_TYPE_8);
return (char *)entry + offset;
}
}
/* Modify the value of this entry and return a pointer to the (potentially new) entry.
* The value is taken by the function and cannot be reused after this function returns. */
entry *entrySetValue(entry *e, sds value) {
if (entryHasValuePtr(e)) {
sds *value_ref = entryGetValueRef(e);
sdsfree(*value_ref);
*value_ref = value;
return e;
} else {
entry *new_entry = entryUpdate(e, value, entryGetExpiry(e));
return new_entry;
}
}
/* Returns the address of the entry allocation. */
void *entryGetAllocPtr(const entry *entry) {
char *buf = sdsAllocPtr(entry);
if (entryHasValuePtr(entry)) buf -= sizeof(sds);
if (entryHasExpiry(entry)) buf -= sizeof(long long);
return buf;
}
/**************************************** Entry Expiry API *****************************************/
/* Returns the entry expiration timestamp.
* In case this entry has no expiration time, will return EXPIRE_NONE. */
long long entryGetExpiry(const entry *entry) {
long long expiry = EXPIRY_NONE;
if (entryHasExpiry(entry)) {
char *buf = entryGetAllocPtr(entry);
debugServerAssert((((uintptr_t)buf & 0x7) == 0)); /* Test that the allocation is indeed 8 bytes aligned
* This is needed since we access the expiry as with pointer casting
* which require the access to be 8 bytes aligned. */
expiry = *(long long *)buf;
}
return expiry;
}
/* Modify the expiration time of this entry and return a pointer to the (potentially new) entry. */
entry *entrySetExpiry(entry *e, long long expiry) {
if (entryHasExpiry(e)) {
char *buf = entryGetAllocPtr(e);
debugServerAssert((((uintptr_t)buf & 0x7) == 0)); /* Test that the allocation is indeed 8 bytes aligned
* This is needed since we access the expiry as with pointer casting
* which require the access to be 8 bytes aligned. */
*(long long *)buf = expiry;
return e;
}
entry *new_entry = entryUpdate(e, NULL, expiry);
return new_entry;
}
/* Return true in case the entry has assigned expiration or false otherwise. */
bool entryIsExpired(entry *entry) {
return timestampIsExpired(entryGetExpiry(entry));
}
/**************************************** Entry Expiry API - End *****************************************/
void entryFree(entry *entry) {
if (entryHasValuePtr(entry)) {
sdsfree(entryGetValue(entry));
}
zfree(entryGetAllocPtr(entry));
}
static inline size_t entryReqSize(const_sds field,
sds value,
long long expiry,
bool *is_value_embedded,
int *field_sds_type,
size_t *field_size,
size_t *expiry_size,
size_t *embedded_value_size) {
size_t expiry_alloc_size = (expiry == EXPIRY_NONE) ? 0 : sizeof(long long);
size_t field_len = sdslen(field);
int embedded_field_sds_type = sdsReqType(field_len);
if (embedded_field_sds_type == SDS_TYPE_5 && (expiry_alloc_size > 0)) {
embedded_field_sds_type = SDS_TYPE_8;
}
size_t field_alloc_size = sdsReqSize(field_len, embedded_field_sds_type);
size_t value_len = value ? sdslen(value) : 0;
size_t embedded_value_alloc_size = value ? sdsReqSize(value_len, SDS_TYPE_8) : 0;
size_t alloc_size = field_alloc_size + expiry_alloc_size;
bool embed_value = false;
if (value) {
if (alloc_size + embedded_value_alloc_size <= EMBED_VALUE_MAX_ALLOC_SIZE) {
/* Embed field and value. Value is fixed to SDS_TYPE_8. Unused
* allocation space is recorded in the embedded value's SDS header.
*
* +------+--------------+---------------+
* | TTL | field | value |
* | | hdr "foo" \0 | hdr8 "bar" \0 |
* +------+--------------+---------------+
*/
embed_value = true;
alloc_size += embedded_value_alloc_size;
} else {
/* Embed field, but not value. Field must be >= SDS_TYPE_8 to encode to
* indicate this type of entry.
*
* +------+-------+---------------+
* | TTL | value | field |
* | | ptr | hdr8 "foo" \0 |
* +------+-------+---------------+
*/
embed_value = false;
alloc_size += sizeof(sds);
if (embedded_field_sds_type == SDS_TYPE_5) {
embedded_field_sds_type = SDS_TYPE_8;
alloc_size -= field_alloc_size;
field_alloc_size = sdsReqSize(field_len, embedded_field_sds_type);
alloc_size += field_alloc_size;
}
}
}
if (expiry_size) *expiry_size = expiry_alloc_size;
if (field_sds_type) *field_sds_type = embedded_field_sds_type;
if (field_size) *field_size = field_alloc_size;
if (embedded_value_size) *embedded_value_size = embedded_value_alloc_size;
if (is_value_embedded) *is_value_embedded = embed_value;
return alloc_size;
}
/* Serialize the content of the entry into the provided buffer buf. Make use of the provided arguments provided by a call to entryReqSize.
* Note that this function will take ownership of the value so user should not assume it is valid after this call. */
static entry *entryWrite(char *buf,
size_t buf_size,
const_sds field,
sds value,
long long expiry,
bool embed_value,
int embedded_field_sds_type,
size_t embedded_field_sds_size,
size_t embedded_value_sds_size,
size_t expiry_size) {
/* Set The expiry if exists */
if (expiry_size) {
*(long long *)buf = expiry;
buf += expiry_size;
buf_size -= expiry_size;
}
if (value) {
if (!embed_value) {
*(sds *)buf = value;
buf += sizeof(sds);
buf_size -= sizeof(sds);
} else {
sdswrite(buf + embedded_field_sds_size, buf_size - embedded_field_sds_size, SDS_TYPE_8, value, sdslen(value));
sdsfree(value);
buf_size -= embedded_value_sds_size;
}
}
/* Set the field data */
entry *new_entry = sdswrite(buf, embedded_field_sds_size, embedded_field_sds_type, field, sdslen(field));
/* Field sds aux bits are zero, which we use for this entry encoding. */
sdsSetAuxBit(new_entry, FIELD_SDS_AUX_BIT_ENTRY_HAS_VALUE_PTR, embed_value ? 0 : 1);
sdsSetAuxBit(new_entry, FIELD_SDS_AUX_BIT_ENTRY_HAS_EXPIRY, expiry_size > 0 ? 1 : 0);
/* Check that the new entry was built correctly */
debugServerAssert(sdsGetAuxBit(new_entry, FIELD_SDS_AUX_BIT_ENTRY_HAS_VALUE_PTR) == (embed_value ? 0 : 1));
debugServerAssert(sdsGetAuxBit(new_entry, FIELD_SDS_AUX_BIT_ENTRY_HAS_EXPIRY) == (expiry_size > 0 ? 1 : 0));
return new_entry;
}
/* Takes ownership of value. does not take ownership of field */
entry *entryCreate(const_sds field, sds value, long long expiry) {
bool embed_value = false;
int embedded_field_sds_type;
size_t expiry_size, embedded_value_sds_size, embedded_field_sds_size;
size_t alloc_size = entryReqSize(field, value, expiry, &embed_value, &embedded_field_sds_type, &embedded_field_sds_size, &expiry_size, &embedded_value_sds_size);
size_t buf_size;
/* allocate the buffer */
char *buf = zmalloc_usable(alloc_size, &buf_size);
return entryWrite(buf, buf_size, field, value, expiry, embed_value, embedded_field_sds_type, embedded_field_sds_size, embedded_value_sds_size, expiry_size);
}
/* Modify the entry's value and/or expiration time.
* In case the provided value is NULL, will use the existing value.
* Note that the value ownership is moved to this function and the caller should assume the
* value is no longer usable after calling this function. */
entry *entryUpdate(entry *e, sds value, long long expiry) {
sds field = (sds)e;
entry *new_entry = NULL;
bool update_value = value ? true : false;
long long curr_expiration_time = entryGetExpiry(e);
bool update_expiry = (expiry != curr_expiration_time) ? true : false;
/* Just a sanity check. If nothing changes, lets just return */
if (!update_value && !update_expiry)
return e;
if (!value) value = entryGetValue(e);
bool embed_value = false;
int embedded_field_sds_type;
size_t expiry_size, embedded_value_size, embedded_field_size;
size_t required_entry_size = entryReqSize(field, value, expiry, &embed_value, &embedded_field_sds_type, &embedded_field_size, &expiry_size, &embedded_value_size);
size_t current_embedded_allocation_size = entryHasValuePtr(e) ? 0 : entryMemUsage(e);
bool expiry_add_remove = update_expiry && (curr_expiration_time == EXPIRY_NONE || expiry == EXPIRY_NONE); // In case we are toggling expiration
bool value_change_encoding = update_value && (embed_value != entryHasEmbeddedValue(e)); // In case we change the way value is embedded or not
/* We will create a new entry in the following cases:
* 1. In the case were we add or remove expiration.
* 2. We change the way value is encoded
* 3. in the case were we are NOT migrating from an embedded entry to an embedded entry with ~the same size. */
bool create_new_entry = (expiry_add_remove) || (value_change_encoding) ||
(update_value && entryHasEmbeddedValue(e) &&
!(required_entry_size <= EMBED_VALUE_MAX_ALLOC_SIZE &&
required_entry_size <= current_embedded_allocation_size &&
required_entry_size >= current_embedded_allocation_size * 3 / 4));
if (!create_new_entry) {
/* In this case we are sure we do not have to allocate new entry, so expiry must already be set. */
if (update_expiry) {
serverAssert(entryHasExpiry(e));
char *buf = entryGetAllocPtr(e);
*(long long *)buf = expiry;
}
/* In this case we are sure we do not have to allocate new entry, so value must already be set or we have enough room to embed it. */
if (update_value) {
if (entryHasValuePtr(e)) {
sds *value_ref = entryGetValueRef(e);
sdsfree(*value_ref);
*value_ref = value;
} else {
/* Skip field content, field null terminator and value sds8 hdr. */
sds old_value = entryGetValue(e);
/* We are using the same entry memory in order to store a potentially new value.
* In such cases the old value alloc was adjusted to the real buffer size part it was embedded to.
* Since we can potentially write here a smaller value, which requires less allocation space, we would like to
* inherit the old value memory allocation size. */
size_t value_size = sdsHdrSize(SDS_TYPE_8) + sdsalloc(old_value) + 1;
sdswrite(sdsAllocPtr(old_value), value_size, SDS_TYPE_8, value, sdslen(value));
sdsfree(value);
}
}
new_entry = e;
} else {
if (!update_value) {
/* Check if the value can be reused. */
int value_was_embedded = !entryHasValuePtr(e);
/* In case the original entry value is embedded WE WILL HAVE TO DUPLICATE IT
* if not we have to duplicate it, remove it from the original entry since we are going to delete it.*/
if (value_was_embedded) {
value = sdsdup(value);
} else {
sds *value_ref = entryGetValueRef(e);
*value_ref = NULL;
}
}
/* allocate the buffer for a new entry */
size_t buf_size;
char *buf = zmalloc_usable(required_entry_size, &buf_size);
new_entry = entryWrite(buf, buf_size, entryGetField(e), value, expiry, embed_value, embedded_field_sds_type, embedded_field_size, embedded_value_size, expiry_size);
debugServerAssert(new_entry != e);
entryFree(e);
}
/* Check that the new entry was built correctly */
debugServerAssert(sdsGetAuxBit(new_entry, FIELD_SDS_AUX_BIT_ENTRY_HAS_VALUE_PTR) == (embed_value ? 0 : 1));
debugServerAssert(sdsGetAuxBit(new_entry, FIELD_SDS_AUX_BIT_ENTRY_HAS_EXPIRY) == (expiry_size > 0 ? 1 : 0));
serverAssert(new_entry);
return new_entry;
}
/* Returns memory usage of a entry, including all allocations owned by
* the entry. */
size_t entryMemUsage(entry *entry) {
size_t mem = 0;
if (entryHasValuePtr(entry)) {
/* In case the value is not embedded we might not be able to sum all the allocation sizes since the field
* header could be too small for holding the real allocation size. */
mem += zmalloc_usable_size(entryGetAllocPtr(entry));
} else {
mem += sdsReqSize(sdslen(entry), sdsType(entry));
if (entryHasExpiry(entry)) mem += sizeof(long long);
}
mem += sdsAllocSize(entryGetValue(entry));
return mem;
}
/* Defragments a hashtable entry (field-value pair) if needed, using the
* provided defrag functions. The defrag functions return NULL if the allocation
* was not moved, otherwise they return a pointer to the new memory location.
* A separate sds defrag function is needed because of the unique memory layout
* of sds strings.
* If the location of the entry changed we return the new location,
* otherwise we return NULL. */
entry *entryDefrag(entry *entry, void *(*defragfn)(void *), sds (*sdsdefragfn)(sds)) {
if (entryHasValuePtr(entry)) {
sds *value_ref = entryGetValueRef(entry);
sds new_value = sdsdefragfn(*value_ref);
if (new_value) *value_ref = new_value;
}
char *allocation = entryGetAllocPtr(entry);
char *new_allocation = defragfn(allocation);
if (new_allocation != NULL) {
/* Return the same offset into the new allocation as the entry's offset
* in the old allocation. */
return new_allocation + ((char *)entry - allocation);
}
return NULL;
}
/* Used for releasing memory to OS to avoid unnecessary CoW. Called when we've
* forked and memory won't be used again. See zmadvise_dontneed() */
void entryDismissMemory(entry *entry) {
/* Only dismiss values memory since the field size usually is small. */
if (entryHasValuePtr(entry)) {
dismissSds(*entryGetValueRef(entry));
}
}

94
src/entry.h Normal file
View File

@ -0,0 +1,94 @@
#ifndef _ENTRY_H_
#define _ENTRY_H_
#include "sds.h"
#include <stdbool.h>
/*-----------------------------------------------------------------------------
* Entry
*----------------------------------------------------------------------------*/
/*
* The entry pointer is the field `sds`. We encode the entry layout type
* in the SDS header.
*
* An entry represents a key–value pair with an optional expiration timestamp.
* The pointer of type `entry *` always points to the VALUE `sds`.
*
* Layout 1: Embedded Field and Value (Compact Form)
*
* +-------------------+-------------------+-------------------+
* | Expiration (opt) | Field (sds) | Value (sds) |
* | 8 bytes (int64_t) | "field" + header | "value" + header |
* +-------------------+-------------------+-------------------+
* ^
* |
* entry pointer
*
* - Both field and value are small and embedded.
* - The expiration is stored just before the first sds.
*
*
* Layout 2: Pointer-Based Value (Large Values)
*
* +-------------------+-------------------+------------------+
* | Expiration (opt) | Value pointer | Field (sds) |
* | 8 bytes (int64_t) | 8 bytes (void *) | "field" + header |
* +-------------------+-------------------+------------------+
* ^
* |
* entry pointer
*
* - The value is stored separately via a pointer.
* - Used for large value sizes. */
typedef void entry;
/* The maximum allocation size we want to use for entries with embedded
* values. */
#define EMBED_VALUE_MAX_ALLOC_SIZE 128
/* Returns the field string (sds) from the entry. */
sds entryGetField(const entry *entry);
/* Returns the value string (sds) from the entry. */
sds entryGetValue(const entry *entry);
/* Sets or replaces the value string in the entry. May reallocate and return a new pointer. */
entry *entrySetValue(entry *entry, sds value);
/* Gets the expiration timestamp (UNIX time in milliseconds). */
long long entryGetExpiry(const entry *entry);
/* Returns true if the entry has an expiration timestamp set. */
bool entryHasExpiry(const entry *entry);
/* Sets the expiration timestamp. */
entry *entrySetExpiry(entry *entry, long long expiry);
/* Returns true if the entry is expired compared to current system time (commandTimeSnapshot). */
bool entryIsExpired(entry *entry);
/* Frees the memory used by the entry (including field/value). */
void entryFree(entry *entry);
/* Creates a new entry with the given field, value, and optional expiry. */
entry *entryCreate(const_sds field, sds value, long long expiry);
/* Updates the value and/or expiry of an existing entry.
* In case value is NULL, will use the existing entry value.
* In case expiry is EXPIRE_NONE, will use the existing entry expiration time. */
entry *entryUpdate(entry *entry, sds value, long long expiry);
/* Returns the total memory used by the entry (in bytes). */
size_t entryMemUsage(entry *entry);
/* Defragments the entry and returns the new pointer (if moved). */
entry *entryDefrag(entry *entry, void *(*defragfn)(void *), sds (*sdsdefragfn)(sds));
/* Advises allocator to dismiss memory used by entry. */
void entryDismissMemory(entry *entry);
/* Internal used for debug. No need to use this function except in tests */
bool entryHasEmbeddedValue(entry *entry);
#endif

103
src/expire.c
View File

@ -537,23 +537,19 @@ int checkAlreadyExpired(long long when) {
return (when <= commandTimeSnapshot() && !server.loading && !server.primary_host && !server.import_mode);
}
#define EXPIRE_NX (1 << 0)
#define EXPIRE_XX (1 << 1)
#define EXPIRE_GT (1 << 2)
#define EXPIRE_LT (1 << 3)
/* Parse additional flags of expire commands
/* Parse additional flags of expire commands up to the specify max_index.
* In case max_index will scan all arguments.
*
* Supported flags:
* - NX: set expiry only when the key has no expiry
* - XX: set expiry only when the key has an existing expiry
* - GT: set expiry only when the new expiry is greater than current one
* - LT: set expiry only when the new expiry is less than current one */
int parseExtendedExpireArgumentsOrReply(client *c, int *flags) {
int parseExtendedExpireArgumentsOrReply(client *c, int *flags, int max_args) {
int nx = 0, xx = 0, gt = 0, lt = 0;
int j = 3;
while (j < c->argc) {
while (j < max_args) {
char *opt = c->argv[j]->ptr;
if (!strcasecmp(opt, "nx")) {
*flags |= EXPIRE_NX;
@ -587,6 +583,32 @@ int parseExtendedExpireArgumentsOrReply(client *c, int *flags) {
return C_OK;
}
int convertExpireArgumentToUnixTime(client *c, robj *arg, long long basetime, int unit, long long *unixtime) {
long long when;
if (getLongLongFromObjectOrReply(c, arg, &when, NULL) != C_OK) return C_ERR;
if (when < 0) {
addReplyErrorExpireTime(c);
return C_ERR;
}
if (unit == UNIT_SECONDS) {
if (when > LLONG_MAX / 1000 || when < LLONG_MIN / 1000) {
addReplyErrorExpireTime(c);
return C_ERR;
}
when *= 1000;
}
if (when > LLONG_MAX - basetime) {
addReplyErrorExpireTime(c);
return C_ERR;
}
when += basetime;
debugServerAssert(unixtime);
*unixtime = when;
return C_OK;
}
/*-----------------------------------------------------------------------------
* Expires Commands
*----------------------------------------------------------------------------*/
@ -607,7 +629,7 @@ void expireGenericCommand(client *c, long long basetime, int unit) {
int flag = 0;
/* checking optional flags */
if (parseExtendedExpireArgumentsOrReply(c, &flag) != C_OK) {
if (parseExtendedExpireArgumentsOrReply(c, &flag, c->argc) != C_OK) {
return;
}
@ -795,3 +817,66 @@ void touchCommand(client *c) {
if (lookupKeyRead(c->db, c->argv[j]) != NULL) touched++;
addReplyLongLong(c, touched);
}
/* Returns 1 if the expire value is expired, 0 otherwise. */
bool timestampIsExpired(mstime_t when) {
if (when < 0) return false; /* no expire */
mstime_t now = commandTimeSnapshot();
/* The time indicated by 'when' is considered expired if the current (virtual or real) time is greater
* than it. */
return now > when;
}
/* This function verifies if the current conditions allow expiration of keys and fields.
* For some cases expiration is not allowed, but we would still like to ignore the key
* so to treat it as "expired" without actively deleting it. */
expirationPolicy getExpirationPolicyWithFlags(int flags) {
if (server.loading) return POLICY_IGNORE_EXPIRE;
/* If we are running in the context of a replica, instead of
* evicting the expired key from the database, we return ASAP:
* the replica key expiration is controlled by the primary that will
* send us synthesized DEL operations for expired keys. The
* exception is when write operations are performed on writable
* replicas.
*
* Still we try to reflect the correct state to the caller,
* that is, POLICY_KEEP_EXPIRED so that the key will be ignored, but not deleted.
*
* When replicating commands from the primary, keys are never considered
* expired, so we return POLICY_IGNORE_EXPIRE */
if (server.primary_host != NULL) {
if (server.current_client && (server.current_client->flag.primary)) return POLICY_IGNORE_EXPIRE;
if (!(flags & EXPIRE_FORCE_DELETE_EXPIRED)) return POLICY_KEEP_EXPIRED;
} else if (server.import_mode) {
/* If we are running in the import mode on a primary, instead of
* evicting the expired key from the database, we return ASAP:
* the key expiration is controlled by the import source that will
* send us synthesized DEL operations for expired keys. The
* exception is when write operations are performed on this server
* because it's a primary.
*
* Notice: other clients, apart from the import source, should not access
* the data imported by import source.
*
* Still we try to reflect the correct state to the caller,
* that is, POLICY_KEEP_EXPIRED so that the key will be ignored, but not deleted.
*
* When receiving commands from the import source, keys are never considered
* expired, so we return POLICY_IGNORE_EXPIRE */
if (server.current_client && (server.current_client->flag.import_source)) return POLICY_IGNORE_EXPIRE;
if (!(flags & EXPIRE_FORCE_DELETE_EXPIRED)) return POLICY_KEEP_EXPIRED;
}
/* In some cases we're explicitly instructed to return an indication of a
* missing key without actually deleting it, even on primaries. */
if (flags & EXPIRE_AVOID_DELETE_EXPIRED) return POLICY_KEEP_EXPIRED;
/* If 'expire' action is paused, for whatever reason, then don't expire any key.
* Typically, at the end of the pause we will properly expire the key OR we
* will have failed over and the new primary will send us the expire. */
if (isPausedActionsWithUpdate(PAUSE_ACTION_EXPIRE)) return POLICY_KEEP_EXPIRED;
return POLICY_DELETE_EXPIRED;
}

47
src/expire.h Normal file
View File

@ -0,0 +1,47 @@
#ifndef EXPIRE_H
#define EXPIRE_H
#include <time.h>
#include <stdbool.h>
#include "monotonic.h"
/* Special Expiry values */
#define EXPIRY_NONE -1
/* Flags for expireIfNeeded */
#define EXPIRE_FORCE_DELETE_EXPIRED 1
#define EXPIRE_AVOID_DELETE_EXPIRED 2
#define ACTIVE_EXPIRE_CYCLE_SLOW 0
#define ACTIVE_EXPIRE_CYCLE_FAST 1
/* Command flags for items expiration update conditions */
#define EXPIRE_NX (1 << 0)
#define EXPIRE_XX (1 << 1)
#define EXPIRE_GT (1 << 2)
#define EXPIRE_LT (1 << 3)
/* Return values for expireIfNeeded */
typedef enum {
KEY_VALID = 0, /* Could be volatile and not yet expired, non-volatile, or even nonexistent key. */
KEY_EXPIRED, /* Logically expired but not yet deleted. */
KEY_DELETED /* The key was deleted now. */
} keyStatus;
/* Return value for getExpirationPolicy */
typedef enum {
POLICY_IGNORE_EXPIRE, /* Ignore expiration time of items and treat them as valid. */
POLICY_KEEP_EXPIRED, /* Ignore items which are expired but do not actively delete them. */
POLICY_DELETE_EXPIRED /* Delete expired keys on access. */
} expirationPolicy;
/* Forward declarations */
typedef struct client client;
typedef struct serverObject robj;
bool timestampIsExpired(mstime_t when);
expirationPolicy getExpirationPolicyWithFlags(int flags);
int parseExtendedExpireArgumentsOrReply(client *c, int *flags, int max_args);
int convertExpireArgumentToUnixTime(client *c, robj *arg, long long basetime, int unit, long long *unixtime);
#endif

27
src/hashtable.c
View File

@ -368,6 +368,12 @@ typedef struct {
/* --- Internal functions --- */
/* --- Access API --- */
static inline bool validateElementIfNeeded(hashtable *ht, void *elem) {
if (ht->type->validateEntry == NULL) return true;
return ht->type->validateEntry(ht, elem);
}
static bucket *findBucketForInsert(hashtable *ht, uint64_t hash, int *pos_in_bucket, int *table_index);
static inline void freeEntry(hashtable *ht, void *entry) {
@ -690,6 +696,9 @@ static inline int checkCandidateInBucket(hashtable *ht, bucket *b, int pos, cons
if (compareKeys(ht, key, elem_key) == 0) {
/* It's a match. */
assert(pos_in_bucket != NULL);
if (!validateElementIfNeeded(ht, entry)) {
return 0;
}
*pos_in_bucket = pos;
if (table_index) *table_index = table;
return 1;
@ -1134,6 +1143,15 @@ hashtableType *hashtableGetType(hashtable *ht) {
return ht->type;
}
/* Set the hashtable type and returns the old type of the hashtable.
* NOTE that changing the hashtable type can lead to unexpected results.
* For example, changing the hash function can impact the ability to correctly fetch elements. */
hashtableType *hashtableSetType(hashtable *ht, hashtableType *type) {
hashtableType *oldtype = ht->type;
ht->type = type;
return oldtype;
}
/* Returns a pointer to the table's metadata (userdata) section. */
void *hashtableMetadata(hashtable *ht) {
return &ht->metadata;
@ -1787,7 +1805,7 @@ size_t hashtableScanDefrag(hashtable *ht, size_t cursor, hashtableScanFunction f
if (b->presence != 0) {
int pos;
for (pos = 0; pos < ENTRIES_PER_BUCKET; pos++) {
if (isPositionFilled(b, pos)) {
if (isPositionFilled(b, pos) && validateElementIfNeeded(ht, b->entries[pos])) {
void *emit = emit_ref ? &b->entries[pos] : b->entries[pos];
fn(privdata, emit);
}
@ -1829,7 +1847,7 @@ size_t hashtableScanDefrag(hashtable *ht, size_t cursor, hashtableScanFunction f
do {
if (b->presence) {
for (int pos = 0; pos < ENTRIES_PER_BUCKET; pos++) {
if (isPositionFilled(b, pos)) {
if (isPositionFilled(b, pos) && validateElementIfNeeded(ht, b->entries[pos])) {
void *emit = emit_ref ? &b->entries[pos] : b->entries[pos];
fn(privdata, emit);
}
@ -1859,7 +1877,7 @@ size_t hashtableScanDefrag(hashtable *ht, size_t cursor, hashtableScanFunction f
do {
if (b->presence) {
for (int pos = 0; pos < ENTRIES_PER_BUCKET; pos++) {
if (isPositionFilled(b, pos)) {
if (isPositionFilled(b, pos) && validateElementIfNeeded(ht, b->entries[pos])) {
void *emit = emit_ref ? &b->entries[pos] : b->entries[pos];
fn(privdata, emit);
}
@ -2049,6 +2067,9 @@ int hashtableNext(hashtableIterator *iterator, void **elemptr) {
/* No entry here. */
continue;
}
if (!(iter->flags & HASHTABLE_ITER_SKIP_VALIDATION) && !validateElementIfNeeded(iter->hashtable, b->entries[iter->pos_in_bucket])) {
continue;
}
/* Return the entry at this position. */
if (elemptr) {
*elemptr = b->entries[iter->pos_in_bucket];

6
src/hashtable.h
View File

@ -31,6 +31,7 @@
#include <stddef.h>
#include <stdint.h>
#include <unistd.h>
#include <stdbool.h>
/* --- Opaque types --- */
@ -57,6 +58,8 @@ typedef struct {
/* Compare function, returns 0 if the keys are equal. Defaults to just
* comparing the pointers for equality. */
int (*keyCompare)(const void *key1, const void *key2);
/* Check for entry access should be masked or not. Masked access will just treat the entry as not-exist. */
bool (*validateEntry)(hashtable *ht, void *entry);
/* Callback to free an entry when it's overwritten or deleted.
* Optional. */
void (*entryDestructor)(void *entry);
@ -77,6 +80,7 @@ typedef struct {
size_t (*getMetadataSize)(void);
/* Flag to disable incremental rehashing */
unsigned instant_rehashing : 1;
} hashtableType;
typedef enum {
@ -96,6 +100,7 @@ typedef void (*hashtableScanFunction)(void *privdata, void *entry);
/* Iterator flags */
#define HASHTABLE_ITER_SAFE (1 << 0)
#define HASHTABLE_ITER_PREFETCH_VALUES (1 << 1)
#define HASHTABLE_ITER_SKIP_VALIDATION (1 << 2)
/* --- Prototypes --- */
@ -113,6 +118,7 @@ hashtable *hashtableCreate(hashtableType *type);
void hashtableRelease(hashtable *ht);
void hashtableEmpty(hashtable *ht, void(callback)(hashtable *));
hashtableType *hashtableGetType(hashtable *ht);
hashtableType *hashtableSetType(hashtable *ht, hashtableType *type);
void *hashtableMetadata(hashtable *ht);
size_t hashtableSize(const hashtable *ht);
size_t hashtableBuckets(hashtable *ht);

8
src/module.c
View File

@ -5350,11 +5350,11 @@ int VM_HashSet(ValkeyModuleKey *key, int flags, ...) {
/* If CFIELDS is active, we can pass the ownership of the
* SDS object to the low level function that sets the field
* to avoid a useless copy. */
if (flags & VALKEYMODULE_HASH_CFIELDS) low_flags |= HASH_SET_TAKE_FIELD;
if (flags & VALKEYMODULE_HASH_CFIELDS) low_flags |= (HASH_SET_TAKE_FIELD);
robj *argv[2] = {field, value};
hashTypeTryConversion(key->value, argv, 0, 1);
int updated = hashTypeSet(key->value, field->ptr, value->ptr, low_flags);
int updated = hashTypeSet(key->value, field->ptr, value->ptr, EXPIRY_NONE, low_flags);
count += (flags & VALKEYMODULE_HASH_COUNT_ALL) ? 1 : updated;
/* If CFIELDS is active, SDS string ownership is now of hashTypeSet(),
@ -11224,8 +11224,8 @@ static void moduleScanKeyHashtableCallback(void *privdata, void *entry) {
key = node->ele;
value = createStringObjectFromLongDouble(node->score, 0);
} else if (o->type == OBJ_HASH) {
key = hashTypeEntryGetField(entry);
sds val = hashTypeEntryGetValue(entry);
key = entryGetField(entry);
sds val = entryGetValue(entry);
value = createStringObject(val, sdslen(val));
} else {
serverPanic("unexpected object type");

2
src/monotonic.h
View File

@ -20,6 +20,8 @@
* variable is associated with the monotonic clock and should not be confused
* with other types of time.*/
typedef uint64_t monotime;
typedef long long mstime_t; /* millisecond time type. */
typedef long long ustime_t; /* microsecond time type. */
/* Retrieve counter of micro-seconds relative to an arbitrary point in time. */
extern monotime (*getMonotonicUs)(void);

9
src/object.c
View File

@ -527,7 +527,10 @@ void freeZsetObject(robj *o) {
void freeHashObject(robj *o) {
switch (o->encoding) {
case OBJ_ENCODING_HASHTABLE: hashtableRelease((hashtable *)o->ptr); break;
case OBJ_ENCODING_HASHTABLE:
hashTypeFreeVolatileSet(o);
hashtableRelease((hashtable *)o->ptr);
break;
case OBJ_ENCODING_LISTPACK: lpFree(o->ptr); break;
default: serverPanic("Unknown hash encoding type"); break;
}
@ -682,7 +685,7 @@ void dismissHashObject(robj *o, size_t size_hint) {
hashtableInitIterator(&iter, ht, 0);
void *next;
while (hashtableNext(&iter, &next)) {
dismissHashTypeEntry(next);
entryDismissMemory(next);
}
hashtableResetIterator(&iter);
}
@ -1203,7 +1206,7 @@ size_t objectComputeSize(robj *key, robj *o, size_t sample_size, int dbid) {
asize = zmalloc_size((void *)o) + hashtableMemUsage(ht);
while (hashtableNext(&iter, &next) && samples < sample_size) {
elesize += hashTypeEntryMemUsage(next);
elesize += entryMemUsage(next);
samples++;
}
hashtableResetIterator(&iter);

56
src/rdb.c
View File

@ -32,6 +32,7 @@
* SPDX-License-Identifier: BSD-3-Clause
*/
#include "hashtable.h"
#include "server.h"
#include "lzf.h" /* LZF compression library */
#include "zipmap.h"
@ -717,7 +718,10 @@ int rdbSaveObjectType(rio *rdb, robj *o) {
if (o->encoding == OBJ_ENCODING_LISTPACK)
return rdbSaveType(rdb, RDB_TYPE_HASH_LISTPACK);
else if (o->encoding == OBJ_ENCODING_HASHTABLE)
return rdbSaveType(rdb, RDB_TYPE_HASH);
if (hashTypeHasVolatileElements(o))
return rdbSaveType(rdb, RDB_TYPE_HASH_2);
else
return rdbSaveType(rdb, RDB_TYPE_HASH);
else
serverPanic("Unknown hash encoding");
case OBJ_STREAM: return rdbSaveType(rdb, RDB_TYPE_STREAM_LISTPACKS_3);
@ -840,7 +844,6 @@ size_t rdbSaveStreamConsumers(rio *rdb, streamCG *cg) {
* Returns -1 on error, number of bytes written on success. */
ssize_t rdbSaveObject(rio *rdb, robj *o, robj *key, int dbid) {
ssize_t n = 0, nwritten = 0;
if (o->type == OBJ_STRING) {
/* Save a string value */
if ((n = rdbSaveStringObject(rdb, o)) == -1) return -1;
@ -963,13 +966,14 @@ ssize_t rdbSaveObject(rio *rdb, robj *o, robj *key, int dbid) {
return -1;
}
nwritten += n;
/* check if need to add expired time for the hash elements */
bool add_expiry = hashTypeHasVolatileElements(o);
hashtableIterator iter;
hashtableInitIterator(&iter, ht, 0);
hashtableInitIterator(&iter, ht, HASHTABLE_ITER_SKIP_VALIDATION);
void *next;
while (hashtableNext(&iter, &next)) {
sds field = hashTypeEntryGetField(next);
sds value = hashTypeEntryGetValue(next);
sds field = entryGetField(next);
sds value = entryGetValue(next);
if ((n = rdbSaveRawString(rdb, (unsigned char *)field, sdslen(field))) == -1) {
hashtableResetIterator(&iter);
@ -981,8 +985,17 @@ ssize_t rdbSaveObject(rio *rdb, robj *o, robj *key, int dbid) {
return -1;
}
nwritten += n;
if (add_expiry) {
long long expiry = entryGetExpiry(next);
if ((n = rdbSaveMillisecondTime(rdb, expiry) == -1)) {
hashtableResetIterator(&iter);
return -1;
}
nwritten += n;
}
}
hashtableResetIterator(&iter);
} else {
serverPanic("Unknown hash encoding");
}
@ -2073,7 +2086,7 @@ robj *rdbLoadObject(int rdbtype, rio *rdb, sds key, int dbid, int *error) {
lpSafeToAdd(NULL, totelelen)) {
zsetConvert(o, OBJ_ENCODING_LISTPACK);
}
} else if (rdbtype == RDB_TYPE_HASH) {
} else if (rdbtype == RDB_TYPE_HASH || rdbtype == RDB_TYPE_HASH_2) {
uint64_t len;
sds field, value;
hashtable *dupSearchHashtable = NULL;
@ -2084,8 +2097,8 @@ robj *rdbLoadObject(int rdbtype, rio *rdb, sds key, int dbid, int *error) {
o = createHashObject();
/* Too many entries? Use a hash table right from the start. */
if (len > server.hash_max_listpack_entries)
/* Too many entries or hash object contains elements with expiry? Use a hash table right from the start. */
if (len > server.hash_max_listpack_entries || rdbtype == RDB_TYPE_HASH_2)
hashTypeConvert(o, OBJ_ENCODING_HASHTABLE);
else if (deep_integrity_validation) {
/* In this mode, we need to guarantee that the server won't crash
@ -2126,21 +2139,23 @@ robj *rdbLoadObject(int rdbtype, rio *rdb, sds key, int dbid, int *error) {
}
/* Convert to hash table if size threshold is exceeded */
if (sdslen(field) > server.hash_max_listpack_value || sdslen(value) > server.hash_max_listpack_value ||
!lpSafeToAdd(o->ptr, sdslen(field) + sdslen(value))) {
if (o->encoding != OBJ_ENCODING_HASHTABLE &&
(sdslen(field) > server.hash_max_listpack_value || sdslen(value) > server.hash_max_listpack_value ||
!lpSafeToAdd(o->ptr, sdslen(field) + sdslen(value)))) {
hashTypeConvert(o, OBJ_ENCODING_HASHTABLE);
hashTypeEntry *entry = hashTypeCreateEntry(field, value);
entry *entry = entryCreate(field, value, EXPIRY_NONE);
sdsfree(field);
if (!hashtableAdd((hashtable *)o->ptr, entry)) {
rdbReportCorruptRDB("Duplicate hash fields detected");
if (dupSearchHashtable) hashtableRelease(dupSearchHashtable);
freeHashTypeEntry(entry);
entryFree(entry);
decrRefCount(o);
return NULL;
}
break;
}
/* Add pair to listpack */
o->ptr = lpAppend(o->ptr, (unsigned char *)field, sdslen(field));
o->ptr = lpAppend(o->ptr, (unsigned char *)value, sdslen(value));
@ -2178,15 +2193,26 @@ robj *rdbLoadObject(int rdbtype, rio *rdb, sds key, int dbid, int *error) {
return NULL;
}
/* Also load the entry expiry */
long long itemexpiry = EXPIRY_NONE;
if (rdbtype == RDB_TYPE_HASH_2) {
itemexpiry = rdbLoadMillisecondTime(rdb, RDB_VERSION);
if (itemexpiry < EXPIRY_NONE || rioGetReadError(rdb)) return NULL;
}
/* Add pair to hash table */
hashTypeEntry *entry = hashTypeCreateEntry(field, value);
entry *entry = entryCreate(field, value, itemexpiry);
sdsfree(field);
if (!hashtableAdd((hashtable *)o->ptr, entry)) {
rdbReportCorruptRDB("Duplicate hash fields detected");
freeHashTypeEntry(entry);
entryFree(entry);
decrRefCount(o);
return NULL;
}
if (rdbtype == RDB_TYPE_HASH_2 && itemexpiry > 0) {
hashTypeTrackEntry(o, entry);
}
}
/* All pairs should be read by now */

50
src/rdb.h
View File

@ -90,32 +90,36 @@ static_assert(RDB_VERSION < RDB_FOREIGN_VERSION_MIN || RDB_VERSION > RDB_FOREIGN
/* Map object types to RDB object types. Macros starting with OBJ_ are for
* memory storage and may change. Instead RDB types must be fixed because
* we store them on disk. */
#define RDB_TYPE_STRING 0
#define RDB_TYPE_LIST 1
#define RDB_TYPE_SET 2
#define RDB_TYPE_ZSET 3
#define RDB_TYPE_HASH 4
#define RDB_TYPE_ZSET_2 5 /* ZSET version 2 with doubles stored in binary. */
#define RDB_TYPE_MODULE_PRE_GA 6 /* Used in 4.0 release candidates */
#define RDB_TYPE_MODULE_2 7 /* Module value with annotations for parsing without \
enum RdbType {
RDB_TYPE_STRING = 0,
RDB_TYPE_LIST = 1,
RDB_TYPE_SET = 2,
RDB_TYPE_ZSET = 3,
RDB_TYPE_HASH = 4,
RDB_TYPE_ZSET_2 = 5, /* ZSET version 2 with doubles stored in binary. */
RDB_TYPE_MODULE_PRE_GA = 6, /* Used in 4.0 release candidates */
RDB_TYPE_MODULE_2 = 7, /* Module value with annotations for parsing without \
the generating module being loaded. */
#define RDB_TYPE_HASH_ZIPMAP 9
#define RDB_TYPE_LIST_ZIPLIST 10
#define RDB_TYPE_SET_INTSET 11
#define RDB_TYPE_ZSET_ZIPLIST 12
#define RDB_TYPE_HASH_ZIPLIST 13
#define RDB_TYPE_LIST_QUICKLIST 14
#define RDB_TYPE_STREAM_LISTPACKS 15
#define RDB_TYPE_HASH_LISTPACK 16
#define RDB_TYPE_ZSET_LISTPACK 17
#define RDB_TYPE_LIST_QUICKLIST_2 18
#define RDB_TYPE_STREAM_LISTPACKS_2 19
#define RDB_TYPE_SET_LISTPACK 20
#define RDB_TYPE_STREAM_LISTPACKS_3 21
/* NOTE: WHEN ADDING NEW RDB TYPE, UPDATE rdbIsObjectType(), and rdb_type_string[] */
RDB_TYPE_HASH_ZIPMAP = 9,
RDB_TYPE_LIST_ZIPLIST = 10,
RDB_TYPE_SET_INTSET = 11,
RDB_TYPE_ZSET_ZIPLIST = 12,
RDB_TYPE_HASH_ZIPLIST = 13,
RDB_TYPE_LIST_QUICKLIST = 14,
RDB_TYPE_STREAM_LISTPACKS = 15,
RDB_TYPE_HASH_LISTPACK = 16,
RDB_TYPE_ZSET_LISTPACK = 17,
RDB_TYPE_LIST_QUICKLIST_2 = 18,
RDB_TYPE_STREAM_LISTPACKS_2 = 19,
RDB_TYPE_SET_LISTPACK = 20,
RDB_TYPE_STREAM_LISTPACKS_3 = 21,
RDB_TYPE_HASH_2 = 22,
RDB_TYPE_LAST
};
/* NOTE: WHEN ADDING NEW RDB TYPE, UPDATE rdb_type_string[] */
/* Test if a type is an object type. */
#define rdbIsObjectType(t) (((t) >= 0 && (t) <= 7) || ((t) >= 9 && (t) <= 21))
#define rdbIsObjectType(t) (((t) >= 0 && (t) <= 7) || ((t) >= 9 && (t) < RDB_TYPE_LAST))
/* Special RDB opcodes (saved/loaded with rdbSaveType/rdbLoadType). */
#define RDB_OPCODE_FUNCTION2 245 /* function library data */

153
src/server.c
View File

@ -664,20 +664,34 @@ hashtableType subcommandSetType = {.entryGetKey = hashtableSubcommandGetKey,
/* Hash type hash table (note that small hashes are represented with listpacks) */
const void *hashHashtableTypeGetKey(const void *entry) {
const hashTypeEntry *hash_entry = entry;
return (const void *)hashTypeEntryGetField(hash_entry);
return (const void *)entryGetField(entry);
}
void hashHashtableTypeDestructor(void *entry) {
hashTypeEntry *hash_entry = entry;
freeHashTypeEntry(hash_entry);
entryFree(entry);
}
size_t hashHashtableTypeMetadataSize(void) {
return sizeof(void *);
}
extern bool hashHashtableTypeValidate(hashtable *ht, void *entry);
hashtableType hashHashtableType = {
.hashFunction = dictSdsHash,
.entryGetKey = hashHashtableTypeGetKey,
.keyCompare = hashtableSdsKeyCompare,
.entryDestructor = hashHashtableTypeDestructor,
.getMetadataSize = hashHashtableTypeMetadataSize,
};
hashtableType hashWithVolatileItemsHashtableType = {
.hashFunction = dictSdsHash,
.entryGetKey = hashHashtableTypeGetKey,
.keyCompare = hashtableSdsKeyCompare,
.entryDestructor = hashHashtableTypeDestructor,
.getMetadataSize = hashHashtableTypeMetadataSize,
.validateEntry = hashHashtableTypeValidate,
};
/* Hashtable type without destructor */
@ -2135,6 +2149,9 @@ void createSharedObjects(void) {
shared.multi = createSharedString("MULTI");
shared.exec = createSharedString("EXEC");
shared.hset = createSharedString("HSET");
shared.hdel = createSharedString("HDEL");
shared.hpexpireat = createSharedString("HPEXPIREAT");
shared.hpersist = createSharedString("HPERSIST");
shared.srem = createSharedString("SREM");
shared.xgroup = createSharedString("XGROUP");
shared.xclaim = createSharedString("XCLAIM");
@ -2167,6 +2184,7 @@ void createSharedObjects(void) {
shared.special_asterisk = createSharedString("*");
shared.special_equals = createSharedString("=");
shared.redacted = createSharedString("(redacted)");
shared.fields = createSharedString("FIELDS");
for (j = 0; j < OBJ_SHARED_INTEGERS; j++) {
shared.integers[j] = makeObjectShared(createObject(OBJ_STRING, (void *)(long)j));
@ -7333,4 +7351,131 @@ __attribute__((weak)) int main(int argc, char **argv) {
aeDeleteEventLoop(server.el);
return 0;
}
/*
* The parseExtendedCommandArgumentsOrReply() function performs the common validation for extended
* command arguments used in STRING and HASH commands.
*
* Get specific command extended options - PERSIST/DEL
* Set specific command extended options - XX/NX/GET/IFEQ
* HSET specific command extended options - FXX/FNX
* Common command extended options - EX/EXAT/PX/PXAT/KEEPTTL
*
* Function takes pointers to client, flags, unit, pointer to pointer of expire obj if needed
* to be determined and command_type which can be COMMAND_GET or COMMAND_SET.
*
* If there are any syntax violations C_ERR is returned else C_OK is returned.
*
* Input flags are updated upon parsing the arguments. Unit and expire are updated if there are any
* EX/EXAT/PX/PXAT arguments. Unit is updated to millisecond if PX/PXAT is set.
*
* max_args provides a way to limit the scan to a specific range of arguments.
*/
int parseExtendedCommandArgumentsOrReply(client *c, int *flags, int *unit, robj **expire, robj **compare_val, int command_type, int max_args) {
int j = command_type == COMMAND_SET ? 3 : 2;
for (; j < max_args; j++) {
char *opt = c->argv[j]->ptr;
robj *next = (j == max_args - 1) ? NULL : c->argv[j + 1];
/* clang-format off */
if ((opt[0] == 'n' || opt[0] == 'N') &&
(opt[1] == 'x' || opt[1] == 'X') && opt[2] == '\0' &&
!(*flags & ARGS_SET_XX || *flags & ARGS_SET_IFEQ) && (command_type == COMMAND_SET))
{
*flags |= ARGS_SET_NX;
} else if ((opt[0] == 'x' || opt[0] == 'X') &&
(opt[1] == 'x' || opt[1] == 'X') && opt[2] == '\0' &&
!(*flags & ARGS_SET_NX || *flags & ARGS_SET_IFEQ) && (command_type == COMMAND_SET))
{
*flags |= ARGS_SET_XX;
} else if ((opt[0] == 'f' || opt[0] == 'F') &&
(opt[1] == 'n' || opt[1] == 'N') &&
(opt[2] == 'x' || opt[2] == 'X') && opt[3] == '\0' &&
!(*flags & ARGS_SET_FXX || *flags & ARGS_SET_IFEQ) && (command_type == COMMAND_HSET))
{
*flags |= ARGS_SET_FNX;
} else if ((opt[0] == 'f' || opt[0] == 'F') &&
(opt[1] == 'x' || opt[1] == 'X') &&
(opt[2] == 'x' || opt[2] == 'X') && opt[3] == '\0' &&
!(*flags & ARGS_SET_FNX || *flags & ARGS_SET_IFEQ) && (command_type == COMMAND_HSET))
{
*flags |= ARGS_SET_FXX;
} else if ((opt[0] == 'i' || opt[0] == 'I') &&
(opt[1] == 'f' || opt[1] == 'F') &&
(opt[2] == 'e' || opt[2] == 'E') &&
(opt[3] == 'q' || opt[3] == 'Q') && opt[4] == '\0' &&
next &&
!(*flags & ARGS_SET_NX || *flags & ARGS_SET_XX || *flags & ARGS_SET_IFEQ) && (command_type == COMMAND_SET))
{
*flags |= ARGS_SET_IFEQ;
*compare_val = next;
j++;
} else if ((opt[0] == 'g' || opt[0] == 'G') &&
(opt[1] == 'e' || opt[1] == 'E') &&
(opt[2] == 't' || opt[2] == 'T') && opt[3] == '\0' &&
(command_type == COMMAND_SET))
{
*flags |= ARGS_SET_GET;
} else if (!strcasecmp(opt, "KEEPTTL") && !(*flags & ARGS_PERSIST) &&
!(*flags & ARGS_EX) && !(*flags & ARGS_EXAT) &&
!(*flags & ARGS_PX) && !(*flags & ARGS_PXAT) && (command_type == COMMAND_SET || command_type == COMMAND_HSET))
{
*flags |= ARGS_KEEPTTL;
} else if (!strcasecmp(opt,"PERSIST") && (command_type == COMMAND_GET || command_type == COMMAND_HGET) &&
!(*flags & ARGS_EX) && !(*flags & ARGS_EXAT) &&
!(*flags & ARGS_PX) && !(*flags & ARGS_PXAT) &&
!(*flags & ARGS_KEEPTTL))
{
*flags |= ARGS_PERSIST;
} else if ((opt[0] == 'e' || opt[0] == 'E') &&
(opt[1] == 'x' || opt[1] == 'X') && opt[2] == '\0' &&
!(*flags & ARGS_KEEPTTL) && !(*flags & ARGS_PERSIST) &&
!(*flags & ARGS_EXAT) && !(*flags & ARGS_PX) &&
!(*flags & ARGS_PXAT) && next)
{
*flags |= ARGS_EX;
*expire = next;
j++;
} else if ((opt[0] == 'p' || opt[0] == 'P') &&
(opt[1] == 'x' || opt[1] == 'X') && opt[2] == '\0' &&
!(*flags & ARGS_KEEPTTL) && !(*flags & ARGS_PERSIST) &&
!(*flags & ARGS_EX) && !(*flags & ARGS_EXAT) &&
!(*flags & ARGS_PXAT) && next)
{
*flags |= ARGS_PX;
*unit = UNIT_MILLISECONDS;
*expire = next;
j++;
} else if ((opt[0] == 'e' || opt[0] == 'E') &&
(opt[1] == 'x' || opt[1] == 'X') &&
(opt[2] == 'a' || opt[2] == 'A') &&
(opt[3] == 't' || opt[3] == 'T') && opt[4] == '\0' &&
!(*flags & ARGS_KEEPTTL) && !(*flags & ARGS_PERSIST) &&
!(*flags & ARGS_EX) && !(*flags & ARGS_PX) &&
!(*flags & ARGS_PXAT) && next)
{
*flags |= ARGS_EXAT;
*expire = next;
j++;
} else if ((opt[0] == 'p' || opt[0] == 'P') &&
(opt[1] == 'x' || opt[1] == 'X') &&
(opt[2] == 'a' || opt[2] == 'A') &&
(opt[3] == 't' || opt[3] == 'T') && opt[4] == '\0' &&
!(*flags & ARGS_KEEPTTL) && !(*flags & ARGS_PERSIST) &&
!(*flags & ARGS_EX) && !(*flags & ARGS_EXAT) &&
!(*flags & ARGS_PX) && next)
{
*flags |= ARGS_PXAT;
*unit = UNIT_MILLISECONDS;
*expire = next;
j++;
} else {
addReplyErrorObject(c,shared.syntaxerr);
return C_ERR;
}
/* clang-format on */
}
return C_OK;
}
/* The End */

80
src/server.h
View File

@ -62,9 +62,6 @@
#define static_assert(expr, lit) extern char __static_assert_failure[(expr) ? 1 : -1]
#endif
typedef long long mstime_t; /* millisecond time type. */
typedef long long ustime_t; /* microsecond time type. */
#include "ae.h" /* Event driven programming library */
#include "sds.h" /* Dynamic safe strings */
#include "dict.h" /* Hash tables (old implementation) */
@ -79,10 +76,13 @@ typedef long long ustime_t; /* microsecond time type. */
#include "sparkline.h" /* ASCII graphs API */
#include "quicklist.h" /* Lists are encoded as linked lists of
N-elements flat arrays */
#include "expire.h" /* Expiration public API */
#include "rax.h" /* Radix tree */
#include "connection.h" /* Connection abstraction */
#include "memory_prefetch.h"
#include "volatile_set.h"
#include "trace/trace.h"
#include "entry.h"
#ifdef USE_LTTNG
#define valkey_fork() do_fork()
@ -162,9 +162,6 @@ struct hdr_histogram;
#define CLIENT_MEM_USAGE_BUCKET_MAX_LOG 33 /* Bucket for largest clients: sizes above 4GB (2^32) */
#define CLIENT_MEM_USAGE_BUCKETS (1 + CLIENT_MEM_USAGE_BUCKET_MAX_LOG - CLIENT_MEM_USAGE_BUCKET_MIN_LOG)
#define ACTIVE_EXPIRE_CYCLE_SLOW 0
#define ACTIVE_EXPIRE_CYCLE_FAST 1
/* Children process will exit with this status code to signal that the
* process terminated without an error: this is useful in order to kill
* a saving child (RDB or AOF one), without triggering in the parent the
@ -220,6 +217,11 @@ struct hdr_histogram;
extern int configOOMScoreAdjValuesDefaults[CONFIG_OOM_COUNT];
#define COMMAND_GET 0
#define COMMAND_SET 1
#define COMMAND_HGET 2
#define COMMAND_HSET 3
/* Command flags. Please check the definition of struct serverCommand in this file
* for more information about the meaning of every flag. */
#define CMD_WRITE (1ULL << 0)
@ -514,9 +516,6 @@ typedef enum {
#define SUPERVISED_SYSTEMD 2
#define SUPERVISED_UPSTART 3
/* Anti-warning macro... */
#define UNUSED(V) ((void)V)
#define ZSKIPLIST_MAXLEVEL 32 /* Should be enough for 2^64 elements */
#define ZSKIPLIST_P 0.25 /* Skiplist P = 1/4 */
#define ZSKIPLIST_MAX_SEARCH 10
@ -719,6 +718,23 @@ typedef enum {
* Data types
*----------------------------------------------------------------------------*/
/* Generic set command string object set flags */
#define ARGS_NO_FLAGS 0
#define ARGS_SET_NX (1 << 0) /* Set if key not exists. */
#define ARGS_SET_XX (1 << 1) /* Set if key exists. */
#define ARGS_EX (1 << 2) /* Set if time in seconds is given */
#define ARGS_PX (1 << 3) /* Set if time in ms in given */
#define ARGS_KEEPTTL (1 << 4) /* Set and keep the ttl */
#define ARGS_SET_GET (1 << 5) /* Set if want to get key before set */
#define ARGS_EXAT (1 << 6) /* Set if timestamp in second is given */
#define ARGS_PXAT (1 << 7) /* Set if timestamp in ms is given */
#define ARGS_PERSIST (1 << 8) /* Set if we need to remove the ttl */
#define ARGS_SET_IFEQ (1 << 9) /* Set if we need compare and set */
#define ARGS_ARGV3 (1 << 10) /* Set if the value is at argv[3]; otherwise it's \
* at argv[2]. */
#define ARGS_SET_FNX (1 << 11) /* Set if key item not exists. */
#define ARGS_SET_FXX (1 << 12) /* Set if key item exists. */
/* An Object, that is a type able to hold a string / list / set */
/* The actual Object */
@ -852,8 +868,9 @@ typedef struct replBufBlock {
* by integers from 0 (the default database) up to the max configured
* database. The database number is the 'id' field in the structure. */
typedef struct serverDb {
kvstore *keys; /* The keyspace for this DB */
kvstore *expires; /* Timeout of keys with a timeout set */
kvstore *keys; /* The keyspace for this DB */
kvstore *expires; /* Timeout of keys with a timeout set */
kvstore *object_with_volatile_elements;
dict *blocking_keys; /* Keys with clients waiting for data (BLPOP)*/
dict *blocking_keys_unblock_on_nokey; /* Keys with clients waiting for
* data, and should be unblocked if key is deleted (XREADEDGROUP).
@ -1361,10 +1378,10 @@ struct sharedObjectsStruct {
*bgsaveerr_variants[2],
*execaborterr, *noautherr, *noreplicaserr, *busykeyerr, *oomerr, *plus, *messagebulk, *pmessagebulk,
*subscribebulk, *unsubscribebulk, *psubscribebulk, *punsubscribebulk, *del, *unlink, *rpop, *lpop, *lpush,
*rpoplpush, *lmove, *blmove, *zpopmin, *zpopmax, *emptyscan, *multi, *exec, *left, *right, *hset, *srem,
*rpoplpush, *lmove, *blmove, *zpopmin, *zpopmax, *emptyscan, *multi, *exec, *left, *right, *hset, *hdel, *hpexpireat, *hpersist, *srem,
*xgroup, *xclaim, *script, *replconf, *eval, *persist, *set, *pexpireat, *pexpire, *time, *pxat, *absttl,
*retrycount, *force, *justid, *entriesread, *lastid, *ping, *setid, *keepttl, *load, *createconsumer, *getack,
*special_asterisk, *special_equals, *default_username, *redacted, *ssubscribebulk, *sunsubscribebulk,
*special_asterisk, *special_equals, *default_username, *redacted, *ssubscribebulk, *sunsubscribebulk, *fields,
*smessagebulk, *select[PROTO_SHARED_SELECT_CMDS], *integers[OBJ_SHARED_INTEGERS],
*mbulkhdr[OBJ_SHARED_BULKHDR_LEN], /* "*<value>\r\n" */
*bulkhdr[OBJ_SHARED_BULKHDR_LEN], /* "$<value>\r\n" */
@ -1609,7 +1626,6 @@ typedef enum childInfoType {
CHILD_INFO_TYPE_RDB_COW_SIZE,
CHILD_INFO_TYPE_MODULE_COW_SIZE
} childInfoType;
struct valkeyServer {
/* General */
pid_t pid; /* Main process pid. */
@ -2607,11 +2623,13 @@ typedef struct {
typedef struct {
robj *subject;
int encoding;
bool volatile_items_iter;
unsigned char *fptr, *vptr;
hashtableIterator iter;
volatileSetIterator viter;
void *next;
} hashTypeIterator;
#include "stream.h" /* Stream data type header file. */
@ -2635,6 +2653,7 @@ extern hashtableType kvstoreKeysHashtableType;
extern hashtableType kvstoreExpiresHashtableType;
extern double R_Zero, R_PosInf, R_NegInf, R_Nan;
extern hashtableType hashHashtableType;
extern hashtableType hashWithVolatileItemsHashtableType;
extern dictType stringSetDictType;
extern dictType externalStringType;
extern dictType sdsHashDictType;
@ -2846,6 +2865,7 @@ int processIOThreadsWriteDone(void);
void releaseReplyReferences(client *c);
void resetLastWrittenBuf(client *c);
int parseExtendedCommandArgumentsOrReply(client *c, int *flags, int *unit, robj **expire, robj **compare_val, int command_type, int max_args);
/* logreqres.c - logging of requests and responses */
void reqresReset(client *c, int free_buf);
@ -3335,16 +3355,14 @@ robj *setTypeDup(robj *o);
/* Hash data type */
#define HASH_SET_TAKE_FIELD (1 << 0)
#define HASH_SET_TAKE_VALUE (1 << 1)
#define HASH_SET_KEEP_EXPIRY (1 << 2)
#define HASH_SET_COPY 0
typedef void hashTypeEntry;
hashTypeEntry *hashTypeCreateEntry(sds field, sds value);
sds hashTypeEntryGetField(const hashTypeEntry *entry);
sds hashTypeEntryGetValue(const hashTypeEntry *entry);
size_t hashTypeEntryMemUsage(hashTypeEntry *entry);
hashTypeEntry *hashTypeEntryDefrag(hashTypeEntry *entry, void *(*defragfn)(void *), sds (*sdsdefragfn)(sds));
void dismissHashTypeEntry(hashTypeEntry *entry);
void freeHashTypeEntry(hashTypeEntry *entry);
void hashTypeFreeVolatileSet(robj *o);
void hashTypeTrackEntry(robj *o, void *entry);
void hashTypeUntrackEntry(robj *o, void *entry);
void hashTypeTrackUpdateEntry(robj *o, void *old_entry, void *new_entry, long long old_expiry, long long new_expiry);
void hashTypeConvert(robj *o, int enc);
void hashTypeTryConversion(robj *subject, robj **argv, int start, int end);
@ -3352,6 +3370,7 @@ int hashTypeExists(robj *o, sds key);
int hashTypeDelete(robj *o, sds key);
unsigned long hashTypeLength(const robj *o);
void hashTypeInitIterator(robj *subject, hashTypeIterator *hi);
void hashTypeInitVolatileIterator(robj *subject, hashTypeIterator *hi);
void hashTypeResetIterator(hashTypeIterator *hi);
int hashTypeNext(hashTypeIterator *hi);
void hashTypeCurrentFromListpack(hashTypeIterator *hi,
@ -3363,8 +3382,10 @@ sds hashTypeCurrentFromHashTable(hashTypeIterator *hi, int what);
sds hashTypeCurrentObjectNewSds(hashTypeIterator *hi, int what);
robj *hashTypeLookupWriteOrCreate(client *c, robj *key);
robj *hashTypeGetValueObject(robj *o, sds field);
int hashTypeSet(robj *o, sds field, sds value, int flags);
int hashTypeSet(robj *o, sds field, sds value, long long expiry, int flags);
robj *hashTypeDup(robj *o);
bool hashTypeHasVolatileElements(robj *o);
size_t hashTypeNumVolatileElements(robj *o);
/* Pub / Sub */
int pubsubUnsubscribeAllChannels(client *c, int notify);
@ -3826,6 +3847,8 @@ void zrankCommand(client *c);
void zrevrankCommand(client *c);
void hsetCommand(client *c);
void hsetnxCommand(client *c);
void hsetexCommand(client *c);
void hgetexCommand(client *c);
void hgetCommand(client *c);
void hmgetCommand(client *c);
void hdelCommand(client *c);
@ -3847,6 +3870,15 @@ void hgetallCommand(client *c);
void hexistsCommand(client *c);
void hscanCommand(client *c);
void hrandfieldCommand(client *c);
void hexpireCommand(client *c);
void hexpireatCommand(client *c);
void hpexpireCommand(client *c);
void hpexpireatCommand(client *c);
void httlCommand(client *c);
void hpttlCommand(client *c);
void hexpiretimeCommand(client *c);
void hpexpiretimeCommand(client *c);
void hpersistCommand(client *c);
void configSetCommand(client *c);
void configGetCommand(client *c);
void configResetStatCommand(client *c);

4
src/serverassert.h
View File

@ -63,4 +63,8 @@
void _serverAssert(const char *estr, const char *file, int line);
void _serverPanic(const char *file, int line, const char *msg, ...);
#ifndef static_assert
#define static_assert(expr, lit) extern char __static_assert_failure[(expr) ? 1 : -1]
#endif
#endif

1408
src/t_hash.c
View File

File diff suppressed because it is too large Load Diff

181
src/t_string.c
View File

@ -55,6 +55,9 @@ static int checkStringLength(client *c, long long size, long long append) {
return C_OK;
}
/* Forward declaration */
static int getExpireMillisecondsOrReply(client *c, robj *expire, int flags, int unit, long long *milliseconds);
/* The setGenericCommand() function implements the SET operation with different
* options and variants. This function is called in order to implement the
* following commands: SET, SETEX, PSETEX, SETNX, GETSET.
@ -70,24 +73,6 @@ static int checkStringLength(client *c, long long size, long long append) {
*
* If ok_reply is NULL "+OK" is used.
* If abort_reply is NULL, "$-1" is used. */
#define OBJ_NO_FLAGS 0
#define OBJ_SET_NX (1 << 0) /* Set if key not exists. */
#define OBJ_SET_XX (1 << 1) /* Set if key exists. */
#define OBJ_EX (1 << 2) /* Set if time in seconds is given */
#define OBJ_PX (1 << 3) /* Set if time in ms in given */
#define OBJ_KEEPTTL (1 << 4) /* Set and keep the ttl */
#define OBJ_SET_GET (1 << 5) /* Set if want to get key before set */
#define OBJ_EXAT (1 << 6) /* Set if timestamp in second is given */
#define OBJ_PXAT (1 << 7) /* Set if timestamp in ms is given */
#define OBJ_PERSIST (1 << 8) /* Set if we need to remove the ttl */
#define OBJ_SET_IFEQ (1 << 9) /* Set if we need compare and set */
#define OBJ_ARGV3 (1 << 10) /* Set if the value is at argv[3]; otherwise it's \
* at argv[2]. */
/* Forward declaration */
static int getExpireMillisecondsOrReply(client *c, robj *expire, int flags, int unit, long long *milliseconds);
void setGenericCommand(client *c,
int flags,
robj *key,
@ -105,7 +90,7 @@ void setGenericCommand(client *c,
return;
}
if (flags & OBJ_SET_GET) {
if (flags & ARGS_SET_GET) {
initDeferredReplyBuffer(c);
if (getGenericCommand(c) == C_ERR) goto cleanup;
}
@ -114,26 +99,26 @@ void setGenericCommand(client *c,
found = existing_value != NULL;
/* Handle the IFEQ conditional check */
if (flags & OBJ_SET_IFEQ && found) {
if (!(flags & OBJ_SET_GET) && checkType(c, existing_value, OBJ_STRING)) {
if (flags & ARGS_SET_IFEQ && found) {
if (!(flags & ARGS_SET_GET) && checkType(c, existing_value, OBJ_STRING)) {
goto cleanup;
}
if (compareStringObjects(existing_value, comparison) != 0) {
if (!(flags & OBJ_SET_GET)) {
if (!(flags & ARGS_SET_GET)) {
addReply(c, abort_reply ? abort_reply : shared.null[c->resp]);
}
goto cleanup;
}
} else if (flags & OBJ_SET_IFEQ && !found) {
if (!(flags & OBJ_SET_GET)) {
} else if (flags & ARGS_SET_IFEQ && !found) {
if (!(flags & ARGS_SET_GET)) {
addReply(c, abort_reply ? abort_reply : shared.null[c->resp]);
}
goto cleanup;
}
if ((flags & OBJ_SET_NX && found) || (flags & OBJ_SET_XX && !found)) {
if (!(flags & OBJ_SET_GET)) {
if ((flags & ARGS_SET_NX && found) || (flags & ARGS_SET_XX && !found)) {
if (!(flags & ARGS_SET_GET)) {
addReply(c, abort_reply ? abort_reply : shared.null[c->resp]);
}
goto cleanup;
@ -144,13 +129,13 @@ void setGenericCommand(client *c,
* If the key already exists, delete it. */
if (expire && checkAlreadyExpired(milliseconds)) {
if (found) deleteExpiredKeyFromOverwriteAndPropagate(c, key);
if (!(flags & OBJ_SET_GET)) addReply(c, shared.ok);
if (!(flags & ARGS_SET_GET)) addReply(c, shared.ok);
goto cleanup;
}
/* When expire is not NULL, we avoid deleting the TTL so it can be updated later instead of being deleted and then
* created again. */
setkey_flags |= ((flags & OBJ_KEEPTTL) || expire) ? SETKEY_KEEPTTL : 0;
setkey_flags |= ((flags & ARGS_KEEPTTL) || expire) ? SETKEY_KEEPTTL : 0;
setkey_flags |= found ? SETKEY_ALREADY_EXIST : SETKEY_DOESNT_EXIST;
setKey(c, c->db, key, &val, setkey_flags);
@ -158,7 +143,7 @@ void setGenericCommand(client *c,
/* By setting the reallocated value back into argv, we can avoid duplicating
* a large string value when adding it to the db. */
c->argv[(flags & OBJ_ARGV3) ? 3 : 2] = val;
c->argv[(flags & ARGS_ARGV3) ? 3 : 2] = val;
incrRefCount(val);
server.dirty++;
@ -167,7 +152,7 @@ void setGenericCommand(client *c,
if (expire) {
/* Propagate as SET Key Value PXAT millisecond-timestamp if there is
* EX/PX/EXAT flag. */
if (!(flags & OBJ_PXAT)) {
if (!(flags & ARGS_PXAT)) {
robj *milliseconds_obj = createStringObjectFromLongLong(milliseconds);
rewriteClientCommandVector(c, 5, shared.set, key, val, shared.pxat, milliseconds_obj);
decrRefCount(milliseconds_obj);
@ -175,13 +160,13 @@ void setGenericCommand(client *c,
notifyKeyspaceEvent(NOTIFY_GENERIC, "expire", key, c->db->id);
}
if (!(flags & OBJ_SET_GET)) {
if (!(flags & ARGS_SET_GET)) {
addReply(c, ok_reply ? ok_reply : shared.ok);
}
/* Propagate without the GET argument (Isn't needed if we had expire since in that case we completely re-written the
* command argv) */
if ((flags & OBJ_SET_GET) && !expire) {
if ((flags & ARGS_SET_GET) && !expire) {
int argc = 0;
int j;
robj **argv = zmalloc((c->argc - 1) * sizeof(robj *));
@ -227,7 +212,7 @@ static int getExpireMillisecondsOrReply(client *c, robj *expire, int flags, int
if (unit == UNIT_SECONDS) *milliseconds *= 1000;
if ((flags & OBJ_PX) || (flags & OBJ_EX)) {
if ((flags & ARGS_PX) || (flags & ARGS_EX)) {
*milliseconds += commandTimeSnapshot();
}
@ -240,118 +225,6 @@ static int getExpireMillisecondsOrReply(client *c, robj *expire, int flags, int
return C_OK;
}
#define COMMAND_GET 0
#define COMMAND_SET 1
/*
* The parseExtendedStringArgumentsOrReply() function performs the common validation for extended
* string arguments used in SET and GET command.
*
* Get specific commands - PERSIST/DEL
* Set specific commands - XX/NX/GET/IFEQ
* Common commands - EX/EXAT/PX/PXAT/KEEPTTL
*
* Function takes pointers to client, flags, unit, pointer to pointer of expire obj if needed
* to be determined and command_type which can be COMMAND_GET or COMMAND_SET.
*
* If there are any syntax violations C_ERR is returned else C_OK is returned.
*
* Input flags are updated upon parsing the arguments. Unit and expire are updated if there are any
* EX/EXAT/PX/PXAT arguments. Unit is updated to millisecond if PX/PXAT is set.
*/
int parseExtendedStringArgumentsOrReply(client *c, int *flags, int *unit, robj **expire, robj **compare_val, int command_type) {
int j = command_type == COMMAND_GET ? 2 : 3;
for (; j < c->argc; j++) {
char *opt = c->argv[j]->ptr;
robj *next = (j == c->argc - 1) ? NULL : c->argv[j + 1];
/* clang-format off */
if ((opt[0] == 'n' || opt[0] == 'N') &&
(opt[1] == 'x' || opt[1] == 'X') && opt[2] == '\0' &&
!(*flags & OBJ_SET_XX || *flags & OBJ_SET_IFEQ) && (command_type == COMMAND_SET))
{
*flags |= OBJ_SET_NX;
} else if ((opt[0] == 'x' || opt[0] == 'X') &&
(opt[1] == 'x' || opt[1] == 'X') && opt[2] == '\0' &&
!(*flags & OBJ_SET_NX || *flags & OBJ_SET_IFEQ) && (command_type == COMMAND_SET))
{
*flags |= OBJ_SET_XX;
} else if ((opt[0] == 'i' || opt[0] == 'I') &&
(opt[1] == 'f' || opt[1] == 'F') &&
(opt[2] == 'e' || opt[2] == 'E') &&
(opt[3] == 'q' || opt[3] == 'Q') && opt[4] == '\0' &&
next && !(*flags & OBJ_SET_NX || *flags & OBJ_SET_XX || *flags & OBJ_SET_IFEQ) && (command_type == COMMAND_SET))
{
*flags |= OBJ_SET_IFEQ;
*compare_val = next;
j++;
} else if ((opt[0] == 'g' || opt[0] == 'G') &&
(opt[1] == 'e' || opt[1] == 'E') &&
(opt[2] == 't' || opt[2] == 'T') && opt[3] == '\0' &&
(command_type == COMMAND_SET))
{
*flags |= OBJ_SET_GET;
} else if (!strcasecmp(opt, "KEEPTTL") && !(*flags & OBJ_PERSIST) &&
!(*flags & OBJ_EX) && !(*flags & OBJ_EXAT) &&
!(*flags & OBJ_PX) && !(*flags & OBJ_PXAT) && (command_type == COMMAND_SET))
{
*flags |= OBJ_KEEPTTL;
} else if (!strcasecmp(opt,"PERSIST") && (command_type == COMMAND_GET) &&
!(*flags & OBJ_EX) && !(*flags & OBJ_EXAT) &&
!(*flags & OBJ_PX) && !(*flags & OBJ_PXAT) &&
!(*flags & OBJ_KEEPTTL))
{
*flags |= OBJ_PERSIST;
} else if ((opt[0] == 'e' || opt[0] == 'E') &&
(opt[1] == 'x' || opt[1] == 'X') && opt[2] == '\0' &&
!(*flags & OBJ_KEEPTTL) && !(*flags & OBJ_PERSIST) &&
!(*flags & OBJ_EXAT) && !(*flags & OBJ_PX) &&
!(*flags & OBJ_PXAT) && next)
{
*flags |= OBJ_EX;
*expire = next;
j++;
} else if ((opt[0] == 'p' || opt[0] == 'P') &&
(opt[1] == 'x' || opt[1] == 'X') && opt[2] == '\0' &&
!(*flags & OBJ_KEEPTTL) && !(*flags & OBJ_PERSIST) &&
!(*flags & OBJ_EX) && !(*flags & OBJ_EXAT) &&
!(*flags & OBJ_PXAT) && next)
{
*flags |= OBJ_PX;
*unit = UNIT_MILLISECONDS;
*expire = next;
j++;
} else if ((opt[0] == 'e' || opt[0] == 'E') &&
(opt[1] == 'x' || opt[1] == 'X') &&
(opt[2] == 'a' || opt[2] == 'A') &&
(opt[3] == 't' || opt[3] == 'T') && opt[4] == '\0' &&
!(*flags & OBJ_KEEPTTL) && !(*flags & OBJ_PERSIST) &&
!(*flags & OBJ_EX) && !(*flags & OBJ_PX) &&
!(*flags & OBJ_PXAT) && next)
{
*flags |= OBJ_EXAT;
*expire = next;
j++;
} else if ((opt[0] == 'p' || opt[0] == 'P') &&
(opt[1] == 'x' || opt[1] == 'X') &&
(opt[2] == 'a' || opt[2] == 'A') &&
(opt[3] == 't' || opt[3] == 'T') && opt[4] == '\0' &&
!(*flags & OBJ_KEEPTTL) && !(*flags & OBJ_PERSIST) &&
!(*flags & OBJ_EX) && !(*flags & OBJ_EXAT) &&
!(*flags & OBJ_PX) && next)
{
*flags |= OBJ_PXAT;
*unit = UNIT_MILLISECONDS;
*expire = next;
j++;
} else {
addReplyErrorObject(c,shared.syntaxerr);
return C_ERR;
}
/* clang-format on */
}
return C_OK;
}
/* SET key value [NX | XX | IFEQ comparison-value] [GET]
* [EX seconds | PX milliseconds |
* EXAT seconds-timestamp | PXAT milliseconds-timestamp | KEEPTTL] */
@ -359,9 +232,9 @@ void setCommand(client *c) {
robj *expire = NULL;
robj *comparison = NULL;
int unit = UNIT_SECONDS;
int flags = OBJ_NO_FLAGS;
int flags = ARGS_NO_FLAGS;
if (parseExtendedStringArgumentsOrReply(c, &flags, &unit, &expire, &comparison, COMMAND_SET) != C_OK) {
if (parseExtendedCommandArgumentsOrReply(c, &flags, &unit, &expire, &comparison, COMMAND_SET, c->argc) != C_OK) {
return;
}
@ -371,17 +244,17 @@ void setCommand(client *c) {
void setnxCommand(client *c) {
c->argv[2] = tryObjectEncoding(c->argv[2]);
setGenericCommand(c, OBJ_SET_NX, c->argv[1], c->argv[2], NULL, 0, shared.cone, shared.czero, NULL);
setGenericCommand(c, ARGS_SET_NX, c->argv[1], c->argv[2], NULL, 0, shared.cone, shared.czero, NULL);
}
void setexCommand(client *c) {
c->argv[3] = tryObjectEncoding(c->argv[3]);
setGenericCommand(c, OBJ_EX | OBJ_ARGV3, c->argv[1], c->argv[3], c->argv[2], UNIT_SECONDS, NULL, NULL, NULL);
setGenericCommand(c, ARGS_EX | ARGS_ARGV3, c->argv[1], c->argv[3], c->argv[2], UNIT_SECONDS, NULL, NULL, NULL);
}
void psetexCommand(client *c) {
c->argv[3] = tryObjectEncoding(c->argv[3]);
setGenericCommand(c, OBJ_PX | OBJ_ARGV3, c->argv[1], c->argv[3], c->argv[2], UNIT_MILLISECONDS, NULL, NULL, NULL);
setGenericCommand(c, ARGS_PX | ARGS_ARGV3, c->argv[1], c->argv[3], c->argv[2], UNIT_MILLISECONDS, NULL, NULL, NULL);
}
/* DELIFEQ key value */
@ -445,9 +318,9 @@ void getCommand(client *c) {
void getexCommand(client *c) {
robj *expire = NULL;
int unit = UNIT_SECONDS;
int flags = OBJ_NO_FLAGS;
int flags = ARGS_NO_FLAGS;
if (parseExtendedStringArgumentsOrReply(c, &flags, &unit, &expire, NULL, COMMAND_GET) != C_OK) {
if (parseExtendedCommandArgumentsOrReply(c, &flags, &unit, &expire, NULL, COMMAND_GET, c->argc) != C_OK) {
return;
}
@ -472,7 +345,7 @@ void getexCommand(client *c) {
/* This command is never propagated as is. It is either propagated as PEXPIRE[AT],DEL,UNLINK or PERSIST.
* This why it doesn't need special handling in feedAppendOnlyFile to convert relative expire time to absolute one. */
if (((flags & OBJ_PXAT) || (flags & OBJ_EXAT)) && checkAlreadyExpired(milliseconds)) {
if (((flags & ARGS_PXAT) || (flags & ARGS_EXAT)) && checkAlreadyExpired(milliseconds)) {
/* When PXAT/EXAT absolute timestamp is specified, there can be a chance that timestamp
* has already elapsed so delete the key in that case. */
deleteExpiredKeyFromOverwriteAndPropagate(c, c->argv[1]);
@ -486,7 +359,7 @@ void getexCommand(client *c) {
signalModifiedKey(c, c->db, c->argv[1]);
notifyKeyspaceEvent(NOTIFY_GENERIC, "expire", c->argv[1], c->db->id);
server.dirty++;
} else if (flags & OBJ_PERSIST) {
} else if (flags & ARGS_PERSIST) {
if (removeExpire(c->db, c->argv[1])) {
signalModifiedKey(c, c->db, c->argv[1]);
rewriteClientCommandVector(c, 2, shared.persist, c->argv[1]);

471
src/unit/test_entry.c Normal file
View File

@ -0,0 +1,471 @@
#include "../entry.h"
#include "test_help.h"
#include "../expire.h"
#include "../monotonic.h"
#include "../server.h"
#include <stdio.h>
#include <limits.h>
#include <string.h>
#include <assert.h>
#include <math.h>
/* Constants for test values */
#define SHORT_FIELD "foo"
#define SHORT_VALUE "bar"
#define LONG_FIELD "k:123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890"
#define LONG_VALUE "v:12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890"
/* Verify entry properties */
static int verify_entry_properties(entry *e, sds field, sds value_copy, long long expiry, bool has_expiry, bool has_valueptr) {
TEST_ASSERT(sdscmp(entryGetField(e), field) == 0);
TEST_ASSERT(sdscmp(entryGetValue(e), value_copy) == 0);
TEST_ASSERT(entryGetExpiry(e) == expiry);
TEST_ASSERT(entryHasExpiry(e) == has_expiry);
TEST_ASSERT(entryHasEmbeddedValue(e) != has_valueptr);
return 0;
}
/**
* Test entryCreate functunallity:
* 1. embedded with expiry
* 2. embedded without expiry
* 3. non-embedded with expiry
* 4. non-embedded without expiry
*/
int test_entryCreate(int argc, char **argv, int flags) {
UNUSED(argc);
UNUSED(argv);
UNUSED(flags);
// Test with embedded value with expiry
sds field1 = sdsnew(SHORT_FIELD);
sds value1 = sdsnew(SHORT_VALUE);
sds value_copy1 = sdsdup(value1); // Keep a copy since entryCreate takes ownership of value
long long expiry1 = 100;
entry *e1 = entryCreate(field1, value1, expiry1);
verify_entry_properties(e1, field1, value_copy1, expiry1, true, false);
// Test with embedded value with no expiry
sds field2 = sdsnew(SHORT_FIELD);
sds value2 = sdsnew(SHORT_VALUE);
sds value_copy2 = sdsdup(value2);
long long expiry2 = EXPIRY_NONE;
entry *e2 = entryCreate(field2, value2, expiry2);
verify_entry_properties(e2, field2, value_copy2, expiry2, false, false);
// Test with non-embedded field and value with expiry
sds field3 = sdsnew(LONG_FIELD);
sds value3 = sdsnew(LONG_VALUE);
sds value_copy3 = sdsdup(value3);
long long expiry3 = 100;
entry *e3 = entryCreate(field3, value3, expiry3);
verify_entry_properties(e3, field3, value_copy3, expiry3, true, true);
// Test with non-embedded field and value with no expiry
sds field4 = sdsnew(LONG_FIELD);
sds value4 = sdsnew(LONG_VALUE);
sds value_copy4 = sdsdup(value4);
long long expiry4 = EXPIRY_NONE;
entry *e4 = entryCreate(field4, value4, expiry4);
verify_entry_properties(e4, field4, value_copy4, expiry4, false, true);
entryFree(e1);
entryFree(e2);
entryFree(e3);
entryFree(e4);
// Free field as entryCreate doesn't take ownership
sdsfree(field1);
sdsfree(field2);
sdsfree(field3);
sdsfree(field4);
sdsfree(value_copy1);
sdsfree(value_copy2);
sdsfree(value_copy3);
sdsfree(value_copy4);
return 0;
}
/**
* Test entryUpdate with various combinations of value and expiry changes:
* 1. Update only the value (keeping embedded)
* 2. Update only the expiry (keeping embedded)
* 3. Update both value and expiry (keeping embedded)
* 4. Update with no changes (should return same entry)
* 5. Update to a value that's too large to be embedded
* 6. Update expiry of a non-embedded entry
* 7. Update from non-embedded back to embedded value
* 8. Update entry to less then 3/4 allocation size
* 9. Update entry to more than 3/4 allocation size
* 8. Update entry to exactly 3/4 allocation size
*/
int test_entryUpdate(int argc, char **argv, int flags) {
UNUSED(argc);
UNUSED(argv);
UNUSED(flags);
// Create embedded entry
sds value1 = sdsnew(SHORT_VALUE);
sds field = sdsnew(SHORT_FIELD);
sds value_copy1 = sdsdup(value1);
long long expiry1 = 100;
entry *e1 = entryCreate(field, value1, expiry1);
verify_entry_properties(e1, field, value_copy1, expiry1, true, false);
// Update only value (keeping embedded)
sds value2 = sdsnew("bar2");
sds value_copy2 = sdsdup(value2);
long long expiry2 = expiry1;
entry *e2 = entryUpdate(e1, value2, expiry2);
verify_entry_properties(e2, field, value_copy2, expiry2, true, false);
// Update only expiry (keeping embedded)
long long expiry3 = 200;
entry *e3 = entryUpdate(e2, NULL, expiry3);
verify_entry_properties(e3, field, value_copy2, expiry3, true, false);
// Update both value and expiry (keeping embedded)
sds value4 = sdsnew("bar4");
long long expiry4 = 300;
sds value_copy4 = sdsdup(value4);
entry *e4 = entryUpdate(e3, value4, expiry4);
verify_entry_properties(e4, field, value_copy4, expiry4, true, false);
// Update with no changes (should return same entry)
entry *e5 = entryUpdate(e4, NULL, expiry4);
verify_entry_properties(e5, field, value_copy4, expiry4, true, false);
TEST_ASSERT(e5 == e4);
// Update to a value that's too large to be embedded
sds value6 = sdsnew(LONG_VALUE);
sds value_copy6 = sdsdup(value6);
long long expiry6 = expiry4;
entry *e6 = entryUpdate(e5, value6, expiry6);
verify_entry_properties(e6, field, value_copy6, expiry6, true, true);
// Update expiry of a non-embedded entry
long long expiry7 = 400;
entry *e7 = entryUpdate(e6, NULL, expiry7);
verify_entry_properties(e7, field, value_copy6, expiry7, true, true);
// Update from non-embedded back to embedded value
sds value8 = sdsnew("bar8");
sds value_copy8 = sdsdup(value8);
long long expiry8 = expiry7;
entry *e8 = entryUpdate(e7, value8, expiry8);
verify_entry_properties(e8, field, value_copy8, expiry8, true, false);
// Update value with identical value (keeping embedded)
sds value9 = sdsnew("bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb");
sds value_copy9 = sdsdup(value9);
long long expiry9 = expiry8;
entry *e9 = entryUpdate(e8, value9, expiry9);
verify_entry_properties(e9, field, value_copy9, expiry9, true, false);
// Update the value so that memory usage is less than 3/4 of the current allocation size
// Ensuring required_embedded_size < current_embedded_allocation_size * 3 / 4, which creates a new entry
size_t current_embedded_allocation_size = entryMemUsage(e9);
sds value10 = sdsnew("xxxxxxxxxxxxxxxxxxxxx");
sds value_copy10 = sdsdup(value10);
long long expiry10 = expiry9;
entry *e10 = entryUpdate(e9, value10, expiry10);
verify_entry_properties(e10, field, value_copy10, expiry10, true, false);
TEST_ASSERT(entryMemUsage(e10) < current_embedded_allocation_size * 3 / 4);
TEST_ASSERT(e10 != e9);
// Update the value so that memory usage is at least 3/4 of the current memory usage
// Ensuring required_embedded_size > current_embedded_allocation_size * 3 / 4 without creating a new entry
current_embedded_allocation_size = entryMemUsage(e10);
sds value11 = sdsnew("yyyyyyyyyyyyy");
sds value_copy11 = sdsdup(value11);
long long expiry11 = expiry10;
entry *e11 = entryUpdate(e10, value11, expiry11);
verify_entry_properties(e11, field, value_copy11, expiry11, true, false);
TEST_ASSERT(entryMemUsage(e11) >= current_embedded_allocation_size * 3 / 4);
TEST_ASSERT(entryMemUsage(e11) <= current_embedded_allocation_size);
TEST_ASSERT(entryMemUsage(e11) <=
EMBED_VALUE_MAX_ALLOC_SIZE);
TEST_ASSERT(e10 == e11);
// Update the value so that memory usage is exactly equal to the current allocation size
// Ensuring required_embedded_size == current_embedded_allocation_size without creating a new entry
current_embedded_allocation_size = entryMemUsage(e11);
sds value12 = sdsnew("zzzzzzzzzzzzz");
sds value_copy12 = sdsdup(value12);
long long expiry12 = expiry11;
entry *e12 = entryUpdate(e11, value12, expiry12);
verify_entry_properties(e11, field, value_copy12, expiry12, true, false);
TEST_ASSERT(entryMemUsage(e12) == current_embedded_allocation_size);
TEST_ASSERT(entryMemUsage(e12) <= EMBED_VALUE_MAX_ALLOC_SIZE);
TEST_ASSERT(e12 == e11);
entryFree(e12);
sdsfree(field);
sdsfree(value_copy1);
sdsfree(value_copy2);
sdsfree(value_copy4);
sdsfree(value_copy6);
sdsfree(value_copy8);
sdsfree(value_copy9);
sdsfree(value_copy10);
sdsfree(value_copy11);
sdsfree(value_copy12);
return 0;
}
/**
* Test setting expiry on an entry:
* 1. No expiry
* 2. Set expiry on entry without expiry
* 3. Update expiry on entry with expiry
* 4. Test with non-embedded entry
* 5. Set expiry on non-embedded entry
*/
int test_entryHasexpiry_entrySetExpiry(int argc, char **argv, int flags) {
UNUSED(argc);
UNUSED(argv);
UNUSED(flags);
// No expiry
sds field1 = sdsnew(SHORT_FIELD);
sds value1 = sdsnew(SHORT_VALUE);
entry *e1 = entryCreate(field1, value1, EXPIRY_NONE);
TEST_ASSERT(entryHasExpiry(e1) == false);
TEST_ASSERT(entryGetExpiry(e1) == EXPIRY_NONE);
// Set expiry on entry without expiry
long long expiry2 = 100;
entry *e2 = entrySetExpiry(e1, expiry2);
TEST_ASSERT(entryHasExpiry(e2) == true);
TEST_ASSERT(entryGetExpiry(e2) == expiry2);
// Update expiry on entry with expiry
long long expiry3 = 200;
entry *e3 = entrySetExpiry(e2, expiry3);
TEST_ASSERT(entryHasExpiry(e3) == true);
TEST_ASSERT(entryGetExpiry(e3) == expiry3);
TEST_ASSERT(e2 == e3); // Should be the same pointer when just updating expiry
// Test with non-embedded entry
sds field4 = sdsnew(LONG_FIELD);
sds value4 = sdsnew(LONG_VALUE);
entry *e4 = entryCreate(field4, value4, EXPIRY_NONE);
TEST_ASSERT(entryHasExpiry(e4) == false);
TEST_ASSERT(entryHasEmbeddedValue(e4) == false);
// Set expiry on entry without expiry
long long expiry5 = 100;
entry *e5 = entrySetExpiry(e4, expiry5);
TEST_ASSERT(entryHasExpiry(e5) == true);
TEST_ASSERT(entryGetExpiry(e5) == expiry5);
// Update expiry on entry with expiry
long long expiry6 = 200;
entry *e6 = entrySetExpiry(e5, expiry6);
TEST_ASSERT(entryHasExpiry(e6) == true);
TEST_ASSERT(entryGetExpiry(e6) == expiry6);
TEST_ASSERT(e5 == e6); // Should be the same pointer when just updating expiry
entryFree(e3);
entryFree(e6);
sdsfree(field1);
sdsfree(field4);
return 0;
}
/**
* Test entryIsExpired:
* 1. No expiry
* 2. Future expiry
* 3. Current time expiry
* 4. Past expiry
* 5. Test with loading mode
* 6. Test with import mode and import source client
* 7. Test with import mode and import source client and import expiry
* 8. Test with import mode and import source client and import expiry and import expiry is in the past
*/
int test_entryIsExpired(int argc, char **argv, int flags) {
UNUSED(argc);
UNUSED(argv);
UNUSED(flags);
// Setup server state
enterExecutionUnit(1, ustime());
long long current_time = commandTimeSnapshot();
// No expiry
sds field1 = sdsnew(SHORT_FIELD);
sds value1 = sdsnew(SHORT_VALUE);
entry *e1 = entryCreate(field1, value1, EXPIRY_NONE);
TEST_ASSERT(entryGetExpiry(e1) == EXPIRY_NONE);
TEST_ASSERT(entryIsExpired(e1) == false);
// Future expiry
sds field2 = sdsnew(SHORT_FIELD);
sds value2 = sdsnew(SHORT_VALUE);
long long future_time = current_time + 10000; // 10 seconds in future
entry *e2 = entryCreate(field2, value2, future_time);
TEST_ASSERT(entryGetExpiry(e2) == future_time);
TEST_ASSERT(entryIsExpired(e2) == false);
// Current time expiry
sds field3 = sdsnew(SHORT_FIELD);
sds value3 = sdsnew(SHORT_VALUE);
entry *e3 = entryCreate(field3, value3, current_time);
TEST_ASSERT(entryGetExpiry(e3) == current_time);
TEST_ASSERT(entryIsExpired(e3) == false);
// Test with past expiry
sds field4 = sdsnew(SHORT_FIELD);
sds value4 = sdsnew(SHORT_VALUE);
long long past_time = current_time - 10000; // 10 seconds ago
entry *e4 = entryCreate(field4, value4, past_time);
TEST_ASSERT(entryGetExpiry(e4) == past_time);
TEST_ASSERT(entryIsExpired(e4) == true);
entryFree(e1);
entryFree(e2);
entryFree(e3);
entryFree(e4);
sdsfree(field1);
sdsfree(field2);
sdsfree(field3);
sdsfree(field4);
exitExecutionUnit();
return 0;
}
/**
* Test entryMemUsage:
* 1. Embedded entry tests:
* - Initial creation without expiry
* - Adding expiry (should increase memory usage)
* - Updating expiry (should not change memory usage)
* - Updating value while keeping it embedded:
* * To smaller value (should not decrease memory usage)
* * To bigger value (should not increase memory usage)
*
* 2. Non-embedded entry tests:
* - Initial creation without expiry
* - Adding expiry (should increase memory usage)
* - Updating expiry (should not change memory usage)
* - Updating value:
* * To smaller value (should decrease memory usage)
* * To bigger value (should increase memory usage)
*/
int test_entryMemUsage_entrySetExpiry_entrySetValue(int argc, char **argv, int flags) {
UNUSED(argc);
UNUSED(argv);
UNUSED(flags);
// Tests with embedded entry
// Embedded entry without expiry
sds field1 = sdsnew(SHORT_FIELD);
sds value1 = sdsnew(SHORT_VALUE);
sds value_copy1 = sdsdup(value1);
long long expiry1 = EXPIRY_NONE;
entry *e1 = entryCreate(field1, value1, expiry1);
size_t e1_entryMemUsage = entryMemUsage(e1);
verify_entry_properties(e1, field1, value_copy1, expiry1, false, false);
TEST_ASSERT(e1_entryMemUsage > 0);
// Add expiry to embedded entry without expiry
// This should increase memory usage by sizeof(long long) + 2 bytes
// (long long for the expiry value, 2 bytes for SDS header adjustment)
long long expiry2 = 100;
entry *e2 = entrySetExpiry(e1, expiry2);
size_t e2_entryMemUsage = entryMemUsage(e2);
verify_entry_properties(e2, field1, value_copy1, expiry2, true, false);
TEST_ASSERT(zmalloc_usable_size((char *)e2 - sizeof(long long) - 3) == e2_entryMemUsage);
// Update expiry on an entry that already has one
// This should NOT change memory usage as we're just updating the expiry value (long long)
long long expiry3 = 10000;
entry *e3 = entrySetExpiry(e2, expiry3);
size_t e3_entryMemUsage = entryMemUsage(e3);
verify_entry_properties(e3, field1, value_copy1, expiry3, true, false);
TEST_ASSERT(e3_entryMemUsage == e2_entryMemUsage);
// Update to smaller value (keeping embedded)
// Memory usage should decrease by the difference in value size (2 bytes)
sds value4 = sdsnew("x");
sds value_copy4 = sdsdup(value4);
entry *e4 = entrySetValue(e3, value4);
size_t e4_entryMemUsage = entryMemUsage(e4);
verify_entry_properties(e4, field1, value_copy4, expiry3, true, false);
TEST_ASSERT(zmalloc_usable_size((char *)e4 - sizeof(long long) - 3) == e4_entryMemUsage);
// Update to bigger value (keeping embedded)
// Memory usage should increase by the difference in value size (1 byte)
sds value5 = sdsnew("xx");
sds value_copy5 = sdsdup(value5);
entry *e5 = entrySetValue(e4, value5);
size_t e5_entryMemUsage = entryMemUsage(e5);
verify_entry_properties(e5, field1, value_copy5, expiry3, true, false);
TEST_ASSERT(zmalloc_usable_size((char *)e5 - sizeof(long long) - 3) == e5_entryMemUsage);
// Tests with non-embedded entry
// Non-embedded entry without expiry
sds field6 = sdsnew(LONG_FIELD);
field6 = sdscat(field6, LONG_FIELD); // Double the length to ensure non-embedded entry
sds value6 = sdsnew(LONG_VALUE);
sds value_copy6 = sdsdup(value6);
long long expiry6 = EXPIRY_NONE;
entry *e6 = entryCreate(field6, value6, EXPIRY_NONE);
size_t e6_entryMemUsage = entryMemUsage(e6);
verify_entry_properties(e6, field6, value_copy6, expiry6, false, true);
TEST_ASSERT(e6_entryMemUsage > 0);
// Add expiry to non-embedded entry without expiry
// For non-embedded entries this increases memory by exactly sizeof(long long)
long long expiry7 = 100;
entry *e7 = entrySetExpiry(e6, expiry7);
size_t e7_entryMemUsage = entryMemUsage(e7);
verify_entry_properties(e7, field6, value_copy6, expiry7, true, true);
size_t expected_e7_entry_mem = zmalloc_usable_size((char *)e7 - sizeof(long long) - sizeof(sds) - 3) + sdsAllocSize(value6);
TEST_ASSERT(expected_e7_entry_mem == e7_entryMemUsage);
// Update expiry on a non-embedded entry that already has one
// This should not change memory usage as we're just updating the expiry value
long long expiry8 = 10000;
entry *e8 = entrySetExpiry(e7, expiry8);
size_t e8_entryMemUsage = entryMemUsage(e8);
verify_entry_properties(e8, field6, value_copy6, expiry8, true, true);
TEST_ASSERT(e8_entryMemUsage == e7_entryMemUsage);
// Update to smaller value (keeping non-embedded)
// Memory usage should increase by at least the difference between LONG_VALUE and "x" (143)
sds value9 = sdsnew("x");
sds value_copy9 = sdsdup(value9);
entry *e9 = entrySetValue(e8, value9);
size_t e9_entryMemUsage = entryMemUsage(e9);
verify_entry_properties(e9, field6, value_copy9, expiry8, true, true);
size_t expected_e9_entry_mem = zmalloc_usable_size((char *)e9 - sizeof(long long) - sizeof(sds) - 3) + sdsAllocSize(value9);
TEST_ASSERT(expected_e9_entry_mem == e9_entryMemUsage);
// Update to bigger value (keeping non-embedded)
// Memory usage increases by the difference in value size (1 byte)
sds value10 = sdsnew("xx");
sds value_copy10 = sdsdup(value10);
entry *e10 = entrySetValue(e9, value10);
size_t e10_entryMemUsage = entryMemUsage(e10);
size_t expected_10_entry_mem = zmalloc_usable_size((char *)e10 - sizeof(long long) - sizeof(sds) - 3) + sdsAllocSize(value10);
TEST_ASSERT(expected_10_entry_mem == e10_entryMemUsage);
entryFree(e5);
entryFree(e10);
sdsfree(field1);
sdsfree(field6);
sdsfree(value_copy1);
sdsfree(value_copy4);
sdsfree(value_copy5);
sdsfree(value_copy6);
sdsfree(value_copy9);
sdsfree(value_copy10);
return 0;
}

7
src/unit/test_files.h
View File

@ -20,6 +20,11 @@ int test_dictDisableResizeReduceTo3(int argc, char **argv, int flags);
int test_dictDeleteOneKeyTriggerResizeAgain(int argc, char **argv, int flags);
int test_dictBenchmark(int argc, char **argv, int flags);
int test_endianconv(int argc, char *argv[], int flags);
int test_entryCreate(int argc, char **argv, int flags);
int test_entryUpdate(int argc, char **argv, int flags);
int test_entryHasexpiry_entrySetExpiry(int argc, char **argv, int flags);
int test_entryIsExpired(int argc, char **argv, int flags);
int test_entryMemUsage_entrySetExpiry_entrySetValue(int argc, char **argv, int flags);
int test_cursor(int argc, char **argv, int flags);
int test_set_hash_function_seed(int argc, char **argv, int flags);
int test_add_find_delete(int argc, char **argv, int flags);
@ -242,6 +247,7 @@ unitTest __test_crc64_c[] = {{"test_crc64", test_crc64}, {NULL, NULL}};
unitTest __test_crc64combine_c[] = {{"test_crc64combine", test_crc64combine}, {NULL, NULL}};
unitTest __test_dict_c[] = {{"test_dictCreate", test_dictCreate}, {"test_dictAdd16Keys", test_dictAdd16Keys}, {"test_dictDisableResize", test_dictDisableResize}, {"test_dictAddOneKeyTriggerResize", test_dictAddOneKeyTriggerResize}, {"test_dictDeleteKeys", test_dictDeleteKeys}, {"test_dictDeleteOneKeyTriggerResize", test_dictDeleteOneKeyTriggerResize}, {"test_dictEmptyDirAdd128Keys", test_dictEmptyDirAdd128Keys}, {"test_dictDisableResizeReduceTo3", test_dictDisableResizeReduceTo3}, {"test_dictDeleteOneKeyTriggerResizeAgain", test_dictDeleteOneKeyTriggerResizeAgain}, {"test_dictBenchmark", test_dictBenchmark}, {NULL, NULL}};
unitTest __test_endianconv_c[] = {{"test_endianconv", test_endianconv}, {NULL, NULL}};
unitTest __test_entry_c[] = {{"test_entryCreate", test_entryCreate}, {"test_entryUpdate", test_entryUpdate}, {"test_entryHasexpiry_entrySetExpiry", test_entryHasexpiry_entrySetExpiry}, {"test_entryIsExpired", test_entryIsExpired}, {"test_entryMemUsage_entrySetExpiry_entrySetValue", test_entryMemUsage_entrySetExpiry_entrySetValue}, {NULL, NULL}};
unitTest __test_hashtable_c[] = {{"test_cursor", test_cursor}, {"test_set_hash_function_seed", test_set_hash_function_seed}, {"test_add_find_delete", test_add_find_delete}, {"test_add_find_delete_avoid_resize", test_add_find_delete_avoid_resize}, {"test_instant_rehashing", test_instant_rehashing}, {"test_bucket_chain_length", test_bucket_chain_length}, {"test_two_phase_insert_and_pop", test_two_phase_insert_and_pop}, {"test_replace_reallocated_entry", test_replace_reallocated_entry}, {"test_incremental_find", test_incremental_find}, {"test_scan", test_scan}, {"test_iterator", test_iterator}, {"test_safe_iterator", test_safe_iterator}, {"test_compact_bucket_chain", test_compact_bucket_chain}, {"test_random_entry", test_random_entry}, {"test_random_entry_with_long_chain", test_random_entry_with_long_chain}, {"test_random_entry_sparse_table", test_random_entry_sparse_table}, {NULL, NULL}};
unitTest __test_intset_c[] = {{"test_intsetValueEncodings", test_intsetValueEncodings}, {"test_intsetBasicAdding", test_intsetBasicAdding}, {"test_intsetLargeNumberRandomAdd", test_intsetLargeNumberRandomAdd}, {"test_intsetUpgradeFromint16Toint32", test_intsetUpgradeFromint16Toint32}, {"test_intsetUpgradeFromint16Toint64", test_intsetUpgradeFromint16Toint64}, {"test_intsetUpgradeFromint32Toint64", test_intsetUpgradeFromint32Toint64}, {"test_intsetStressLookups", test_intsetStressLookups}, {"test_intsetStressAddDelete", test_intsetStressAddDelete}, {NULL, NULL}};
unitTest __test_kvstore_c[] = {{"test_kvstoreAdd16Keys", test_kvstoreAdd16Keys}, {"test_kvstoreIteratorRemoveAllKeysNoDeleteEmptyHashtable", test_kvstoreIteratorRemoveAllKeysNoDeleteEmptyHashtable}, {"test_kvstoreIteratorRemoveAllKeysDeleteEmptyHashtable", test_kvstoreIteratorRemoveAllKeysDeleteEmptyHashtable}, {"test_kvstoreHashtableIteratorRemoveAllKeysNoDeleteEmptyHashtable", test_kvstoreHashtableIteratorRemoveAllKeysNoDeleteEmptyHashtable}, {"test_kvstoreHashtableIteratorRemoveAllKeysDeleteEmptyHashtable", test_kvstoreHashtableIteratorRemoveAllKeysDeleteEmptyHashtable}, {NULL, NULL}};
@ -268,6 +274,7 @@ struct unitTestSuite {
{"test_crc64combine.c", __test_crc64combine_c},
{"test_dict.c", __test_dict_c},
{"test_endianconv.c", __test_endianconv_c},
{"test_entry.c", __test_entry_c},
{"test_hashtable.c", __test_hashtable_c},
{"test_intset.c", __test_intset_c},
{"test_kvstore.c", __test_kvstore_c},

2
src/util.c
View File

@ -59,8 +59,6 @@
#include <immintrin.h>
#endif
#define UNUSED(x) ((void)(x))
/* Glob-style pattern matching. */
static int stringmatchlen_impl(const char *pattern,
int patternLen,

11
src/util.h
View File

@ -33,6 +33,17 @@
#include <stdint.h>
#include "sds.h"
/* Anti-warning macro... */
#ifndef UNUSED
#define UNUSED(V) ((void)V)
#endif
/* min/max */
#undef min
#undef max
#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))
/* The maximum number of characters needed to represent a long double
* as a string (long double has a huge range of some 4952 chars, see LDBL_MAX).
* This should be the size of the buffer given to ld2string */

3
src/valkey-check-rdb.c
View File

@ -146,8 +146,11 @@ char *rdb_type_string[] = {
"stream-v2",
"set-listpack",
"stream-v3",
"hash-volatile-items",
};
static_assert(sizeof(rdb_type_string) / sizeof(rdb_type_string[0]) == RDB_TYPE_LAST, "Mismatch between enum and string table");
char *type_name[OBJ_TYPE_MAX] = {"string", "list", "set", "zset", "hash", "module", /* module type is special */
"stream"};

79
src/volatile_set.c Normal file
View File

@ -0,0 +1,79 @@
#include <string.h>
#include "volatile_set.h"
#include "zmalloc.h"
#include "config.h"
#include "endianconv.h"
#include "serverassert.h"
#define EXPIRY_HASH_SIZE 16
volatile_set *createVolatileSet(volatileEntryType *type) {
volatile_set *set = zmalloc(sizeof(volatile_set));
set->etypr = type;
set->expiry_buckets = raxNew();
return set;
}
void freeVolatileSet(volatile_set *b) {
raxFree(b->expiry_buckets);
zfree(b);
}
int volatileSetAddEntry(volatile_set *set, void *entry, long long expiry) {
unsigned char buf[EXPIRY_HASH_SIZE];
expiry = htonu64(expiry);
memcpy(buf, &expiry, sizeof(expiry));
memcpy(buf + 8, &entry, sizeof(entry));
if (sizeof(entry) == 4) memset(buf + 12, 0, 4); /* Zero padding for 32bit target. */
return raxTryInsert(set->expiry_buckets, buf, sizeof(buf), NULL, NULL);
}
int volatileSetRemoveEntry(volatile_set *set, void *entry, long long expiry) {
unsigned char buf[EXPIRY_HASH_SIZE];
expiry = htonu64(expiry);
memcpy(buf, &expiry, sizeof(expiry));
memcpy(buf + 8, &entry, sizeof(entry));
if (sizeof(entry) == 4) memset(buf + 12, 0, 4); /* Zero padding for 32bit target. */
return raxRemove(set->expiry_buckets, buf, sizeof(buf), NULL);
}
int volatileSetUpdateEntry(volatile_set *set, void *old_entry, void *new_entry, long long old_expiry, long long new_expiry) {
if (old_entry == new_entry && old_expiry == new_expiry) return 1;
if (old_entry && old_expiry != -1) {
assert(volatileSetRemoveEntry(set, old_entry, old_expiry));
}
if (new_entry && new_expiry != -1) {
assert(volatileSetAddEntry(set, new_entry, new_expiry));
}
return 1;
}
int volatileSetExpireEntry(volatile_set *set, void *entry) {
volatileSetRemoveEntry(set, entry, set->etypr->getExpiry(entry));
if (set->etypr->expire) {
set->etypr->expire(entry);
return 1;
}
return 0;
}
size_t volatileSetNumEntries(volatile_set *set) {
assert(set && set->expiry_buckets);
return set->expiry_buckets->numele;
}
void volatileSetStart(volatile_set *set, volatileSetIterator *it) {
raxStart(&it->bucket, set->expiry_buckets);
}
int volatileSetNext(volatileSetIterator *it, void **entryptr) {
if (raxNext(&it->bucket)) {
assert(it->bucket.key_len == EXPIRY_HASH_SIZE);
memcpy(entryptr, it->bucket.key + sizeof(long long), sizeof(*entryptr));
return 1;
}
return 0;
}
void volatileSetReset(volatileSetIterator *it) {
raxStop(&it->bucket);
}

40
src/volatile_set.h Normal file
View File

@ -0,0 +1,40 @@
#ifndef VOLATILESET_H
#define VOLATILESET_H
#include <stddef.h>
#include "rax.h"
#include "sds.h"
typedef struct {
sds (*entryGetKey)(const void *entry);
long long (*getExpiry)(const void *entry);
int (*expire)(void *entry);
} volatileEntryType;
typedef struct {
volatileEntryType *etypr;
rax *expiry_buckets;
} volatile_set;
typedef struct volatileSetIterator {
raxIterator bucket;
} volatileSetIterator;
int volatileSetRemoveEntry(volatile_set *set, void *entry, long long expiry);
int volatileSetAddEntry(volatile_set *set, void *entry, long long expiry);
int volatileSetExpireEntry(volatile_set *set, void *entry);
int volatileSetUpdateEntry(volatile_set *set, void *old_entry, void *new_entry, long long old_expiry, long long new_expiry);
size_t volatileSetNumEntries(volatile_set *set);
void volatileSetStart(volatile_set *set, volatileSetIterator *it);
int volatileSetNext(volatileSetIterator *it, void **entryptr);
void volatileSetReset(volatileSetIterator *it);
void freeVolatileSet(volatile_set *b);
volatile_set *createVolatileSet(volatileEntryType *type);
#endif

2639
tests/unit/hashexpire.tcl Normal file
View File

File diff suppressed because it is too large Load Diff