* chore: add active time to stream consumers
Adjust XINFO command to output active-time property.
Store active-time and switch to RDB_TYPE_STREAM_LISTPACKS_3 if FLAGS_stream_rdb_encode_v2
is enabled.
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Now rdb_load supports RDB_TYPE_STREAM_LISTPACKS, RDB_TYPE_STREAM_LISTPACKS_2 and RDB_TYPE_STREAM_LISTPACKS_3 formats.
rdb_save still saves with RDB_TYPE_STREAM_LISTPACKS format - we want to release the DF version that can load everything first, and
then update the replicaion format in the next versions.
Also, update rdb_test.cc
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: let resp parser provide more useful logs
1. More warning logs around bad BAD_ARRAYLEN messages
2. Lift the restriction around big bulk strings and log a warning instead.
3. Pull helio
Probably fixes#4213
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* remove DbSlice mutex
* add ConditionFlag in SliceSnapshot
* disable compression when big value serialization is on
* add metrics
---------
Signed-off-by: kostas <kostas@dragonflydb.io>
The bug:
calling lua_error does not return, instead it unwinds the Lua call stack until an error handler is found or the
script exits. This lead to memory leak on object that should release memory in destructor.
Specific example is the absl::FixedArray<string_view, 4> args(argc); which allocates on heap if argc > 4. The free was not called leading to memory leak.
The fix:
Add scoping to to the function so that the destructor is called before calling raise error
Signed-off-by: adi_holden <adi@dragonflydb.io>
This PR syncs some of the improvements that were introduced in streams in Redis 7.2.3 OSS.
1. verify xsetid against max deleted id in the stream
2. Implement precise memory measurement of streams for "memory usage" command.
3. Always compact nodes in stream listpacks after creating new nodes.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Specifically:
* `INFO REPLICATION` does not list the replicas, but does still show
`connected_slaves`
* `INFO SERVER` does not show `thread_count` and `os`
Fixes#4173
There are actually a few failures fixed in this PR, only one of which is a test bug:
* `db_slice_->Traverse()` can yield, causing `fiber_cancelled_`'s value to change
* When a migration is cancelled, it may never finish `WaitForInflightToComplete()` because it has `in_flight_bytes_` that will never reach destination due to the cancellation
* `IterateMap()` with numeric key/values overrode the key's buffer with the value's buffer
Fixes#4207
* fix: bugs in stream code
1. Memory leak in streamGetEdgeID
2. Addresses CVE-2022-31144
3. Fixes XAUTOCLAIM bugs and adds tests.
4. Limits the count argument in XAUTOCLAIM command to 2^18 (CVE-2022-35951)
Also fixes#3830
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Signed-off-by: Roman Gershman <romange@gmail.com>
Co-authored-by: Shahar Mike <chakaz@users.noreply.github.com>
1. Use transaction time in streams code, similarly to how we do it in other commands.
Stop using mstime() and delete unused redis code.
2. Check for sequence overflow issue when passing huge sequence ids.
Add a test.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Also fix "debug objhist" so that its value histogram will show effective malloc
used distributions for all types.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
After running `debug POPULATE 100 list 100 rand type list elements 10000`
with `--list_experimental_v2=false`:
```
type_used_memory_list:16512800
used_memory:105573120
```
When running with `--list_experimental_v2=true`:
```
used_memory:105573120
type_used_memory_list:103601700
```
TODO: does not yet handle compressed entries correctly but we do not enable compression by default.
Fixes#3800
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: Add more qlist tests
Also fix a typo bug in NodeAllowMerge.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: fix build
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
fix(search_family): fix(search_family): Fix crash when no SEPARATOR is specified in the FT.CREATE command
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
* chore: get back on the decision to put a hard limit on command interface
Limiting commands to only Transaction* and SinkReplyBuilder does not hold.
We need sometimes to access context fields for multitude of reasons.
But I do not want to pass the huge ConnectionContext object because, it's hard
then to track unusual access patterns.
The compromise: to introduce CommandContext that currently has tx, rb and extended fields.
It will be relatively easy to identify irregular access patterns by tracking the extended field.
This commit is the first one in series of probably 10-15 commits. No functional changes here.
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
