1
0
Fork 0
mirror of https://github.com/dragonflydb/dragonfly.git synced 2024-12-15 17:51:06 +00:00
Commit graph

2619 commits

Author SHA1 Message Date
Borys
efa4efd2bf
refactor: use CmdArgParser for XGROUP command (#3739) 2024-09-18 22:30:37 +03:00
Borys
bbaa2669f9
test: unskip test for debugging purpose (#3738) 2024-09-18 14:13:07 +00:00
Borys
f122a19a02
test: add tests for replication (#3734)
* test: add tests for replication
2024-09-18 16:32:21 +03:00
Kostas Kyrimis
6e45c9c3e2
fix: properly track json memory usage (#3641)
* add JsonMemTracker
* add logic based on MiMallocResource deltas that calculates json object usage
* add test

Signed-off-by: kostas <kostas@dragonflydb.io>
2024-09-18 13:08:43 +00:00
Stepan Bagritsevich
b235617a0d
fix(json_family): Fix out of bound ranges for the JSON.ARR* commands (#3712)
* fix(json_family): Fix out of bound ranges for theJSON.ARR* commands

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* refactor(json_family): address comments

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* refactor(json_family): address comments 2

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

---------

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2024-09-18 14:31:17 +02:00
Vladislav
41ba864924
chore: Remove ReqSerializer (#3721)
Signed-off-by: Vladislav <vladislav.oleshko@gmail.com>
2024-09-18 14:31:47 +03:00
Shahar Mike
ffb4c2b601
fix: Fix test_take_over_seeder (#3733)
The test assumed any shutdown will take not more than 1s. This doesn't
always hold, and also waiting for 1s isn't ideal because usually it
takes less than that.

Changed to use `assert_eventually` instead.

Fixes #3684
2024-09-18 14:17:09 +03:00
Shahar Mike
1c6be62a0b
fix: Fix cluster_mgr.py (#3730)
We updated the reply of `SLOT-MIGRATION-STATUS`, so `cluster_mgr.py`
needs to be adjusted as well.
2024-09-18 11:44:15 +03:00
Shahar Mike
a115bc2b9f
fix: Fix test test_client_pause_with_replica (#3729)
There are 2 minor issues with this test:
1. It specified `cmdstat_replconf` as `cmd_stats` instead of `cmd`,
   that's clearly a typo as `cmd_stats` is a map with stats, while
   `replconf` is a Dragonfly command
2. Command `MULTI` is allowed to run even when the server is in paused
   state, see
   [here](https://github.com/dragonflydb/dragonfly/blob/main/src/server/main_service.cc#L1197):

   ```
   // Don't interrupt running multi commands or admin connections.
   ```

Fixes #3675
2024-09-18 09:40:26 +03:00
Stepan Bagritsevich
ae5ce9b497
fix(json_family): Separate double and int values during the comparison of the JSON objects (#3711)
* fix(json_family): Separate the double and int values in JSON commands

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* refactor(json_family): Address comments

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

---------

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2024-09-18 07:24:48 +02:00
Stepan Bagritsevich
824af02f6f
fix(json_family): Fix JSON.ARRPOP command in legacy mode should not return WRONGTYPE error (#3683)
* fix(json_family): Fix WRONGTYPE error for the JSON legacy mode in the JSON.ARRPOP command

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* refactor(json_family): address comments

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* refactor(json_family): address comments 2

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

---------

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2024-09-18 07:24:18 +02:00
Andy Dunstall
a64fc74ce1
tests: fix and enable s3 snapshot test (#3720)
* test: fix s3 snapshot test

* ci: configure s3 regression test

* tests: only run s3 snapshot test if bucket not empty
2024-09-17 17:35:53 +03:00
Kostas Kyrimis
8a34b3e730
chore: enable ReplyGuard in ReplyBuilder2 (#3705)
* add ReplyGuard in ReplyBuilder2

Signed-off-by: kostas <kostas@dragonflydb.io>
2024-09-17 13:37:23 +03:00
Kostas Kyrimis
6f84115152
chore: add log info on failed commands (#3694)
* log errors on failed commands

Signed-off-by: kostas <kostas@dragonflydb.io>
2024-09-17 13:07:46 +03:00
Shahar Mike
51746d99c7
fix(cluster): Do not Pause() replication / migrations (#3716)
Pre-this change, whenever Dragonfly was paused (either by a user or by
internal processes like takeover or slot migration finalization),
migrations and replications were also paused.

This could cause timing issues, which sometime result in migration
failures. Specifically, when 2 nodes have migrations from one to the
other **in parallel** (A->B and B->A), the `Pause()` that happens on A
(which happens because it's a source node) will stop it from processing
incoming traffic from B (incoming because it is also a target node).

If timed correctly, it will be locked until it times out, and so the
migration will fail.

The fix is to prevent replications and migrations from adhering to
`Pause()`s, which I think should not have happened in the first place
because they should use the admin port anyway.

Fixes #3319
2024-09-17 10:47:55 +03:00
Andy Dunstall
b9ff6934e8
fix: fix s3 load snapshot (#3717) 2024-09-17 07:17:24 +01:00
romange
6f3da56e75 chore(helm-chart): update to v1.22.2 2024-09-16 20:06:02 +00:00
Roman Gershman
e21ba0b3d9
chore: symbolize stack traces in tests upon crash (#3714)
We disable address space randomization when building the binary
and use addr2line to symbolize the stacktrace if it exists.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-09-16 13:43:16 +03:00
Borys
e2a852b7e4
fix: add default value has_mc_flag field (#3710) 2024-09-16 10:03:08 +03:00
Roman Gershman
267bd431e4
chore: add clone benchmark (#3709) 2024-09-15 13:10:43 +03:00
Borys
93de559977
Update dflycluster slot-migration-status reply (#3707)
* feat: update DFLYCLUSTER SLOT-MIGRATION-STATUS reply
2024-09-15 09:44:40 +03:00
Kostas Kyrimis
b5929f0162
fix: allow parsing extra spaces on acl files (#3703)
* allow parsing extra whitespace characters in acl files

Signed-off-by: kostas <kostas@dragonflydb.io>
2024-09-13 10:17:20 +03:00
Yang Hau
35c70dbfe1
feat(core): Support RISCV RVV (#3655) 2024-09-12 18:40:46 +03:00
Kostas Kyrimis
4988831b6f
chore: pull latest helio (#3700)
Signed-off-by: kostas <kostas@dragonflydb.io>
2024-09-12 15:51:23 +03:00
Stepan Bagritsevich
3815cda26d
fix(json_family) Add NOESCAPE option to the JSON.GET command (#3685)
* fix(json_family) Add NOESCAPE option to the JSON.GET command

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* refactor(json_family): address comments

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* refactor(json_family): address comments 2

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

---------

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2024-09-12 13:04:46 +02:00
Kostas Kyrimis
5819755af1
fix: test_replicaof_reject_on_load assert failure (#3697)
* increase snapshot size for the test

Signed-off-by: kostas <kostas@dragonflydb.io>
2024-09-12 09:57:48 +03:00
Borys
bae2767707
test: fix test_cluster_replication_migration (#3699) 2024-09-11 23:00:53 +03:00
Kostas Kyrimis
d041386184
fix: test_acl_revoke_pub_sub_while_subscribed (#3680)
* add logs
* add asyncio sleep to avoid producer stalls

Signed-off-by: kostas <kostas@dragonflydb.io>
2024-09-11 12:00:49 +03:00
Stepan Bagritsevich
3ece1725a1
fix(json_family): Fix the JSON.SET bug if the path is in legacy mode and is not the root (#3693)
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2024-09-10 15:50:32 +02:00
Borys
9b8aa8eab4
fix: join for cancel incoming migration (#3692) 2024-09-10 14:42:37 +03:00
adiholden
e71f679386
fix(server): fix replication master deadlock on cancelation flow (#3686)
* fix server: fix replication deadlock on cancelation

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-09-10 14:13:38 +03:00
Roman Gershman
bdc578acef
chore: limit number of descriptors in the exec map (#3688)
For some cases, this map can grow indefinitely.
This change makes it less detailed by makes sure that number of possible keys is bounded.
Still it can provide a good summary of nature of exec transactions.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-09-10 07:50:30 +00:00
Roman Gershman
3cdc8fa128
chore: add a script that parses allocator tracking logs (#3687) 2024-09-10 07:26:44 +00:00
Roman Gershman
257749263b
chore: adjust RdbChannel sizes (#3676)
Fixes #3658

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-09-09 22:21:42 +03:00
Borys
35c287b813
test: unskip cluster tests and add debug info (#3681) 2024-09-09 22:21:17 +03:00
Stepan Bagritsevich
2fad54e41f
fix(search_family): Fix LOAD option behavior in the FT.AGGREGATE command (#3660)
* fix(search_family): Fix LOAD option behavior in the FT.AGGREGATE command

fixes dragonflydb#3646

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* Update src/server/search/search_family.cc

Co-authored-by: Shahar Mike <chakaz@users.noreply.github.com>
Signed-off-by: Stepan Bagritsevich <43710058+BagritsevichStepan@users.noreply.github.com>

---------

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
Signed-off-by: Stepan Bagritsevich <43710058+BagritsevichStepan@users.noreply.github.com>
Co-authored-by: Shahar Mike <chakaz@users.noreply.github.com>
2024-09-09 09:21:48 +00:00
adiholden
c34f2b7eeb
server logs: change script error to warning (#3670)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-09-09 11:23:37 +03:00
Shahar Mike
b10a4a5348
feat(server): Support CLIENT SETINFO (#3673)
Add support for `CLIENT SETINFO <LIB-NAME | LIB-VER>` and also return
that as part of `CLIENT LIST`, like Valkey.

Fixes #3137
2024-09-09 11:03:05 +03:00
Roman Gershman
b7b96424e4
deprecate RecordsPopper and serialize channel records during push (#3667)
chore: deprecate RecordsPopper and serialize channel records during push

Records channel is redundant for DFS/replication because we have single producer/consumer
scenario and both running on the same thread. Unfortunately we need it for RDB snapshotting.

For non-rdb cases we could just pass a io sink to the snapshot producer,
so that it would use it directly instead of StringFile inside FlushChannelRecord.

This would reduce memory usage, eliminate yet another memory copy and generally would make everything simpler.
For that to work, we must serialize the order of FlushChannelRecord, and that's implemented by
this PR. Also fixes #3658.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-09-09 06:19:04 +00:00
Shahar Mike
1306a91bda
chore: Add CLIENT ID command (#3672)
We already adhere to all requirements, we just need to return the id
when the command is issued :)

Fixes #3651
2024-09-08 22:00:53 +03:00
Stepan Bagritsevich
0e7c12f5d6
fix(search_family): Fix FT.AGGREGATE GROUPBY option (#3657) 2024-09-08 17:29:09 +02:00
romange
14ac055a47 chore(helm-chart): update to v1.22.1 2024-09-08 09:29:23 +00:00
Roman Gershman
264835e9c4
chore: cosmetic changes around Snapshot functions (#3652)
* chore: cosmetic changes around Snapshot functions

Some renames and added comments. Refactored StartIncremental into a separate function
without any functional changes.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

* chore: fix comments

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-09-08 09:25:41 +03:00
Roman Gershman
1d34bf735e
fix: recursive calls in the allocation tracker (#3665)
Also, remove dependence of absl::TimeZone bloated monstrosity, which was required by
absl::FormatTime api, even though we do not actually format a timezone.

When absl::LocalTimeZone is accessed it allocates hundreds of thousands of bytes
for each shard thread (maybe due to lack thread safety during lazy initialization).

At the end, strftime does a great job without any shenanigans.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-09-08 06:24:09 +00:00
Joe Zhou
8e064cd619
chore: add some community suggestions to README (#3664) 2024-09-07 14:09:02 +03:00
Borys
1ed3702af7
test: fix MC test_expiration (#3663) 2024-09-06 14:20:21 +03:00
Borys
894bf4735b
fix: fix multi mget exec error message (#3662) 2024-09-06 13:06:02 +03:00
Borys
2cc2a23247
fix: deadlock in the cluster migration process (#3653) 2024-09-05 21:55:15 +03:00
Borys
a1e9ee1b6d
CmdArgParser improvement (#3633)
* feat: add processing of tail args into CmdArgParser::Check
* refactor: rename CmdArgParser::Switch to Map
* feat: add CheckMap method into CmdArgParser
2024-09-05 15:30:54 +03:00
Roman Gershman
3461419088
chore: allow disabling io_uring registered buffers (#3650) 2024-09-05 14:47:42 +03:00