1
0
Fork 0
mirror of https://github.com/dragonflydb/dragonfly.git synced 2024-12-14 11:58:02 +00:00
Commit graph

358 commits

Author SHA1 Message Date
Borys
e57067d2fb
refactor(cluster): #2652 initiate migration process from CONFIG cmd (#2667)
* refactor(cluster): #2652 initiate migration process from CONFIG cmd
2024-02-29 16:08:53 +02:00
adiholden
7e4527098b
fix(server): client pause work while blocking commands run (#2584)
fix #2576
fix #2661

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-02-28 11:07:03 +00:00
Vladislav
5ac5e3bdac
fix: replace localhost with 127.0.0.1 in cluster tests (#2665) 2024-02-27 12:53:54 +03:00
Roman Gershman
91c299b33e
chore: Del and NUMINCRBY use json::Path (#2655)
* chore: Del and NUMINCRBY use json::Path

Also, fix various protocol bugs when we sent simple string
instead of sending bulk strings.

Fixed a typo in path.cc that lead to a data race bug.

Finally, flip the flag in regression tests to start covering json::Path code
and added test coverage for the data race bug

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-26 14:06:21 +00:00
Vladislav
5ee61db0f3
feat(connection): Support pipelining with Memcached (#2648)
* feat(connection): Support pipelining with Memcached

Adds support for pipelining to Memcached, enhances Memcached pytests

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-02-23 20:18:25 +03:00
adiholden
9e66ec5833
fix(bug): server crash info memory while saving (#2637)
* bug: crash info memory while saving

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-02-21 13:39:57 +00:00
Kostas Kyrimis
f32156788e
feat(regTests): upload only failed test logs on ci and clean up logging (#2547)
* upload only failed test logs
* remove printing log names for passed tests
* print slow tests with --duration
* separate regression and unit logs for CI workflow
2024-02-21 10:35:07 +02:00
Kostas Kyrimis
a1950038b6
fix: loading state error type to be compatible with redis (#2629)
* add -LOADING prefix for loading errors
* replace -ERR with -LOADING for loading errors
2024-02-20 12:45:41 +00:00
Vladislav
75eaeb32db
feat(pytest): More types for seeder (#2577)
* feat(pytest): More types for seeder

Add more types to the seeder and refactor replication test

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-02-20 15:31:08 +03:00
adiholden
1ef8795611
fix(bug): crash when starting replica while saving (#2618)
The bug: crash when starting replica while saving
The problem: accessing the wrong allocator on snapshot class destruction as it was destructed not in the thread of the shard
The fix: call snapshot destructor when we finish snapshot on the correct thread 

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-02-20 10:50:46 +00:00
Borys
491538e170
feat(cluster): #2448 add new DFLYMIGRATE ACK cmd (#2582)
* feat(cluster): #2448 add new DFLYMIGRATE ACK cmd
2024-02-19 19:21:14 +02:00
Leonardo Mello
07a8411ee7
fix(server_family): GetMetrics should show commands in lowercase (#2601)
* replace with lowercase cmdstat_info and cmdstat_replconf

Signed-off-by: Leonardo Mello <lsvmello@gmail.com>
2024-02-19 18:50:59 +02:00
adiholden
15b3fb13b6
fix(server): saving is not a server state (#2613)
* fix(server): saving is not a server state

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-02-19 15:20:48 +00:00
Vladislav
d035111bed
fix: fix cluster test (#2612)
* fix: fix cluster test
2024-02-19 11:03:04 +03:00
Shahar Mike
28800df071
fix(test): Use less memory for STRING and HASH memory tests (#2593)
While at it, also register the opt_only mark
2024-02-15 10:08:45 +02:00
Shahar Mike
b18fe8c0a8
test(cluster): Fix and uncomment counter sanity checks (#2591) 2024-02-14 14:06:34 +02:00
Vladislav
4e3be726c5
chore(pytest): Refactor snapshot test (#2583)
* chore(pytest): Refactor snapshot test

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-02-14 12:13:52 +03:00
Roman Gershman
4000adf57f
fix: do not migrate during connection close (#2570)
* fix: do not migrate during connection close

Fixes #2569
Before the change we had a corner case where Dragonfly would call
OnPreMigrateThread but would not call CancelOnErrorCb because OnBreakCb has already been called
(it resets break_cb_engaged_)

On the other hand in OnPostMigrateThread we called RegisterOnErrorCb if breaker_cb_ which resulted in double registration.
This change simplifies the logic by removing break_cb_engaged_ flag since CancelOnErrorCb is safe to call if nothing is registered.
Moreover, we now skip Migrate flow if a socket is being closed.

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-12 16:03:34 +02:00
Shahar Mike
6d11f86091
test(cluster-migration): Fix some bugs and add cluster migration fuzzy tests (#2572)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
Co-authored-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-02-12 13:47:34 +02:00
Roman Gershman
24fcf8d883
chore: Recognize exit status in regression tests (#2571)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-12 09:03:56 +02:00
Shahar Mike
8ead569b2f
test(memory): Unaccounted memory test + add DEBUG POPULATE TYPE <type> (#2561)
* test(memory): Test memory accounting for all types

* slightly faster

* WIP

* working

* Document

* Update test to use DEBUG POPULATE

* Nothing much

* Working

* fix

* yaml

* explicit capture

* fix ci?

* stub tx
2024-02-12 08:09:48 +02:00
Vladislav
881edb501e
feat(pytest): Gen2 seeder, part 1 (#2556)
* feat(pytest): Gen2 seeder

Implement new seeder that uses lua scripts to improve performance

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-02-09 19:20:25 +03:00
Vladislav
9537cbdb0b
fix(pytest): Disable flaky interpreter test monitoring (#2558) 2024-02-07 12:51:24 +03:00
Vladislav
ed59a439d1
fix(tests): increase interpreter test load, lower notice limit, include 2553 (#2554)
* fix: increase load + lower notice limit

* chore: include 2553

* fix: lower metric EVEN more

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-02-06 16:57:32 +02:00
Vladislav
83a12b99c6
fix: fix interpreter acquisition with MULTI (#2549)
* fix: fix interpreter acquisition with MULTI

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-02-06 12:54:14 +02:00
Shahar Mike
bc9b214ae4
fix(server): Do not yield in journal if not allowed (#2540)
* fix(server): Do not yield in journal if not allowed

* Add pytest

* Compare keys

* check_all_replicas_finished
2024-02-06 12:35:00 +02:00
Kostas Kyrimis
7b4591cc7d
fix: wrongly printing timedout emoji on test failures (#2546)
* fix wrongly printing timedout emoji on non timedout test failures
* add pytest-timeout as dependency in tests/dragonfly/requirements
2024-02-06 10:25:12 +02:00
Borys
eaca66b02f
fix(cluster): fix #2532 tests redis client close errors (#2542)
* fix(cluster): fix #2532 tests redis client close errors

* refactor: address comments
2024-02-05 22:42:03 +02:00
Roman Gershman
5c0029978e
chore: remove redis sorted set implementation (#2522)
Also remove unused code.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-05 10:29:11 +02:00
Vladislav
e0f86697f9
fix: fix script replication (#2531)
* fix: fix script replication

Single key script replication was previously broken because the EXEC entry wasn't sent. Send it manually

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-02-04 20:28:44 +03:00
Borys
5189dae118
feat(cluster): add migration finalization (#2507)
* feat(cluster): add migration finalization
2024-02-01 17:24:54 +02:00
Shahar Mike
ad90602bc2
test: Disable flaky test test_policy_based_eviction_propagation (#2505) 2024-01-30 10:28:00 +02:00
Roman Gershman
97bde35a0f
Fix redis->DF replication tests (#2495)
fix: redis_replication_test fails due to long time for df replicas takes to catch up.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-01-29 03:25:58 +02:00
Kostas Kyrimis
754a186f5a
feat(lua): add missing no-op redis.replicate_commands() (#2472)
fixes #2468
add no-op lua function redis.replicate_commands()
2024-01-28 13:05:52 +02:00
Borys
43808da27f
fix(cluster): fix slot filtration to RestoreStreamer (#2477)
* fix(cluster): fix slot filtration to RestoreStreamer

* test: add cluster data migration test
2024-01-28 12:29:54 +02:00
Vladislav
a5b9401449
fix: reduce test_pipeline_batching_while_migrating flakiness (#2475)
* fix: reduce test_pipeline_batching_while_migrating flakiness
2024-01-25 17:55:12 +03:00
Kostas Kyrimis
ba972923b3
feat(lua): add no-op redis.log command (#2476)
* add no-op redis.log command
2024-01-25 15:45:47 +02:00
Shahar Mike
e6f418575b
test(cluster): Enable seeder to work against a Dragonfly cluster (#2462) 2024-01-24 20:02:04 +02:00
Borys
a16b940a65
feat(cluster): add tx execution in cluster_shard_migration (#2385)
* feat(cluster): add tx execution in cluster_shard_migration
refactor(replication): move code that is common for cluster and
replica into a separate file, add full-sync-cut cmd
2024-01-22 21:19:39 +02:00
Shahar Mike
2f0287429d
fix(replication): Correctly replicate commands even when OOM (#2428)
* fix(replication): Correctly replicate commands even when OOM

Before this change, OOM in shard callbacks could have led to data
inconsistency between the master and the replica. For example, commands
which mutated data on 1 shard but failed on another, like `LMOVE`.

After this change, callbacks that result in an OOM will correctly
replicate their work (none, partial or complete) to replicas.

Note that `MSET` and `MSETNX` required special handling, in that they are
the only commands that can _create_ multiple keys, and so some of them
can fail.

Fixes #2381

* fixes

* test fix

* RecordJournal

* UNDO idiotnessness

* 2 shards

* fix pytest
2024-01-18 12:29:59 +02:00
Kostas Kyrimis
39e7e5ad87
fix: missing error reply to client after AddOrFind throw std::bad_alloc (#2411)
* Handle properly and reply on execution paths that throw std::bad_alloc within AddOrFind
2024-01-15 10:13:10 +02:00
Shahar Mike
13718699d8
feat(server): Implement CLIENT KILL (#2404)
* feat(server): Implement `CLIENT KILL`

Currently, it supports the following syntax:

* `CLIENT KILL <addr>:<port>`
* `CLIENT KILL ID <id>`
* `CLIENT KILL ADDR <addr>:<port>`
* `CLIENT KILL LADDR <addr>:<port>`

It will not allow killing an admin-connection from a non-admin port.

There are a few parameters of `CLIENT KILL` that Redis supports but this
PR does not yet add. Let's add them as needed.

Fixes #1614

* Add tests

* fixes
2024-01-15 09:49:23 +02:00
Vladislav
484b4de216
Fix flush when migrating connection (#2407)
fix: don't miss flush for control messages
2024-01-13 09:57:33 +03:00
Yue Li
8d09478474
bug(server): log evicted keys in journal in PrimeEvictionPolicy. (#2302)
fixes #2296

added a regression test that tests both policy based eviction as well as heart beat eviction.

---------

Signed-off-by: Yue Li <61070669+theyueli@users.noreply.github.com>
2024-01-11 01:45:29 -08:00
adiholden
f37c57c704
fix(server): crash on rename save command on background save (#2375)
* fix(server): crash on rename save command on baground save

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-01-07 12:21:09 +02:00
Yue Li
6f9107291e
test: Adding integration test using Relay benchmark (#2348)
Adding integration test using Relay benchmark
2024-01-02 12:44:22 -08:00
Borys
03f69ff6c3
feat: add SLOT-MIGRATION-STATUS cmd for source node (#2349)
* feat: add SLOT-MIGRATION-STATUS cmd for source node
implements #2232
add ability using SLOT-MIGRATION-STATUS without args
to print info about all migration processes for the current node
2024-01-02 12:10:06 +02:00
adiholden
5d67c95797
bug(server): reject replicaof while loading from snapshot (#2338)
fix #2337
The bug:
replicaof was not rejected while loading snapshot
The fix:
replicaof is allowed while server is in loading state to allow replicaof while replication in full sync mode
I now reject replicaof if the server is in loading state and it is master

Another bug fix:
allow cron snapshot if --replicaof flag was set

Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-12-27 13:57:49 +02:00
Shahar Mike
a360b308c9
refactor(server): Privatize PreUpdate() and PostUpdate() (#2322)
* refactor(server): Privatize `PreUpdate()` and `PostUpdate()`

While at it:
* Make `PreUpdate()` not decrease object size
* Remove redundant leftover call to `PreUpdate()` outside `DbSlice`

* Add pytest

* Test delete leads to 0 counters

* Improve test

* fixes

* comments
2023-12-25 07:49:57 +00:00
Roman Gershman
700a65ece5
chore: refactor VersionMonitor into a separate file (#2326)
* chore: refactor VersionMonitor into a separate file
---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-12-24 22:06:57 +02:00