Borys
39dd73fc71
fix: fix bug in cluster/slot_set ( #3143 )
...
* fix: fix bug in cluster/slot_set
* fix: fix slot flushes
2024-06-07 14:31:11 +03:00
Borys
66a524a026
test: skip test_cluster_migration_cancel, it is broken ( #3146 )
2024-06-06 16:33:08 +03:00
Borys
7606af706f
fix: fix RestoreStreamer to prevent buckets skipping #2830 ( #3119 )
...
* fix: fix RestoreStreamer to prevent bucket skipping #2830
2024-06-04 11:50:03 +03:00
Borys
644dc3f139
New test for cluster migration: connection issue ( #3102 )
...
* test: update test_config_consistency,
update test_cluster_data_migration,
new cluster migration test for network issues
2024-06-02 09:16:03 +03:00
Borys
b02a789ebf
fix: add timeout for DFLYMIGRATE ACK to prevent deadlock ( #3093 )
...
* fix: add timeout for DFLYMIGRATE ACK to prevent deadlock
2024-05-28 17:41:51 +03:00
Borys
0dea257f41
fix: fix cluster incorrect keys status ( #3083 )
...
* fix: fix cluster incorrect keys status
2024-05-26 15:10:01 +03:00
Shahar Mike
082aba02ef
fix(cluster-migration): Support cancelling migration right after starting it ( #2992 )
...
* fix(cluster-migration): Support cancelling migration right after starting it
This fixes a few small places, but most importantly it does not allow a
migration to start before both the outgoing and incoming side received
the updated config. This solves a few edge cases.
Fixes #2968
* add TODO
* fix test
* gh comments and fixes
* add comment
2024-05-02 15:50:42 +03:00
Borys
415839df79
fix: fix deadlock and slot flush for migration cancel #2968 ( #2972 )
...
* fix: fix deadlock and slot flush for migration cancel #2968
2024-04-30 08:44:05 +00:00
Borys
654ec9f1c4
feat: add slot migration error processing ( #2957 )
...
* feat: add slot migration error processing
2024-04-29 10:51:23 +03:00
Shahar Mike
322b2e7ac1
fix(test): Unflake fuzzy cluster migration test ( #2927 )
...
* WIP WIP WIP: Test if fuzzy migration test is still flaky
* tune down
* rm ci changes
2024-04-19 23:04:01 +03:00
Borys
9a6a9ec198
feat: add ability reaply config with migration #2924 ( #2926 )
...
* feat: add ability reaply config with migration #2924
2024-04-19 16:21:54 +03:00
Shahar Mike
56965edbe1
feat(cluster): Migration cancellation support ( #2869 )
2024-04-17 13:19:31 +03:00
Borys
d99b0eda16
feat: retry ACK if the configs are different #2833 ( #2906 )
...
* feat: retry ACK if the configs are different #2833
2024-04-16 15:03:30 +03:00
adiholden
9cbe69576e
fix(cluster_replication): replicate redis cluster node bug fix ( #2876 )
...
* fix redis replication error handling and set cntx as journal emulated
Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-04-14 22:49:00 +03:00
Shahar Mike
b8693b4805
feat(cluster): Send number of keys for incoming and outgoing migrations. ( #2858 )
...
The number of keys in an _incoming_ migration indicates how many keys
were received, while for _outgoing_ it shows the total number. Combining
the two can provide the control plane with percentage.
This slightly modified the format of the response.
Fixes #2756
2024-04-08 21:17:03 +03:00
Borys
482bd58787
feat(cluster): add migration removing by config #2835 ( #2844 )
2024-04-05 11:03:54 +03:00
Borys
7b419c6d10
refactor(cluster): replace sync_id with node_id for slot migration #2835 ( #2838 )
2024-04-04 10:14:03 +03:00
Kostas Kyrimis
095ed161d2
chore: disable test_cluster_slot_migration ( #2788 )
...
* disable test_cluster_slot_migration because it needs refactoring -- design has changed
2024-03-29 09:51:24 +00:00
adiholden
2ad7439128
feat(server): support cluster replication ( #2748 )
...
* feat(server): support cluster replication
Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-03-26 15:26:19 +02:00
Borys
d6b7df94bb
refactor: remove start-slot-migration cmd #2727 ( #2728 )
...
* refactor: remove start-slot-migration cmd #2727
2024-03-21 13:37:05 +02:00
Shahar Mike
5c69f5552c
fix(cluster): Save replica ID per replica ( #2735 )
...
Fixes #2734
2024-03-19 13:42:48 +02:00
Kostas Kyrimis
a96d4e4dc8
chore: disable flaky fuzzy migration test ( #2716 )
2024-03-12 14:18:43 +02:00
Shahar Mike
8b31195798
feat(cluster): Add --cluster_id flag ( #2695 )
...
* feat(cluster): Add `--cluster_id` flag
This flag sets the unique ID of a node in a cluster.
It is UB (and bad) to set the same IDs to multiple nodes in the same
cluster.
If unset (default), the `master_replid` (previously known as `master_id`) is used.
Fixes #2643
Related to #2636
* gh comments
* oops - revert line removal
* fix
* replica
* disallow cluster_node_id in emulated mode
* fix replica test
2024-03-10 20:16:31 +02:00
Borys
98616755c0
feat(cluster): automatic slot migration finalization #2697 ( #2698 )
...
* feat(cluster): automatic slot migration finalization #2697
2024-03-07 20:29:21 +02:00
Borys
dfedaf7e6e
refactor: remove FULL-SYNC-CUT cmd #2687 ( #2688 )
...
* refactor: remove FULL-SYNC-CUT cmd #2687
2024-03-06 14:26:35 +02:00
Borys
e57067d2fb
refactor(cluster): #2652 initiate migration process from CONFIG cmd ( #2667 )
...
* refactor(cluster): #2652 initiate migration process from CONFIG cmd
2024-02-29 16:08:53 +02:00
Vladislav
5ac5e3bdac
fix: replace localhost with 127.0.0.1 in cluster tests ( #2665 )
2024-02-27 12:53:54 +03:00
Borys
491538e170
feat(cluster): #2448 add new DFLYMIGRATE ACK cmd ( #2582 )
...
* feat(cluster): #2448 add new DFLYMIGRATE ACK cmd
2024-02-19 19:21:14 +02:00
Vladislav
d035111bed
fix: fix cluster test ( #2612 )
...
* fix: fix cluster test
2024-02-19 11:03:04 +03:00
Shahar Mike
b18fe8c0a8
test(cluster): Fix and uncomment counter sanity checks ( #2591 )
2024-02-14 14:06:34 +02:00
Shahar Mike
6d11f86091
test(cluster-migration): Fix some bugs and add cluster migration fuzzy tests ( #2572 )
...
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
Co-authored-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-02-12 13:47:34 +02:00
Borys
eaca66b02f
fix(cluster): fix #2532 tests redis client close errors ( #2542 )
...
* fix(cluster): fix #2532 tests redis client close errors
* refactor: address comments
2024-02-05 22:42:03 +02:00
Borys
5189dae118
feat(cluster): add migration finalization ( #2507 )
...
* feat(cluster): add migration finalization
2024-02-01 17:24:54 +02:00
Borys
43808da27f
fix(cluster): fix slot filtration to RestoreStreamer ( #2477 )
...
* fix(cluster): fix slot filtration to RestoreStreamer
* test: add cluster data migration test
2024-01-28 12:29:54 +02:00
Shahar Mike
e6f418575b
test(cluster): Enable seeder to work against a Dragonfly cluster ( #2462 )
2024-01-24 20:02:04 +02:00
Borys
a16b940a65
feat(cluster): add tx execution in cluster_shard_migration ( #2385 )
...
* feat(cluster): add tx execution in cluster_shard_migration
refactor(replication): move code that is common for cluster and
replica into a separate file, add full-sync-cut cmd
2024-01-22 21:19:39 +02:00
Borys
03f69ff6c3
feat: add SLOT-MIGRATION-STATUS cmd for source node ( #2349 )
...
* feat: add SLOT-MIGRATION-STATUS cmd for source node
implements #2232
add ability using SLOT-MIGRATION-STATUS without args
to print info about all migration processes for the current node
2024-01-02 12:10:06 +02:00
Borys
fd76c51310
feat: add command flow for slot migration process ( #2292 )
...
* feat(cluster): add command flow for slot migration process
fixes #2295
DFLYMIGRATE FLOW command was added to establish
connections for every shard replication process.
Slow serialization step is the separate issue so
for now only eof_token is sent for reply to
DFLYMIGRATE FLOW command.
Expected state for START-SLOT-MIGRATION is FULL_SYNC now.
2023-12-20 18:47:11 +02:00
Vladislav
aaf01d4244
feat(cluster): Cancel blocking commands on cluster update ( #2255 )
...
Handle blocking commands during cluster config update
---------
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-12-17 15:32:35 +03:00
Borys
24b13434cf
feat: add slot-migration-status command ( #2239 )
...
* feat: add slot-migration-status command
2023-12-04 12:47:46 +02:00
Borys
bfb1b3b624
Start slot migration ( #2218 )
...
* feat: add new command START-SLOT-MIGRATION
2023-11-29 13:38:13 +02:00
Roman Gershman
7aa3dba423
chore: use decode_responses when creating a redis client ( #2109 )
...
* chore: use decode_responses when creating a redis client
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-11-03 11:00:26 +02:00
Roy Jacobson
7ad29ab919
fix(regtests): Limit infinite loop in replication_test ( #1961 )
2023-09-28 11:45:03 +03:00
Roy Jacobson
b55316c0e6
tests: choose open ports randomally ( #1569 )
...
* Implement changes to the testing infrastructure to use random ports
* Use psutil to find out the random ports
2023-09-18 10:23:49 +03:00
Kostas Kyrimis
7944af3c62
feat: Add black formatter to the project ( #1544 )
...
Add black formatter and run it on pytests
2023-07-17 13:13:12 +03:00
Shahar Mike
a6745850ab
Fix regression test failures. ( #1529 )
...
The issue was that, sometimes, the ID generated for one of the nodes
contained the slot ID that was used in the test (either 5259 or 5260).
This caused the test to replace the "slot" part of the id, which in turn
caused the node to think that it no longer owns any slot.
2023-07-10 10:59:58 +03:00
Shahar Mike
5c11beb919
feat(cluster): Send flush slots cmd from masters to replicas. ( #1484 )
...
This fixes potential data diffs between master and replica upon slot moving
Fixes #1320
2023-06-28 22:04:51 +03:00
Chaka
de21afc6e6
test(cluster): Test that cluster works with a standard cluster client. ( #1336 )
...
In this case, `redis.RedisCluster`.
To be double sure I also looked at the actual packets and saw that the
client asks for `CLUSTER SLOTS`, and then after the redistribution of
slots, following a few `MOVED` replied, it asks for the new slots again.
2023-06-06 16:03:09 +03:00
Chaka
124437bb77
feat(cluster): Allow actions on non-owned keys on replication port. ( #1334 )
...
This allows masters to send data of non-owned keys to their replicas,
which is useful when:
1. Config is temporarily different between master and replica
2. Preparing for taking ownership over currently not-owned slots (in the upcoming migration feature)
Fixed #1319
2023-06-04 16:41:21 +03:00
Chaka
67ca8c8615
test(cluster): Add basic cluster mode pytests. ( #1328 )
2023-06-01 15:13:18 +03:00
