1
0
Fork 0
mirror of https://github.com/dragonflydb/dragonfly.git synced 2024-12-14 11:58:02 +00:00
Commit graph

406 commits

Author SHA1 Message Date
Vladislav
fd5772a186
chore(tiering): Lots of metrics (#2977)
* chore(tiering): Lots of metrics
2024-04-30 22:25:45 +03:00
Borys
415839df79
fix: fix deadlock and slot flush for migration cancel #2968 (#2972)
* fix: fix deadlock and slot flush for migration cancel #2968
2024-04-30 08:44:05 +00:00
Kostas Kyrimis
652d4da932
chore(acl): allow multiple users in acl deluser (#2945)
* add allow multiple users in acl deluser command
2024-04-29 17:03:54 +03:00
Borys
654ec9f1c4
feat: add slot migration error processing (#2957)
* feat: add slot migration error processing
2024-04-29 10:51:23 +03:00
Kostas Kyrimis
bbe6c8579a
fix(pytests): replace proc.wait() with proc.communicate() to avoid deadlocks (#2964)
* replace proc.wait() with proc.communicate() to avoid deadlocks
2024-04-26 13:22:33 +03:00
adiholden
d5cd0ed204
fixes for v1.18.0 (#2956)
* fix server: change table_growth_margin default value

---------

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-04-24 18:21:14 +03:00
Joe Zhou
84aa237ba7
chore(acl): adjust some ACL command responses (#2943)
* change ACL DELUSER, ACL WHOAMI, and some ACL DRYRUN string/integer responses.
* change ACL GETUSER response, when the user does not exist, it should reply (nil).
2024-04-22 18:14:47 +03:00
Vladislav
e78b909b96
feat(tiering): Get, GetSet, Set test (#2921)
Get, GetSet, Set test

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-04-22 13:55:17 +03:00
Shahar Mike
322b2e7ac1
fix(test): Unflake fuzzy cluster migration test (#2927)
* WIP WIP WIP: Test if fuzzy migration test is still flaky

* tune down

* rm ci changes
2024-04-19 23:04:01 +03:00
Borys
9a6a9ec198
feat: add ability reaply config with migration #2924 (#2926)
* feat: add ability reaply config with migration #2924
2024-04-19 16:21:54 +03:00
Shahar Mike
56965edbe1
feat(cluster): Migration cancellation support (#2869) 2024-04-17 13:19:31 +03:00
Vladislav
4fe00a071e
chore(tiering): Update Get, Set, Del (#2897)
* chore(tiering): Update Get, Set and Del


---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-04-16 19:20:24 +03:00
Borys
d99b0eda16
feat: retry ACK if the configs are different #2833 (#2906)
* feat: retry ACK if the configs are different #2833
2024-04-16 15:03:30 +03:00
Vladislav
468942ccbb
chore: pull helio and add ipv6 replication test (#2889)
* chore: pull helio and add ipv6 replication test

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-04-15 15:37:22 +03:00
adiholden
9cbe69576e
fix(cluster_replication): replicate redis cluster node bug fix (#2876)
* fix redis replication error handling and set cntx as journal emulated


Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-04-14 22:49:00 +03:00
Kostas Kyrimis
c2f13993d9
fix(acl): authentication with UDS socket (#2895)
* disable authentication on UDS socket
* add a test so the bug won't happen again
2024-04-12 16:01:12 +03:00
Roman Gershman
2a4d3ae08e
chore: clean up REPLTAKEOVER flow (#2887)
* chore: clean up REPLTAKEOVER flow

1. Factor out the catchup function.
2. Simplify the flow and make the second parameters - integer.
3. Return OK if the server is already a master (and do nothing underneath).

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-04-11 16:18:06 +03:00
Kostas Kyrimis
1bcbb41ec3
chore: relax repltakeover constraints to only exclude write commands (#2873)
* relax repltakeover constraints
2024-04-10 16:58:34 +03:00
adiholden
eb164be596
fix(redis replication): remove partial sync flow ,not supported yet (#2865)
* fix redis replicaiton: remove partial sync flow ,not supported yet

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-04-10 09:27:02 +03:00
adiholden
b1e688b33f
bug(server): set connection flags block/pause flag on all blocking commands (#2816)
* bug((server)): set connecttion blocking and puash flags on all blocking commands

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-04-09 09:49:33 +03:00
Shahar Mike
b8693b4805
feat(cluster): Send number of keys for incoming and outgoing migrations. (#2858)
The number of keys in an _incoming_ migration indicates how many keys
were received, while for _outgoing_ it shows the total number. Combining
the two can provide the control plane with percentage.

This slightly modified the format of the response.

Fixes #2756
2024-04-08 21:17:03 +03:00
Roman Gershman
604e9c6e97
fix: authorize the http connection to call commands (#2863)
fix: authorize the http connection to call DF commands

The assumption is that basic-auth already covers the authentication part.
And thanks to @sunneydev for finding the bug and providing the tests.
The tests actually uncovered another bug where we may parse partial http requests.
This one is handled by https://github.com/romange/helio/pull/243

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-04-08 13:19:01 +03:00
adiholden
ee8e5a53bf
fix(pytest): dont check process return code on kill (#2862)
fix pytest: dont check process return code on kill

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-04-08 07:08:05 +00:00
adiholden
a5ea47f2d9
fix(pytest): make pytests fail if server crash on shutdown (#2827)
* fix pytests: make pytests fail if server crash on shutdown

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-04-07 15:30:04 +03:00
Vladislav
76729d6e4c
fix(tests): Fix numsub test (#2852)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-04-07 09:48:57 +03:00
Borys
482bd58787
feat(cluster): add migration removing by config #2835 (#2844) 2024-04-05 11:03:54 +03:00
Borys
7b419c6d10
refactor(cluster): replace sync_id with node_id for slot migration #2835 (#2838) 2024-04-04 10:14:03 +03:00
Roman Gershman
8937e28bc2
chore: Fix build by disabling the tests. (#2821)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-04-02 15:21:28 +03:00
Roman Gershman
d3b90c8210
fix: correct json response for errors (#2813)
Fixes #2811

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-04-01 22:56:26 +03:00
Kostas Kyrimis
b2e2ad6e04
feat(server): check master journal lsn in replica (#2778)
Send journal lsn to replica and compare the lsn value against number of records received in replica side

Signed-off-by: kostas <kostas@dragonflydb.io>
Co-authored-by: adi_holden <adi@dragonflydb.io>
2024-04-01 17:51:31 +03:00
Kostas Kyrimis
095ed161d2
chore: disable test_cluster_slot_migration (#2788)
* disable test_cluster_slot_migration because it needs refactoring -- design has changed
2024-03-29 09:51:24 +00:00
Vladislav
c8724adddf
chore: Fix memcached flags not updated (#2787)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-03-29 12:10:58 +03:00
Kostas Kyrimis
cd20c4003d
chore(replication-tests): add cache_mode on test replication all (#2685)
* add cache_mode cases on test_replication_all
* fix CVCOnBumpUp to not skip some of the modified buckets
2024-03-27 14:28:52 +02:00
Kostas Kyrimis
7b7291c001
chore(ci): open last_log_file in append mode (#2776)
* open last_log_file in append mode
2024-03-26 19:01:26 +02:00
adiholden
2ad7439128
feat(server): support cluster replication (#2748)
* feat(server): support cluster replication

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-03-26 15:26:19 +02:00
Kostas Kyrimis
3abee8a361
fix(ci): do not overwrite last_log_file among tests (#2759)
* add clean up semantics for tests such that we don't loose log files
2024-03-26 14:24:02 +02:00
Roman Gershman
966d7f55ba
chore: preparation for basic http api (#2764)
* chore: preparation for basic http api

The goal is to provide very basic support for simple commands,
fancy stuff like pipelining, blocking commands won't work.

1. Added optional registration for /api handler.
2. Implemented parsing of post body.
3. Added basic formatting routine for the response. It does not cover all the commands but should suffice for
   basic usage.

The API is a POST method and the body of the request should contain command arguments formatted as json array.
For example, `'["set", "foo", "bar", "ex", "100"]'`.
The response is a json object with either `result` field holding the response of the command or
`error` field containing the error message sent by the server.
See `test_http` test in tests/dragonfly/connection_test.py for more details.


* chore: cover iouring with enable_direct_fd

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-03-25 12:12:31 +02:00
Shahar Mike
b830a71e66
feat(replication): Do not auto replicate different master (#2753)
* feat(replication): Do not auto replicate different master

Until now, replicas would re-connect and re-replicate a master after the
master will restart. This is problematic in case the master loses its
data, which will cause the replica to flush all and lose its data as
well.

This is a breaking change though, in that whoever controls the replica
now has to explicitly issue a `REPLICAOF X Y` in order to re-establish
a connection to a new master. This is true even if the master loaded an
up to date RDB file.

It's not necessary if the replica lost connection to the master and the
master was always alive, and the connection is re-established.

Fixes #2636

* fix test

* fixes

* proxy proxy java java

* better comment

* fix comments

* replica_reconnect_on_master_restart

* proxy.close()
2024-03-24 12:04:37 +02:00
Borys
d6b7df94bb
refactor: remove start-slot-migration cmd #2727 (#2728)
* refactor: remove start-slot-migration cmd #2727
2024-03-21 13:37:05 +02:00
Roman Gershman
2d246adbbb
chore: better error reporting when connecting to tls with plain socket (#2740)
* chore: better error reporting when connecting to tls with plain socket

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-03-19 17:20:23 +02:00
Shahar Mike
5c69f5552c
fix(cluster): Save replica ID per replica (#2735)
Fixes #2734
2024-03-19 13:42:48 +02:00
Kostas Kyrimis
76d697aaff
feat: add flag masteruser (#2693)
* add flag masteruser
* fix ack reply on acl validation failure
2024-03-15 08:41:24 +02:00
Kostas Kyrimis
094df3ef3f
fix(BgSave): async from sync (#2702)
* make BgSave command async
2024-03-14 13:31:58 +02:00
Kostas Kyrimis
a96d4e4dc8
chore: disable flaky fuzzy migration test (#2716) 2024-03-12 14:18:43 +02:00
Shahar Mike
8b31195798
feat(cluster): Add --cluster_id flag (#2695)
* feat(cluster): Add `--cluster_id` flag

This flag sets the unique ID of a node in a cluster.

It is UB (and bad) to set the same IDs to multiple nodes in the same
cluster.

If unset (default), the `master_replid` (previously known as `master_id`) is used.

Fixes #2643
Related to #2636

* gh comments

* oops - revert line removal

* fix

* replica

* disallow cluster_node_id in emulated mode

* fix replica test
2024-03-10 20:16:31 +02:00
Borys
98616755c0
feat(cluster): automatic slot migration finalization #2697 (#2698)
* feat(cluster): automatic slot migration finalization #2697
2024-03-07 20:29:21 +02:00
Borys
dfedaf7e6e
refactor: remove FULL-SYNC-CUT cmd #2687 (#2688)
* refactor: remove FULL-SYNC-CUT cmd #2687
2024-03-06 14:26:35 +02:00
Roman Gershman
93debc754c
chore: more fixes for macos (#2677)
1.Add back the search files to MacOs build (linker errors are fixed now).
2. Add default maxmemory argument (if not present already) when launching dragonfly process in regression tests.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-03-02 23:14:33 +02:00
Borys
e57067d2fb
refactor(cluster): #2652 initiate migration process from CONFIG cmd (#2667)
* refactor(cluster): #2652 initiate migration process from CONFIG cmd
2024-02-29 16:08:53 +02:00
adiholden
7e4527098b
fix(server): client pause work while blocking commands run (#2584)
fix #2576
fix #2661

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-02-28 11:07:03 +00:00