1
0
Fork 0
mirror of https://github.com/dragonflydb/dragonfly.git synced 2024-12-14 11:58:02 +00:00
Commit graph

576 commits

Author SHA1 Message Date
Kostas Kyrimis
15fce9df2d
chore: logs on assert fail for test_acl_cat_commands_multi_exec_squash (#3749)
* print result if assertion fails

Signed-off-by: kostas <kostas@dragonflydb.io>
2024-09-23 09:51:58 +03:00
Roman Gershman
29b18f0dcb
fix: tune test_replicaof_reject_on_load parameters (#3762)
Reduce the snapshot size by 20% and increase the timeout to avoid failures due to slow loads.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-09-23 09:50:10 +03:00
Roman Gershman
e09ebe0c5c
fix: test deadlock with processing the stdout of sed (#3735)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-09-22 15:40:27 +03:00
adiholden
4d38271efa
feat(server): introduce rss oom limit (#3702)
* introduce rss denyoom limit

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-09-22 13:28:24 +03:00
adiholden
5cf917871c
feat(server): introduce oom_deny_commands flag (#3718)
* server: introduce oom_deny_commands flag

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-09-22 09:32:18 +03:00
Kostas Kyrimis
ed21867fe9
chore: add missing await in test_take_over_seeder (#3744)
* add missing await

Signed-off-by: kostas <kostas@dragonflydb.io>
2024-09-19 17:03:11 +00:00
Shahar Mike
55e3647248
chore: Switch ports for cluster_mgr_test.py (#3741)
We saw failures due to port already in use
2024-09-19 12:32:31 +03:00
adiholden
409c2a3beb
test: add test for replication deadlock on replication timeout (#3691)
* test: add test for replication deadlock on replication timeout

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-09-19 12:11:28 +03:00
Borys
bbaa2669f9
test: unskip test for debugging purpose (#3738) 2024-09-18 14:13:07 +00:00
Borys
f122a19a02
test: add tests for replication (#3734)
* test: add tests for replication
2024-09-18 16:32:21 +03:00
Kostas Kyrimis
6e45c9c3e2
fix: properly track json memory usage (#3641)
* add JsonMemTracker
* add logic based on MiMallocResource deltas that calculates json object usage
* add test

Signed-off-by: kostas <kostas@dragonflydb.io>
2024-09-18 13:08:43 +00:00
Shahar Mike
ffb4c2b601
fix: Fix test_take_over_seeder (#3733)
The test assumed any shutdown will take not more than 1s. This doesn't
always hold, and also waiting for 1s isn't ideal because usually it
takes less than that.

Changed to use `assert_eventually` instead.

Fixes #3684
2024-09-18 14:17:09 +03:00
Shahar Mike
1c6be62a0b
fix: Fix cluster_mgr.py (#3730)
We updated the reply of `SLOT-MIGRATION-STATUS`, so `cluster_mgr.py`
needs to be adjusted as well.
2024-09-18 11:44:15 +03:00
Shahar Mike
a115bc2b9f
fix: Fix test test_client_pause_with_replica (#3729)
There are 2 minor issues with this test:
1. It specified `cmdstat_replconf` as `cmd_stats` instead of `cmd`,
   that's clearly a typo as `cmd_stats` is a map with stats, while
   `replconf` is a Dragonfly command
2. Command `MULTI` is allowed to run even when the server is in paused
   state, see
   [here](https://github.com/dragonflydb/dragonfly/blob/main/src/server/main_service.cc#L1197):

   ```
   // Don't interrupt running multi commands or admin connections.
   ```

Fixes #3675
2024-09-18 09:40:26 +03:00
Andy Dunstall
a64fc74ce1
tests: fix and enable s3 snapshot test (#3720)
* test: fix s3 snapshot test

* ci: configure s3 regression test

* tests: only run s3 snapshot test if bucket not empty
2024-09-17 17:35:53 +03:00
Roman Gershman
e21ba0b3d9
chore: symbolize stack traces in tests upon crash (#3714)
We disable address space randomization when building the binary
and use addr2line to symbolize the stacktrace if it exists.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-09-16 13:43:16 +03:00
Borys
93de559977
Update dflycluster slot-migration-status reply (#3707)
* feat: update DFLYCLUSTER SLOT-MIGRATION-STATUS reply
2024-09-15 09:44:40 +03:00
Kostas Kyrimis
b5929f0162
fix: allow parsing extra spaces on acl files (#3703)
* allow parsing extra whitespace characters in acl files

Signed-off-by: kostas <kostas@dragonflydb.io>
2024-09-13 10:17:20 +03:00
Kostas Kyrimis
5819755af1
fix: test_replicaof_reject_on_load assert failure (#3697)
* increase snapshot size for the test

Signed-off-by: kostas <kostas@dragonflydb.io>
2024-09-12 09:57:48 +03:00
Borys
bae2767707
test: fix test_cluster_replication_migration (#3699) 2024-09-11 23:00:53 +03:00
Kostas Kyrimis
d041386184
fix: test_acl_revoke_pub_sub_while_subscribed (#3680)
* add logs
* add asyncio sleep to avoid producer stalls

Signed-off-by: kostas <kostas@dragonflydb.io>
2024-09-11 12:00:49 +03:00
Borys
35c287b813
test: unskip cluster tests and add debug info (#3681) 2024-09-09 22:21:17 +03:00
Shahar Mike
b10a4a5348
feat(server): Support CLIENT SETINFO (#3673)
Add support for `CLIENT SETINFO <LIB-NAME | LIB-VER>` and also return
that as part of `CLIENT LIST`, like Valkey.

Fixes #3137
2024-09-09 11:03:05 +03:00
Roman Gershman
b7b96424e4
deprecate RecordsPopper and serialize channel records during push (#3667)
chore: deprecate RecordsPopper and serialize channel records during push

Records channel is redundant for DFS/replication because we have single producer/consumer
scenario and both running on the same thread. Unfortunately we need it for RDB snapshotting.

For non-rdb cases we could just pass a io sink to the snapshot producer,
so that it would use it directly instead of StringFile inside FlushChannelRecord.

This would reduce memory usage, eliminate yet another memory copy and generally would make everything simpler.
For that to work, we must serialize the order of FlushChannelRecord, and that's implemented by
this PR. Also fixes #3658.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-09-09 06:19:04 +00:00
Borys
1ed3702af7
test: fix MC test_expiration (#3663) 2024-09-06 14:20:21 +03:00
Borys
2cc2a23247
fix: deadlock in the cluster migration process (#3653) 2024-09-05 21:55:15 +03:00
Daniel M
eef5be1729
test: remove redundant tests (#3642)
* test: remove redundant tests

* remove more tests

* add test_bitmap_commands

* add test_bitmap_commands and test_geo_commands to ignore list
2024-09-04 23:55:35 +03:00
Roman Gershman
e6f5a2073c
fix: crash when passing empty arguments (#3627)
* fix: crash when passing empty arguments

Fix the case where we pass an empty argument, which then is parsed as an
empty string view with null pointer. The null pointer is not handled correctly
by our low level c code, hence switch to using ""sv for that.


* chore: add more list asserts + improve test_hypothesis
---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-09-03 12:43:12 +03:00
Kostas Kyrimis
f8f8c69e6a
chore: disable big value ser on reg tests (#3629)
* disable big value ser on reg tests

Signed-off-by: kostas <kostas@dragonflydb.io>
2024-09-03 08:04:27 +00:00
Roman Gershman
1a0d44354f
fix: limit parsing in zrange commands (#3626)
1. The offset value can be negative, in that case we should return an empty array.
2. Fix edge cases of inf*0 and -inf + inf, so they will result in 0 and non NaN (similarily to Valkey).

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-09-03 10:08:45 +03:00
Borys
8fca7dd9f8
test: fix search tests (#3625) 2024-09-03 09:21:47 +03:00
Roman Gershman
879f2950e5
fix: edge cases around mismatched path in json code (#3609)
For legacy mode:
1. For mutate commands, an empty result should throw an error
2. For read commands, it returns nil if path was not found, but if it was matched
   but only with a wrong types, it will throw an error.

For non-legacy mode, objlen should throw an error for non existing key.

Simplified JsonCallbackResult a bit and made sure more fakeredis tests are passing.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-09-02 21:37:59 +03:00
Kostas Kyrimis
959b96e7cc
fix(test_auth_resp3_bug): release build failing (#3621)
* remove problematic assertion

Signed-off-by: kostas <kostas@dragonflydb.io>
2024-09-02 14:32:53 +00:00
adiholden
658243fd09
fix pytest: use generic random dbfilename in tests (#3617)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-09-02 09:27:22 +03:00
Borys
8e9b097b9d
fix: fix expiration processing for set command (#3607)
* fix: fix expiration processing for set command
2024-09-02 08:44:11 +03:00
Shahar Mike
de5ecc7447
chore: Split --cluster_announce_ip and --replica_announce_ip (#3615)
chore: Split `cluster_announce_ip` and `replica_announce_ip`

This PR partially reverts #3421

Fixes #3541
2024-09-01 12:43:44 +00:00
Roman Gershman
10de338926
chore: run fakeredis flow with the debug build (#3612)
1. Fix html publishing code
2. Upload dragonfly logs
3. Run it on pull requests

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-09-01 14:51:00 +03:00
Roman Gershman
5c48320496
fix: debug crash inside parsing of ZRANGE (#3611)
Also, fix error msg for EXEC command and finally tune more fakeredis tests.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-08-31 08:02:21 +03:00
Roman Gershman
dd0effac6f
feat: add slave_repl_offset to the replication section. (#3596)
* feat: add slave_repl_offset to the replication section.

In Valkey slave_repl_offset denotes the replication offset on replica site during stable sync phase.
During fullsync phase it appears with 0 value.

In Dragonfly this field appears only after full sync has completed, thus it allows
to check whether Dragonfly reached stable sync phase. The value of this field describes the cumulative progress
of all the replication flows and it does not directly correspond to master side metrics.

In addition, this PR fixes the bug in wait_available_async() function in our replication tests.
This function is intended to wait until a replica reaches stable state and it did by sending pings until they do not
respond with LOADING error, hence the assumption is that the replica is in full sync state already.

However it can happen that master_link_status is "up" but replica has not reached full sync state, and the PING will succeed
just because wait_available_async() was called before full sync started. The whole approach of polling the state is fragile.

Now we use `slave_repl_offset` explicitly to see if the replica reaches stable state.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

* chore: simplify wait_available_async

* chore: comments

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-08-30 18:58:07 +03:00
Kostas Kyrimis
0705bbb536
feat(acl): add pub/sub (#3574)
* add support for pub/sub
* add tests
---------

Signed-off-by: kostas <kostas@dragonflydb.io>
2024-08-30 15:41:28 +03:00
Roman Gershman
20336805f3
chore: enable experimental_new_io by default. (#3605)
* chore: enable experimental_new_io by default.

It has been running for weeks with the flag on, so enabled it also for community.

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
Co-authored-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-08-29 23:30:26 +03:00
Roman Gershman
fa2d67b8a8
fix: xreadgroup replies as a map for RESP3 (#3576)
* fix: xreadgroup replies as a map for RESP3

Moreover, it returns data for all the strings, irrespective whether they have results or not
(unlike with XREAD)

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

* fix: properly handle xpending with 0 results

Also reject ENTRIESREAD instead of silently accepting it.

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-08-29 20:05:57 +00:00
Roman Gershman
b74bd5a7b2
fix: JSON.STRAPPEND (#3597)
* fix: JSON.STRAPPEND

JSON.STRAPPEND was completely broken.

First, it accepts exactly 3 arguments, i.e. a single value to append.
Secondly, the value must be a json string, not the regular string. Meaning it must be in double quotes.
So, before we parsed: `JSON.STRAPPEND key $.field bar` and now we parse:
`JSON.STRAPPEND key $.field "bar"`

In addition fixed the behavior of JSON.STRLEN to return "no such key" error in case the
json key does not exist and path is specified.
---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-08-29 12:29:34 +00:00
dependabot[bot]
72fc0391f0
chore(deps): bump aiohttp from 3.8.4 to 3.10.2 in /tests/dragonfly (#3600)
Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.8.4 to 3.10.2.
- [Release notes](https://github.com/aio-libs/aiohttp/releases)
- [Changelog](https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst)
- [Commits](https://github.com/aio-libs/aiohttp/compare/v3.8.4...v3.10.2)

---
updated-dependencies:
- dependency-name: aiohttp
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-08-29 14:03:51 +03:00
Kostas Kyrimis
9c25ca028c
fix: skip test_replica_reconnections_after_network_disconnect (#3589)
* skip the test because it's flaky

---------

Signed-off-by: kostas <kostas@dragonflydb.io>
2024-08-28 11:11:49 +00:00
Roman Gershman
dc040b53ad
fix: return an error when invalid number of arguments is passed. (#3584)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-08-28 06:20:29 +00:00
Roman Gershman
80e1dbb304
chore: allow running dragonfly pytests in repeat (#3577) 2024-08-27 11:35:47 +03:00
Kostas Kyrimis
238bf3ee85
fix: disable test_cluster_flushall_during_migration (#3573)
* disable test_cluster_flushall_during_migration
2024-08-26 17:50:49 +03:00
Kostas Kyrimis
450ec7549b
fix: disable failing test_cron_snapshot_failed_saving (#3571)
* disable test_cron_snapshot_failed_saving

---------

Signed-off-by: kostas <kostas@dragonflydb.io>
2024-08-26 12:20:34 +00:00
Roman Gershman
908290a268
chore: improve compatibility of set and ping commands (#3569)
* chore: improve compatibility of set and ping commands

smismember should return an array of longs and not array of strings.
ping in subscribe mode returns an array for resp2.
Also, fix double rounding for legacy float mode.
---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-08-26 13:33:03 +03:00