1
0
Fork 0
mirror of https://github.com/dragonflydb/dragonfly.git synced 2024-12-14 11:58:02 +00:00
Commit graph

1864 commits

Author SHA1 Message Date
Roman Gershman
06d88ddc88
chore: parse a simple jsonpath (#2566)
Also, add an ubuntu-20 reusable devcontainer.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-09 15:41:22 +02:00
adiholden
c2ee348d2c
fix(server): do not print warning on replica stop/ takeover (#2560)
* fix(server): do not print warning on replica stop/ takeover

---------

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-02-09 08:10:41 +02:00
Roman Gershman
4860c8eac2
chore: enable ContendExpire (#2563)
chore: enable ContendExpire unit test

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-08 15:30:54 +00:00
Shahar Mike
9912df09ae
fix(server): Init tx time for all multi/lua transactions (#2562)
* fix(server): Return correct `TIME` under unscheduled tx

Fixes #2555

* Init tx time in all multi / lua cases

* init ctor
2024-02-08 14:47:07 +02:00
Borys
72f651d527
feat(cluster): add slots blocking when we finalize migration (#2484)
* feat(cluster): add slots blocking when we finalize the migration
2024-02-07 15:19:35 +02:00
Vladislav
9537cbdb0b
fix(pytest): Disable flaky interpreter test monitoring (#2558) 2024-02-07 12:51:24 +03:00
romange
5b96e269a3 chore(helm-chart): update to v1.14.3 2024-02-07 06:19:10 +00:00
Roman Gershman
4b4ff4688b
feat: introduce a skeleton for jsonpath parser (#2521)
Not finished yet, mostly a boilerplate plus minimal bindings for the test.
Motivation: as long as we rely on jsoncons jsonpath implementation we won't be able
to change json implementation as well. jsoncons path is quite qood but it will only
work with jsoncons json library. By implementing this ourselves we will gain the freedom
to change json implementation in the future.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-06 19:21:40 +02:00
Vladislav
ed59a439d1
fix(tests): increase interpreter test load, lower notice limit, include 2553 (#2554)
* fix: increase load + lower notice limit

* chore: include 2553

* fix: lower metric EVEN more

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-02-06 16:57:32 +02:00
Vladislav
83a12b99c6
fix: fix interpreter acquisition with MULTI (#2549)
* fix: fix interpreter acquisition with MULTI

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-02-06 12:54:14 +02:00
Shahar Mike
bc9b214ae4
fix(server): Do not yield in journal if not allowed (#2540)
* fix(server): Do not yield in journal if not allowed

* Add pytest

* Compare keys

* check_all_replicas_finished
2024-02-06 12:35:00 +02:00
Roman Gershman
336d6ff181
Update helio (#2538)
chore: UpdateHelio dependency

Add support for asan/ubsan checkers in our dev environment.
Remove more clang warnings.

Once we fix all the problems we will enable them in our CI as well.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-06 11:57:26 +02:00
Kostas Kyrimis
4c0055ddf0
feat: move lint test chart to seperate action and trigger it on reg tests (#2536)
* move lint test chart to a separate action
* use it in regression tests
2024-02-06 10:27:10 +02:00
Kostas Kyrimis
7b4591cc7d
fix: wrongly printing timedout emoji on test failures (#2546)
* fix wrongly printing timedout emoji on non timedout test failures
* add pytest-timeout as dependency in tests/dragonfly/requirements
2024-02-06 10:25:12 +02:00
Michael Primeaux
951b0897e6
bug: DFLY_PASSWORD environment variable deprecation (#2539)
* If the image.tag version is equal to 14.0.0 or greater then specify the DFLY_requirepass environment variable. Otherwise use the DFLY_PASSWORD environment variable.

Signed-off-by: Michael Primeaux <michael.primeaux@mac.com>

* Corrected version logic and added golden test

Signed-off-by: Michael Primeaux <michael.primeaux@mac.com>

* Updated golden file for TLS + image.tag

Signed-off-by: Michael Primeaux <michael.primeaux@mac.com>

* fix CI for chart linting test

* rename new test specific to password

* use v1.13.0

* update golden chart

---------

Signed-off-by: Michael Primeaux <michael.primeaux@mac.com>
Signed-off-by: Michael Primeaux <mprimeaux@users.noreply.github.com>
Co-authored-by: Tarun Pothulapati <tarun@dragonflydb.io>
2024-02-06 13:42:05 +05:30
Borys
eaca66b02f
fix(cluster): fix #2532 tests redis client close errors (#2542)
* fix(cluster): fix #2532 tests redis client close errors

* refactor: address comments
2024-02-05 22:42:03 +02:00
Kostas Kyrimis
f707f6d821
fix: add emoji when regression tests timeout (#2543)
* add emoji when regression tests timeout
2024-02-05 16:33:37 +02:00
romange
97982eef2c chore(helm-chart): update to v1.14.2 2024-02-05 12:21:23 +00:00
Roman Gershman
5c0029978e
chore: remove redis sorted set implementation (#2522)
Also remove unused code.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-05 10:29:11 +02:00
Vladislav
e0f86697f9
fix: fix script replication (#2531)
* fix: fix script replication

Single key script replication was previously broken because the EXEC entry wasn't sent. Send it manually

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-02-04 20:28:44 +03:00
Andy Dunstall
562aa151a7
fix(zset): ignore cancelled op status in BZPopMinMax (#2527)
fix(zset): ignore cancelled op status
2024-02-03 09:35:05 +00:00
Vladislav
40d08a3c67
fix(transaction): Add special barrier for blocking tx (#2512)
Refactor blocking transaction code. Introduce BatonBarrier for managing atomic and exclusive wakeup notifications that don't conflict with neither expiration nor cancelling

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-02-01 23:37:36 +03:00
s-shiraki
7e875bdafe
feat(server): Implement CONFIG HELP command (#2510)
* feat(server): Implement CONFIG HELP command

* fix
2024-02-01 21:35:31 +02:00
Shahar Mike
b71d0f6383
feat(server): Add MEMORY TRACK ADDRESS command (#2513)
* feat(server): Add `MEMORY TRACK ADDRESS` command

Given a hex address, it replies with whether or not this address is
currently allocated by mimalloc's backing heap.

* init!

* Add HELP message
2024-02-01 21:31:08 +02:00
Vladislav
c987e64332
fix: handle null length strings in stored commands (#2518)
* fix: handle null length strings in stored commands

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-02-01 19:25:44 +03:00
Roman Gershman
b5b5093165
fix: fix BLOCKING/REVERSE_MAPPING flags for some commands (#2516)
* fix: BLOCKING/REVERSE_MAPPING flags for some commands

Also, simplify interfaces around REVERSE_MAPPING in the internal tx code.
---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-01 15:39:14 +00:00
Borys
5189dae118
feat(cluster): add migration finalization (#2507)
* feat(cluster): add migration finalization
2024-02-01 17:24:54 +02:00
Roman Gershman
adeac6bd27
Pr1 (#2517)
* fix: Remove a stale reference to blocking watch queue

1. Remove the duplicated FinalizeWatched function
2. Identify the case where we delete the watched queue while we may still have awakedened_keys pointing to it.
3. Add a test reproducing the issue of having in awakened_keys an untangled key.

Properly fixes #2514

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-01 14:19:08 +02:00
Roman Gershman
2b0310db32
fix: do not crash with inconsistent watch queue (#2515)
Output more info about the state of things.
Skip the non existent key and continue the execution.
Fixes #2514

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-01 09:48:42 +02:00
s-shiraki
73fe5a4eb2
fix(server): Implement SCRIPT FLUSH command (#2493)
* fix(server): Implement SCRIPT FLUSH command
2024-01-31 08:31:06 +02:00
Vladislav
90a9f05e36
chore(transaction): Use PhasedBarrier for easier synchronization (#2455)
chore(transaction): Use PhasedBarrier for easier synchronization

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-01-30 19:43:06 +03:00
adiholden
503891b1fa
fix(server): update post updater iterator in tiering (#2497)
* fix(server): update post updater iterator in tiering

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-01-30 11:46:00 +00:00
Shahar Mike
b2bdb0f683
feat(server): Memory tracker (#2501)
* feat(server): Memory tracker

* PR comments
2024-01-30 12:44:07 +02:00
Shahar Mike
ad90602bc2
test: Disable flaky test test_policy_based_eviction_propagation (#2505) 2024-01-30 10:28:00 +02:00
Shahar Mike
87fb07abe3
fix(server): Fix crash in RENAME (#2503)
Crash was a `CHECK()` failure when renaming to an existing key within
the same shard.

Specifically, the problem was that we ran post updater after deletion,
which is illegal as it could invalidate iterators.

Fixes #2502
2024-01-30 10:24:49 +02:00
Borys
1dee082f86
fix(cluster): fix incorrect version checking and resource double free (#2499)
* fix(cluster): fix incorrect version checking and resource double free
2024-01-29 14:14:21 +02:00
Roman Gershman
e4862f227d
chore: introduce TaskQueue for executing shard local tasks (#2498)
TaskQueue replaces fb2::FiberQueue and allows running multiple fibers on the consumer side.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-01-29 11:40:14 +03:00
Roman Gershman
97bde35a0f
Fix redis->DF replication tests (#2495)
fix: redis_replication_test fails due to long time for df replicas takes to catch up.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-01-29 03:25:58 +02:00
Roman Gershman
d7604c1bd0
chore: remove batch reply statistics (#2490)
1. average batch latency will always be 0, because even in cases we have outliers they will be dominated by
   small CPU only copies that take dozens of ns.
2. Measuring an operation like kBatch, which is solely CPU-based, necessitated the use of a clock. According to the CPU profiler,
   this contributed to approximately 5% of the CPU usage.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-01-29 03:23:38 +02:00
Kostas Kyrimis
754a186f5a
feat(lua): add missing no-op redis.replicate_commands() (#2472)
fixes #2468
add no-op lua function redis.replicate_commands()
2024-01-28 13:05:52 +02:00
Borys
43808da27f
fix(cluster): fix slot filtration to RestoreStreamer (#2477)
* fix(cluster): fix slot filtration to RestoreStreamer

* test: add cluster data migration test
2024-01-28 12:29:54 +02:00
Roman Gershman
d608ec9c62
chore: Introduce LockKey for LockTable (#2463)
This should reduce allocations in a common case (not multi).
In addition, rename Transaction::args_ to kv_args_.

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
Co-authored-by: Vladislav <vlad@dragonflydb.io>
2024-01-28 12:19:15 +02:00
adiholden
9f4c4353b5
fix(server): mget crash on same key get (#2474)
* fix(server): mget crash on same key get

fix: #2465
the bug: on cache mode mget bumps up items. When executing mget with the same key several times i.e mget key key we will invalidate the iterator when we bump up the item in dash table.
the fix: bump up/down items only once by using bumped_items set
This PR also reverts c225113
and updates the bumped stats and bumped_items set if the item was bumped

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-01-28 11:45:35 +02:00
Roman Gershman
3ebb32df3f
chore: lock keys when going through fast-path execution (#2491)
This is needed if we want to allow asynchronous transactional operations during the callback execution.
Also update actions versions.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-01-27 19:02:53 +02:00
Vladislav
675b3889a4
chore(transaction): Launder copied keys in multi transactions (#2478)
* chore(transaction): Launder copied keys in multi transactions

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-01-27 12:24:42 +02:00
Roman Gershman
8a3ccff0bc
chore: fix clang warnings (#2485) 2024-01-26 18:40:02 +03:00
Vladislav
7b6181641c
chore(transaction): Simplify PollExecution() (#2457)
* chore(transaction): Simplify PollExecution()

Remove seqlock_ from transaction. This change is possible because:
- We don't re-use shard_data[0] for multi transactions anymore
- We disarm atomically and poll callbacks are stateless

This makes it safe to call PollExecution() unconditionally that will determine on it's own whether the caller needs to run or is already expired

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-01-26 14:34:17 +03:00
Roman Gershman
ea5955962e
chore: switch json object to pmr allocator (#2483)
* chore: switch json object to pmr allocator

1. Move json object creation code to the shard-thread inside rdb_load
2. Now json_family never references "json" type, always dfly::JsonType
3. Switch JsonType to pmr::json.
4. Make sure we pass the correct memory_resource when creating json object from string.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-01-26 12:47:46 +02:00
Vladislav
a5b9401449
fix: reduce test_pipeline_batching_while_migrating flakiness (#2475)
* fix: reduce test_pipeline_batching_while_migrating flakiness
2024-01-25 17:55:12 +03:00
Kostas Kyrimis
ba972923b3
feat(lua): add no-op redis.log command (#2476)
* add no-op redis.log command
2024-01-25 15:45:47 +02:00