1
0
Fork 0
mirror of https://github.com/dragonflydb/dragonfly.git synced 2024-12-14 11:58:02 +00:00
Commit graph

756 commits

Author SHA1 Message Date
Roman Gershman
8f1a2a49b5
docs: Add how to build dragonfly on Fedora (#822)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-02-20 17:16:52 +02:00
adiholden
d738d8d976
bug(snapshot): Fix unwriten entries in multiple snapshotting
* bug(snapshot): Multiple snapshots at the same time skips serializing some entries
---------

Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-02-20 14:46:43 +02:00
ashotland
1f82f9af73
fix(regression-test): Sentinel test stabilization (#826)
* Ditch docker whcih is complex on CI in favour of local redis binary

Signed-off-by: ashotland <ari@dragonflydb.io>

* Fix typo

Signed-off-by: ashotland <ari@dragonflydb.io>

* Wait for sentinel termination

Signed-off-by: ashotland <ari@dragonflydb.io>

* fix(regression-tests): sentinel test increase timeout waiting for key to
exist in replica

* debug sentinel test

Signed-off-by: ashotland <ari@dragonflydb.io>

* add pytest repeat - tmp comment out failure notficicaiton

Signed-off-by: ashotland <ari@dragonflydb.io>

* fix typo

Signed-off-by: ashotland <ari@dragonflydb.io>

* repeat 100

Signed-off-by: ashotland <ari@dragonflydb.io>

* Increase timeout for debuging

Signed-off-by: ashotland <ari@dragonflydb.io>

* Debug prints

Signed-off-by: ashotland <ari@dragonflydb.io>

* fix

Signed-off-by: ashotland <ari@dragonflydb.io>

* increase overall timeout

Signed-off-by: ashotland <ari@dragonflydb.io>

* Debug

Signed-off-by: ashotland <ari@dragonflydb.io>

* cleanup

Signed-off-by: ashotland <ari@dragonflydb.io>

---------

Signed-off-by: ashotland <ari@dragonflydb.io>
2023-02-20 14:21:27 +02:00
Vladislav
03e99a5d96
EVAL multi modes + non atomic modes (#818)
- Implement multi modes for eval
- Implement non atomic mode
- Enhance tests
2023-02-20 09:43:31 +03:00
Roman Gershman
dd952c3c52
feat(server): add debug information about running transactions (#820)
Specifically, provide a total length over all shards of pending txs.
Also, provide number of "free" transactions pending in the queue -
those that could progress immediately but do not because
they are not first in the queue.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-02-19 21:20:14 +02:00
Ali-Akber Saifee
914dd23cdb
fix(zset): Make count optional for ZPOP{MIN,MAX} (#821)
fix(zset): Make count optional for zpop{min,max}

The commands should allow count to be optional and
default to 1 as per the official redis command documentation.

Additionally update command flags to write + fast

Signed-off-by: Ali-Akber Saifee <ali@indydevs.org>
2023-02-19 21:19:45 +02:00
Vladislav
4ef06e759a
Basic multi modes for MULTI/EXEC (#796)
feat(server): Basic multi transaction modes

This commit adds the notion of multi transaction modes that allow controlling the execution and
locking behaviour of multi transactions.
In general, there are four modes:
- GLOBAL: all commands run within a global transaction. There is no need for recording locks. Lua scripts can theoretically run with undeclared keys.
- LOCK_AHEAD: the transaction locks all keys ahead likewise to a regular transaction and schedules itself.
- LOCK_INCREMENTAL: the transaction determines what shards it has keys in and schedules itself on those shards, but locks only when accessing a new key. This allows other transactions to run ooo alonside with a big multi-transaction that accesses a contended key only at its very end.
- NON_ATOMIC: all commands run separately, no atomicity is provided, likewise to a pipeline

This commit only adds support for the first 3 modes to EXEC commands.

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-02-18 20:18:28 +03:00
ashotland
4b3dc87ba0
Fix(regression-tests): Sentinel test - wait for sentinel termination (#816)
* Ditch docker whcih is complex on CI in favour of local redis binary

Signed-off-by: ashotland <ari@dragonflydb.io>

* Fix typo

Signed-off-by: ashotland <ari@dragonflydb.io>

* Wait for sentinel termination

Signed-off-by: ashotland <ari@dragonflydb.io>

---------

Signed-off-by: ashotland <ari@dragonflydb.io>
2023-02-17 16:45:39 +02:00
ashotland
d6043c7d6d
feat(charts): Update helm chart to support password from secret (#799)
* feat(server): Enable overriding --requirepass form env var

Signed-off-by: ashotland <ari@dragonflydb.io>

* Change precednce order
Elaborate test

Signed-off-by: ashotland <ari@dragonflydb.io>

* Update helm chart to support passowrd from secret

Using recently added capability of setting dragonfly passowrd with the
DFLY_PASSWORD environment variable

Signed-off-by: ashotland <ari@dragonflydb.io>

* rename to existingSecret* + CI file

Signed-off-by: ashotland <ari@dragonflydb.io>

* fix typo in comment

Signed-off-by: ashotland <ari@dragonflydb.io>

---------

Signed-off-by: ashotland <ari@dragonflydb.io>
2023-02-17 16:19:02 +02:00
Vladislav
2b40a7f324
feat(server): Check locks in heartbeat, allow multiple tx in replica (#815) 2023-02-17 16:50:44 +03:00
Roman Gershman
f92403fb09
feat(core): Add Ttl semantics to string_map (#813)
Now string_map respects the optional ttl argument that accepts ttl time in seconds.
Upon traversing and accessing expirerd elements they will be transparently removed and deleted
from the map.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-02-17 10:49:21 +02:00
Vladislav
18ce8c3a5b
tests(server): Add multi_test (#810) 2023-02-16 14:47:23 +03:00
ashotland
2f89cdc6af
chore(regressions-tests): Update regression-tests.yml (#805)
* Update regression-tests.yml

Run every 3 hours
Add -s to pytest

Signed-off-by: ashotland <ari@dragonflydb.io>

* Update regression-tests.yml

Signed-off-by: ashotland <ari@dragonflydb.io>

---------

Signed-off-by: ashotland <ari@dragonflydb.io>
2023-02-16 12:16:39 +02:00
Roman Gershman
8739f811ea
chore(server): reduce number of allocations with multi-exec commands (#807)
chore(server): reduce number of allocations when storing multi-exec commands

This is useful when DF is used to store large blobs and MULTI is used.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-02-15 18:03:40 +02:00
adiholden
ae6687e563
fix(regression-tests): increase timeout of regression-tests (#806)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-02-15 12:39:23 +02:00
Vladislav
83256bd4e1
fix(pytest): fix periodic test (#804)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-02-15 11:12:41 +02:00
Vladislav
6dedb33f85
fix(pytest): SAVE to SAVE RDB (#803) 2023-02-15 11:15:38 +03:00
adiholden
50f50c8380
feat(server): write journal record with optional await based on flag… (#791)
* feat(server): write journal recorod with optional await based on flag issue #788

Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-02-15 09:34:24 +02:00
Roman Gershman
8068e1a2ae
feat(server): save dragonfly snapshot in a new format by default (#802)
Introduce a new flag `df_snapshot_format` that controls the default behavior.
The flag itself is defaulted to save snapshots in a new Dragonfly
specific format.

BREAKING CHANGE: before that Dragonfly saved snapshots in rdb format by default.
Now the flag controls which format is chosen when issuing SAVE/BGSAVE commands.
In addition, "SAVE DF", "SAVE RDB" can be used to save using dragonfly/redis formats
accordingly.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-02-15 06:19:12 +02:00
Nick Gottlieb
15c856726a
update readme (#794)
* Add files via upload

Signed-off-by: Nick Gottlieb <ngottlieb1@gmail.com>

* Update README.md

Signed-off-by: Nick Gottlieb <ngottlieb1@gmail.com>

* Delete logo-full.svg

Signed-off-by: Nick Gottlieb <ngottlieb1@gmail.com>

* Update README.md

Signed-off-by: Nick Gottlieb <ngottlieb1@gmail.com>

* Update README.md

Signed-off-by: Nick Gottlieb <ngottlieb1@gmail.com>

* Update README.md

Signed-off-by: Nick Gottlieb <ngottlieb1@gmail.com>

* Update README.md

Signed-off-by: Nick Gottlieb <ngottlieb1@gmail.com>

* Rename logo.svg to logo-full.svg

Signed-off-by: Nick Gottlieb <ngottlieb1@gmail.com>

* Update README.md

Signed-off-by: Nick Gottlieb <ngottlieb1@gmail.com>

* Update README.md

Signed-off-by: Nick Gottlieb <ngottlieb1@gmail.com>

---------

Signed-off-by: Nick Gottlieb <ngottlieb1@gmail.com>
2023-02-14 23:08:32 +02:00
Roman Gershman
81f0b5034d
feat(facade): Limit request cache using runtime flag (#793) 2023-02-14 17:35:01 +02:00
adiholden
72bad6c5ab
fix(replica) : replica will not sync execution multi shard commands as default (#800)
-sfix(replica) : replica will not sync execution multi shard commands as default
2023-02-14 16:30:14 +02:00
adiholden
c5e8a627b5
test(regression-tests): stop replication regression-tests schedual run (#795)
* regression replication will not run on github schedual
* allow running regression replication manually

Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-02-14 14:48:39 +02:00
ashotland
49b1ba5b6d
feat(server): Enable overriding --requirepass form env var (#792)
Signed-off-by: ashotland <ari@dragonflydb.io>
2023-02-14 13:19:33 +02:00
Vladislav
25db011afc
fix(server): Fix bugs (#797) 2023-02-14 13:42:20 +03:00
Vladislav
3e46fd1318
Refactor initialization phase of transaction (#790)
* refactor(server): Split transaction init

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-02-14 08:32:02 +02:00
Vladislav
6837c38488
fix(server): Fix transaction bug (#787)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-02-13 14:38:21 +02:00
adiholden
ee2807c9bc
feat(replica): add debug command - replica offset (#786)
* feat(replica): add debug command - replica offset

Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-02-13 13:00:46 +02:00
adiholden
9c9ae84493
feat(server): add dfly replica offset command (#780)
* feat(server): add dfly replica offset command

Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-02-13 11:11:33 +02:00
Boaz Sade
e7a5d583d0
fix(server): monitor lua at script execution (#767)
Signed-off-by: Boaz Sade <boaz@dragonflydb.io>
2023-02-12 18:22:40 +02:00
Boaz Sade
b62a590a3c
test(server): adding unit tests for reply builder (#764)
Signed-off-by: Boaz Sade <boaz@dragonflydb.io>
2023-02-12 13:37:49 +02:00
ashotland
1dd4148f64
Update README.md (#778)
* Update README.md

Signed-off-by: ashotland <ari@dragonflydb.io>

* Document installing from remote OCI package

Signed-off-by: ashotland <ari@dragonflydb.io>

---------

Signed-off-by: ashotland <ari@dragonflydb.io>
2023-02-12 13:26:43 +02:00
Roman Gershman
6f1e49120e
feat(server): track pipeline requests cache capacity (#776)
1. now pipeline_cache_capacity tracks how many bytes are allocated via free_req_pool_ cache.
   it's been shown now via "pipeline_cache_bytes" stat in "info memory" command.
2. a small refactoring of server_state code into server_state.cc
3. Reduce dependency of dfly_transaction on facade lib.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-02-12 11:58:21 +02:00
Vladislav
6e612e7545
feat(server): Async unlock multi (#774)
* feat(server):Async unlock multi

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-02-11 18:57:02 +02:00
Roman Gershman
3ba46a5097
feat(server): Extend malloc-stats command functionality. (#775)
1. Remove dangerous check-fail in SegmentAllocator.
2. Allow printing heap stats for both backing and data heaps.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-02-11 07:17:48 +02:00
Roman Gershman
83837532e9
chore(server): General improvements
1. Update helio dependency
2. Add SLOWLOG command stub that returns empty data.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-02-10 22:37:19 +02:00
Vladislav
07973d40eb
Update transaction and enable OOO for regular transactions (#769)
* refactor(server): Update ScheduleSingleHop

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-02-09 19:36:55 +02:00
adiholden
41c1ebab18
bug(replica): execute expire within multi command only if its belong … (#766)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-02-08 14:33:05 +02:00
Vladislav
b2958b5381
feat(server): Pool lua interpreters (#765) 2023-02-08 14:56:32 +03:00
Roman Gershman
46b42e571f
chore(server): track ooo transactions via metrics. (#763)
This change allows to track which transactions are run as out of order.
OOO txs are more performant and inhibit substantially less latency.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-02-07 19:40:25 +02:00
adiholden
4a826fdb7b
bug(transaction): local result needs to be reset on InitByArgs Fixes … (#762)
* bug(transaction): local result needs to be reset on InitByArgs Fixes #752

Signed-off-by: adi_holden <adi@dragonflydb.io>

* add unit test

Signed-off-by: adi_holden <adi@dragonflydb.io>

---------

Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-02-06 15:48:12 +02:00
Roman Gershman
448508a23a
feat(server): add total_connections_received statistics (#757)
In addition:

1. improve logging of received commands
2. provide stable ordering of error stats and commandstats

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-02-06 11:00:42 +02:00
adiholden
cc74594c2f
bug(replication): BLPOP fix write to shard journal of popped key (#761)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-02-06 10:04:48 +02:00
adiholden
69bca570f0
bug(server): fix deadlock in BufferedStreamerBase notify all producers (#760)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-02-06 10:04:22 +02:00
Roman Gershman
4c725b6654
feat(server): add latency statistics for lua script calls (#758)
Also provide new subcommands "script list" and "script latency"
Fixes #754

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-02-05 21:38:59 +02:00
ashotland
aef8631ab2
fix(regression-tests): sentinel_test.py: Ditch docker whcih is complex on CI in favour of local redis binary (#755)
* Ditch docker whcih is complex on CI in favour of local redis binary

Signed-off-by: ashotland <ari@dragonflydb.io>

* Fix typo

Signed-off-by: ashotland <ari@dragonflydb.io>

---------

Signed-off-by: ashotland <ari@dragonflydb.io>
2023-02-05 11:58:09 +02:00
Roman Gershman
da40b63433
feat(list): Add BLMove command (#753)
Also update api_status with ZUNION and BRPOPLPUSH
Fixes #751

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-02-05 10:55:20 +02:00
romange
8d095d00fa chore(helm-chart): update to v0.15.0 2023-02-05 08:33:11 +00:00
adiholden
d30ebc5b0d
feat(server): replication should not evict items (#743)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-02-05 09:04:08 +02:00
Vladislav
3449ba8a7b
Exclude single threaded test now that expiry is tested (#750) 2023-02-03 23:08:29 +03:00