1
0
Fork 0
mirror of https://github.com/dragonflydb/dragonfly.git synced 2024-12-14 11:58:02 +00:00
Commit graph

1989 commits

Author SHA1 Message Date
Vladislav
5ac5e3bdac
fix: replace localhost with 127.0.0.1 in cluster tests (#2665) 2024-02-27 12:53:54 +03:00
Roman Gershman
df39d3a286
chore: move jsoncons path code into dedicated files (#2666)
It's only a code move, without functional changes.
This is in preparation to implementing the same path functionality
for flexbuffers objects.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-27 11:50:24 +02:00
Roman Gershman
91c299b33e
chore: Del and NUMINCRBY use json::Path (#2655)
* chore: Del and NUMINCRBY use json::Path

Also, fix various protocol bugs when we sent simple string
instead of sending bulk strings.

Fixed a typo in path.cc that lead to a data race bug.

Finally, flip the flag in regression tests to start covering json::Path code
and added test coverage for the data race bug

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-26 14:06:21 +00:00
Kostas Kyrimis
d54f2201ae
feat: add current_fork_perc in info all command (#2640)
* add field current_snapshot_perc (instead of current_fork_perc)
* add field current_save_keys_processed
* add field current_save_keys_total
2024-02-26 11:17:31 +02:00
Roman Gershman
d54b600f31
chore: switch to self-built flatbuffers (#2657)
* chore: switch to self-built flatbuffers

This should solve the linker errors on MacOs build.

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-26 10:03:17 +02:00
Vladislav
58f284c4f4
chore: Move CallFromScript log to level 2 (#2656) 2024-02-25 12:25:28 +03:00
Roman Gershman
050649d762
chore: fully switch to dual parsing with json::Path (#2654)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-24 13:26:31 +02:00
Roman Gershman
2b30f69fe3
chore: JSON.GET now works with our own jsonpath (#2653) 2024-02-24 13:23:41 +02:00
Vladislav
5ee61db0f3
feat(connection): Support pipelining with Memcached (#2648)
* feat(connection): Support pipelining with Memcached

Adds support for pipelining to Memcached, enhances Memcached pytests

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-02-23 20:18:25 +03:00
Andy Dunstall
47171c4c8e
fix: fix zmalloc_size on macos (#2646)
* fix: fix zmalloc_size on macos

---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Co-authored-by: Roman Gershman <roman@dragonflydb.io>
2024-02-23 17:39:58 +02:00
Kostas Kyrimis
3d27913c46
fix: data race in save_stages_controller (#2647)
* fix: data race in save_stages_controller
2024-02-23 17:35:20 +02:00
Borys
8771ab32a6
refactor: create one type for slots set #2459 (#2645)
* refactor: create one type for slot ranges #2459
2024-02-23 14:10:42 +02:00
Roman Gershman
bcae2dfb46
a test with flat buffers (#2520)
feat: a test with flat buffers

Also, add an experimental flag `--experimental_flat_json` that allows writing json objects as flat strings using
flexibuffers.

The experiment shows that `debug populate 100000 a 10 type json elements 30`
uses almost 3 times less memory than with native jsoncons objects.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-23 13:53:41 +02:00
Roman Gershman
a06d40567b
fix: avoid crashing during the connection traversal (#2651)
* fix: avoid crashing during the connection traversal

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-23 10:12:36 +00:00
Roman Gershman
6fce3fca9f
chore: fix build for MacOs (#2635)
Also update actions versions to Node 20.
This change allows dragonfly to be built on MacOs.
However, we still have multiple failing tests on MacOS.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-22 12:02:20 +02:00
romange
ea98513a30 chore(helm-chart): update to v1.14.5 2024-02-22 08:17:26 +00:00
Shahar Mike
9baf7c2645
fix(streamer): Do not yield from the Traverse callback. (#2638)
* fix(streamer): Do not yield from the Traverse callback.

Yielding inside the callback can move entries within the bucket, which
is unsupported.

* fix
2024-02-21 20:26:16 +02:00
adiholden
9e66ec5833
fix(bug): server crash info memory while saving (#2637)
* bug: crash info memory while saving

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-02-21 13:39:57 +00:00
Kostas Kyrimis
52548fe3a0
refactor: remove goto statements from replica.cc (#2633)
* remove goto statements from replica.cc
2024-02-21 13:02:36 +02:00
Kostas Kyrimis
f32156788e
feat(regTests): upload only failed test logs on ci and clean up logging (#2547)
* upload only failed test logs
* remove printing log names for passed tests
* print slow tests with --duration
* separate regression and unit logs for CI workflow
2024-02-21 10:35:07 +02:00
thweetkomputer
8fd880c844
fix: crash when using nonexistent global variable in lua (#2627)
fix: check whether key exists in FetchKey()

Signed-off-by: zc <2320092610@qq.com>
2024-02-21 08:50:59 +02:00
Roman Gershman
cbfd5bb7c5
chore: doc_accessors now parse with json::Path as well (#2615)
chore: replace search path parsing with json::Path

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-20 19:58:22 +02:00
romange
8cce14ce85 chore(helm-chart): update to v1.14.4 2024-02-20 16:55:13 +00:00
Roman Gershman
ea6f059b30
fix: provide more context in case we have duplicate keys (#2619)
Partially addresses #2500 and avoid crashing the process.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-20 16:04:49 +02:00
Shahar Mike
ebca523166
fix(cluster_mgr): Disable CPU affinity (#2632) 2024-02-20 13:43:17 +00:00
Kostas Kyrimis
a1950038b6
fix: loading state error type to be compatible with redis (#2629)
* add -LOADING prefix for loading errors
* replace -ERR with -LOADING for loading errors
2024-02-20 12:45:41 +00:00
Vladislav
75eaeb32db
feat(pytest): More types for seeder (#2577)
* feat(pytest): More types for seeder

Add more types to the seeder and refactor replication test

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-02-20 15:31:08 +03:00
Shahar Mike
c7750b9d58
feat(cluster_mgr): Add support for migrate action (#2626)
Example usage:

```bash
# Create a 2-node cluster
./cluster_mgr.py --action=create --replicas_per_master=1 --num_master=2

# Move (no migration) all slots to first node
./cluster_mgr.py --action=move --target_port=7001 --slot_start=8192 --slot_end=16383

# Fill data - like run memtier

# Migrate all slots to 2nd node. One could measure how long this step takes.
./cluster_mgr.py --action=migrate --target_port=7002 --slot_start=0 --slot_end=16383
```
2024-02-20 13:58:13 +02:00
Shahar Mike
ebe83c820c
test: Run slow tests in regression runs (again^2) (#2623)
* test: Run slow tests in regression runs (again^2)

* better comment
2024-02-20 13:57:40 +02:00
adiholden
1ef8795611
fix(bug): crash when starting replica while saving (#2618)
The bug: crash when starting replica while saving
The problem: accessing the wrong allocator on snapshot class destruction as it was destructed not in the thread of the shard
The fix: call snapshot destructor when we finish snapshot on the correct thread 

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-02-20 10:50:46 +00:00
Shahar Mike
5f3e9a88c9
test: Run slow tests in regression runs (again:) (#2616) 2024-02-19 20:44:11 +02:00
Borys
491538e170
feat(cluster): #2448 add new DFLYMIGRATE ACK cmd (#2582)
* feat(cluster): #2448 add new DFLYMIGRATE ACK cmd
2024-02-19 19:21:14 +02:00
Leonardo Mello
07a8411ee7
fix(server_family): GetMetrics should show commands in lowercase (#2601)
* replace with lowercase cmdstat_info and cmdstat_replconf

Signed-off-by: Leonardo Mello <lsvmello@gmail.com>
2024-02-19 18:50:59 +02:00
adiholden
15b3fb13b6
fix(server): saving is not a server state (#2613)
* fix(server): saving is not a server state

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-02-19 15:20:48 +00:00
Kostas Kyrimis
58dda3bc4b
fix(rdb_load): EnsureRead(min) requesting more bytes than min (#2604)
* fixes EnsureRead(min) to request exactly min bytes
2024-02-19 14:41:25 +02:00
Vladislav
d035111bed
fix: fix cluster test (#2612)
* fix: fix cluster test
2024-02-19 11:03:04 +03:00
Roman Gershman
c8332edfa9
chore: allow aggregate functions for jsonpath (#2611)
Specifically, allows expressions in the form of
`max($.objs[*].score`, i.e. where a function aggregates 0 or more values
produced by the path expression. The final value is passed to a PathCallback.
In case no matches are found, json null value is passed. In any case, the callback
is called exactly once for such expressions.

In addition, fixed a parser bug that does not allow fields after the bracketing expression.

Finally, introduced an utility function for easier parsing of json::Path.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-19 10:02:15 +02:00
Roman Gershman
e7c96925be
chore: clean up redis code (#2614)
Remove unused redis files and file parts.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-19 08:38:21 +02:00
Roman Gershman
fa75360227
chore: get rid of object.c and robj* in cc code (#2610)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-18 16:52:23 +02:00
Roman Gershman
417ca952d6
add loadtest benchmarking program (#2603)
* feat: add loadtest skeleton
---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Co-authored-by: shahar <shahar@dragonflydb.io>
2024-02-18 16:51:38 +02:00
Roman Gershman
750f039316
Add mutate method to jsonpath (#2606)
* Add mutate method to jsonpath

Plug it in into OpToggle op.

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-18 16:06:33 +02:00
Vladislav
1b51e82e55
chore(transaction): Add debug stats for fail printing (#2600)
* chore(transaction): Add debug stats for per shard data

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-02-18 15:36:14 +03:00
Vladislav
4d4fed6fec
chore(transaction): Untie scheduling from multi status (#2590)
* chore(transaction): Untie scheduling from multi status

Idea: We decide whether we have to schedule not based on our multi status (atomic multi), but solely based on the fact if COORD_SCHED is set

Goal: Being able to use ScheduleSingleHop()/Schedule() for multi transactions, and thus later allow single hop multi transactions
---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-02-18 12:25:57 +03:00
Roman Gershman
d802a2181a
chore: consolidate json code under core/json (#2609)
Fix linker dependencies - now dfly_core depends on jsonpath.
jsonpath does not depend on CompactObject anymore.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-18 09:22:34 +00:00
Roman Gershman
1ba59e9179
chore: remove ImportRObj routine and reduce reliance on object.c (#2607)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-18 10:07:00 +02:00
Roman Gershman
6aafe3dc5e
chore: cleanup of redis implementation of sets/hashmaps (#2605)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-17 18:35:28 +02:00
Roman Gershman
3ec56452f9
chore: refactor DfsItem to allow mutability semantics (#2602)
No functional changes are made.
This change is needed to allow traversal functionality that will mutate json object during the traversal.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-16 10:13:20 +02:00
Roman Gershman
888a437bf5
wire json::Path into json_family (#2587)
* feat: wire json::Path into the server

The feature is disabled under a flag and it covers read operations for now.
---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-15 17:05:53 +00:00
Roman Gershman
1e06c63727
feat: Add dbfilename to config registry (#2598)
Also make "dir" read-only parameter.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-15 18:00:36 +02:00
Roman Gershman
f847a3f4cd
feat: support descent path operator (#2594)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-02-15 15:03:07 +02:00