Roman Gershman
f8b3fa0d7b
feat: support GCS storage for saving/loading snapshots ( #4064 )
2024-11-06 12:11:59 +02:00
adiholden
ae3faf59fb
feat(server): dont use channel for replication / save df ( #4041 )
...
* feat server: dont use channel for replication / save df
Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-11-05 16:50:01 +02:00
Roman Gershman
7df8c268d8
chore: eliminate redundant ConnectionContext arguments ( #4065 )
2024-11-05 10:40:04 +02:00
dependabot[bot]
54c67a9198
chore(deps): bump pytest-repeat from 0.9.1 to 0.9.3 in /tests/dragonfly ( #4057 )
...
Bumps [pytest-repeat](https://github.com/pytest-dev/pytest-repeat ) from 0.9.1 to 0.9.3.
- [Release notes](https://github.com/pytest-dev/pytest-repeat/releases )
- [Changelog](https://github.com/pytest-dev/pytest-repeat/blob/main/CHANGES.rst )
- [Commits](https://github.com/pytest-dev/pytest-repeat/compare/v0.9.1...v0.9.3 )
---
updated-dependencies:
- dependency-name: pytest-repeat
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-04 23:47:24 +02:00
dependabot[bot]
aa4a61894d
chore(deps): bump docker/build-push-action from 5 to 6 ( #4055 )
...
Bumps [docker/build-push-action](https://github.com/docker/build-push-action ) from 5 to 6.
- [Release notes](https://github.com/docker/build-push-action/releases )
- [Commits](https://github.com/docker/build-push-action/compare/v5...v6 )
---
updated-dependencies:
- dependency-name: docker/build-push-action
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-04 22:38:06 +02:00
dependabot[bot]
d6ec5910d1
chore(deps): bump github/codeql-action from 3.24.9 to 3.27.0 ( #4054 )
...
Bumps [github/codeql-action](https://github.com/github/codeql-action ) from 3.24.9 to 3.27.0.
- [Release notes](https://github.com/github/codeql-action/releases )
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md )
- [Commits](1b1aada464...662472033e
2024-11-04 22:37:51 +02:00
dependabot[bot]
8dc12a39e5
chore(deps): bump ossf/scorecard-action from 2.3.1 to 2.4.0 ( #4053 )
...
Bumps [ossf/scorecard-action](https://github.com/ossf/scorecard-action ) from 2.3.1 to 2.4.0.
- [Release notes](https://github.com/ossf/scorecard-action/releases )
- [Changelog](https://github.com/ossf/scorecard-action/blob/main/RELEASE.md )
- [Commits](0864cf1902...62b2cac7ed
2024-11-04 22:37:07 +02:00
dependabot[bot]
f67103f9b0
chore(deps): bump mozilla-actions/sccache-action from 0.0.4 to 0.0.6 ( #4051 )
...
Bumps [mozilla-actions/sccache-action](https://github.com/mozilla-actions/sccache-action ) from 0.0.4 to 0.0.6.
- [Release notes](https://github.com/mozilla-actions/sccache-action/releases )
- [Commits](https://github.com/mozilla-actions/sccache-action/compare/v0.0.4...v0.0.6 )
---
updated-dependencies:
- dependency-name: mozilla-actions/sccache-action
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-04 22:36:20 +02:00
dependabot[bot]
18f7d9bbd5
chore(deps): bump mikepenz/action-junit-report from 4 to 5 ( #4052 )
...
Bumps [mikepenz/action-junit-report](https://github.com/mikepenz/action-junit-report ) from 4 to 5.
- [Release notes](https://github.com/mikepenz/action-junit-report/releases )
- [Commits](https://github.com/mikepenz/action-junit-report/compare/v4...v5 )
---
updated-dependencies:
- dependency-name: mikepenz/action-junit-report
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-04 22:35:47 +02:00
dependabot[bot]
94ee96afd0
chore(deps): bump redis-om from 0.2.2 to 0.3.3 in /tests/dragonfly ( #4060 )
...
Bumps [redis-om](https://github.com/redis/redis-om-python ) from 0.2.2 to 0.3.3.
- [Release notes](https://github.com/redis/redis-om-python/releases )
- [Commits](https://github.com/redis/redis-om-python/compare/v0.2.2...v0.3.3 )
---
updated-dependencies:
- dependency-name: redis-om
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-04 22:35:18 +02:00
dependabot[bot]
eedb04509d
chore(deps): bump github.com/gruntwork-io/terratest from 0.41.11 to 0.47.2 in /contrib/charts/dragonfly ( #4058 )
...
chore(deps): bump github.com/gruntwork-io/terratest
Bumps [github.com/gruntwork-io/terratest](https://github.com/gruntwork-io/terratest ) from 0.41.11 to 0.47.2.
- [Release notes](https://github.com/gruntwork-io/terratest/releases )
- [Commits](https://github.com/gruntwork-io/terratest/compare/v0.41.11...v0.47.2 )
---
updated-dependencies:
- dependency-name: github.com/gruntwork-io/terratest
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-04 22:34:48 +02:00
dependabot[bot]
b70f9703f4
chore(deps): bump tomli from 2.0.1 to 2.0.2 in /tests/dragonfly ( #4059 )
...
Bumps [tomli](https://github.com/hukkin/tomli ) from 2.0.1 to 2.0.2.
- [Changelog](https://github.com/hukkin/tomli/blob/master/CHANGELOG.md )
- [Commits](https://github.com/hukkin/tomli/compare/2.0.1...2.0.2 )
---
updated-dependencies:
- dependency-name: tomli
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-04 22:31:29 +02:00
dependabot[bot]
86b64d910a
chore(deps): bump github.com/redis/go-redis/v9 from 9.5.1 to 9.7.0 in /tools/replay ( #4062 )
...
chore(deps): bump github.com/redis/go-redis/v9 in /tools/replay
Bumps [github.com/redis/go-redis/v9](https://github.com/redis/go-redis ) from 9.5.1 to 9.7.0.
- [Release notes](https://github.com/redis/go-redis/releases )
- [Changelog](https://github.com/redis/go-redis/blob/master/CHANGELOG.md )
- [Commits](https://github.com/redis/go-redis/compare/v9.5.1...v9.7.0 )
---
updated-dependencies:
- dependency-name: github.com/redis/go-redis/v9
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-04 22:31:01 +02:00
dependabot[bot]
ceb474fbda
chore(deps): bump numpy from 1.24.1 to 2.1.3 in /tools ( #4063 )
...
Bumps [numpy](https://github.com/numpy/numpy ) from 1.24.1 to 2.1.3.
- [Release notes](https://github.com/numpy/numpy/releases )
- [Changelog](https://github.com/numpy/numpy/blob/main/doc/RELEASE_WALKTHROUGH.rst )
- [Commits](https://github.com/numpy/numpy/compare/v1.24.1...v2.1.3 )
---
updated-dependencies:
- dependency-name: numpy
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-04 22:30:34 +02:00
Roman Gershman
8d61a91200
chore: pass reply_builder explicitly to pubsub module ( #4021 )
...
Also, deprecate `reply_builder()` access method.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-11-04 19:20:12 +02:00
BLANKatGITHUB
fb7ea6c827
feat(dependabot): Add dependabot to check for vulnerabilies and updat… ( #4035 )
...
* feat(dependabot): Add dependabot to check for vulnerabilies and update dependencies
* feat(dependabot): Adds dependabot to update dependencies automatically
Signed-off-by: BLANKatGITHUB <131886247+BLANKatGITHUB@users.noreply.github.com>
* feat(dependabot): adds support for dependabot
---------
Signed-off-by: BLANKatGITHUB <131886247+BLANKatGITHUB@users.noreply.github.com>
2024-11-04 19:19:04 +02:00
Stepan Bagritsevich
7ac853567b
fix(search_family): Fix crash in FT.PROFILE command for invalid queries ( #4043 )
...
* refactor(search_family): Remove unnecessary std::move in FT.SEARCH
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
* fix(search_family): Fix crash in FT.PROFILE command for invalid queries
fixes dragonflydb#3983
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
* refactor(search_family_test): address comments
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
---------
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2024-11-04 18:18:12 +01:00
Kostas Kyrimis
9c2fc3fe63
chore: add ForceUpdate to BPTree ( #3993 )
...
ReallocIfNeeded deleted and reinserted the same key in the BPTree stored in SortedMap. This is not really needed since the key is actually the same and we can just update the pointer within BPTree instead of doing the deletion + insertion chore.
---------
Signed-off-by: kostas <kostas@dragonflydb.io>
2024-11-04 15:53:13 +02:00
Kostas Kyrimis
4859077122
chore(DenseSet): defrag all links in a chain ( #4019 )
...
* add defrag for all nodes in a link chain
* add tests
---------
Signed-off-by: kostas <kostas@dragonflydb.io>
2024-11-04 08:34:22 +00:00
Roman Gershman
cc6fdd7fbf
chore: add retry to test_noreply_pipeline test ( #4045 )
2024-11-04 08:32:48 +00:00
Roman Gershman
76b1f537b5
chore: refactor part of s3 logic for loading a snapshot. ( #4044 )
...
Is done as preparation to share some of its code with GCS implementation.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-11-04 10:13:53 +02:00
Kostas Kyrimis
e103254cf9
chore: add tx queue head debug info in AnalyzeTxQueue ( #4026 )
...
* add tx queue head debug info in AnalyzeTxQueue
Signed-off-by: kostas <kostas@dragonflydb.io>
2024-11-04 09:41:18 +02:00
Borys
e4b468d953
fix: reduce memory consumption during migration ( #4017 )
...
* refactor: reduce memory consumption for RestoreStreamer
* fix: add Throttling into RestoreStreamer::WriteBucket
2024-11-03 17:03:45 +02:00
Borys
5a597cf6cc
test: update test_big_containers ( #4025 )
2024-11-03 16:20:11 +02:00
BLANKatGITHUB
121a01f78a
feat(pre-commit): adds senstive data check like api keys and linting … ( #4030 )
...
* feat(pre-commit): adds senstive data check like api keys
---------
Signed-off-by: BLANKatGITHUB <131886247+BLANKatGITHUB@users.noreply.github.com>
2024-11-02 09:44:00 +02:00
Borys
cd2ab4a585
chore: decrease logs severity for OOM from error to warning ( #4031 )
2024-11-01 11:12:28 +02:00
Roman Gershman
30728021d0
chore: pull helio ( #4036 )
2024-11-01 11:09:49 +02:00
Roman Gershman
d10e76b408
chore: support load/save from GCS ( #4006 )
...
Not everything is supported but manual load save is supported.
1. Run dragonfly with `--dir gs://bucket/path`
2. In redis-cli:
a. SET foo bar
b. SAVE DF gsdump
c. DFLY LOAD gs://bucket/path/gsdump-summary.dfs
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-10-30 13:57:58 +02:00
Kostas Kyrimis
daf8604175
chore: flush journal to channel for non auto journal commands ( #3945 )
...
* flush journal to channel for non auto journal commands
2024-10-30 12:09:01 +02:00
Roman Gershman
c8b56b69b4
chore: print info stats if test_noreply_pipeline fails ( #4016 )
2024-10-30 08:17:15 +00:00
Roman Gershman
6f6897cef1
chore: pass RedisReplyBuilder explicitly from dragonfly connection ( #4009 )
...
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-10-29 14:52:09 +02:00
Kostas Kyrimis
566f246cee
chore: skip AggregateWithLoadOptionHard on sanitizers (false positive) ( #4013 )
...
The test fails because of false positive so I disabled it.
Signed-off-by: kostas <kostas@dragonflydb.io>
2024-10-29 14:26:50 +02:00
Roman Gershman
7737bb46f8
chore: improve verbosity logging during shutdown ( #4010 )
...
Increase verbosity levels of uring_proactor if deadlock was detected in debug mode.
Also allow optionally to disable periodic fiber, which sometimes helps during the debugging.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-10-29 13:45:57 +02:00
Kostas Kyrimis
4b495182e8
fix: separate Heartbeat and ShardHandler to fibers ( #3936 )
...
* separate shard_handler from Heartbeat
* add test
---------
Signed-off-by: kostas <kostas@dragonflydb.io>
2024-10-29 09:22:53 +02:00
Diskein
f16a32582e
feat(zset_family): support WITHSCORE in zrevrank/zrank commands ( #3921 ) ( #4001 )
...
Signed-off-by: Diskein <kalantaevskii@gmail.com>
2024-10-29 04:44:27 +02:00
Roman Gershman
92be74f4e4
fix: build break in search_family ( #4008 )
...
Also perform additional clean-up of cntx->reply_builder() - Part11
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-10-28 17:40:27 +02:00
Roman Gershman
1bdd56c973
chore: pass SinkReplyBuilder and Transaction explicitly. Part10 ( #3998 )
2024-10-28 16:18:52 +02:00
Roman Gershman
41d8c66a15
fix: flaky test_failover test ( #4007 )
...
Prevent from checking replica too early,
to avoid "Loading" exception.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-10-28 14:58:36 +02:00
Roman Gershman
b9c1aaec90
chore: introduce GcsSnapshotStorage ( #4004 )
...
It's not been used yet so no functional changes.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-10-28 09:57:58 +00:00
Roman Gershman
c2710604de
chore: refactor snapshot expanding logic ( #4003 )
...
S3 and file expansion logic had some duplicate code.
this PR refactors it before adding GCS support.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-10-28 10:08:10 +02:00
Roman Gershman
db6504564d
chore: clean up conn_use_incoming_cpu heuristic ( #3978 )
...
Add a comment explaining when it can be useful.
It usually won't work without tuning the host first.
See https://docs.kernel.org/networking/scaling.html
for more details.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-10-28 09:54:44 +02:00
Roman Gershman
b0d52c69ba
feat: introduce metrics/logs of when pipelining is being throttled ( #4000 )
...
* feat: introduce metrics/logs of when pipelining is being throttled
Fixes #3999 following up on discussion at #3997 .
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-10-28 09:15:04 +02:00
Stepan Bagritsevich
5dcad859b0
fix(search_family): Fix indexes loading in the FT.SEARCH and Ft.AGGREGATE commands ( #3955 )
...
fixes dragonflydb#3782, dragonflydb#3659
Signed-off-by: Stsiapan Bahrytsevich <stefan@dragonflydb.io>
2024-10-27 21:14:02 +01:00
BLANKatGITHUB
851e43211e
refactor(workflow): changes default permissions to read for few workflows increasing security ( #3905 )
...
* refactor: makes read default permision
---------
Signed-off-by: BLANKatGITHUB <131886247+BLANKatGITHUB@users.noreply.github.com>
2024-10-26 19:58:23 +03:00
Roman Gershman
c895b96274
fix: keep the ttl bit when dense_set grows with expiring items ( #3995 )
...
Add a test that triggerred a corner case with assertion failure.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-10-25 22:29:54 +03:00
Roman Gershman
408a8a33f2
chore: pass SinkReplyBuilder and Transaction explicitly. Part9 ( #3996 )
2024-10-25 16:51:37 +03:00
Roman Gershman
ef09052482
chore: pass SinkReplyBuilder and Transaction explicitly. Part7 ( #3988 )
...
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-10-25 13:32:51 +03:00
Roman Gershman
a2a5e3179a
chore: pass SinkReplyBuilder and Transaction explicitly. Part8 ( #3992 )
2024-10-25 13:32:28 +03:00
Vladislav
eb9b6890b0
feat(search): Prefix search for tags ( #3972 )
2024-10-25 08:47:27 +03:00
Borys
0a62f6b6ba
fix: prevent starting migrations from replica ( #3985 )
2024-10-25 08:41:26 +03:00
