1
0
Fork 0
mirror of https://github.com/dragonflydb/dragonfly.git synced 2024-12-14 11:58:02 +00:00
Commit graph

130 commits

Author SHA1 Message Date
adiholden
4a5d2f2a9a
feat(regression test) : add regression test that run in optimiztion mode (#900)
* feat(regression test) : add run in opimiztion mode

Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-03-02 16:33:52 +02:00
Roman Gershman
edd93a086c
fix: ci (#893)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-03-01 16:22:33 +02:00
Tarun Pothulapati
5dfaa1d061
test(chart): add golden file output rendering tests (#882)
This commit adds a new `golden_test.go` file into the helm chart
which essentially renders the chart with a set of values and
compares that with the expected golden file, and errors if
they don't match.

This builds on the existing CI values files in `ci/` directory. As
this is the first time, The golden files are rendered and added. This
means for all the future changes, The golden files can be updated
by running `go test -update` and the CI will fail if the golden files
are not updated. By doing this, Both the committer and reviewer
can be sure that the changes are intentional, without having to render
the chart manually.
2023-02-28 18:02:15 +05:30
adiholden
6a2b152e8b
feat(db slice): add fiber atomic gaurd (#878)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-02-26 13:38:37 +02:00
ashotland
1f82f9af73
fix(regression-test): Sentinel test stabilization (#826)
* Ditch docker whcih is complex on CI in favour of local redis binary

Signed-off-by: ashotland <ari@dragonflydb.io>

* Fix typo

Signed-off-by: ashotland <ari@dragonflydb.io>

* Wait for sentinel termination

Signed-off-by: ashotland <ari@dragonflydb.io>

* fix(regression-tests): sentinel test increase timeout waiting for key to
exist in replica

* debug sentinel test

Signed-off-by: ashotland <ari@dragonflydb.io>

* add pytest repeat - tmp comment out failure notficicaiton

Signed-off-by: ashotland <ari@dragonflydb.io>

* fix typo

Signed-off-by: ashotland <ari@dragonflydb.io>

* repeat 100

Signed-off-by: ashotland <ari@dragonflydb.io>

* Increase timeout for debuging

Signed-off-by: ashotland <ari@dragonflydb.io>

* Debug prints

Signed-off-by: ashotland <ari@dragonflydb.io>

* fix

Signed-off-by: ashotland <ari@dragonflydb.io>

* increase overall timeout

Signed-off-by: ashotland <ari@dragonflydb.io>

* Debug

Signed-off-by: ashotland <ari@dragonflydb.io>

* cleanup

Signed-off-by: ashotland <ari@dragonflydb.io>

---------

Signed-off-by: ashotland <ari@dragonflydb.io>
2023-02-20 14:21:27 +02:00
Vladislav
4ef06e759a
Basic multi modes for MULTI/EXEC (#796)
feat(server): Basic multi transaction modes

This commit adds the notion of multi transaction modes that allow controlling the execution and
locking behaviour of multi transactions.
In general, there are four modes:
- GLOBAL: all commands run within a global transaction. There is no need for recording locks. Lua scripts can theoretically run with undeclared keys.
- LOCK_AHEAD: the transaction locks all keys ahead likewise to a regular transaction and schedules itself.
- LOCK_INCREMENTAL: the transaction determines what shards it has keys in and schedules itself on those shards, but locks only when accessing a new key. This allows other transactions to run ooo alonside with a big multi-transaction that accesses a contended key only at its very end.
- NON_ATOMIC: all commands run separately, no atomicity is provided, likewise to a pipeline

This commit only adds support for the first 3 modes to EXEC commands.

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-02-18 20:18:28 +03:00
ashotland
2f89cdc6af
chore(regressions-tests): Update regression-tests.yml (#805)
* Update regression-tests.yml

Run every 3 hours
Add -s to pytest

Signed-off-by: ashotland <ari@dragonflydb.io>

* Update regression-tests.yml

Signed-off-by: ashotland <ari@dragonflydb.io>

---------

Signed-off-by: ashotland <ari@dragonflydb.io>
2023-02-16 12:16:39 +02:00
adiholden
ae6687e563
fix(regression-tests): increase timeout of regression-tests (#806)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-02-15 12:39:23 +02:00
Nick Gottlieb
15c856726a
update readme (#794)
* Add files via upload

Signed-off-by: Nick Gottlieb <ngottlieb1@gmail.com>

* Update README.md

Signed-off-by: Nick Gottlieb <ngottlieb1@gmail.com>

* Delete logo-full.svg

Signed-off-by: Nick Gottlieb <ngottlieb1@gmail.com>

* Update README.md

Signed-off-by: Nick Gottlieb <ngottlieb1@gmail.com>

* Update README.md

Signed-off-by: Nick Gottlieb <ngottlieb1@gmail.com>

* Update README.md

Signed-off-by: Nick Gottlieb <ngottlieb1@gmail.com>

* Update README.md

Signed-off-by: Nick Gottlieb <ngottlieb1@gmail.com>

* Rename logo.svg to logo-full.svg

Signed-off-by: Nick Gottlieb <ngottlieb1@gmail.com>

* Update README.md

Signed-off-by: Nick Gottlieb <ngottlieb1@gmail.com>

* Update README.md

Signed-off-by: Nick Gottlieb <ngottlieb1@gmail.com>

---------

Signed-off-by: Nick Gottlieb <ngottlieb1@gmail.com>
2023-02-14 23:08:32 +02:00
adiholden
72bad6c5ab
fix(replica) : replica will not sync execution multi shard commands as default (#800)
-sfix(replica) : replica will not sync execution multi shard commands as default
2023-02-14 16:30:14 +02:00
adiholden
c5e8a627b5
test(regression-tests): stop replication regression-tests schedual run (#795)
* regression replication will not run on github schedual
* allow running regression replication manually

Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-02-14 14:48:39 +02:00
Roman Gershman
83837532e9
chore(server): General improvements
1. Update helio dependency
2. Add SLOWLOG command stub that returns empty data.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-02-10 22:37:19 +02:00
ashotland
eb0bef4c33
Notify on cancelled too (#749)
Signed-off-by: ashotland <ari@dragonflydb.io>
2023-02-02 19:12:03 +03:00
Boaz Sade
c6e96a12c0
feat(action): build debian package for arm64 arch (#719) (#738)
Signed-off-by: Boaz Sade <boaz@dragonflydb.io>
2023-02-01 11:43:21 +02:00
ashotland
353aa344ba
Add -x to pytest invocation (#735)
Signed-off-by: ashotland <ari@dragonflydb.io>
2023-01-31 12:23:32 +02:00
adiholden
152f16bc14
test(regression test): run regression tests on github actions every 6… (#734)
test(regression test): run regression tests on github actions every 6 hours

Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-01-31 09:41:06 +02:00
ashotland
20a309ebd0
chore(ci):Reduce --gtest_repeat to 10 (#730)
Reduce --gtest_repeat to 10

Signed-off-by: ashotland <ari@dragonflydb.io>
2023-01-28 20:38:26 +02:00
Roman Gershman
90eb1d81b7
fix(lua): Fix a deadlock happenning when calling a lua script (#726)
The scenario is described in a unit test that reproduces the issue with high chance.
Also added dragonfly_test in repeat=100 mode to CI.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-01-25 10:16:52 +02:00
Boaz Sade
3360ff1704
feat(action): build debian package for arm64 arch (#719)
Signed-off-by: Boaz Sade <boaz@dragonflydb.io>
2023-01-24 14:43:47 +02:00
Boaz Sade
a130b71cd9
feat(actions): add support for Debian packaging - add missing dependency (#701)
feat(actions): add support for debian packaging - add missing dependency

Signed-off-by: Boaz Sade <boaz@dragonflydb.io>

Signed-off-by: Boaz Sade <boaz@dragonflydb.io>
2023-01-18 13:28:13 +02:00
Boaz Sade
be74fa0a5b
feat(actions): add support for debian packaging (#696)
Signed-off-by: Boaz Sade <boaz@dragonflydb.io>

Signed-off-by: Boaz Sade <boaz@dragonflydb.io>
2023-01-18 09:29:28 +02:00
Philipp B
c3de3ef2f3
chore: overhaul chart (#620)
This should greatly improve readability of the chart itself
- moved the `Pod` spec for both the `Deployment` and `StatefulSet` into a common template in `_pod.yaml`
- replace a bunch of `if $value; print $value`-type blocks with `with $value; print .`
- replaced `command.set` in `values.yaml` with `command` directly
  - this was broken anyways, as the chart wrongly referenced `command.cmd` for both `Deployment` and `StatefulSet`
- populated contrib/charts/dragonfly/ci/ folder for development/CI purposes

Signed-off-by: Philipp Born <git@pborn.eu>

Signed-off-by: Philipp Born <git@pborn.eu>
2022-12-30 08:30:11 +02:00
Philipp B
589a6ffd00
feat(ci test): add testing for helm chart (#622)
feat: add lint-test-chart job to CI

Signed-off-by: Philipp Born <git@pborn.eu>

Signed-off-by: Philipp Born <git@pborn.eu>
2022-12-29 18:22:09 +02:00
Philipp Born
98b92a0073 feat: build and publish weekly alpha/development container image
- make use of docker buildx caching when possible (helpful with local docker builds)
- introduce a reusable container workflow which is triggered by docker-release and docker-weekly workflows
- added an alpine-dev Dockerfile
- split release.sh contents into different Makefile targets
- make use of job matrix to build alpine + ubuntu in parallel
- make alpine build optional by checking for Dockerfile presence
-- as the pre-built binaries don't work with alpine, because of glibc <-> musl incompatibilities

Signed-off-by: Philipp Born <git@pborn.eu>
2022-12-26 20:41:53 +02:00
Roman Gershman
bcafd7e25d
feat: introduce simd algorithm for bitpacking (#568)
My benchmark shows a x3.5 improvement when compressing a 1KB string.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-12-17 19:22:40 +02:00
adiholden
53e18673a9
feat(ci test): add build and test on release mode (#556) 2022-12-11 13:52:45 +02:00
Roman Gershman
2d83aa995e
fix(github): add libzstd-dev dep to aarch64 release pipeline (#539) 2022-12-06 21:19:15 +02:00
adiholden
74d1839f97
feat(regression test): add timeout cpu info and inc frequecy (#524)
Signed-off-by: adi_holden <adi@dragonflydb.io>

Signed-off-by: adi_holden <adi@dragonflydb.io>
2022-12-01 14:04:26 +02:00
Roman Gershman
cd40bd76e4 fix: Use dev image for running regression tests (#521) 2022-11-30 15:43:17 +02:00
Boaz Sade
26b03145f9 feat(server): acrtive memory defrag high level flow
Signed-off-by: Boaz Sade <boaz@dragonflydb.io>
2022-11-30 14:27:59 +02:00
Philipp B
96989b2124
feat(actions): push helm chart as OCI image (#500)
Signed-off-by: Philipp Born <git@pborn.eu>

Signed-off-by: Philipp Born <git@pborn.eu>
2022-11-17 17:02:18 +02:00
zNNiz
57a313cf5a
A dedicated github actions pipeline that build dragonfly and runs pyt… (#490)
Adding a regression pipeline

A dedicated github actions pipeline that build dragonfly and runs pytests located under "tests/dragonfly" every 3 days

Fixes #291

Signed-off-by: zNNiz <96877065+zNNiz@users.noreply.github.com>
2022-11-16 13:05:43 +02:00
Leonardo Mello
e46e5819c9
ci: added clang pre-commit hook (#461)
Signed-off-by: Leonardo Mello <lsvmello@gmail.com>
2022-11-05 22:16:57 +02:00
Ran Ribenzaft
63f8e00273
ci(workflow): fixing discord notification link (#419)
Signed-off-by: Ran Ribenzaft <ranrib@users.noreply.github.com>
2022-10-23 21:44:06 +03:00
Ran Ribenzaft
f1e4e6f08c
build(ci): add discord message on docker release (#375)
Signed-off-by: Ran Ribenzaft <ranrib@users.noreply.github.com>
2022-10-12 08:03:43 +03:00
Roman Gershman
b616b1e1fd
feat(server): support epoll linux api (#351) 2022-10-04 11:11:09 +03:00
odedponcz
a00c205b5b
docs: Improve wording in issue reporting (#232)
* docs: Improve wording in issue reporting

Signed-off-by: odedponcz <oded@poncz.com>

* docs: Change contribution doc flow

Signed-off-by: odedponcz <oded@poncz.com>
2022-08-08 10:55:44 +03:00
Roman Gershman
567af06d2c
fix(action): switch git push to dedicated action (#192)
Docker release pipeline can not push to protected main branch using
the service token. This switches to PAT secret.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-07-06 12:55:10 +03:00
Philipp B
7446b85bb3
feat(actions): bump helm-chart version when new container image is published (#162)
Signed-off-by: Philipp Born <mail@philipp-born.eu>
2022-06-20 20:25:14 +03:00
Roman Gershman
46220183ae chore(actions): fix docker release pipeline
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-06-16 09:00:11 +03:00
Ryan Russell
f739300415
chore(issues): Add issue templates with links to Discord/Discussions (#152)
Signed-off-by: Ryan Russell <git@ryanrussell.org>
2022-06-15 16:42:26 +03:00
Roman Gershman
f24005407b fix(actions): Fix pre-release docker pipeline 2022-06-13 08:23:29 +03:00
Ryan Russell
d0af04d427
docs(PR Template): Add PR Template Enforcing Conventional Commits #107 (#137)
Signed-off-by: Ryan Russell <git@ryanrussell.org>
2022-06-12 10:40:17 +03:00
Roman Gershman
e2c52c47a5 chore(actions): Minor improvements to the release scripts. 2022-06-08 14:46:46 +03:00
Roman Gershman
e646209da4 Stability fixes to PUBSUB code 2022-06-08 08:46:48 +03:00
Roman Gershman
62b6ae48c0 Fix prerelease docker pipeline 2022-06-06 18:32:19 +03:00
Roman Gershman
1fe3862e57 Update docker pipeline to support pre-releases as well 2022-06-06 16:47:59 +03:00
Roman Gershman
d3f598eb88 More printings in rdb_test in order to catch the snapshotting bug 2022-06-02 12:32:10 +03:00
Roman Gershman
d5f2c23922
Fixes #66. (#70)
* Fixes #66.

1. Introduce `hz` flag to control the frequency of periodically running tasks.
2. Update helio dependency.

* Fix CI caching for PRs.
2022-06-01 13:19:41 +03:00
Olle Jonsson
c7534dfb6a
CI: Use v3 of Actions "upload-artifacts", "checkout" and "cache" (#62)
CI: Use v3 of GitHub Actions actions

Affects "checkout", "upload-artifacts", and "cache".
2022-05-31 17:42:05 +03:00
Philipp B
8aa8f2b0b0
Update docker-release workflow to tag image with release version (#60) 2022-05-31 14:04:08 +03:00
Roman Gershman
f23c552bd3 Fix docker pipeline 2022-05-30 15:51:48 +03:00
Roman Gershman
29575d00ee Fix aarch64 pipeline and update docker path in README 2022-05-30 12:30:00 +03:00
Roman Gershman
e239fc68b4 Added glue code to expose various http handlers from dragonfly 2022-05-30 11:53:39 +03:00
Roman Gershman
5af2fe0145 docker-release pipeline now uses release assets for building dockers 2022-05-30 11:24:56 +03:00
Roman Gershman
6dd1552506
Some clean-ups in rdb_save code. Add verbosity printings for CI build (#47) 2022-05-29 20:23:05 +03:00
Roman Gershman
6e7d3d215f A fix for the release pipeline 2022-05-26 13:44:54 +03:00
odedponcz
caeb60c22a
Fixit (#40)
* Easy getting started

Co-authored-by: Roman Gershman <romange@gmail.com>
2022-05-26 11:15:44 +03:00
Roman Gershman
e494fb30e9 Upload assets to the release 2022-05-26 08:00:38 +03:00
Roman Gershman
45029dae9a Add a skeleton for the release pipeline 2022-05-25 21:14:04 +03:00
Roman Gershman
1de6f5317d Update README with more info. 2022-05-21 00:23:23 +03:00
Roman Gershman
280ce351a8 Fix CI after moving the repo 2022-05-11 20:37:25 +03:00
Roman Gershman
3dce1d33fc Update README.md. Disable alpine docker for now.
Update organization references across the docs.
2022-05-11 12:49:19 +03:00
Roman Gershman
2d6251ca83 Add circleci config.yml 2022-05-05 13:38:22 +03:00
Roman Gershman
5568205b34 More work on tiered storage.
1. Reads from external storage support now o_direct mode.
2. Simplify write unloading logic. Make pending buffer a ring buffer with
   a predefined capacity.
3. Add more tiered stats to info command
2022-05-05 12:05:05 +03:00
Roman Gershman
b6ed08edb7 Fix a typo 2022-05-03 11:58:09 +03:00
Roman Gershman
2c8cb23098 Few improvements.
1. Docker build now builds for arm64 as well.
2. Add bind option to specify on which interface the server should listen.
3. Automatically deduce maxmemory setting.
2022-04-25 12:06:31 +03:00
Roman Gershman
bfcefd932d Support rdb loading.
1. Basic support of sets and hsets. No intset or dict-based hset yet.
2. Make sure that rdbloader actually loads the data into shards.
3. Support object metadata like expiry time.
4. Support lists loading.
2022-04-12 16:39:25 +03:00
Roman Gershman
b2c50a6fe3 CI fixes 2022-03-27 18:28:53 +03:00
Roman Gershman
d1291be0b7 Add ubuntu flavoured docker image of dragonfly 2022-02-25 20:19:01 +02:00
Roman Gershman
a859cf2fc8 Add docker release action 2022-02-25 18:45:12 +02:00
Roman Gershman
b8521828e3 Add Dockerfile for prod container. Reorganize source tree to be docker-build friendly. 2022-02-25 10:03:42 +02:00
Roman Gershman
29f5052c4d Add alpine-dev container to test matrix 2022-02-24 14:11:51 +02:00
Roman Gershman
b82fb72fc5 Add basic metrics to INFO command 2022-02-24 14:11:51 +02:00
Roman Gershman
ee15f98142 Use container in CI 2022-02-24 14:11:50 +02:00
Roman Gershman
286da5e409 Improve ccache config 2022-02-24 14:11:50 +02:00
Roman Gershman
e88d995618 Fix consistency bug in watched code
CI improvements
2022-02-24 14:11:50 +02:00
Roman Gershman
ea399e3e5a Identify multi-blpop interaction bug.
Now it fails on missing functionality without deadlocking.
2022-01-13 13:10:34 +02:00
Roman Gershman
a0dfb3171a Fix checkout directive 2021-12-21 11:29:59 +02:00
Roman Gershman
1b52488ee9 Add github ci workflow 2021-12-21 10:50:46 +02:00