1
0
Fork 0
mirror of https://github.com/dragonflydb/dragonfly.git synced 2024-12-14 11:58:02 +00:00
Commit graph

615 commits

Author SHA1 Message Date
ashotland
f204f1c670
Cache logs player batching (#623)
* feat(tools):cache log player batching all the way optimization

Signed-off-by: ashotland <ashotland@gmail.com>

* feat(tools): cache log player add one last print stats after completion

Signed-off-by: ashotland <ashotland@gmail.com>

Signed-off-by: ashotland <ashotland@gmail.com>
2022-12-30 12:29:11 +03:00
Philipp B
c3de3ef2f3
chore: overhaul chart (#620)
This should greatly improve readability of the chart itself
- moved the `Pod` spec for both the `Deployment` and `StatefulSet` into a common template in `_pod.yaml`
- replace a bunch of `if $value; print $value`-type blocks with `with $value; print .`
- replaced `command.set` in `values.yaml` with `command` directly
  - this was broken anyways, as the chart wrongly referenced `command.cmd` for both `Deployment` and `StatefulSet`
- populated contrib/charts/dragonfly/ci/ folder for development/CI purposes

Signed-off-by: Philipp Born <git@pborn.eu>

Signed-off-by: Philipp Born <git@pborn.eu>
2022-12-30 08:30:11 +02:00
Philipp B
589a6ffd00
feat(ci test): add testing for helm chart (#622)
feat: add lint-test-chart job to CI

Signed-off-by: Philipp Born <git@pborn.eu>

Signed-off-by: Philipp Born <git@pborn.eu>
2022-12-29 18:22:09 +02:00
ashotland
cee5af1baf
feat(tools):cache log player batching all the way optimisation (#617)
feat(tools):cache log player batching all the way optimization

Signed-off-by: ashotland <ashotland@gmail.com>

Signed-off-by: ashotland <ashotland@gmail.com>
2022-12-29 16:01:01 +02:00
Pierre Jambet
febcda7935
docs(quickstart): Clarify docker command for macOS (#612)
Clarify docker command for macOS

I had issues when running the command on my m2 macbook air. [It turns
out][1] that apparently docker desktop on macOS doesn't really support
the host option.

Additionally, it looks like I had to explicitly pass the port mapping
options for the ports to be actually mapped.

There might be a better and more efficient solution, but this is what I
came up with and it finally worked.

[1]:https://github.com/docker/for-mac/issues/1031
2022-12-28 18:43:50 +02:00
ashotland
92454c6fc8
fix(tools): cache logs player handle append as redis append (#615)
Signed-off-by: ashotland <ari@dragonflydb.io>

Signed-off-by: ashotland <ari@dragonflydb.io>
2022-12-28 17:04:53 +03:00
ashotland
5638f65226
feat(tools):Cache logs player (#614)
Signed-off-by: ashotland <ari@dragonflydb.io>

Signed-off-by: ashotland <ari@dragonflydb.io>
2022-12-28 14:58:31 +03:00
Roman Gershman
63b83c5b99
chore: Add tiered_storage_test. (#613)
1. Support tiered deletion.
2. Add notion of tiered entity in "DEBUG OBJECT" output.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-12-28 10:37:55 +02:00
ashotland
e39d266abe
fix(server):set of existing object with expiry (issue#607) (#609)
fix(server):set of existing object with expriy (issue#607)

Signed-off-by: ashotland <ari@dragonflydb.io>

Signed-off-by: ashotland <ari@dragonflydb.io>
2022-12-27 18:09:01 +02:00
adiholden
aacab66434
feat(replica): atomicity for multi shard commands (#598) 2022-12-27 16:13:05 +02:00
Vladislav
e6721d8160
feat(server): Improved cancellation (#599) 2022-12-27 16:01:54 +03:00
ashotland
b48f7557b7
feat(server): track hits and misses stats. (#608)
feat: track hits and misses stats.

Signed-off-by: ashotland <ari@dragonflydb.io>
2022-12-27 13:57:23 +02:00
Roman Gershman
75d35bf987
life of a transaction (#278)
docs(transactions): Add explanations on how Dragonfly transactions work.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-12-27 06:01:11 +02:00
Philipp Born
98b92a0073 feat: build and publish weekly alpha/development container image
- make use of docker buildx caching when possible (helpful with local docker builds)
- introduce a reusable container workflow which is triggered by docker-release and docker-weekly workflows
- added an alpine-dev Dockerfile
- split release.sh contents into different Makefile targets
- make use of job matrix to build alpine + ubuntu in parallel
- make alpine build optional by checking for Dockerfile presence
-- as the pre-built binaries don't work with alpine, because of glibc <-> musl incompatibilities

Signed-off-by: Philipp Born <git@pborn.eu>
2022-12-26 20:41:53 +02:00
Boaz Sade
8802fb4a3c chore(server): upgrade mimalloc for v2.0.7
Signed-off-by: Boaz Sade <boaz@dragonflydb.io>
2022-12-26 13:08:54 +02:00
Boaz Sade
044d4a4044 fix(server): defrag unit test fix
Signed-off-by: Boaz Sade <boaz@dragonflydb.io>
2022-12-26 12:46:17 +02:00
adiholden
5d39521de3
feat(replica): Support FlushDB command for replication #580 (#591) 2022-12-25 14:03:49 +02:00
Boaz Sade
8d86e9b014
feat(server): adding version monitoring task (#586)
feat(server): adding version monoitoring task

Signed-off-by: Boaz Sade <boaz@dragonflydb.io>

Signed-off-by: Boaz Sade <boaz@dragonflydb.io>
2022-12-25 10:05:30 +02:00
Roman Gershman
bbc809a637 feat(server): integrate string_map into rdb code to support save/load
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-12-24 17:21:57 +02:00
Roman Gershman
69b00b9cdd feat(server): Integrate set_map into hset_family
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-12-24 17:21:57 +02:00
Roman Gershman
5d34f271e9 feat(server): Extend string_map api, fix AddOrSet method.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-12-24 17:21:57 +02:00
Roman Gershman
9375a48f71 feat(core): Add StringMap data structure.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-12-24 17:21:57 +02:00
Roman Gershman
adb3266825
feat(server): Redesign and simplify tiered storage module. (#589)
1. Allow offloading blobs larger than 2KB.
2. Totally redesign the offloading algorithm for blobs smaller than 2KB.
3. Fix bugs around IO request cancelations.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-12-22 03:58:58 +02:00
Philipp B
b5f6629d55
fix(chart): probe indentation (#590)
No idea why it worked before.. Fixes #588

Signed-off-by: Philipp Born <git@pborn.eu>

Signed-off-by: Philipp Born <git@pborn.eu>
2022-12-21 22:03:12 +02:00
Vladislav
2386b02234
feat(server): Use new journal format (#563) 2022-12-20 16:38:19 +03:00
Roman Gershman
cdc31fc98c
fix: handle corner case when comitted memory is negative (#570)
Set mem_defrag_threshold so that defrag will be trigerred when memory grows.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-12-19 13:56:20 +02:00
Roman Gershman
5f572f00f1
fix: choose an appropriate SIMD implementation for aarch64 (#579) 2022-12-19 12:18:41 +02:00
Roman Gershman
69d9ef2c03
chore: Improve the implementation of simd based packing (#577)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-12-18 19:59:34 +02:00
adiholden
d11b0d10da
bug(rdb load): Rdb loader tasks stop running on failure #567 (#576)
Signed-off-by: adi_holden <adi@dragonflydb.io>

Signed-off-by: adi_holden <adi@dragonflydb.io>
2022-12-18 18:25:13 +02:00
Roman Gershman
c18cb8208e
feat: implement ascii_unpack using SIMD instructions (#573)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-12-18 17:27:30 +02:00
Roman Gershman
bcafd7e25d
feat: introduce simd algorithm for bitpacking (#568)
My benchmark shows a x3.5 improvement when compressing a 1KB string.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-12-17 19:22:40 +02:00
Roman Gershman
adc89c7592
chore: Adding a mpsc intrusive queue based on Vyukov's design (#562)
feat(server): Speed up rdb load by using object pool for parsing objects.

1. Add a mpsc intrusive queue based on Vyukov's design.
2. Use it as a object pool when we pull from the queue to reuse the existing object
   and push into it in order to return the object back to the pool.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-12-16 17:54:55 +02:00
Roman Gershman
d4cad11f86
chore: Refactor RdbLoad (#564)
Specifically, pass OpaqueObj to parsing routines so that they could fill it instead of creating it.
In addition, this change improves the interface between generic_family and RdbLoad code:
it removes reliance on the internal Item class.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-12-16 16:41:47 +02:00
Vladislav
08803e664c
feat(server): New auto-journal types/read/write (#560) 2022-12-15 23:42:00 +03:00
Boaz Sade
3d610ee2d7
feat(server): JSON family using JSON type (#561)
feat(server): json family using json type

Signed-off-by: Boaz Sade <boaz@dragonflydb.io>

Signed-off-by: Boaz Sade <boaz@dragonflydb.io>
2022-12-14 19:25:10 +02:00
Boaz Sade
dba4c6bd54
feat(server): active memory defrag test application (#552)
Signed-off-by: Boaz Sade <boaz@dragonflydb.io>

Signed-off-by: Boaz Sade <boaz@dragonflydb.io>
2022-12-13 14:34:32 +02:00
Roman Gershman
444f7e3f03
Support offloading of blobs longer than 2KB (#559)
feat: Support offloading of blobs longer than 2KB
2022-12-13 04:30:03 +02:00
Tomato6966
0efa1c92c4
added example of start script (#557)
* added example of start script

+ removed duplicated explanation

Signed-off-by: Tomato6966 <chris.pre03@gmail.com>
2022-12-11 17:52:42 +02:00
adiholden
53e18673a9
feat(ci test): add build and test on release mode (#556) 2022-12-11 13:52:45 +02:00
Vladislav
f98d6f3357
feat(server): Implement robust error & cancellation on replica (#531) 2022-12-11 12:11:25 +03:00
Vladislav
a0fe3c3dcc
chore: Update helio dependency (#555)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2022-12-10 18:02:28 +02:00
Roman Gershman
23c902d8e9
chore: Update helio dependency (#553)
Switch to using fibers_ext::Fiber.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-12-09 16:43:41 +02:00
Boaz Sade
c34270c02d
feat(server): json set type support (#546)
Signed-off-by: Boaz Sade <boaz@dragonflydb.io>
2022-12-08 15:32:58 +02:00
adiholden
ae9e3c45d9
Support lz4 compression (#545)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2022-12-08 09:41:55 +02:00
Philipp B
2a67dc307e
feat: add healtcheck for container (#544)
feat: add netcat based healtcheck

(similar to `redis-cli ping`)

Signed-off-by: Philipp Born <git@pborn.eu>
2022-12-07 16:44:07 +02:00
arukiidou
c698e9c25e
fix(helm): Chart.yaml - Add home, keywords, sources, kubeVersion (#542)
fix(helm): Chart.yaml - add home, keywords, sources, kubeVersion

Signed-off-by: junya koyama <arukiidou@yahoo.co.jp>
2022-12-07 12:37:35 +02:00
adiholden
90bc3ab8b8
feat(rdb save): Create compressor interface (#538)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2022-12-07 10:12:18 +02:00
romange
9aa9a78ea5 chore(helm-chart): update to v0.12.0 2022-12-07 04:23:34 +00:00
Roman Gershman
2d83aa995e
fix(github): add libzstd-dev dep to aarch64 release pipeline (#539) 2022-12-06 21:19:15 +02:00
adiholden
e21a212688
feat(rdb save): add Lz4Compressor (#536)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2022-12-06 15:46:06 +02:00