1
0
Fork 0
mirror of https://github.com/dragonflydb/dragonfly.git synced 2024-12-14 11:58:02 +00:00
Commit graph

440 commits

Author SHA1 Message Date
Roman Gershman
29575d00ee Fix aarch64 pipeline and update docker path in README 2022-05-30 12:30:00 +03:00
Roman Gershman
e239fc68b4 Added glue code to expose various http handlers from dragonfly 2022-05-30 11:53:39 +03:00
Roman Gershman
5af2fe0145 docker-release pipeline now uses release assets for building dockers 2022-05-30 11:24:56 +03:00
Roman Gershman
4d276c7ef0 Integrate version metadata into build. Fixes #46. 2022-05-30 08:49:19 +03:00
Roman Gershman
bb7b205bff Switch to using absl flags 2022-05-30 06:45:09 +03:00
Roman Gershman
6dd1552506
Some clean-ups in rdb_save code. Add verbosity printings for CI build (#47) 2022-05-29 20:23:05 +03:00
odedponcz
e5107c2047
Add license (#45)
* Add BSL license
* set change date

Co-authored-by: Roman Gershman <romange@gmail.com>
2022-05-29 18:24:29 +03:00
Roman Gershman
776276ccd4
Add more benchmarks to README (#44) 2022-05-29 17:47:34 +03:00
Roman Gershman
ccf051742d Execute commands from the replicated log 2022-05-28 23:20:52 +03:00
Roman Gershman
1490eb5c3c Implement parsing of replication stream from redis 2022-05-28 21:24:07 +03:00
Roman Gershman
46929e9c52
Update README: add benchmarks (#43) 2022-05-27 16:15:12 +03:00
Roman Gershman
114e8bec5d Fixes #41.
1. Found dangling transaction pointers that where left in the watch queue. Fix the state machine there.
2. Improved transaction code a bit, merged duplicated code into RunInShard function, got rid of RunNoop.
3. Improved BPopper::Run flow.
4. Added 'DEBUG WATCH' command. Also 'DEBUG OBJECT' now returns shard id and the lock status of the object.
2022-05-27 12:20:01 +03:00
Roman Gershman
3a38576bbb Introduce regression tests 2022-05-27 00:26:51 +03:00
Roman Gershman
4a2e84b975 In order to support Debian/Bullseye, we know require minimal kernel version - 5.10 2022-05-26 20:04:33 +03:00
Roman Gershman
6e7d3d215f A fix for the release pipeline 2022-05-26 13:44:54 +03:00
odedponcz
caeb60c22a
Fixit (#40)
* Easy getting started

Co-authored-by: Roman Gershman <romange@gmail.com>
2022-05-26 11:15:44 +03:00
Roman Gershman
e494fb30e9 Upload assets to the release 2022-05-26 08:00:38 +03:00
Roman Gershman
45029dae9a Add a skeleton for the release pipeline 2022-05-25 21:14:04 +03:00
Roman Gershman
5ad6352ad7 Update dashtable doc 2022-05-23 19:41:55 +03:00
Roman Gershman
8bed96d20b Memory and persistence fixes.
1. Fix memory counting stats. Use memory_usable_size(ptr) for counting used memory.
2. Fix crash when there is I/O error when writing snapshot. Now we return error message.
2022-05-23 16:17:13 +03:00
Roman Gershman
c087f7c61b Update helio version. Add more gc related stats 2022-05-23 08:52:04 +03:00
Roman Gershman
166ca73559 Rdb save opens file for writes in direct mode. More work on dashtable docs 2022-05-22 22:34:05 +03:00
odedponcz
869cd19897
Fixit (#39)
* Add more documentation about dashtable. Tune expiry heuristics a bit

* Minor lang change

* minor lng changes

Co-authored-by: Roman Gershman <romange@gmail.com>
2022-05-22 09:46:49 +03:00
Roman Gershman
3717079172 Add more tests 2022-05-21 21:45:41 +03:00
Roman Gershman
c7d1893478 Add more documentation about dashtable. Tune expiry heuristics a bit 2022-05-21 20:35:19 +03:00
Roman Gershman
543979875a Fixes #37. 2022-05-21 07:09:46 +03:00
Roman Gershman
1de6f5317d Update README with more info. 2022-05-21 00:23:23 +03:00
Roman Gershman
343dd22ce5 Fixes #21.
1. Snapshot now passed db index and rdb save handles by inserting select opcodes when needed.
2. Fix few more bugs in CVCUponInsert.
3. Save lua scripts.
2022-05-20 23:25:41 +03:00
Roman Gershman
599c786c16 This CR fixes #36.
1. Fix a bug in dash table related to snapshotting.
2. Rewrite GlobalState code and make state transitions atomic and well defined.
3. Fix Save/Flush semantics by capturing snapshotted tables together with the snapshot.
2022-05-20 12:35:28 +03:00
Roman Gershman
439070d00d Few improvements.
1. Stabilized naming scheme for snapshot files.
   The behavior is determined by dbfilename flag.
   By default it's 'dump'. Dragonfly checks if the flag has an extension
   if not, it automatically saves timestamped files (dump*.rdb) and loads the latest file that is available.
   In case the flag has extension like 'dump.rdb' it fallbacks to redis behavior of loading and saving to the
   same file.

2. Updated the logo url.
2022-05-19 15:50:42 +03:00
Roman Gershman
9eb13b5163 Cover more type families with OOM checks 2022-05-19 12:00:47 +03:00
Roman Gershman
5e4aa0a1a8 Handle OOM errors for more types. Fix clang warnings and errors 2022-05-18 21:53:49 +03:00
Roman Gershman
30cf9541c2 Add reference counting to DbTable. The will help us to implement flushing the tables in parallel with snapshotting 2022-05-17 14:21:28 +03:00
Roman Gershman
5849d09f73 Reuse vectors on a hot-path of parsing requests in dragonfly_connection. 2022-05-17 11:22:08 +03:00
Roman Gershman
cfaf173236 Allow cache mode - intelligent eviction of less likely to be used items.
This id one by shifting right slots in a stash bucket of the full segment.
In addition, I added eviction related stats to memory and stats section.
I also updated README with the changes. Finally, I added a flag that allows
to disable http-admin console in dragonfly.
2022-05-16 19:39:17 +03:00
Roman Gershman
797c8121b1 Limit table growth according to maxmemory.
1. Make EngineShardSet to be a process singleton instead of a variable passed everywhere.
2. Implement a heuristic of updating free memory per shard.
3. Make sure that SET command returns OOM when a database reaches its limits.
2022-05-16 08:19:32 +03:00
Roman Gershman
6d44d72a56 Add more debugging tools for object introspection 2022-05-15 23:11:45 +03:00
Roman Gershman
038868802f Add caching mode to dragonfly - part 1.
The heart of this feature is ability to bump up entries that has been searched for.
When we bump up an entry withing a dash-table it's moved out of stash buckets or to lower slot ids
within normal buckets. The entry they replace is moved to the place of bumped-up entries.
The next change will be to implement the actual eviction heuristic that will evict keys from the stash buckets.
This will give stash buckets an addition responsibility for caching mode - they will serve as a probation buffer
that holds low-priority or newly added items.
2022-05-15 21:03:56 +03:00
Roman Gershman
6e2e5fcf67 Add support for reordering items within dash table - needed for cache evictions.
1. Simplify versioning scheme. This also reduces metadata space and increases table capacity
   for same memory reservation by ~%5.
2. Fix snapshotting bugs. Recognize duplicate keys during loading.
3. Introduce DEBUG LOAD command to perform manual loading of files.
2022-05-15 14:17:45 +03:00
Roman Gershman
1f953a4ca8 Delete expired items in the background. 2022-05-12 15:52:58 +03:00
Roman Gershman
2b553535b2 Upon full segment try unloading stash buckets back to regular ones 2022-05-12 13:43:03 +03:00
Roman Gershman
4be8c25711 Configure keep-alive for server sockets.
Minor fixes to docker build.
Provide a workaround for #35 that should suffice for the initial release.
2022-05-12 09:13:56 +03:00
Roman Gershman
280ce351a8 Fix CI after moving the repo 2022-05-11 20:37:25 +03:00
Roman Gershman
3dce1d33fc Update README.md. Disable alpine docker for now.
Update organization references across the docs.
2022-05-11 12:49:19 +03:00
Roman Gershman
1aa2a171ac Stats and debugging improvements. Provide some keyspace stats 2022-05-11 09:57:47 +03:00
Roman Gershman
e9dda3aa64 Implement zinterstore 2022-05-10 21:48:24 +03:00
Roman Gershman
53fb11b2fd Tiered storage code cleanups 2022-05-10 20:15:03 +03:00
Roman Gershman
0c5e2a5ecd Implement CLIENT LIST and CLIENT SETNAME. 2022-05-10 06:35:37 +03:00
Roman Gershman
9e0c705d16 More work on tiered storage.
1. Set min block size back to 4k.
2. Free external entities that are being overriden.
3. Fix accounting stats.
4. Support entities batching.
2022-05-09 07:39:25 +03:00
Roman Gershman
c34e7c6d44 Implement ZUNIONSTORE.
Fix some bugs in transactional framework to support irregular commands.
Lay out groundwork for supporting XXX-STORE other commands in zsets.
2022-05-07 20:38:18 +03:00