1
0
Fork 0
mirror of https://github.com/dragonflydb/dragonfly.git synced 2024-12-14 11:58:02 +00:00
dragonflydb-dragonfly/README.md

290 lines
6 KiB
Markdown
Raw Normal View History

# Dragonfly
2021-11-24 12:17:44 +00:00
2021-12-21 09:07:29 +00:00
[![ci-tests](https://github.com/romange/dragonfly/actions/workflows/ci.yml/badge.svg)](https://github.com/romange/dragonfly/actions/workflows/ci.yml)
2022-03-03 07:34:53 +00:00
A novel memory store that supports Redis and Memcached commands.
For more detailed status of what's implemented - see below.
2021-11-24 12:17:44 +00:00
2022-03-03 07:34:53 +00:00
Features include:
2021-11-30 08:11:59 +00:00
1. High throughput reaching millions of QPS on a single node.
2. TLS support.
3. Pipelining mode.
2022-03-03 07:34:53 +00:00
4. A novel cache design, which does not require specifying eviction policies.
5. Memory efficiency that can save 20-40% for regular workloads and even more for cache like
workloads
2021-11-30 08:11:59 +00:00
## Building from source
I've tested the build on Ubuntu 21.04+.
2022-03-24 09:30:22 +00:00
Requires: CMake, Ninja, boost, libunwind8-dev
2021-11-30 08:11:59 +00:00
2022-03-24 09:30:22 +00:00
```
sudo apt install ninja-build
sudo apt install libunwind-dev
sudo apt-get install libboost-all-dev
```
2021-11-30 08:11:59 +00:00
```
git clone --recursive https://github.com/romange/dragonfly
cd dragonfly && ./helio/blaze.sh -release
cd build-opt && ninja dragonfly
2021-11-30 08:11:59 +00:00
```
## Running
```
./dragonfly --logtostderr
2021-11-30 08:11:59 +00:00
```
for more options, run `./dragonfly --help`
## Milestone Egg 🥚
API 1.0
- [X] String family
- [X] SET
- [ ] SETNX
- [X] GET
- [X] DECR
- [X] INCR
- [X] DECRBY
2022-01-09 18:43:49 +00:00
- [X] GETSET
- [X] INCRBY
- [X] MGET
- [X] MSET
- [ ] MSETNX
- [ ] SUBSTR
- [ ] Generic family
- [X] DEL
- [X] ECHO
- [X] EXISTS
- [X] EXPIRE
2022-01-09 18:43:49 +00:00
- [X] EXPIREAT
- [X] PING
- [X] RENAME
- [X] RENAMENX
- [X] SELECT
2022-01-09 18:43:49 +00:00
- [X] TTL
- [X] TYPE
- [ ] SORT
- [X] Server Family
- [X] QUIT
- [X] DBSIZE
- [ ] BGSAVE
- [X] SAVE
2022-01-09 18:43:49 +00:00
- [X] DBSIZE
- [X] DEBUG
- [X] EXEC
- [X] FLUSHALL
2022-01-09 18:43:49 +00:00
- [X] FLUSHDB
- [X] INFO
2022-01-09 18:43:49 +00:00
- [X] MULTI
- [X] SHUTDOWN
- [X] LASTSAVE
- [X] SLAVEOF/REPLICAOF
- [ ] SYNC
- [ ] Set Family
- [x] SADD
- [x] SCARD
- [X] SDIFF
- [X] SDIFFSTORE
- [X] SINTER
- [X] SINTERSTORE
- [X] SISMEMBER
- [X] SMOVE
- [X] SPOP
- [ ] SRANDMEMBER
- [X] SREM
- [X] SMEMBERS
- [X] SUNION
- [X] SUNIONSTORE
- [X] List Family
- [X] LINDEX
- [X] LLEN
- [X] LPOP
- [X] LPUSH
2022-03-15 17:52:11 +00:00
- [X] LRANGE
- [X] LREM
- [X] LSET
- [X] LTRIM
- [X] RPOP
- [ ] RPOPLPUSH
- [X] RPUSH
2022-03-06 06:46:48 +00:00
- [X] SortedSet Family
- [X] ZADD
- [X] ZCARD
- [ ] ZINCRBY
- [X] ZRANGE
- [X] ZRANGEBYSCORE
2022-03-06 06:46:48 +00:00
- [X] ZREM
2022-03-22 21:50:47 +00:00
- [X] ZREMRANGEBYSCORE
- [X] ZREVRANGE
2022-03-06 06:46:48 +00:00
- [X] ZSCORE
- [ ] Not sure whether these are required for the initial release.
2022-02-20 11:49:37 +00:00
- [X] AUTH
- [ ] BGREWRITEAOF
- [ ] KEYS
- [ ] MONITOR
- [ ] RANDOMKEY
- [ ] MOVE
In addition, we want to support efficient expiry (TTL) and cache eviction algorithms.
We should implement basic memory management support. For Master/Slave replication we should design
a distributed log format.
2022-02-20 11:49:37 +00:00
### Memchache API
- [X] set
- [X] get
- [X] replace
- [X] add
- [X] stats (partial)
- [x] append
- [x] prepend
- [x] delete
- [x] flush_all
- [x] incr
- [x] decr
- [x] version
- [x] quit
2022-02-20 11:49:37 +00:00
API 2.0
- [ ] List Family
- [X] BLPOP
- [ ] BRPOP
- [ ] BRPOPLPUSH
- [ ] BLMOVE
- [ ] LINSERT
2022-03-24 21:04:02 +00:00
- [X] LPUSHX
- [X] RPUSHX
2022-03-07 21:11:43 +00:00
- [X] String Family
- [X] SETEX
- [X] APPEND
- [X] PREPEND (dragonfly specific)
- [ ] BITCOUNT
- [ ] BITFIELD
- [ ] BITOP
- [ ] BITPOS
- [ ] GETBIT
2022-03-24 21:04:02 +00:00
- [X] GETRANGE
- [ ] INCRBYFLOAT
- [ ] PSETEX
- [ ] SETBIT
- [ ] SETRANGE
2022-03-24 21:04:02 +00:00
- [X] STRLEN
2022-03-03 07:34:53 +00:00
- [X] HashSet Family
- [X] HSET
- [X] HMSET
2022-03-03 07:34:53 +00:00
- [X] HDEL
- [X] HEXISTS
- [X] HGET
- [X] HMGET
2022-03-03 07:34:53 +00:00
- [X] HLEN
- [ ] HINCRBY
- [ ] HINCRBYFLOAT
- [X] HGETALL
- [X] HKEYS
- [ ] HSETNX
- [ ] HVALS
- [ ] HSCAN
- [ ] PubSub family
- [ ] PUBLISH
- [ ] PUBSUB
- [ ] PUBSUB CHANNELS
- [ ] SUBSCRIBE
- [ ] UNSUBSCRIBE
- [ ] Server Family
- [ ] WATCH
- [ ] UNWATCH
- [ ] DISCARD
- [ ] CLIENT KILL/LIST/UNPAUSE/PAUSE/GETNAME/SETNAME/REPLY/TRACKINGINFO
- [X] COMMAND
- [ ] COMMAND COUNT/GETKEYS/INFO
- [ ] CONFIG GET/REWRITE/SET/RESETSTAT
- [ ] MIGRATE
- [ ] ROLE
- [ ] SLOWLOG
- [ ] PSYNC
- [ ] TIME
- [ ] LATENCY...
- [X] Generic Family
- [X] SCAN
- [X] PEXPIREAT
- [ ] PEXPIRE
- [ ] DUMP
- [X] EVAL
- [X] EVALSHA
- [ ] OBJECT
- [ ] PERSIST
2022-03-24 21:04:02 +00:00
- [X] PTTL
- [ ] RESTORE
- [X] SCRIPT LOAD
- [ ] SCRIPT DEBUG/KILL/FLUSH/EXISTS
- [ ] TOUCH
- [X] Set Family
- [X] SSCAN
- [X] SMISMEMBER
- [X] Sorted Set Family
- [ ] ZCOUNT
- [ ] ZINTERSTORE
- [ ] ZLEXCOUNT
- [ ] ZRANGEBYLEX
- [ ] ZRANK
- [ ] ZREMRANGEBYLEX
- [X] ZREMRANGEBYRANK
- [ ] ZREVRANGEBYSCORE
- [ ] ZREVRANK
- [ ] ZUNIONSTORE
- [ ] ZSCAN
- [ ] HYPERLOGLOG Family
- [ ] PFADD
- [ ] PFCOUNT
- [ ] PFMERGE
- [ ] PUBSUB Family
- [ ] PSUBSCRIBE
- [ ] PUBSUB
- [ ] PUBLISH
- [ ] PUNSUBSCRIBE
- [ ] SUBSCRIBE
- [ ] UNSUBSCRIBE
Commands that I prefer avoid implementing before launch:
- PUNSUBSCRIBE
- PSUBSCRIBE
- HYPERLOGLOG
- SCRIPT DEBUG
- OBJECT
- DUMP/RESTORE
- CLIENT
Also, I would omit keyspace notifications. For that I would like to deep dive and learn
exact use-cases for this API.
### Random commands we implemented along the way
- [X] ROLE (2.8) decorator for for master withour replicas
- [X] UNLINK (4.0) decorator for DEL command
- [X] BGSAVE
## Milestone Nymph
2022-01-09 18:43:49 +00:00
API 2,3,4 without cluster support, without modules, without memory inspection commands.
Without support for keyspace notifications.
2022-01-09 18:43:49 +00:00
2022-01-08 18:36:52 +00:00
Design config support. ~140 commands overall...
## Milestone Molt
2022-01-08 18:36:52 +00:00
API 5,6 - without cluster and modules. Streams support. ~80 commands overall.
## Milestone Adult
2022-03-05 19:35:49 +00:00
TBD.
## Design decisions along the way
### Expiration deadlines with relative accuracy
I decided to limit the expiration range to 365 days. Moreover, expiration deadlines
2022-03-05 19:35:49 +00:00
with millisecond precision (PEXPIRE/PSETEX etc) will be rounded to closest second
**for deadlines greater than 33554431ms (approximately 560 minutes). In other words,
2022-03-05 19:35:49 +00:00
expiries of `PEXPIRE key 10010` will expire exactly after 10 seconds and 10ms. However,
`PEXPIRE key 34000300` will expire after 34000 seconds (i.e. 300ms earlier). Similarly,
`PEXPIRE key 34000800` will expire after 34001 seconds, i.e. 200ms later.
2022-03-05 19:35:49 +00:00
Such rounding has at most 0.002% error which I hope is acceptable for large ranges.
2022-03-24 09:30:22 +00:00
If it breaks your use-cases - talk to me or open an issue and explain your case.