1
0
Fork 0
mirror of https://github.com/dragonflydb/dragonfly.git synced 2024-12-14 11:58:02 +00:00
dragonflydb-dragonfly/README.md

278 lines
5.8 KiB
Markdown
Raw Normal View History

# Dragonfly
2021-11-24 12:17:44 +00:00
2021-12-21 09:07:29 +00:00
[![ci-tests](https://github.com/romange/dragonfly/actions/workflows/ci.yml/badge.svg)](https://github.com/romange/dragonfly/actions/workflows/ci.yml)
2022-03-03 07:34:53 +00:00
A novel memory store that supports Redis and Memcached commands.
For more detailed status of what's implemented - see below.
2021-11-24 12:17:44 +00:00
2022-03-03 07:34:53 +00:00
Features include:
2021-11-30 08:11:59 +00:00
1. High throughput reaching millions of QPS on a single node.
2. TLS support.
3. Pipelining mode.
2022-03-03 07:34:53 +00:00
4. A novel cache design, which does not require specifying eviction policies.
5. Memory efficiency that can save 20-40% for regular workloads and even more for cache like
workloads
2021-11-30 08:11:59 +00:00
## Building from source
I've tested the build on Ubuntu 21.04+.
2022-03-24 09:30:22 +00:00
Requires: CMake, Ninja, boost, libunwind8-dev
2021-11-30 08:11:59 +00:00
2022-03-24 09:30:22 +00:00
```
sudo apt install ninja-build
sudo apt install libunwind-dev
sudo apt-get install libboost-all-dev
```
2021-11-30 08:11:59 +00:00
```
git clone --recursive https://github.com/romange/dragonfly
cd dragonfly && ./helio/blaze.sh -release
cd build-opt && ninja dragonfly
2021-11-30 08:11:59 +00:00
```
## Running
```
./dragonfly --logtostderr
2021-11-30 08:11:59 +00:00
```
for more options, run `./dragonfly --help`
2022-04-04 09:07:27 +00:00
## Milestone - Source Available
API 1.0
- [X] String family
- [X] SET
- [ ] SETNX
- [X] GET
- [X] DECR
- [X] INCR
- [X] DECRBY
2022-01-09 18:43:49 +00:00
- [X] GETSET
- [X] INCRBY
- [X] MGET
- [X] MSET
2022-04-01 17:43:56 +00:00
- [X] MSETNX
- [X] SUBSTR
- [ ] Generic family
- [X] DEL
- [X] ECHO
- [X] EXISTS
- [X] EXPIRE
2022-01-09 18:43:49 +00:00
- [X] EXPIREAT
2022-04-19 08:38:23 +00:00
- [X] KEYS
- [X] PING
- [X] RENAME
- [X] RENAMENX
- [X] SELECT
2022-01-09 18:43:49 +00:00
- [X] TTL
- [X] TYPE
- [ ] SORT
- [X] Server Family
2022-04-19 08:38:23 +00:00
- [X] AUTH
- [X] QUIT
- [X] DBSIZE
- [ ] BGSAVE
- [X] SAVE
2022-01-09 18:43:49 +00:00
- [X] DEBUG
- [X] EXEC
- [X] FLUSHALL
2022-01-09 18:43:49 +00:00
- [X] FLUSHDB
- [X] INFO
2022-01-09 18:43:49 +00:00
- [X] MULTI
- [X] SHUTDOWN
- [X] LASTSAVE
- [X] SLAVEOF/REPLICAOF
- [ ] SYNC
- [ ] Set Family
- [x] SADD
- [x] SCARD
- [X] SDIFF
- [X] SDIFFSTORE
- [X] SINTER
- [X] SINTERSTORE
- [X] SISMEMBER
- [X] SMOVE
- [X] SPOP
- [ ] SRANDMEMBER
- [X] SREM
- [X] SMEMBERS
- [X] SUNION
- [X] SUNIONSTORE
- [X] List Family
- [X] LINDEX
- [X] LLEN
- [X] LPOP
- [X] LPUSH
2022-03-15 17:52:11 +00:00
- [X] LRANGE
- [X] LREM
- [X] LSET
- [X] LTRIM
- [X] RPOP
- [ ] RPOPLPUSH
- [X] RPUSH
2022-03-06 06:46:48 +00:00
- [X] SortedSet Family
- [X] ZADD
- [X] ZCARD
2022-03-30 11:25:42 +00:00
- [X] ZINCRBY
- [X] ZRANGE
- [X] ZRANGEBYSCORE
2022-03-06 06:46:48 +00:00
- [X] ZREM
2022-03-22 21:50:47 +00:00
- [X] ZREMRANGEBYSCORE
- [X] ZREVRANGE
2022-03-06 06:46:48 +00:00
- [X] ZSCORE
- [ ] Not sure whether these are required for the initial release.
- [ ] BGREWRITEAOF
- [ ] MONITOR
- [ ] RANDOMKEY
- [ ] MOVE
API 2.0
2022-04-04 09:07:27 +00:00
- [X] List Family
- [X] BLPOP
2022-03-31 11:26:33 +00:00
- [X] BRPOP
- [ ] BRPOPLPUSH
2022-04-18 16:45:48 +00:00
- [X] LINSERT
2022-03-24 21:04:02 +00:00
- [X] LPUSHX
- [X] RPUSHX
2022-03-07 21:11:43 +00:00
- [X] String Family
- [X] SETEX
- [X] APPEND
- [X] PREPEND (dragonfly specific)
- [ ] BITCOUNT
- [ ] BITFIELD
- [ ] BITOP
- [ ] BITPOS
- [ ] GETBIT
2022-03-24 21:04:02 +00:00
- [X] GETRANGE
2022-04-18 16:45:48 +00:00
- [X] INCRBYFLOAT
2022-03-26 17:06:46 +00:00
- [X] PSETEX
- [ ] SETBIT
2022-03-26 17:06:46 +00:00
- [X] SETRANGE
2022-03-24 21:04:02 +00:00
- [X] STRLEN
2022-03-03 07:34:53 +00:00
- [X] HashSet Family
- [X] HSET
- [X] HMSET
2022-03-03 07:34:53 +00:00
- [X] HDEL
- [X] HEXISTS
- [X] HGET
- [X] HMGET
2022-03-03 07:34:53 +00:00
- [X] HLEN
2022-04-02 08:03:35 +00:00
- [X] HINCRBY
2022-04-19 19:21:54 +00:00
- [X] HINCRBYFLOAT
- [X] HGETALL
- [X] HKEYS
2022-04-02 08:03:35 +00:00
- [X] HSETNX
- [X] HVALS
2022-04-19 19:21:54 +00:00
- [X] HSCAN
- [ ] PubSub family
2022-03-30 11:25:42 +00:00
- [X] PUBLISH
- [ ] PUBSUB
- [ ] PUBSUB CHANNELS
2022-03-30 11:25:42 +00:00
- [X] SUBSCRIBE
- [X] UNSUBSCRIBE
- [ ] PSUBSCRIBE
- [ ] PUNSUBSCRIBE
- [ ] Server Family
- [ ] WATCH
- [ ] UNWATCH
2022-03-31 11:26:33 +00:00
- [X] DISCARD
- [ ] CLIENT KILL/LIST/UNPAUSE/PAUSE/GETNAME/SETNAME/REPLY/TRACKINGINFO
- [X] COMMAND
- [ ] COMMAND COUNT/GETKEYS/INFO
- [ ] CONFIG GET/REWRITE/SET/RESETSTAT
- [ ] MIGRATE
- [ ] ROLE
- [ ] SLOWLOG
- [ ] PSYNC
- [ ] TIME
- [ ] LATENCY...
- [X] Generic Family
- [X] SCAN
- [X] PEXPIREAT
- [ ] PEXPIRE
- [ ] DUMP
- [X] EVAL
- [X] EVALSHA
- [ ] OBJECT
- [ ] PERSIST
2022-03-24 21:04:02 +00:00
- [X] PTTL
- [ ] RESTORE
- [X] SCRIPT LOAD
- [ ] SCRIPT DEBUG/KILL/FLUSH/EXISTS
- [X] Set Family
- [X] SSCAN
- [X] Sorted Set Family
- [X] ZCOUNT
- [ ] ZINTERSTORE
- [X] ZLEXCOUNT
- [X] ZRANGEBYLEX
- [X] ZRANK
- [ ] ZREMRANGEBYLEX
- [X] ZREMRANGEBYRANK
- [ ] ZREVRANGEBYSCORE
- [X] ZREVRANK
- [ ] ZUNIONSTORE
- [ ] ZSCAN
- [ ] HYPERLOGLOG Family
- [ ] PFADD
- [ ] PFCOUNT
- [ ] PFMERGE
2022-04-04 09:07:27 +00:00
In addition, we want to support efficient expiry (TTL) and cache eviction algorithms.
We should implement basic memory management support. For Master/Slave replication we should design
a distributed log format.
### Memchache API
- [X] set
- [X] get
- [X] replace
- [X] add
- [X] stats (partial)
- [x] append
- [x] prepend
- [x] delete
- [x] flush_all
- [x] incr
- [x] decr
- [x] version
- [x] quit
Commands that I prefer avoid implementing before launch:
- PUNSUBSCRIBE
- PSUBSCRIBE
- HYPERLOGLOG
- SCRIPT DEBUG
- OBJECT
- DUMP/RESTORE
- CLIENT
Also, I would omit keyspace notifications. For that I would like to deep dive and learn
exact use-cases for this API.
2022-04-04 09:07:27 +00:00
### Random commands we implemented as decorators along the way
- [X] ROLE (2.8) decorator for for master withour replicas
- [X] UNLINK (4.0) decorator for DEL command
- [X] BGSAVE
- [X] FUNCTION FLUSH
2022-03-05 19:35:49 +00:00
2022-04-04 09:07:27 +00:00
## Milestone Stability
APIs 3,4,5 without cluster support, without modules, without memory introspection commands.
Without geo commands and without support for keyspace notifications, without streams.
Design config support. ~10-20 commands overall...
Probably implement cluster-API decorators to allow cluster-configured clients to connect to a single
instance.
2022-03-05 19:35:49 +00:00
2022-04-05 05:36:00 +00:00
- [X] HSTRLEN
2022-03-05 19:35:49 +00:00
## Design decisions along the way
### Expiration deadlines with relative accuracy
Expiration ranges are limited to ~4 years. Moreover, expiration deadlines
2022-03-05 19:35:49 +00:00
with millisecond precision (PEXPIRE/PSETEX etc) will be rounded to closest second
**for deadlines greater than 134217727ms (approximately 37 hours)**.
Such rounding has less than 0.001% error which I hope is acceptable for large ranges.
2022-03-24 09:30:22 +00:00
If it breaks your use-cases - talk to me or open an issue and explain your case.
2022-04-04 09:07:27 +00:00
For more detailed differences between this and Redis implementations [see here](doc/differences.md).