1
0
Fork 0
mirror of https://github.com/dragonflydb/dragonfly.git synced 2024-12-14 11:58:02 +00:00
dragonflydb-dragonfly/README.md
2022-03-15 19:52:11 +02:00

4.6 KiB

Dragonfly

ci-tests

A novel memory store that supports Redis and Memcached commands. For more detailed status of what's implemented - see below.

Features include:

  1. High throughput reaching millions of QPS on a single node.
  2. TLS support.
  3. Pipelining mode.
  4. A novel cache design, which does not require specifying eviction policies.
  5. Memory efficiency that can save 20-40% for regular workloads and even more for cache like workloads

Building from source

I've tested the build on Ubuntu 21.04+.

git clone --recursive https://github.com/romange/dragonfly
cd dragonfly && ./helio/blaze.sh -release
cd build-opt && ninja dragonfly

Running

./dragonfly --logtostderr

for more options, run ./dragonfly --help

Milestone Egg 🥚

API 1.0

  • String family
    • SET
    • SETNX
    • GET
    • DECR
    • INCR
    • DECRBY
    • GETSET
    • INCRBY
    • MGET
    • MSET
    • MSETNX
    • SUBSTR
  • Generic family
    • DEL
    • ECHO
    • EXISTS
    • EXPIRE
    • EXPIREAT
    • PING
    • RENAME
    • RENAMENX
    • SELECT
    • TTL
    • TYPE
    • SORT
  • Server Family
    • QUIT
    • DBSIZE
    • BGSAVE
    • SAVE
    • DBSIZE
    • DEBUG
    • EXEC
    • FLUSHALL
    • FLUSHDB
    • INFO
    • MULTI
    • SHUTDOWN
    • LASTSAVE
    • SLAVEOF/REPLICAOF
    • SYNC
  • Set Family
    • SADD
    • SCARD
    • SDIFF
    • SDIFFSTORE
    • SINTER
    • SINTERSTORE
    • SISMEMBER
    • SMOVE
    • SPOP
    • SRANDMEMBER
    • SREM
    • SMEMBERS
    • SUNION
    • SUNIONSTORE
  • List Family
    • LINDEX
    • LLEN
    • LPOP
    • LPUSH
    • LRANGE
    • LREM
    • LSET
    • LTRIM
    • RPOP
    • RPOPLPUSH
    • RPUSH
  • SortedSet Family
    • ZADD
    • ZCARD
    • ZINCRBY
    • ZRANGE
    • ZRANGEBYSCORE
    • ZREM
    • ZREMRANGEBYSCORE
    • ZREVRANGE
    • ZSCORE
  • Not sure whether these are required for the initial release.
    • AUTH
    • BGREWRITEAOF
    • KEYS
    • MONITOR
    • RANDOMKEY
    • MOVE

In addition, we want to support efficient expiry (TTL) and cache eviction algorithms. We should implement basic memory management support. For Master/Slave replication we should design a distributed log format.

Memchache API

  • set
  • get
  • replace
  • add
  • stats (partial)
  • append
  • prepend
  • delete
  • flush_all
  • incr
  • decr
  • version
  • quit

API 2.0

  • List Family
    • BLPOP
    • BRPOP
    • BRPOPLPUSH
  • String Family
    • SETEX
  • HashSet Family
    • HSET
    • HMSET
    • HDEL
    • HEXISTS
    • HGET
    • HMGET
    • HLEN
    • HINCRBY
    • HINCRBYFLOAT
    • HGETALL
    • HKEYS
  • PubSub family
    • PUBLISH
    • PUBSUB
    • PUBSUB CHANNELS
    • SUBSCRIBE
    • UNSUBSCRIBE
  • Server Family
    • WATCH
    • UNWATCH
    • DISCARD
  • Generic Family
    • SCAN
    • PEXPIREAT
  • String Family
    • APPEND
    • PREPEND (dragonfly specific)

Commands that I prefer not implement before launch:

  • PUNSUBSCRIBE
  • PSUBSCRIBE

Also, I would omit keyspace notifications. For that I would like to deep dive and learn exact use-cases for this API.

Random commands we implemented along the way

  • ROLE (2.8) decorator for for master withour replicas
  • UNLINK (4.0) decorator for DEL command
  • BGSAVE

Milestone Nymph

API 2,3,4 without cluster support, without modules, without memory inspection commands. Without support for keyspace notifications.

Design config support. ~140 commands overall...

Milestone Molt

API 5,6 - without cluster and modules. Streams support. ~80 commands overall.

Milestone Adult

TBD.

Design decisions along the way

Expiration deadlines with relative accuracy

I decided to limit the expiration range to 365 days. Moreover, expiration deadlines with millisecond precision (PEXPIRE/PSETEX etc) will be rounded to closest second **for deadlines greater than 33554431ms (approximately 560 minutes). In other words, expiries of PEXPIRE key 10010 will expire exactly after 10 seconds and 10ms. However, PEXPIRE key 34000300 will expire after 34000 seconds (i.e. 300ms earlier). Similarly, PEXPIRE key 34000800 will expire after 34001 seconds, i.e. 200ms later.

Such rounding has at most 0.002% error which I hope is acceptable for large ranges. If it breaks your use-cases - talk to me or open an issue and explain your case.