* feat(server): Add `RestoreSerializer`
This utility class serializes `CompactObj`s as `RESTORE` commands, and
has a similar interface (and a common base class) as `RdbSerializer`
* RETURN_ON_ERR
* fixes
Issue was that in `ServerFamilyTest.SlowLogLen` we set the threshold to
be 0 microseconds and make sure that all commands are logged as slow.
However, in opt, some commands sometimes take 0 microseconds, which
fails the test.
Confirmed via:
```
./server_family_test --gtest_repeat=100 --gtest_filter=ServerFamilyTest.SlowLogLen
```
* chore: consolidate facade stats under a single struct
Remove connection stats from server state and move them under FacadeStats.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: fixing comments
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* feat: introduce user timeout
* feat: introduce tcp_user_timeout flag.
See TCP_USER_TIMEOUT flag in tcp(7) man page.
This linux-only setting allows fail faster during the send operation
if for some reason the remote socket is unresponsive and does not send ACKs for
the transmitted segments.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* Update src/facade/dragonfly_listener.cc
Co-authored-by: Shahar Mike <chakaz@users.noreply.github.com>
Signed-off-by: Roman Gershman <romange@gmail.com>
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Signed-off-by: Roman Gershman <romange@gmail.com>
Co-authored-by: Shahar Mike <chakaz@users.noreply.github.com>
* feat: add SLOT-MIGRATION-STATUS cmd for source node
implements #2232
add ability using SLOT-MIGRATION-STATUS without args
to print info about all migration processes for the current node
fix: fix "debug exec" command
It used mutex lock inside Await callback which is prohibited.
In addition, we improved loggings across the transaction code.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
chore: add send state to client list
This can be done only via a differrent fiber by inspecting SinkReplyBuilder state.
Also, get rid of awkward reply builder statistics and use instead thread local stats for that.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
chore: simplify transaction multi-locking
Also, add the ananlysis routine that determines whether the schewduled transaction is contended with other transaction in a
shard thread.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
We parse the container limits for heuristics that deduces memory/cpu capacities automatically.
It's ok if we fail on some less common systems since there are manual overrides that allows specifying these
limits explicitly.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Before this PR, using `make` for building releases used `-g1 -gz` to
provide some level of debug symbols (mostly function names in stack
traces).
We want to be able to have binaries with full debug symbols (for
coredumps etc), but not in the public releases.
This PR builds with full debug symbols, but changes `make package` to
strip most debug symbols, while keeping `.debug_line` (and keeping them
compressed as if with `-gz`). This will allow a single build command for
public releases as well as releases with full symbols.
1. Fix AnalyzeTxQueue to stop crashing for various transaction types.
2. Pass exec command length to slowlog
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
fix#2337
The bug:
replicaof was not rejected while loading snapshot
The fix:
replicaof is allowed while server is in loading state to allow replicaof while replication in full sync mode
I now reject replicaof if the server is in loading state and it is master
Another bug fix:
allow cron snapshot if --replicaof flag was set
Signed-off-by: adi_holden <adi@dragonflydb.io>
* feat: track differrent patterns of multi/exec transactions
This information is exposed via "DEBUG EXEC" command.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: address comments + add basic squasher stats
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
This command shows the current state of transaction queues,
specifically how many armed (ready to run) transactions there,
how loaded these queue are and how many locks there are in each shard.
In addition, if a tx queue becomes too long, we will output warning logs about
the state of the queue, in order to be able to identify
the bottlenecks post-factum.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* refactor(server): Privatize `PreUpdate()` and `PostUpdate()`
While at it:
* Make `PreUpdate()` not decrease object size
* Remove redundant leftover call to `PreUpdate()` outside `DbSlice`
* Add pytest
* Test delete leads to 0 counters
* Improve test
* fixes
* comments
This metric shows how much the transaction queue of dragonfly is loaded.
Also, output free text, human intended replies as verbatim.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* feat: add keyspace_mutations metric
Currently we expose hits/misses for read only commands only (compatible with redis).
`keyyspace_mutations` complement this providing number of key operations for write commands.
It's interesting because now we can learn the number of key_ops vs API ops, where
key_ops = misses + hits + mutations
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: address fixes
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
1. How many transactions we processed by type
2. How many transactions we processed by width (number of unique shards).
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
partly fixes#2294
bug: id ">" means that the consumer want to receive only messages
that were never delivered to any other consumer
fix: added temporary tests results, the correct ones are commented out
until we fix the blocking mechanism. Fix requested range for blocking
operations
fixes https://github.com/dragonflydb/dragonfly/issues/2139
This is part two that implements the logic which notifies tracking clients by sending invalidation messages:
- The client tracking state is set by CLIENT TRACKING subcommand as well
as upon client disconnection.
- Track the keys of a readonly command by maintaining mapping that maps
keys to the sets of tracking clients.
- Send invalidation messages to clients when their tracked keys are
updated.
- Make PerformDeletion a member function of DbSlice, and send
invalidation message within the function.
- Mock the function for sending invalidation message to avoid test
crash due to lack of real listener in the testing framework.
- Add functional (some) tests for client tracking based on the mocked interfaces.
---------
Signed-off-by: Yue Li <61070669+theyueli@users.noreply.github.com>
* feat(cluster): add command flow for slot migration process
fixes#2295
DFLYMIGRATE FLOW command was added to establish
connections for every shard replication process.
Slow serialization step is the separate issue so
for now only eof_token is sent for reply to
DFLYMIGRATE FLOW command.
Expected state for START-SLOT-MIGRATION is FULL_SYNC now.
The bug:
When running dragonfly in cache mode we bump up items on dash table when we find them. If we access few items on the callback that reside next to each other we will invalidate the first found iterator.
The fix:
After we bump up entry we insert the prime table ref to bump set. When checking if we can bump down an item we check the item is not in this set. Once we finish running the transaction callback we clear the set.
Signed-off-by: adi_holden <adi@dragonflydb.io>
