1. average batch latency will always be 0, because even in cases we have outliers they will be dominated by
small CPU only copies that take dozens of ns.
2. Measuring an operation like kBatch, which is solely CPU-based, necessitated the use of a clock. According to the CPU profiler,
this contributed to approximately 5% of the CPU usage.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
This should reduce allocations in a common case (not multi).
In addition, rename Transaction::args_ to kv_args_.
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
Co-authored-by: Vladislav <vlad@dragonflydb.io>
* fix(server): mget crash on same key get
fix: #2465
the bug: on cache mode mget bumps up items. When executing mget with the same key several times i.e mget key key we will invalidate the iterator when we bump up the item in dash table.
the fix: bump up/down items only once by using bumped_items set
This PR also reverts c225113
and updates the bumped stats and bumped_items set if the item was bumped
Signed-off-by: adi_holden <adi@dragonflydb.io>
This is needed if we want to allow asynchronous transactional operations during the callback execution.
Also update actions versions.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore(transaction): Simplify PollExecution()
Remove seqlock_ from transaction. This change is possible because:
- We don't re-use shard_data[0] for multi transactions anymore
- We disarm atomically and poll callbacks are stateless
This makes it safe to call PollExecution() unconditionally that will determine on it's own whether the caller needs to run or is already expired
---------
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
* chore: switch json object to pmr allocator
1. Move json object creation code to the shard-thread inside rdb_load
2. Now json_family never references "json" type, always dfly::JsonType
3. Switch JsonType to pmr::json.
4. Make sure we pass the correct memory_resource when creating json object from string.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* fix: fixes for v1.14.0
Stop writing to the replication ring_buffer
Stop allocating in TopKeys
Tighter CHECKs around tx execution.
---------
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
* feat: allow reading offloaded strings without loading to the store
* Add disk stats to IoMgr
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* feat(cluster): add tx execution in cluster_shard_migration
refactor(replication): move code that is common for cluster and
replica into a separate file, add full-sync-cut cmd
Previously, transactions would run out of order only when all shards determined that the keys locks were free. With this change, each shard might decide to run out of order independently if the locks are free. COORD_OOO is now deprecated and the OUT_OF_ORDER per-shard flag should is used to indicate it
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
Motivation - after we submitted #2429 some smart-ass clients
prevent users from accessing single-node commands like "SELECT".
This PR fixes it by allowing consistent sharding based on hashtags
even with cluster mode disabled.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
feat: measure the latency of pipelined commands
Unlike with regular latency measurement that measures the execution time + the reply time,
here we measure the overall latency from the point in time when a command was parsed.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
We had a place in tools/packaging/generate_debian_package.sh that relied on the existence of build-opt,
moreover, if it did not exist the script deadlocked.
1. Added more loggings
2. Removed the loop
3. Removed unnecessary dependency in the script on the build-dir name.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* fix(replication): Correctly replicate commands even when OOM
Before this change, OOM in shard callbacks could have led to data
inconsistency between the master and the replica. For example, commands
which mutated data on 1 shard but failed on another, like `LMOVE`.
After this change, callbacks that result in an OOM will correctly
replicate their work (none, partial or complete) to replicas.
Note that `MSET` and `MSETNX` required special handling, in that they are
the only commands that can _create_ multiple keys, and so some of them
can fail.
Fixes#2381
* fixes
* test fix
* RecordJournal
* UNDO idiotnessness
* 2 shards
* fix pytest
The (subtle) bug is that the previous code uses an `initializer_list` c'tor, which copies the
`string_view` locally. Then it keeps that reference to the `string_view`,
but it goes out of scope in the next line
* feat: allow throttling tiered writes
The throttling is controlled by tiered_storage_throttle_us flag
and can be disabled by passing `--tiered_storage_throttle_us=0`.
This introduces a soft back-pressure during writes.
On my machine `debug POPULATE 10000000 key 1000 RAND` with tiered_storage_throttle_us=0
offloads 12% of all the entries, but with tiered_storage_throttle_us=1 it offloads
almost 100% by prolonging the operation from 0.96s to 1.72s.
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* feat(server): Implement `CLIENT KILL`
Currently, it supports the following syntax:
* `CLIENT KILL <addr>:<port>`
* `CLIENT KILL ID <id>`
* `CLIENT KILL ADDR <addr>:<port>`
* `CLIENT KILL LADDR <addr>:<port>`
It will not allow killing an admin-connection from a non-admin port.
There are a few parameters of `CLIENT KILL` that Redis supports but this
PR does not yet add. Let's add them as needed.
Fixes#1614
* Add tests
* fixes
