* Support flow-controlled gRPC client requests
- Split client request DATA frames according to the peer's connection and
stream flow-control windows.
- Buffer unsent DATA and resume transmission when WINDOW_UPDATE restores
capacity.
- Track pending request bytes per H2 connection and apply
socket_max_unwritten_bytes as an upper bound.
- Reject or reroute new requests once the pending DATA limit is reached.
- Release buffered DATA when an RPC fails, times out, or its stream is
removed.
- Add tests for fragmented transmission, deferred DATA flushing,
pending-byte accounting, and buffer cleanup.
* Fix pending HTTP/2 data limit race
- Check pending DATA capacity atomically with client stream insertion.
- Leave stream and window state unchanged when the limit is exceeded.
- Use an ephemeral port in the gRPC flow-control test.
- Avoid accessing an empty payload buffer in H2 frame tests.
ParseSofaMessage only checked body_size against max_body_size, while
meta_size and the total frame size were left unbounded. A frame with a
large meta_size and zero body_size passed the body_size check and made
the connection keep buffering far beyond the configured limit before the
invalid metadata was rejected. Bound meta_size by max_body_size as well,
consistent with other protocols such as baidu_std and hulu_pbrpc.
Add unit tests covering oversized body and oversized metadata.
* Limit mcpack2pb array item count to the actual payload size
The item count in an mcpack array header is read directly from the
request and was used as-is by the generated parsing code to Reserve()
memory for repeated protobuf fields. A malformed request could claim an
item count up to INT32_MAX and force the server to preallocate ~16GB of
virtual memory, which may abort the process on memory-constrained hosts.
Cap the item count by the remaining bytes of the array (each item
occupies at least one byte) so that the preallocation is bounded by the
request size.
* Fix underflow in mcpack2pb array item count clamping
Add the missing BRPC_WITH_UBRING Bazel configuration, wire it through
the library and example targets, and document the supported build
commands.
Enable RDMA and UBRING in CI jobs whose all-options configurations did
not exercise those features. Run both feature suites in the existing
Bazel unit-test jobs and enable RDMA in the Make unit-test job.
Use unsigned literals for UBRING atomic counter operations to match the
counter type and avoid template deduction failures on stricter
compilers.
Signed-off-by: Zhengwei Zhu <141622927+ZhengweiZhu@users.noreply.github.com>
Wait for the asynchronous span collector to release root client spans
after Controller reset or submission. This prevents LeakSanitizer from
reporting pending test spans when the short-lived test binary exits
before the collector’s first polling interval.
Signed-off-by: Zhengwei Zhu <141622927+ZhengweiZhu@users.noreply.github.com>
Keep the current RPC span alive from Controller until the RPC finishes,
SubmitSpan runs, or the Controller is reset. This lets root client spans
without a local parent be submitted to rpcz instead of being destroyed
after the caller-side temporary shared_ptr goes out of scope.
Child client spans remain linked to their parent through weak local-parent
references and parent-owned client lists, so they are still serialized
under their parent without introducing shared_ptr cycles.
Co-authored-by: lh2debug <lh2debug@163.com>
PollCq only re-polled send_cq after arming both CQs, so a recv CQE
arriving in the one-shot notification race window of recv_cq was left
in the CQ and the RPC timed out. Restart the re-poll from recv_cq so
that both CQs are covered.
* build(cxx): align all builds on a C++14 baseline
CMake, Make, tests, tools, and examples now require C++14 so every supported build path shares the same minimum language level. The compiler checks and user-facing documentation reflect that baseline.
* fix(test): keep cores scoped to the failing test
Clear prior cores before each test binary. A passing death test can leave a core that would otherwise be used to diagnose an unrelated later failure.
* revert src/butil/type_traits.h
* cputime optimize for arm64
Update src/butil/time.cpp
modify using __attribute__((constructor)) and merge init_invariant_cpu_freq to inside function read_invariant_cpu_frequency
* Add the macro switch BUTIL_USE_CPU_FREQUENCY while preserving the original default behavior for ARM64
* 1、Activation options: CMake -DWITH_CPU_FREQUENCY=ON, Bazel --define BUTIL_USE_CPU_FREQUENCY=true, script argument --with-cpu-frequency.
2、Disabled by default for consistent legacy behavior.
* fix compile failed
---------
Co-authored-by: seekdwh <dongweihao@huawei.com>
Make the handshake state atomic and publish RDMA_OFF before
FALLBACK_TCP. This prevents TCP event handling from observing a
partially published fallback state. Use a relaxed load for the
completion-path check because it does not consume related data.
Signed-off-by: Lijin Xiong <legion.xiong@gmail.com>
Co-authored-by: Lijin Xiong <lijin.xiong@zstack.io>
Controller::set_request/response_checksum_type() previously only
covered the serialized protobuf body; the attachment (if any) was
never protected. Add set_request/response_checksum_attachment(bool)
so callers can opt the attachment into the same checksum.
- baidu_rpc_meta.proto: add RpcMeta.checksum_with_attachment so the
receiver knows whether to fold the attachment into verification.
Defaults to false, so old peers that don't understand the field
keep verifying against the body only (backward compatible).
- Controller: add the two setters/getters, thread the flag through
ClientSettings (Save/ApplyClientSettings) so ParallelChannel/
SelectiveChannel sub-controllers inherit it correctly, and reset it
in ResetPods().
- ChecksumIn: add an optional `attachment' field consumed by checksum
handlers.
- crc32c_checksum.cpp: extend the crc32c over body then attachment (in
that fixed order) when requested.
- baidu_rpc_protocol.cpp: wire checksum_attachment through
SerializeRpcMessage/DeserializeRpcMessage and every client/server
send/receive path; skip it when progressive attachment reading is
enabled since there's no single complete IOBuf to checksum in that
case.
Add brpc_checksum_unittest.cpp covering Crc32cCompute/Crc32cVerify
directly (including corruption/omission/order sensitivity) and an
end-to-end Server/Channel test for both request- and response-side
attachment checksums.
* guard negative value_size in memcache PopStore
Signed-off-by: ubeddulla khan <ubed@bugqore.com>
* memcache: drop only the declared message when value_size is negative
Move the value_size < 0 check ahead of the pop_front so a malformed
reply no longer pops sizeof(header) + extras_length + key_length past
the declared message boundary. Discard exactly
sizeof(header) + total_body_length instead, keeping the following
pipelined responses aligned.
Signed-off-by: ubeddulla khan <ubed@bugqore.com>
---------
Signed-off-by: ubeddulla khan <ubed@bugqore.com>
Example CMakeLists duplicated the same dependency discovery and link
setup. Move that into brpc_example_find_common_deps in
BrpcExample.cmake, migrate MySQL and other examples to use it, and
implement the helper as a macro so CMAKE_PREFIX_PATH, include paths,
and DYNAMIC_LIB propagate to the caller scope.
Example-specific deps stay local (e.g. readline/ncurses/thriftnb for
MySQL, gperftools for redis/http). LINK_SO behavior is unchanged.
* add ubring transport
* fix the bug for ub ring transport
* fix the bug for ub ring transport and other
* add the license for ub transport
* Modifying the variable naming style
* optimize the log message and some field name
* fix some bug for ubring
* add some log ubring endpoint
* add todo
* fix the bug for handshake for ub endpoint
* fix the bug for client ub endpoint
* optimize the iobuf file code
* modify the log level
* fix the declare_shm_ubs define not found bug
* add the timer_mgr support for macos and format code style
* fix the bug for macos epoll
* fix the timespece bug
* adaptor the itimerspec for macos platform
* optimize the cmakelist config
* add the ubring docs for ubring transport
* modify some file name and directory structure
* modify some code style and optimize code logical
* add the cmake ubshm transport ci/testing
* add the dependency header
* bug fix the code
* Fix UB shm allocation cleanup crashes (#11)
Co-authored-by: 郭业昌 <lvpengfei@MacBook-Air.local>
* remove the brpc_ubshm_unittest
* Found and fixed two stability issues (#13)
* 修复ubring server端关闭连接coredump问题
* 修复PollIn/PollOut解引用已释放Socket指针的问题
PollIn/PollOut通过ep->_socket(裸指针)读取data socket,当data socket
被销毁时该指针悬空,导致Socket::Address读到垃圾id触发SIGSEGV。
改为存储_socket_id(SocketId),用Address获取引用计数的Socket,
并在整个回调期间持有该引用,避免解引用悬空指针。
* 修复client非正常退出导致UBRING shm残留的问题
client被强杀(SIGTERM/崩溃/OOM)时teardown没跑完,localShm(_C)的
shm_unlink未执行,导致/dev/shm残留_C文件。server的remoteShm只munmap
不unlink(正确),无法清理client的名字。
在握手ESTABLISHED时(client/server都确认对方已mmap自己的localShm)
立即unlink localShm名字。此时对端已持有mmap引用,unlink只删名字不
影响通信;进程任意时刻退出都不会残留文件名。
* Address chenBright's review: use English comments and BAIDU_CACHELINE_ALIGNMENT
- Convert all Chinese comments in ubshm to English (per chenBright's
'Please use English' on ub_endpoint.cpp:723, ub_ring.cpp:337,
shm_ubs.cpp:316, and similar)
- Replace __attribute__((aligned(64))) with BAIDU_CACHELINE_ALIGNMENT
in ubr_msg.h (per chenBright's comment on ubr_msg.h:41)
- Remove unnecessary TODO comment in ub_ring.cpp:551 (per chenBright's
'Unnecessary comments, please delete')
* Remove unused lock macros in thread_lock.h
Per chenBright's review, the functions and macros defined in
thread_lock.h are largely unused. Verified usage across ubshm:
- LOCK_GUARD / UnlockMutex: 8 call sites in shm_ubs.cpp and
ub_ring_manager.cpp, kept.
- SPIN_LOCK_GUARD, R_LOCK_GUARD, W_LOCK_GUARD, SEMAPHORE_WAIT_GUARD,
SEMAPHORE_WAIT_GUARD_WITH_CLOSE and their helper functions
(UnlockSpinLock, UnlockRWLock, PostSem, PostSemWithClose): 0 call
sites, removed.
* Apply chenBright's review on timer_mgr globals
Per chenBright's review on timer_mgr.cpp:32-37:
- Add explicit default values to uninitialized globals
(g_total_timer_num=0, g_max_system_fd=0, g_epoll_execute_thread=0,
g_timer_module_initialized=0)
- Rename globals to snake_case (g_epollFd -> g_epoll_fd,
g_totalTimerNum -> g_total_timer_num, g_timerFdCtxMap ->
g_timer_fd_ctx_map, maxSystemFd -> g_max_system_fd,
g_epollExecuteThread -> g_epoll_execute_thread,
g_timerModuleInitialized -> g_timer_module_initialized)
- maxSystemFd also gains the g_ prefix to match global naming style
Also fix the missing std:: qualifier on atomic_fetch_sub/add/load
(per chenBright's earlier comment on timer_mgr.cpp:80).
* Change CloseTimerFd fd type from uint32_t to int
Per chenBright's review on timer_mgr.cpp:399 (uint32_t -> int).
fd is a system file descriptor; POSIX APIs use int and -1 denotes an
invalid fd, which uint32_t cannot represent. Changed the CloseTimerFd
signature (header + definition) and removed the now-unnecessary
(uint32_t) casts at the two call sites.
* Use BAIDU_LIKELY/BAIDU_UNLIKELY instead of custom __builtin_expect
Per chenBright's review on common.h:27. Rather than redefine the
macros with __builtin_expect directly, forward LIKELY/UNLIKELY to
brpc's standard BAIDU_LIKELY/BAIDU_UNLIKELY (from butil/compiler_specific.h).
The 122 call sites keep using LIKELY()/UNLIKELY() unchanged; only the
macro bodies change, preserving semantics.
* Add unit tests for UBShmEndpoint
Per chenBright's request to add unit tests for UBShmTransport in this
PR (rather than a follow-up).
Adds test/brpc_ubring_unittest.cpp with tests covering the public
interface of UBShmEndpoint under the g_skip_ub_init=true mode (which
skips real shared-memory/poller setup):
- construct_and_destruct: lifecycle safety
- is_writable_false_when_skip_init: skip-mode behavior
- reset_is_idempotent: Reset() is safe to call repeatedly
The file follows the brpc_*_unittest.cpp naming convention so it is
auto-collected by test/CMakeLists.txt's file(GLOB). Verified: compiles,
links, and all 3 tests pass (g++ 15.2, C++17, gtest, BRPC_WITH_UBRING=ON).
* Rewrite UBShmEndpoint unit tests with real coverage
Per chenBright's feedback that the previous tests were too simple and
did not cover the main methods.
Source changes to enable testing:
- Move HelloMessage struct declaration from ub_endpoint.cpp to
ub_endpoint.h so tests can access it
- Expose private members under #ifdef UNIT_TEST (precedent:
butil/containers/stack_container.h) so tests can call
AllocateClientResources without -Dprivate=public (which breaks
GCC 15 + new libstdc++ <any>/<sstream>)
Tests (9, all passing on Ubuntu 26.04 g++ 15.2 C++17 gtest):
HelloMessageTest (5): serialize/deserialize roundtrip, network byte
order verification, uint64 max boundary, full shm_name, toString
UBShmEndpointTest (4): construct, real IPC shm
AllocateClientResources (g_skip_ub_init=false), reset cleanup, reset
idempotency
* rename the variable to snake_case style
---------
Co-authored-by: YeChang Guo <52730608+YChange01@users.noreply.github.com>
Co-authored-by: 郭业昌 <lvpengfei@MacBook-Air.local>
Co-authored-by: gure <740684863@qq.com>
Previously BringUpQp only did a local ibv_query_ece + ibv_set_ece
roundtrip and never exchanged ECE capabilities with the peer.
This patch wires up the standard requestor/responder ECE negotiation
flow on top of the existing v3 handshake without adding any extra
round trip:
1. Client queries local ECE, advertises it in its v3 hello.
2. Server applies the client's ECE in INIT->RTR (set_ece), then
after RTS queries the reduced/negotiated ECE and sends it back
in the reply hello.
3. Client applies the server's reduced ECE in INIT->RTR.
Replace the function-like macros ADD_TLS_PTHREAD_LOCK_COUNT and
SUB_TLS_PTHREAD_LOCK_COUNT with inline functions
AddTlsPthreadLockCount() and SubTlsPthreadLockCount() (with matching
no-op definitions in the BRPC_DEBUG_BTHREAD_SCHE_SAFETY=0 branch).
Macros lack type safety and scoping, so encapsulating the logic in
regular functions is clearer and safer while keeping the same
behavior. All call sites are updated accordingly.
tls_unique_user_ptr was read directly without going through the
BAIDU_GET_VOLATILE_THREAD_LOCAL macro in TaskGroup::sched_to, while
the paired write later in the same function correctly used
BAIDU_SET_VOLATILE_THREAD_LOCAL. sched_to contains jump_stack, a
suspend-point where GCC (aarch64) and Clang may incorrectly cache
the address of a thread_local variable, which is exactly the hazard
BAIDU_GET_VOLATILE_THREAD_LOCAL is designed to avoid. Fix the read to
use the macro for consistency and correctness.
Weibing Wang