master
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Bright Chen
|
271b98fef7 | Refactor NULL with nullptr in example (#3466) | ||
|
darion-yaphet
|
f094566f54 |
build(example): extract shared CMake dependency helper for examples (#3391)
Example CMakeLists duplicated the same dependency discovery and link setup. Move that into brpc_example_find_common_deps in BrpcExample.cmake, migrate MySQL and other examples to use it, and implement the helper as a macro so CMAKE_PREFIX_PATH, include paths, and DYNAMIC_LIB propagate to the caller scope. Example-specific deps stay local (e.g. readline/ncurses/thriftnb for MySQL, gperftools for redis/http). LINK_SO behavior is unchanged. |
||
|
rajvarun77
|
7684356982 |
Add MySQL client protocol (text protocol, transactions, prepared statements) on clean-room auth (#3330)
* feat(mysql): clean-room MySQL authentication codec Clean-room implementation of the MySQL connection-phase authentication handshake, derived from the public MySQL protocol documentation with no GPL lineage: mysql_native_password and caching_sha2_password scrambles, HandshakeV10/ HandshakeResponse41 codec, and length-encoded integer/string plus packet-header wire helpers. Handles the lenenc NULL (0xFB) marker and rejects an oversize auth_response. * feat(mysql): full MySQL text protocol with transactions and prepared statements Port the MySQL protocol client (issue #2093) onto the clean-room auth codec and protobuf 3.21 (NonreflectableMessage): COM_QUERY text protocol, interactive transactions via connection affinity, and prepared statements. Wire caching_sha2_password (fast-auth, full-auth RSA, and secure-transport cleartext) into the live client. Fix the lenenc 9-byte length marker in pack_encode_length (0xFD -> 0xFE) per the MySQL protocol spec. * test(mysql): clean-room integration tests + prepared-stmt error fix + Controller cleanup - Add clean-room integration tests (transactions, prepared statements, pooled connection concurrency, connection-type) run against a self-spawned mysqld. - Fix: a failed COM_STMT_PREPARE now returns the ERR packet to the caller and keeps the connection alive, instead of closing the socket. - Warn when a prepared statement runs on a 'short' connection (re-prepares on every execute; prefer 'pooled'). - Replace Controller's mysql-specific _mysql_stmt with a generic opaque per-RPC slot so no protocol type leaks onto the shared Controller. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(mysql): consolidate sources under policy/mysql/; drop inherited images Move all MySQL sources (mysql.*, mysql_command/reply/common/transaction/ statement*, mysql_protocol.*, mysql_authenticator.*) into src/brpc/policy/mysql/ alongside the clean-room auth codec; update all includes, build globs, and install rules. Remove three benchmark images inherited from the #2093 port and the doc section referencing them. No behavior change: full build green; all 19 mysql unit/integration tests pass; a 30-case standalone end-to-end run was independently verified against mysqld. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(mysql): address code-review findings (binary protocol, auth, edge cases) + ASF headers - Binary DATETIME/TIME: gate the microsecond bytes on the packet length, not the column's declared decimals (over-read / result-set desync). - COM_STMT_SEND_LONG_DATA: frame stmt_id/param_id inside the packet; fix chunk offset. - COM_STMT_EXECUTE: emit the trailing 0-length packet for 16MiB-aligned payloads. - OK/EOF status & warnings: decode via mysql_uint2korr (big-endian safe). - Row NULL-bitmap: arena-allocate instead of a stack VLA; cap column_count. - Auth: bounds-check the parsed auth string; size-bound StringPiece uses. - Prepared stmt: prune stale per-socket stmt_id map entries; count only real '?' placeholders (skip quotes/comments). - MysqlResponse::Clear and MysqlRequest copy/Swap: reset/copy all members. - Controller::ResetPods: release _bind_sock on controller reuse. - Standardize mysql file license headers to the ASF form; clarify auth comment. All 19 unit/integration tests still pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(mysql): address review findings — license header, flag typo, header-global, doc/comment drift - example/mysql_c++/mysql_go_press.go: add ASF Apache-2.0 license header (the last file failing the License Check; other 6 already had headers). - mysql_common.{h,cpp}: move the MysqlCollations map definition out of the header into the .cpp behind an `extern` declaration, so each translation unit no longer gets its own copy (C++11-safe; avoids the header-defined global flagged in review). - mysql_statement.{cpp,inl.h}: rename the misspelled gflag mysql_statment_map_size -> mysql_statement_map_size (user-facing name). - docs/cn/mysql_client.md: prepared statements ARE supported now — drop the stale "不支持Prepared statement". - brpc_mysql_connection_type_unittest.cpp: rewrite the stale header comment that described a removed "MustError" test; the prepared-statement path now transparently re-prepares under CONNECTION_TYPE_SHORT and succeeds, matching the actual PreparedStatementUnderShortRePreparesAndSucceeds test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(mysql): use the standard ASF license header on new example/test files The new MySQL example sources and brpc_mysql_unittest.cpp carried the old "Copyright (c) Baidu, Inc." Apache-2.0 header, which skywalking-eyes (the repo's License Check, configured copyright-owner = Apache Software Foundation) does not accept — so all 7 files failed the gate. Replace with the canonical ASF header used by the other 549 sources in the tree, and drop the stale Baidu copyright/date attribution lines. Verified locally with `license-eye -c .licenserc.yaml header check`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(mysql): add diagnostic error logs on silent failure paths Extend the failure-path logging from the auth codec to the full client protocol. Parsers and request/response handlers returned false/0/nullptr silently on malformed wire data, truncated packets, bad state, or missing prepared statements, giving no clue why a query/connection failed. Add a LOG(ERROR) at each silent failure path naming the function and the concrete cause. - mysql_auth_handshake/packet.cpp: handshake + lenenc codec failure paths (truncated <field>, pre-4.1 server, reserved 0xFF marker, length mismatch). - mysql.cpp: request command/param guards. - mysql_reply.cpp: result-set parse (column/row/error packet truncation, arena alloc failure, bad binary-row header). - mysql_protocol.cpp: serialize/process type + serialization failures. - mysql_statement.cpp: statement-id lookup misses (not-prepared / stale conn). Logic unchanged — logs inserted only. Normal control-flow returns (need-more-data, lenenc NULL 0xFB, short-connection no-cache) are not logged. mysql_authenticator.cpp and mysql_transaction.cpp already logged every failure path and were left unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(mysql): demote non-fatal diagnostic logs to LOG(WARNING) Tune the severity of the failure-path logs. LOG(ERROR) is reserved for total failures — handshake/connection corruption, authentication failure, out-of-memory, and fundamental request/response type or serialization misuse. Operational and statement-data failures are demoted to LOG(WARNING): - mysql_auth_packet.cpp: lenenc int/string/header decode failures (these fire during normal resultset parsing). - mysql.cpp: request command/param API guards. - mysql_reply.cpp: column/row/field/ERR-packet truncation (statement data). The arena out-of-memory and Auth::Parse auth-plugin failures stay ERROR. - mysql_statement.cpp: prepared-statement-id lookup misses (recoverable, trigger a re-prepare). handshake parsing and the protocol serialize/process type checks remain LOG(ERROR). Severity-only change; messages and logic unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(test): make AuthCase a C++11 aggregate so push_back{...} compiles CI compiles unittests with -std=c++0x (C++11), where a class with a default member initializer is not an aggregate, so g_auth_cases.push_back({...}) fails to compile. Drop the default member initializer on AuthCase::use_ssl; all init sites pass it explicitly. Mirrors the same fix on the #3310 codec branch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(mysql,ci): run integration tests against a real CI-provisioned mysqld The MySQL integration tests were not exercising a live server in CI: - Merge master to pick up #3323, which installs mysql-server in the install-essential-dependencies action used by the unittest lanes, so the mysqld binary is present for the self-spawning test/mysql/* suites. - Delete the legacy test/brpc_mysql_unittest.cpp: it hardcoded an external host (db4free.net) with embedded credentials and asserted on connect failure (no skip), so it timed out and hard-failed in CI. The test/mysql/*_integration suites supersede it, spawning a throwaway local mysqld and GTEST_SKIP-ing when none is available (the redis precedent). - Build the test/mysql/* suites in the make lane: extend test/Makefile's source glob and add a mysql/-prefixed link rule; add the subdir to run_tests.sh with nullglob so it runs them. - Bazel: build one cc_test per mysql test file via generate_unittests instead of globbing them all into a single brpc_mysql_test target, which duplicated main() and the FLAGS_mysql_* definitions (ld: duplicate symbol). Drop the per-file main()s so every suite relies on gtest_main, matching brpc_redis_unittest; tag the server-spawning suites external+local so bazel runs them unsandboxed with the real mysqld. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * address review comments: bind-sock flag, non-copyable response, naming - Controller: store the mysql-transaction BindSockAction in two bits of _flags (FLAGS_BIND_SOCK_RESERVE / FLAGS_BIND_SOCK_USE) via set_bind_sock_action()/bind_sock_action(), instead of a dedicated member, per review. No behavior change. - MysqlResponse: make it explicitly non-copyable (= delete copy ctor and assignment) and turn the previously no-op MergeFrom into a hard CHECK-failure, so an accidental copy/CopyFrom is caught instead of silently dropping parsed replies. - MysqlRequest: rename the trivial getters get_tx()/get_stmt() to tx()/stmt() per review. - ControllerPrivateAccessor: add set_mysql_statement_type() as a clearly named alias over the pipelined_count slot the mysql protocol reuses, and call it from the mysql protocol instead of set_pipelined_count() directly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(mysql): move unittests from mysql/ subdir into test/ Move all seven mysql unittests (auth handshake/packet/scramble, connection_type, pool_concurrency, prepared/txn integration) and the test-plan doc out of test/mysql/ into test/, per review on apache/brpc#3310. The existing brpc_*_unittest glob in the Makefile and CMakeLists.txt now picks them up, so both revert to master with no mysql-specific lines. In BUILD.bazel they likewise join the brpc_unittests glob; the five tests that fork a real mysqld keep their ["external", "local"] tags by folding those entries into that target's per_test_tags instead of a separate mysql target. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(mysql): drop run_tests.sh mysql/ subdir entry after move The mysql unittests now live in test/ and match the existing brpc*unittest glob in run_tests.sh, so revert the script to master. The removed mysql/brpc*unittest entry needed `shopt -s nullglob` to skip a missing subdir, but CI runs the script under `sh` (dash), where shopt is absent; nullglob stayed off and the now-empty glob was executed as a literal path, exiting 127. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Fix backup/retry call hang: default-initialize bind_sock_action The new bind_sock_action member of Controller::Call had no in-class default, and the Call(Call*) copy constructor used for backup requests and retries did not initialize it in its member-init list. The backup call therefore read an indeterminate value in Controller::Call::OnComplete, which branches on it before the normal pool-return / SetFailed path; when the garbage matched BIND_SOCK_RESERVE/BIND_SOCK_USE the backup call's socket was diverted to _bind_sock or held instead of returned, leaving the in-flight RPC unconcluded and hanging ChannelTest.backup_request until the test timeout (nondeterministic, hence flaky-looking). Give the member an in-class default initializer (BIND_SOCK_NONE) so every Call construction path inherits it; a backup/retry never inherits transaction connection-affinity. This closes the whole class of "new init path forgets the member" rather than the single copy-ctor instance. * Initialize Call::bind_sock_action explicitly in every constructor Per review: drop the in-class default initializer for Call::bind_sock_action and set it explicitly in each Call constructor (the copy ctor used for backup/retry) and Call::Reset(). Leaving it uninitialized was the cause of the backup/retry-request hang; explicit per-constructor init keeps every path safe. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Address review: remove redundant comments and dead code, move _bind_sock.reset() to ResetNonPods() * Remove commented-out code in mysql_stmt example --------- Co-authored-by: rajvarun77 <287367605+rajvarun77@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |