Commit Graph

410 Commits

Author SHA1 Message Date
Dominic Hamon 31fae1bb24 rename to avoid gtest warnings 2026-01-20 10:51:58 +00:00
Jesse Rosenstock 90ebbdedf0 Make the Benchmark class public (#2101)
Move ::benchmark::internal::Benchmark to ::benchmark::Benchmark.

It's a bit odd that the documented way to pass arguments to a
benchmark is with `benchmark::internal::Benchmark`.  Make this
public instead.

https://github.com/google/benchmark/blob/v1.9.4/docs/user_guide.md#passing-arguments

https://raw.githubusercontent.com/google/benchmark/refs/tags/v1.9.4/docs/user_guide.md#:~:text=void%20CustomArguments(benchmark%3A%3A-,internal%3A%3ABenchmark,-*%20b)%20%7B%0A%20%20for%20(int

Keep ::benchmark::internal::Benchmark as a deprecated forwarding alias.
Uses of Benchmark in the internal namespace need to explicitly use
::benchmark::Benchmark to avoid the deprecation warning.
2026-01-14 21:04:17 +03:00
Frank Rosner 3e7dac665b #2080: Fix rate and thread rate counter aggregates (#2081)
* Update counter.cc

* User counters: normalize time by thread count

Fixes https://github.com/google/benchmark/issues/2080

* docs

---------

Co-authored-by: Roman Lebedev <lebedev.ri@gmail.com>
2025-12-10 21:38:32 +03:00
Roman Lebedev 1b150e2fb0 Add tests for user counters w/ threads (#2089) 2025-12-08 18:59:07 +00:00
wieDasDing 8498a2a039 Fix warnings from Clang (#2062)
* Fix warnings from Clang

This patch addresses a few issues:
- Wformat-nonliteral from colour printing functions
- Wmissing-prototypes from a few internal functions

Signed-off-by: Xiangfei Ding <dingxiangfei2009@protonmail.ch>

* Wrap tests in anonymous namespaces

Technically tests should be using internal linkage.

Signed-off-by: Xiangfei Ding <dingxiangfei2009@protonmail.ch>

---------

Signed-off-by: Xiangfei Ding <dingxiangfei2009@protonmail.ch>
2025-10-23 11:58:38 +03:00
dominic 2948b6a2e6 Adding explicit cast to aid in template argument deduction. (#2042)
* Adding explicit cast to aid in template argument deduction.

The `BENCHMARK(my_func)` macro needs to register the function my_func
so the library can run it. It uses `std::make_unique` to create an
object that stores metadata about the benchmark, including its name
("my_func") and a pointer to the function itself (my_func).

The specific constructor being called via `std::make_unique` is for
`benchmark::internal::FunctionBenchmark`.

When a benchmark function name is overloaded, the compiler can't
determine which function is being referred to until the call is made
with specific arguments, or enough context is provided.

The addition of a `static_cast` to the expected `Function*` type
resolves any ambiguity as the compiler is forced to pick the one with
the matching signature.

* clang-format

* add a test

* clang-format again
2025-09-16 16:14:09 +01:00
Roman Lebedev b2959a733d JSON reporter: make int dumping always happen in C locale (#2040)
The test fails without the fix.

Fixes https://github.com/google/benchmark/issues/2039
2025-09-15 09:53:26 +01:00
Roman Lebedev f02794a8bd Automatic ASLR disablement (#1978)
While ASLR is a useful security hardening feature,
it introduces unreproducible noise into benchmarks,
and we really really really don't want any noise,
especially easily avoidable one.

Unless prevented by some other security hardening,
we can disable ASLR for the current process,
and restart it, thus eliminating this noise.

Fixes https://github.com/google/benchmark/issues/461
2025-05-01 17:34:09 +01:00
Roman Lebedev b616354a76 Improve user UX on empty benchmarks (#1976)
Fixes https://github.com/google/benchmark/issues/1962
2025-05-01 13:46:23 +01:00
krzikalla f828d71c59 Method templates for Fixtures introduced (#1967) 2025-04-11 12:25:46 +01:00
krzikalla 0da57b85cf Threading api refactor (#1955)
Refactor the multi-threading api to support
using custom user-provided thread factory
instead of always spawning POSIX Threads.
2025-03-29 10:49:25 +03:00
Roman Lebedev 1bc59dce27 Finish cxx version bump (#1948)
* `CMakeLists.txt`: drop hopefully obsolete code

* README.md: update

* Unbreak `BENCHMARK_HAS_CXX11` macro

835365f99a stopped defining it,
but didn't un-conditionalize the code guarded under it...

* Drop `BENCHMARK_HAS_NO_VARIADIC_REGISTER_BENCHMARK`

We no longer support such an old gcc version

* `docs/user_guide.md`: proofread

* Add a test to ensure that `benchmark.h` remains C++14 header

* Revert `[[maybe_unused]]` changes - it requires C++17

* Also support C++11 standard for using the library

I don't think we want to support C++03 though,
but i suppose C++11 is palatable, at least right now.
2025-03-17 09:18:19 +00:00
EfesX 5a4c459548 fix memory manager result bug (#1941)
* fix memory manager result bug

* change is_valid to memory_iterations

* fix test

* some fixes

* fix test

...for msvc

* fix test

* fix test

add the correct explicitly casts

* fix msvc failure

* some fixes

* remove unnecessary include
2025-03-12 10:15:28 +00:00
EfesX ff5c94d860 change setup and teardown callback type (#1934)
Change type of callbacks to take `std::function`
2025-02-20 16:16:32 +03:00
dominic 951429282a [clang-tidy] resolve some global clang-tidy issues (#1933)
* [clang-tidy] resolve some global clang-tidy issues

* more nolint and some global fixes
2025-02-18 10:07:57 +00:00
dominic 57efbfb3a0 use smart pointers (#1935)
* use smart pointers
* use vectors
* size_t
2025-02-18 09:59:27 +00:00
dominic adbda82db3 [clang-tidy] autofix readability issues (#1931)
* [clang-tidy] autofix readability issues

* more modern clang format
2025-02-12 19:40:49 +00:00
dominic 2d4c8dd21a [clang-tidy] autofix cppcoreguidelines (#1932)
* [clang-tidy] autofix cppcoreguidelines

* better than automation maybe
2025-02-12 19:32:45 +00:00
dominic f8db7f6c07 [clang-tidy] fix missing braces (#1928)
* [clang-tidy] fix missing braces
2025-02-11 23:56:06 +00:00
dominic c68e308b4f [clang-tidy] fix warning about decaying array to pointer (#1926)
* [clang-tidy] fix warning about decaying array to pointer

* fix a different warning (old style cast)

* use string_view instead of old-style const char* strings

* ensure bazel windows is using c++17

* learn to use bazel

* and tests

* precommit fix

* more string_view creation and casting

* format

* format

* [clang-tidy] use unique_ptr for benchmark registration (#1927)

* use unique_ptr for benchmark registration
2025-02-11 23:15:20 +00:00
dominic 835365f99a remove cxx03 test, fully unblocking c++1X development (#1903)
* remove cxx03 test, fully unblocking c++1X development

* remove unnecessary macros

* pre-commit

* remove opt-in analyzer warnings from clang-tidy

* revert some changes, flush streams

* replace abort with exit(1) to call atexit and dtors

* remove more endl and put in explicit flush
2025-02-10 22:17:49 +00:00
Dominic Hamon 4642758438 fix some clang-tidy issues 2025-01-30 09:52:07 +00:00
xdje42 049f6e79cc [BUG] Run external profiler (ProfilerManager) same number of iterations #1913 (#1914)
Run the external profiler the same number of iterations as the
benchmark was run normally.
This makes, for example, a trace collected via ProfilerManager
consistent with collected PMU data.
2025-01-29 09:53:56 +00:00
dominic 62a321d6dc update standard to C++17 per C++ build support (#1875)
* update standard to C++17 per C++ build support

* disable deadcode checks from clang-tidy

* fix redundant definition of constexpr
2024-11-13 13:06:48 +00:00
xdje42 ffc727a859 Verify RegisterProfilerManager doesn't overwrite an existing registration (#1837)
* Verify RegisterProfilerManager doesn't overwrite an existing registration

Tested:
Add a second registration to test/profiler_manager_test.cc and
verify the test crashes as expected.

* Verify RegisterProfilerManager doesn't overwrite an existing registration

Tested:
Configure with:
cmake -GNinja -DCMAKE_BUILD_TYPE=Debug -DBENCHMARK_DOWNLOAD_DEPENDENCIES=on
Then run:
ctest -R profiler_manager_gtest
Before change test fails (expected), after change test passes (expected)

---------

Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
2024-10-24 10:22:58 +01:00
dominic 498714357f upgrade bazel mods. requires c++14 for tests (#1867) 2024-10-23 10:27:18 +01:00
xdje42 ebb5e3922d Move ProfilerManager Start/Stop routines closer to actual benchmark #1807 (#1818)
Previously, the Start/Stop routines were called before the benchmark function
was called and after it returned. However, what we really want is for them
to be called within the core of the benchmark:

  for (auto _ : state) {
    // This is what we want traced, not the entire BM_foo function.
  }
2024-08-01 08:42:41 +01:00
xdje42 7c8ed6b082 [FR] Add API to provide custom profilers #1807 (#1809)
This API is akin to the MemoryManager API and lets tools provide
their own profiler which is wrapped in the same way MemoryManager is
wrapped. Namely, the profiler provides Start/Stop methods that are called
at the start/end of running the benchmark in a separate pass.

Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
2024-07-16 09:56:40 +01:00
dominic c64b144f42 mitigate clang build warnings -Wconversion (#1763)
* mitigate clang build warnings -Wconversion

* ensure we have warnings set everywhere and fix some
2024-03-07 12:19:56 +00:00
Roman Lebedev ef88520d6f Revert "fix some warnings" (#1762)
This reverts commit 1576991177.
2024-03-06 12:40:31 +00:00
Dominic Hamon 1576991177 fix some warnings 2024-02-20 16:51:06 +00:00
Roman Lebedev 3d85343d65 Rewrite complexity_test to use (hardcoded) manual time (#1757)
* Rewrite complexity_test to use (hardcoded) manual time

This test is fundamentally flaky, because it tried to read tea leafs,
and is inherently misbehaving in CI environments,
since there are unmitigated sources of noise.

That being said, the computed Big-O also depends on the `--benchmark_min_time=`

Fixes https://github.com/google/benchmark/issues/272

* Correctly compute Big-O for manual timings. Fixes #1758.

* complexity_test: do more stuff in empty loop

* Make all empty loops be a bit longer empty

Looks like on windows, some of these tests still fail,
i guess clock precision is too small.
2024-02-19 15:22:35 +00:00
Sam James 385033bd11 CycleClock: Add support for Alpha architecture (#1753)
* Add support for Alpha architecture

As documented, the real cycle counter is unsafe to use here, because it
is a 32-bit integer which wraps every ~4s.  Use gettimeofday instead,
which has a limitation of a low-precision real-time-clock (~1ms), but no
wrapping.  Passes test suite.

Support parsing /proc/cpuinfo on Alpha

tabular_test: add a missing DoNotOptimize call
2024-02-14 00:04:44 +03:00
Roman Lebedev b04cec1bf9 Deflake CI (#1751)
* `complexity_test`: deflake, same as https://github.com/google/benchmark/issues/272

As it can be seen in e.g. https://github.com/google/benchmark/actions/runs/7711328637/job/21016492361
We may get `65: BM_Complexity_O1_BigO                           0.00 N^2        0.00 N^2  `

* `user_counters_tabular_test`: deflake

We were still getting zero times there. Perhaps this is better?
2024-02-02 18:39:46 +03:00
Roman Lebedev e990563876 Add BENCHMARK_TEMPLATE[12]_CAPTURE, fusion of BENCHMARK_CAPTURE and BENCHMARK_TEMPLATE (#1747)
Test coverage isn't great, but not worse than the existing one.

You'd think `BENCHMARK_CAPTURE` would suffice,
but you can't pass `func<targs>` to it (due to the `<` and `>`),
and when passing `(func<targs>)` we get issues with brackets.
So i'm not sure if we can fully avoid this helper.

That being said, if there is only a single template argument,
`BENCHMARK_CAPTURE()` works fine if we avoid using function name.
2024-01-30 12:44:36 +00:00
Roman Lebedev 17bc235ab3 Output library / schema versions in JSON context block (#1742)
* CMake: `get_git_version()`: just use `--dirty` flag of `git describe`

* CMake: move version normalization out of `get_git_version()`

Mainly, i want `get_git_version()` to return true version,
not something sanitized.

* JSON reporter: store library version and schema version in `context`

* Tools: discard inputs with unexpected `json_schema_version`

* Extract version string into `GetBenchmarkVersiom()`

---------

Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
2024-01-29 13:15:43 +00:00
hamptonm1 e523e454f2 Update perf_counters_gtest.cc (#1728) 2024-01-04 09:11:07 +00:00
IS 6b7e86c5c8 Fix mis-matching argument in closing tag for cmake macro (#1714) (#1715)
Co-authored-by: Iakov Sergeev <yahontu@gmail.com>
2023-12-20 09:54:55 +00:00
Roman Lebedev 1e96bb0ab5 Support windows MSYS2 environments (#1704)
* [CI] Attempt to add windows MSYS2-based coverage

* Mark decl of `State::KeepRunningInternal()` as `inline`

Maybe helps with
```
D:\a\_temp\msys64\ucrt64\bin\g++.exe -DHAVE_STD_REGEX -DHAVE_STEADY_CLOCK -DTEST_BENCHMARK_LIBRARY_HAS_NO_ASSERTIONS -ID:/a/benchmark/benchmark/include -Wall  -Wextra  -Wshadow  -Wfloat-equal  -Wold-style-cast  -Werror  -pedantic  -pedantic-errors  -fstrict-aliasing  -Wno-deprecated-declarations  -Wno-deprecated  -Wstrict-aliasing  -Wno-unused-variable -std=c++11 -fvisibility=hidden -fno-keep-inline-dllexport   -UNDEBUG -MD -MT test/CMakeFiles/benchmark_test.dir/benchmark_test.cc.obj -MF test\CMakeFiles\benchmark_test.dir\benchmark_test.cc.obj.d -o test/CMakeFiles/benchmark_test.dir/benchmark_test.cc.obj -c D:/a/benchmark/benchmark/test/benchmark_test.cc
In file included from D:/a/benchmark/benchmark/test/benchmark_test.cc:1:
D:/a/benchmark/benchmark/include/benchmark/benchmark.h:1007:37: error: 'bool benchmark::State::KeepRunningInternal(benchmark::IterationCount, bool)' redeclared without dllimport attribute after being referenced with dll linkage [-Werror]
 1007 | inline BENCHMARK_ALWAYS_INLINE bool State::KeepRunningInternal(IterationCount n,
      |                                     ^~~~~
```

* Mark more `State`'s member function decls as `inline`

```
[27/110] Building CXX object test/CMakeFiles/spec_arg_verbosity_test.dir/spec_arg_verbosity_test.cc.obj
FAILED: test/CMakeFiles/spec_arg_verbosity_test.dir/spec_arg_verbosity_test.cc.obj
D:\a\_temp\msys64\clang32\bin\clang++.exe -DHAVE_STD_REGEX -DHAVE_STEADY_CLOCK -DHAVE_THREAD_SAFETY_ATTRIBUTES -DTEST_BENCHMARK_LIBRARY_HAS_NO_ASSERTIONS -ID:/a/benchmark/benchmark/include -Wall  -Wextra  -Wshadow  -Wfloat-equal  -Wold-style-cast  -Werror  -pedantic  -pedantic-errors  -Wshorten-64-to-32  -fstrict-aliasing  -Wno-deprecated-declarations  -Wno-deprecated  -Wstrict-aliasing  -Wthread-safety  -Wno-unused-variable -std=c++11 -fvisibility=hidden -fvisibility-inlines-hidden   -UNDEBUG -MD -MT test/CMakeFiles/spec_arg_verbosity_test.dir/spec_arg_verbosity_test.cc.obj -MF test\CMakeFiles\spec_arg_verbosity_test.dir\spec_arg_verbosity_test.cc.obj.d -o test/CMakeFiles/spec_arg_verbosity_test.dir/spec_arg_verbosity_test.cc.obj -c D:/a/benchmark/benchmark/test/spec_arg_verbosity_test.cc
In file included from D:/a/benchmark/benchmark/test/spec_arg_verbosity_test.cc:5:
D:/a/benchmark/benchmark/include/benchmark/benchmark.h:999:44: error: 'benchmark::State::KeepRunning' redeclared inline; 'dllimport' attribute ignored [-Werror,-Wignored-attributes]
  999 | inline BENCHMARK_ALWAYS_INLINE bool State::KeepRunning() {
      |                                            ^
D:/a/benchmark/benchmark/include/benchmark/benchmark.h:1003:44: error: 'benchmark::State::KeepRunningBatch' redeclared inline; 'dllimport' attribute ignored [-Werror,-Wignored-attributes]
 1003 | inline BENCHMARK_ALWAYS_INLINE bool State::KeepRunningBatch(IterationCount n) {
      |                                            ^
D:/a/benchmark/benchmark/include/benchmark/benchmark.h:1075:60: error: 'benchmark::State::begin' redeclared inline; 'dllimport' attribute ignored [-Werror,-Wignored-attributes]
 1075 | inline BENCHMARK_ALWAYS_INLINE State::StateIterator State::begin() {
      |                                                            ^
D:/a/benchmark/benchmark/include/benchmark/benchmark.h:1078:60: error: 'benchmark::State::end' redeclared inline; 'dllimport' attribute ignored [-Werror,-Wignored-attributes]

 1078 | inline BENCHMARK_ALWAYS_INLINE State::StateIterator State::end() {

      |                                                            ^

```

* StatisticsTest.CV: don't require precise FP match, tolerate some abs error

We get ever so slightly different results on windows with GCC.
```
71: Test command: D:\a\benchmark\benchmark\_build\test\statistics_gtest.exe
71: Working Directory: D:/a/benchmark/benchmark/_build/test
71: Test timeout computed to be: 10000000
71: Running main() from gmock_main.cc
71: [==========] Running 4 tests from 1 test suite.
71: [----------] Global test environment set-up.
71: [----------] 4 tests from StatisticsTest
71: [ RUN      ] StatisticsTest.Mean
71: [       OK ] StatisticsTest.Mean (0 ms)
71: [ RUN      ] StatisticsTest.Median
71: [       OK ] StatisticsTest.Median (0 ms)
71: [ RUN      ] StatisticsTest.StdDev
71: [       OK ] StatisticsTest.StdDev (0 ms)
71: [ RUN      ] StatisticsTest.CV
71: D:/a/benchmark/benchmark/test/statistics_gtest.cc:31: Failure
71: Expected equality of these values:
71:   benchmark::StatisticsCV({2.5, 2.4, 3.3, 4.2, 5.1})
71:     Which is: 0.32888184094918088
71:   0.32888184094918121
71: [  FAILED  ] StatisticsTest.CV (0 ms)
71: [----------] 4 tests from StatisticsTest (0 ms total)
```

* Fix DLL path discovery for tests
2023-11-23 17:47:04 +03:00
illbegood 4a2e34ba73 Fix CMakeLists.txt for perf_counters_test (#1701) 2023-11-16 09:55:59 +00:00
Tiago Freire a543fcd410 Fixed compiler warnings (#1697)
* fixed warnings
used proper math functions

* ran clang format

* used a more up-to-date clang-format

* space twedling

* reveretd CMakeLists.txt
2023-11-10 10:09:50 +00:00
dominic 6a16cee366 Add no-unititialized to tests (#1683) 2023-10-23 08:54:08 +00:00
Vy Nguyen 7495f83e2a Set -Wno-unused-variable for tests (#1682)
We  used assert() a lot in tests and that can cause build breakages in some of the opt builds (since assert() are removed)

it's not practical to sprinkle "(void)" everywhere so I think setting this warning option is the best option for now.
2023-10-20 12:51:32 +00:00
Andreas Abel 7736df0304 Make json and csv output consistent. (#1662)
* Make json and csv output consistent.

Currently, the --benchmark_format=csv option does not output the correct value for the cv statistics. Also, the json output should not contain a time unit for the cv statistics.

* fix formatting

* undo json change

---------

Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
2023-09-26 13:43:23 +01:00
Mészáros Gergely c9106a79fa Audit MSVC references in cmake files to consider clang++ (#1669)
There are three major compilers on Windows targeting the MSVC ABI (i.e.
linking with microsofts STL etc.):
  - `MSVC`
  - `clang-cl` aka clang with the MSVC compatible CLI
  - `clang++` aka clang with gcc compatible CLI

The cmake variable `MSVC` is only set for the first two as it defined in
terms of the CLI interface provided:

> Set to true when the compiler is some version of Microsoft Visual
> C++ or another compiler simulating the Visual C++ cl command-line syntax.

(from cmake docs)

For many of the tests in the library its the ABI that matters not the
cmdline, so check `CMAKE_CXX_SIMULATE_ID` too, if it is `MSVC` the
current compiler is targeting the MSVC ABI. This handles `clang++`
2023-09-26 12:31:24 +01:00
Jesse Rosenstock dfc8a92abc test: Use gtest_main only when needed (#1657)
* test: Use gtest_main only when needed

There are two types of tests.  `*_gtest.cc` files use `gtest` and
`gtest_main`.  `*_test.cc` files define their own main.

Only depend on `gtest`/`gtest_main` when needed.  This is similar
to what `CMakeLists.txt` does.

* comment-only: gunit => gtest

* Fix typo
2023-08-24 13:43:50 +01:00
Jesse Rosenstock e73915667c State: Initialize counters with kAvgIteration in constructor (#1652)
* State: Initialize counters with kAvgIteration in constructor

Previously, `counters` was updated in `PauseTiming()` with
`counters[name] += Counter(measurement, kAvgIteration)`.

The first `counters[name]` call inserts a counter with no flags.

There is no `operator+=` for `Counter`, so the insertion is done
by converting the `Counter` to a `double`, then constructing a
`Counter` to insert from the `double`, which drops the flags.

Pre-insert the `Counter` with the correct flags, then only
update `Counter::value`.

Introduced in 1c64a36 ([perf-counters] Fix pause/resume (#1643)).

* perf_counters_test.cc: Don't divide by iterations

Perf counters are now divided by iterations, so dividing again
in the test is wrong.

* State: Fix shadowed param error

* benchmark.cc: Fix clang-tidy error

---------

Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
2023-08-21 15:35:42 +01:00
Jesse Rosenstock e441a8cb11 perf-counters: Make tests pass on Android (#1653)
* perf_counters_gtest: Make test pass on Android

Tested on Pixel 3 and Pixel 6.  Reduce test to the intersection of
what passes on all platforms.

Pixel 6 doesn't support BRANCHES, and only supports two perf
counters.


---------

Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
2023-08-21 15:04:50 +01:00
Mircea Trofin 1c64a36c5b [perf-counters] Fix pause/resume (#1643)
* [perf-counters] Fix pause/resume

Using `state.PauseTiming() / state.ResumeTiming()` was broken.

Thanks [@virajbshah] for the the repro testcase.

* ran clang-format over the whole perf_counters_test.cc

* Remove check that perf counters are 0 on `Pause`, since `Pause`/`Resume`
sequences would cause a non-0 counter value

* both upper and lower bound for the with/without resume counters

---------

Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
2023-08-11 12:46:36 +01:00
देवांश वार्ष्णेय 02a354f3f3 bug: Inconsistent suffixes console reporter 1009 (#1631)
* removed appendHumanReadable as it was not used anywhere

---------

Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
2023-08-01 08:47:09 +01:00