Skip to main content
A Performance Verification Period (PVP) checks that the site performs acceptably under real traffic after a platform change goes live. That kind of check needs something real to compare against - this page documents the benchmarking methodology, where the captured baseline lives, and how to rerun the same methodology later so a comparison stays sound. This is a manual, periodic exercise, not a CI check, so it lives here rather than in Automated tests. The scripts themselves are in tools/benchmarks/ - see that folder’s README.md for the full script-by-script reference and the captured results.

​
Reading the numbers

Every metric on this page is reported as p50 and p95 - percentiles across many repeated runs, not a single measurement. p50 (the median) is the typical experience: half the runs were faster than this, half slower. p95 is the near-worst-case: only 5% of runs were slower than this - useful for catching occasional bad experiences that an average would smooth over and hide. A page with a fast p50 but a much slower p95 is inconsistent (fast most of the time, but with a real minority of slow loads). A page where p50 and p95 sit close together is consistently fast or consistently slow, with little variance either way.

​
Setup and running

The scripts are plain Node - nothing about them is tied to the machine they were first run on. To run them (on any machine with a clone of this repo):
Then run any script directly, for example:
Each writes into results/<label>/, defaulting to today’s date - set RUN_LABEL to control it explicitly (RUN_LABEL=2026-09-05 node run-pageload-benchmark.mjs). Never set RUN_LABEL=baseline for a later run - results/baseline/ is the reference capture and must stay untouched for any later comparison to mean anything. npm install and npx playwright install chromium both download their own tooling fresh each time (from npm’s registry and Playwright’s browser CDN respectively) - nothing from a previous run needs to be copied over, and nothing under tools/benchmarks/ outside of results/ needs to survive between machines.

​
Scope

A full PVP’s scope (page load, search latency, navigation, CDN geography across the EU, concurrency/traffic-based performance, error rates under load) needs real production traffic and infrastructure that doesn’t exist ahead of a launch. This page’s baseline scopes that down to the three metrics that can be measured meaningfully without that infrastructure: page load (p50/p95), search latency, and navigation responsiveness.

​
How the current site’s search actually works

Before choosing a search-latency method, it’s worth knowing what’s actually being measured. docs.superoffice.com’s search calls a separate back-end service rather than using a client-side search index.
That back end can have significant cold-start latency after a period of inactivity. Any script measuring it must run one untimed warmup request first, or a cold start will wreck the p95.
Mintlify’s search after cutover is a completely different back end - the current one goes away entirely. That means a raw server-response-time comparison won’t carry over, but an end-to-end perceived-latency measurement (type a query, click Search, measure until results render) is back-end-agnostic - it measures the same user-visible behavior regardless of what powers it. That’s why two separate search scripts exist: one DocFx-only diagnostic, one number that actually carries forward to a later comparison.

​
Tools and metrics

Lighthouse CI was chosen over WebPageTest or the PageSpeed Insights API because CDN geography and multi-location testing are explicitly out of scope for this exercise - a single consistent vantage point is enough, and Lighthouse CI needs no signup or API key and stays fully checked into this repo. Lighthouse’s default simulated-mobile throttling is left as-is deliberately - whatever settings are used for the baseline must stay identical for the post-cutover rerun, not tuned for “more realistic” numbers now.

​
Sample set

tools/benchmarks/urls.json, queries.json, and nav-paths.json hold the representative pages, search terms, and click-paths. The scripts just read from this JSON config, so it can be swapped or extended without touching any script. Keep this config identical between the pre- and post-cutover runs - that is what makes the comparison valid. urls.json’s page-load sample is chosen for content-type diversity rather than nav breadth: a generated database-reference table, an integration subpage, a release-notes index, a page with both images and an embedded video, and a page with tables but no images, alongside a few plain landing pages. Different content shapes stress page load differently (a video embed versus a large generated table versus a plain landing page), and content type is invariant across the DocFx-to-Mintlify cutover even though the URLs themselves may move. nav-paths.json covers the navigation shapes that behave differently depending on the site’s routing model, not just a handful of arbitrary click-throughs:
  • A deep multi-hop click into content (home-to-company-card-variable, 5 hops) - stress-tests a realistic “user drills down through several levels” journey.
  • Navigating between two sibling pages within the same left-side TOC (learn-toc-sibling).
  • Navigating across a TOC boundary - Learn to Integrations (top nav) and Learn to API (via the Developer Guide dropdown) - learn-to-integrations-cross-toc / learn-to-api-cross-toc.
  • Switching language via the language dropdown (switch-language) - a DocFx full-page reload today; likely a much cheaper operation on Mintlify.
The last three all start from a click-toggled dropdown menu (Bootstrap dropdown-toggle, not a CSS :hover menu - confirmed by inspecting the live DOM, not assumed). nav-paths.json’s hop schema supports this via an openSelector field: clicked first, untimed, to reveal the menu, before the real (timed) navigation click on selector. See that file’s own comment for the exact mechanics, and nav-responsiveness.mjs for the implementation.

​
Where the actual numbers live

The captured baseline (p50/p95 for every page, query, and nav hop) is not reproduced on this page. Captured figures live in tools/benchmarks/results/baseline/ in this repo - raw JSON plus a summary .md table per script - alongside notes on anything that stood out (an unusually slow content area, an outlier run worth re-checking, and so on). tools/benchmarks/README.md explains how to read them and how results/<label>/ keeps a rerun from ever overwriting results/baseline/.

​
Rerunning the benchmark later

  1. Confirm tools/benchmarks/urls.json, queries.json, and nav-paths.json are unchanged from the baseline capture (or, if the sample set was deliberately updated, note that in the new results write-up - a changed sample set is no longer an apples-to-apples comparison).
  2. nav-paths.json’s hop selectors match DocFx’s literal href markup - they will need re-verifying against the equivalent Mintlify pages first, since Mintlify’s markup won’t match.
  3. Run all four scripts per tools/benchmarks/README.md against the live Mintlify site.
  4. Compare against results/baseline/ (see above). Flag the server-side search-latency script’s numbers as non-comparable (different back end) - only the end-to-end search number, the page-load metrics, and the navigation numbers are valid before/after comparisons.

​
Related content

  • Automated tests - the per-PR and scheduled checks this page’s manual exercise complements.

Related topics

Automated testsAcceptance testing