How to Benchmark Website Speed Before and After Optimization: A Guide to Performance Comparison Testing

After optimizing a website, many people's first instinct is to reopen the page and see if it feels faster. If it seems smoother than before, or if the score in a speed testing tool has improved, they conclude that the optimization worked. But when you're actually troubleshooting website performance, this kind of judgment isn't reliable.

The network itself fluctuates, the local browser may have already cached images and scripts, and access routes differ across regions and ISPs. For example, before optimization you tested from a Shanghai Telecom environment, and after optimization you switched to Guangzhou Mobile. The first test had no cache, while the second time the browser had already loaded the page. Even if the numbers change, it's hard to tell whether the website optimization actually made a difference or whether the test conditions simply changed.

So when speed testing before and after website optimization, what really matters isn't "testing once each time," but keeping the test environment as consistent as possible, then comparing data such as DNS, TCP, TLS, TTFB, page load time, resource size, and Core Web Vitals. Only when the before-and-after tests are comparable do the optimization conclusions have any reference value.

1. Why must conditions be kept consistent when speed testing before and after website optimization?

Website performance data is easily affected by the external environment: the same website tested at different times can differ by hundreds of milliseconds; the same page accessed from Beijing Telecom versus Guangzhou Mobile can have completely different latency. If the conditions of the two tests vary too much, the resulting data doesn't really have much comparative meaning.

For example:

Before optimization
Shanghai Telecom
Desktop
No cache
Tested at 10 AM

After optimization
Guangzhou Mobile
Mobile
Cache present
Tested at 8 PM

Even if the load time dropped from 3 seconds to 2 seconds after optimization, you can't directly claim the optimization brought a 1-second improvement, because the node, network, device, and cache conditions all changed.

A more reasonable approach is to keep the following conditions as fixed as possible:

If you use an online speed testing tool, it's best to choose the same node before and after. If you test locally in a browser, also try to keep the network environment consistent, and be clear about whether you're testing the first visit or a repeat visit with cache. When comparing before and after website optimization, the more stable the test environment, the more valuable the data.

2. What data should be recorded before optimization?

Before optimizing a website, it's best to run a baseline test first, meaning you record the website's current real performance. Without this data, even if the site feels faster later, it's hard to say exactly where the improvement came from. Generally you don't need to record every performance metric; focusing on a few data points that have a big impact on actual access is enough.

First is DNS resolution time. DNS happens before the website connection is established. If the site just changed DNS, CDN, or resolution services, you can watch whether this stage changes.

Next are TCP and TLS. TCP connection time reflects the network route and server distance, while TLS relates to HTTPS connection setup. For websites that are cross-region, cross-border, or just started using a CDN, these two are usually worth referencing.

TTFB is also important.

If this optimization involves the server, database, PHP, page cache, CDN origin pull, or dynamic APIs, TTFB is often one of the metrics most likely to change.

In addition, you can also record:

Page load time
Request count
Total page size
LCP
INP
CLS

For example, record before optimization:

TTFB: 620 ms
Page load: 3.8 s
Request count: 126
Page size: 4.6 MB
LCP: 3.1 s

These data points can then be used directly as a comparison baseline.

3. How should you speed test before and after website optimization?

When actually doing a comparison, I recommend following the order of "test baseline first, then optimize, then retest under the same conditions," rather than changing the website first and then looking back for old data.

Before optimization, you can run multiple tests using a fixed URL.

If the website mainly serves domestic users, you can use Chahu's website speed test to check access performance across different regions and ISPs. If you want to keep observing network latency, you can also combine it with Chahu's online Ping for testing.

On the first test, don't just save a single "total load time." It's best to record DNS, connection time, TTFB, page load, and results from different regions together.

For example, testing the same node five times in a row:

1.82 s
1.76 s
1.91 s
1.79 s
1.84 s

In this case, rather than just taking the fastest 1.76 seconds, it's better to look at the median or the overall range. This helps minimize the impact of single network fluctuations on the results. After completing the baseline test, proceed with website optimization.

For example:

Enable CDN
Turn on page cache
Compress images
Combine or reduce JS
Enable Brotli
Optimize database
Upgrade server
Adjust DNS

After optimization is complete, retest using the original URL, the same node, and the same test conditions. If before optimization you used a Beijing Telecom node, desktop, no cache, and five consecutive tests, then after optimization try to keep it consistent. Only then do the two sets of data truly have comparative meaning.

4. Which metrics should you focus on before and after website optimization?

Different types of optimization require attention to different metrics.

For example, a complete set of before-and-after data might be:

From a table like this, you can see fairly intuitively at which stage the performance improvement occurred.

If you optimized the server or backend, focus on:

TTFB
Server response time
Dynamic API response

If you optimized the CDN, it's better to observe:

RTT
TCP
TLS
TTFB
Cache HIT
Node performance in different regions

If you did frontend optimization, focus on:

Page size
Request count
LCP
Page load time

If you only compressed images, then focus on resource size, download time, and whether LCP improved.

Website performance optimization isn't about staring at a single "how many seconds did it load" for every project. Different optimization goals should correspond to different judgment metrics.

5. Why does speed testing sometimes get slower after website optimization?

This situation is actually quite common. Sometimes right after optimization, retesting shows the page is a few hundred milliseconds slower, but that doesn't necessarily mean the optimization failed.

First, check whether the test nodes changed before and after.

For example, using Shanghai Telecom before optimization and Guangzhou Mobile after optimization can itself cause differences of tens or even hundreds of milliseconds.

Browser cache state also affects results.

The first visit needs to re-download resources, while subsequent visits may read directly from browser cache. If the cache state differs between the two tests, the final load times naturally can't be compared directly.

In CDN scenarios, you also need to consider whether the cache has been established.

Right after switching CDN or refreshing the cache, the first access to a node may result in a MISS, requiring origin pull to fetch resources. Only after the cache is established will subsequent requests truly reflect the edge cache effect.

DNS is the same: after just changing DNS or CNAME, recursive DNS cache update times may differ across regions, so short-term fluctuations in results are normal.

Also watch out for third-party resources, such as analytics code, ads, live chat, maps, third-party fonts, videos, third-party JS, and so on. These resources aren't fully controlled by your own website. If a third-party service responds slowly during one speed test, it will also drag up the overall page load time. So before and after website optimization, it's best not to look at just one result. One slower test doesn't directly mean the optimization failed; what's more valuable is looking at the overall trend after multiple tests.

6. Don't just look at "total load time"

When comparing before and after website optimization, the most common mistake is looking at only one number.

For example:

Before optimization: 3.2 seconds
After optimization: 2.4 seconds

This result certainly shows the page is somewhat faster overall, but it doesn't tell you exactly what changed.

If we break it down further:

Before optimization
DNS: 20 ms
TTFB: 850 ms
Page load: 3.5 s

After optimization
DNS: 21 ms
TTFB: 220 ms
Page load: 2.3 s

Here it's obvious that DNS barely changed, and what actually improved was the server response.

If the data looks like this instead:

Before optimization
TTFB: 180 ms
Page size: 5.2 MB
Load time: 4.1 s

After optimization
TTFB: 175 ms
Page size: 2.8 MB
Load time: 2.0 s

That tells you server speed hardly changed, and the main performance gain came from optimizing front-end resources like images, CSS, and JS.

Another example:

Before optimization
TCP: 130 ms
TLS: 190 ms
TTFB: 350 ms

After optimization
TCP: 42 ms
TLS: 65 ms
TTFB: 140 ms

If a CDN was just brought in this time, then the improvement most likely comes mainly from users being closer to edge nodes and the network connection path being shortened. This is also why, when testing speed before and after website optimization, it's best to break the access path down rather than only looking at the final "total time."

Conclusion

When testing speed before and after website optimization, what really matters isn't how much the score improved after optimization, but whether you can confirm under identical conditions which metrics actually changed. If you use different nodes, different devices, or different cache states before and after, even if the numbers look very different, the results may not be meaningful. A more reliable approach is to save a set of baseline data before optimization, then retest using the same URL, same region, and same testing method.

During testing, you can look at DNS, TCP, TLS, TTFB, page load time, resource size, and Core Web Vitals together, then judge which data is most worth paying attention to based on the goal of this optimization. This way you not only know "whether the website got faster," but can also determine whether the performance gain came from the server, CDN, caching, network route, or front-end resource optimization. For websites that operate long-term, this repeatable and comparable testing method also has more practical value than simply judging by feel that "the page seems faster."

Related Q&A

1. What are the essential differences in speed metrics between dynamically rendered websites (such as React/Vue SSR) and static web pages (HTML)?

Because static HTML pages don't involve complex server-side logic calculations, their TTFB is usually extremely short, and the performance bottleneck is mainly in CDN transmission and front-end resource size. For SSR (server-side rendering) websites, TTFB includes the time for database queries, template assembly, and API calls, so TTFB tends to fluctuate more. In addition, even if an SSR page renders HTML very quickly (with excellent LCP), the page still can't respond to interactions until client-side JavaScript finishes Hydration, so the INP (Interaction to Next Paint) metric deserves special attention.

2. When doing global multi-node speed tests, does the test node's network bandwidth affect latency data?

Yes. The test node's own outbound bandwidth and concurrent processing capacity directly determine the throughput ceiling when downloading large resources. If the edge node used by the speed test provider has narrow bandwidth (for example, limited to 10 Mbps), then downloading several megabytes of images or JS bundles will take artificially longer, distorting the Fully Loaded metric. Therefore, when evaluating cross-border network performance, you should prioritize metrics less affected by bandwidth, such as Ping, RTT, and TTFB, or choose professional tools with dedicated high-bandwidth test nodes.

3. Why doesn't the LCP metric improve noticeably after converting images to WebP format?

Smaller image size doesn't equal faster LCP rendering. Besides file size, the key factors affecting LCP are loading priority and render blocking. If a WebP image is set to loading="lazy" (lazy loading), or is loaded after a large number of render-blocking CSS/JS scripts, the browser still can't preload that resource early. To truly improve LCP, you need to compress the size while also using <link rel="preload"> to preload key above-the-fold images and removing the lazy loading attribute from above-the-fold images.

4. For high-traffic websites doing performance evaluation, how should single-point concurrent testing and stress testing work together?

Single-node speed tests (such as Lighthouse/WebPageTest) mainly target "optimization of the rendering and transmission chain for a single request," suitable for troubleshooting front-end resources and CDN configuration; while stress tests (such as k6, JMeter) simulate "server throughput and response capability under concurrent access by many users." If the website is fast under low concurrency but TTFB spikes or even 504 errors appear as the number of online users rises, the bottleneck is in the back-end database connection pool, CPU limits, or PHP-FPM process count limits. Performance evaluation must combine both: first use stress testing to find the server's load limit, then conduct front-end performance testing under stable load.