SEO experiment log: what if several changes happen together?

Suppose a team changes titles, improves internal links and launches a new navigation menu in the same week. Traffic rises afterward. The result is worth investigating, but attributing it to the title change alone would go beyond the evidence.

My example experiment log would record the hypothesis, affected pages, precise changes, deployment dates, comparison approach, expected delay, and other events that could influence the outcome. Also define what result would make the hypothesis less convincing.

The log is useful even when a clean experiment is impossible. It prevents a later report from forgetting how many things changed.

How would you describe the outcome when you cannot isolate the cause?

Record when changes actually became visible, not just when a ticket was marked complete. A deployment can roll out gradually or leave cached pages unchanged.

Keep a few before-and-after examples and confirm the crawler-facing version. Otherwise the analysis may assume every page received the same treatment on the same day.

I would list content updates as changes too. A rewritten introduction or new example can affect the page's usefulness even if the ticket was labelled “title optimisation.”

If the intervention has several parts, name it as a combined change. That is more honest than extracting one small component and calling it the winner.

The report could say the change package was followed by an improvement in the chosen measures, while the individual contribution of each component remains uncertain.

Then propose a narrower follow-up on a suitable group of pages. Keep the original evidence intact so a second test can refine the explanation rather than quietly rewriting the first result.