A statistical relationship is not yet usable market information.

We specify precisely:

• when the information is generated;
• when it is actually available;
• how it is converted into a measurable signal;
• which horizon the hypothesis is intended to explain;
• which conditions would invalidate it.

Information that was revised later or became available only at a later date must not be used as though it had been known earlier.

We test whether the observed change is related to:

• financial results;
• deviation from expectations;
• price behaviour;
• volume or liquidity;
• relative behaviour versus the market or industry.

For events such as earnings releases, we may use an event study appropriate to the research question and measure price behaviour relative to a relevant market or industry model.

We test when the signal begins to contain information and how quickly that value decays.

The same hypothesis may be relevant over a horizon of several days and entirely uninformative over several months.

We therefore test multiple predefined time horizons and track the decay of informational value.

The result must hold in a period on which it was not developed.

When working across multiple companies and periods, we use a panel structure. The number of tested variants must be proportionate to the amount of genuinely available data.

Before interpreting the result, we also assess statistical power. We test whether the number and structure of observations are sufficient to detect the effect we are looking for with reasonable reliability.

Selecting the best result from dozens or hundreds of attempts without adjustment is not evidence.

A backtest must not assume that a trade occurs instantly and without cost.

Depending on the market, the simulation therefore accounts for factors such as:

• delay between signal generation and possible execution;
• bid-ask spread and fees;
• liquidity;
• limits on trade size relative to typical daily volume;
• estimated market impact;
• turnover;
• strategy capacity.

We report results both before costs and after a realistic estimate of costs.

We do not assess risk separately from volatility, liquidity or uncertainty in the estimate itself.

Depending on the market, the simulation may use approaches such as volatility targeting and limits on position size.

A mathematically optimal position under perfectly known probabilities is not the same as a reasonable position in an environment where the estimate itself is uncertain.

One historical sequence of results is only one realisation.

We therefore test how the outcome changes under conditions such as:

• a different ordering of trades;
• preservation of blocks of high-volatility periods;
• higher costs;
• lower liquidity;
• a change in market regime.

The output is not just one historical maximum drawdown. We are interested in the distribution of possible outcomes and what may happen beyond the original historical path.

The more model variants we test, the greater the risk that one will produce a good result by chance.

We therefore track:

• the number of tested variants;
• stability outside the training sample;
• sensitivity to parameter changes;
• signs of structural change in performance.

We do not consider a model robust if it works only under one precise combination of parameters.

We define in advance what must happen for us to reject the hypothesis.

For example:

• the effect disappears after controlling for an industry or market factor;
• the lead time is not stable;
• the result disappears after realistic costs;
• the strategy is not feasible because of liquidity;
• the result exists only within one narrow period or market regime.

Without conditions for rejection, the process is not validation. It is a search for confirmation.