simple statistical test - to check difference between two treatments

Last reviewed 01/2018

In many trials the primary outcome is a disease event

• outcomes might be a particular event e.g. death, or a composite outcome such as death, myocardial infarction, or stroke
• standard statistical methods - Cox proportional hazard models and log rank tests - take account of variation in patient follow-up times - in order to assess the differences in treatment groups. However, if events relate to a fixed follow-up time then methods for comparing two proportions (for example, the Chi squared test) may be used
• this page in GPnotebook describes an easy method for quickly assessing the strength of evidence for a treatment difference in an event outcome

This test is a statistical test used to check for a difference between treatments

• the key data are the numbers of patients with the event in each group. This statistical test (simple/simplest statistical test) compares these two numbers
• if considering a randomised clinical trial with two treatment groups of roughly equal size. In this context the outcome of interest is a clinical event (e.g. myocardial infarction whilst on a particular treament) and the key data are the numbers of patients experiencing the event by treatment group (e.g. number of individuals who experienced a myocardial infarction in each group)

Calculate the difference in the two numbers of events and divide by the square root of their sum. Call the resulting number Z

• Z = (a - b) / square root of (a + b)
  • where a and b represent the number of events in each treatment group
• the null hypothesis is that the two treatments have identical influence on the risk of an event. Therefore z is approximately a standardised normal deviate, i.e. has a normal distribution with mean 0 and variance 1. Therefore the value of Z can be converted to a P value via normal distribution tables
  • for example
    • a Z value of > 1.96 is equivalent to a P value < 0.05
    • a Z value of > 2.58 represents a P value of < 0.01
  • this test is approximate
    • however it generally provides reliable results because
      • with randomisation, the number of patients in the two treatment groups will be almost equal, as will be the length of patient follow-up
      • event rates are usually quite low - for example, less than 20% of patients (and often much lower). This allows the number of patients having an event in each group can be considered to have Poisson distribution. Thus if the total number of events is not too small (for example, not less than 20) then the normal approximation for the comparison of two Poisson random variables leads to the formula in the figure
      • key information involved in the calculation are the numerators (the numbers with an event); size of the denominators being unimportant
    • calculation has two limitations
      • if there is a non-neglible difference in the amount in the treatment groups then the test will become biased in respect of the larger treatment group
      • if there are a high number of event rates then the test becomes conservative, i.e. P values are larger than they should be
• simple test is also relevant to meta-analyses, provided that all included trials have equal randomisation
  • for example a meta-analysis studied the incidence of target lesion revascularisation in six trials comparing two different types of stent (A, B)
    • combining the data on 3669 patients in all six trials, target lesion revascularisation was done in 95 and 142 patients respectively in the stent A group and stentl groups. Hencev Z= (142 - 95)/ square root (142+95)= 3.05, giving P = 0.002

Reference:

1. BMJ 2006; 332:1256-8.