Frequently Asked Questions

On this page are questions which have been put to me over the last few years, plus several which perhaps should have been. The answers are my own views, sometimes lighthearted, for what they are worth.
Click on the questions below to view my answers.

Peter Wilkinson

1. How should I select a CRO?

The million dollar question. There are as many opinions on this as there are people in the industry, and I may be no better qualified to answer than many others.

The answers below should help....

  • Which would you rather pay for: sales and marketing backed by junior staff, or high quality expertise?
  • Most CROs put their highest emphasis (in terms of status and salary) onto the sales and marketing (business development) and project management functions because these are the people that the client initially sees. They are required to bring large quantities of business in order to pay for large running costs. Once the contract is signed they fade away and the work is performed by more junior staff.
  • How important is cost to you?
  • Only select a large CRO if cost is no a factor.
  • Do you get what you pay for?
  • An old cliche. The experience of many Pharmaceutical and Venture capital companies indicates that the cliche certainly does not apply to large CROs. I think that there is a lot to be said for selection based on whether you like and get on with the people, but not the salesman, the people who you will be working with.
    Many CROs have able, highly paid project managers who are the client interface. They may also have an able statistician. However the work is frequently delegated to a junior statistician, and inadequately reviewed. You really want a statistician of at least 10 years experience.
  • Staff Turnover?
  • Are you concerned about staff turnover in the CRO? Do you want to be dealing with the same contact and project manager at the start and end of the study? Find out what is the average rate of staff turnover in the CRO? Ignore quoted figures such as 5% turnover - these are often ill-defined and made up! Why not ask the people you will be dealing with how long they have been with the company?

2. What distinguishes you from other Biometrics CROs?

Personal service. It is what distinguishes us from the widely advertised names in medical biostatistics. If you have a specific development program or clinical study you can talk to me and be given advice which is relevant to you. I am happy to provide initial advice on study design etc free of charge and provide early costings for budgetary purposes.

3. How would you describe Wilkinson Associates?

It's difficult to put it succinctly, but we are a professional firm which at the same time is user friendly, approachable and pragmatic. Our attitude is that we look after people who are involved with clinical studies, as well as just data and statistics with inconvenient things called human beings attached to them. We're here to help.

4. Is dual data entry necessary?

Yes. Wherever possible it is much better than single entry plus QA. This observation is based on years of experience.

5. What do you think of Electronic Data Capture (EDC)

If you are considering this route it is worth considering the possible benefits to you. "the way of the future", "quicker results", "more efficient query handing" have been suggested. "The way of the future": Although EDC probably does represent the future the products currently available have been disappointing in the haste to get them to market. "quicker results": I was involved in a study whose main objective was to test in-house EDC versus in-house paper data management in order to test which methodology resulted in the earlier database lock. It was important to some in the company that EDC was seen to win, which it did, by 2 days! However as project statistician I was party to the documentation that showed that this was a pyrrhic victory for EDC because the EDC database had to be locked with a huge number of unresolved queries! "more efficient query handing": Unfortunately in practice the reverse appears to be true. For example if the investigator fails to address an EDC query properly then this can remain as faulty data. Issues with current EDC products are:
  • Inflexibility: Any changes in design can result in prohibitively costly changes to the EDC system. eg adjustment to validation check ranges, changes to the CRF. I worked as project statistician on a study where modest changes to a very few checks resulted in a Change of Scope for $30,000 - ouch! By comparison we never charge for changes to electronic validation checks.

  • Data quality: This sometimes leaves something to be desired. The explanation possibly relates to the fact that paper data management involves data entry and data management staff being closely involved with the data on a frequent basis. This is lacking in EDC, and quality can suffer as a result.

  • Inadequate documentation: Often no provision is made in EDC for adequate documentation to be provided to the statistician.

  • It is often said that costs of EDM are similar to conventional data management. The reason for this is that most of the companies selling EDM are large CROs and comparisons are being made with their changes for conventional data management, which tend to be considerably higher than a smaller specialist data management unit would charge.

6. How many patients do I need?

Many statisticians make the mistake of basing the power calculation on a treatment difference which has been supplied by a clinician, often the investigator. This may be either an expected or hoped for treatment difference. The treatment difference which the statistician uses as an input to the sample size calculation must be the least clinically important treatment difference. An error in this area can lead to a commercially disastrous study which is also unethical.

7. When is the best time to involve a statistician in a clinical study?

This is an obvious one - at the study design stage. This is probably the best value a company can get for statistician time. A very small number of hours at this stage can make a massive different to the study and its chance of success. Someone really experienced is needed for this work. The sample size calculation only represents a small part of this work.

8. Who should review the CRF?

In addition to the clinical and medical reviews the statistician and data manager should review the CRF. We have a lot of experience of what does, and does not work in a CRF design.

To be really cynical if you get statistical and data management input to the CRF, then they can't complain about the design later!

9. What statistical package should be used?

This must be SAS. Any CRO or statistician who does not use SAS is not serious about the Pharmaceutical Industry and regulatory work!

10. I have a study which is to be used for regulatory purposes, and needs statistical input. I am in contact with a statistician in my hospital/university.

Please don't! I am not trying to deny these worthy statisticians work - they usually have too must anyway! Academic statisticians whilst often brilliant are just not the people to work on studies which are to be used for regulatory purposes. Theirs is a different specialisation, and without industry experience they often don't have any understanding of what is required. It is like asking a psychiatrist to perform surgery - however able they may be they are not the right people for the job!

11. To you what is the most satisfying aspect of statistical consultancy?

Probably seeing a study which initially would have had little chance of achieving its objectives being revised following my suggestions to an innovative design with every chance of success.

12. What else is satisfying in statistical consultancy?

Performing the statistical analysis for a study that has been well designed, and carried out. When the result in a treatment difference which is shown to be statistically and also clinically significant there is a huge satisfaction in a job well done.

13. What is your pet hate in the clinical trials area of the pharmaceutical industry?

Underpowered clinical studies.

Following on from the exhortations of the late (and great) medical statistician Ken McRae, Professor of Medical Statistics, I seem to spend a lot of my time trying to persuade people to carry out correctly powered studies. This does not always mean more patients. It can often be achieved by improvements in study design. Under-powered studies do not make sense commercially, and are also unethical.

14. How should a study be powered?

Underpowered studies represent a poor investment commercially. If the results fail to demonstrate efficacy the study often fails to demonstrate lack of efficacy either! If this is an efficacy study this will mean that the study has probably been a waste of time and money. The alternatives are either to abandon the project or to then carry out the correctly powered study which should have been performed in the first place!

It is also unethical to subject patients to a clinical trial which has little chance of proving anything. See question 'How many patients do I need?'

15. My study has an active control. Should I be using an equivalence study?

Probably yes, or at least a non-inferiority study. Although many published studies involving a positive control group are designed as comparative studies, this will not wash with the MHRA or FDA. In order to avoid huge sample sizes a lot of high quality statistical help is usually required