Interpreting research results

Been a bit quiet lately - the shingles actually took more out of me than I expected.  I'm going to post some reflections on that experience shortly, so stay tuned for that.

In the meantime, I want to take the time to share an email conversation I had with a colleague of mine, Frederic Wellens PT, who has given me permission to post it on this website.  Here's the conversation:

Hi Dave,

I was re-reading your study tonight (A Descriptive Study of Pressure Pain Threshold at 2 Standardized Sites in People With Acute or Subacute Neck Pain) and found interesting your result that PPT where not statistically different between the affected and non affected side.  The reason I found it interesting is because it is at odds with my own clinical observations since I have been using my pressure algometer.

I find more often than not the PPT on the affected side to be somewhat lower (from 1 lbs to 3 pound) on average at least on the initial assessment (I would say it's mostly true when I suspect some central sensitization to be present).  This observation is on a wide range of painfull conditions and not just whiplash, so perhaps the symetry of PPT is a feature of whiplash and not other painful conditions.  To that effect, both the references you cited finding the same result you did refer to neck pain. (I do also take other areas than TA and UFT and I do focus on areas of the affected limbs distal and proximal to the pain but distant enough from the painfull site to avoid a confound with peripheral sensitization issues)

It could also well be that I am biased in finding lower PPT on the affected side and thus my measures are affected.  But I try to take every measurements without looking at the score before the patient's response.  Perhaps it's how I explain the test to patients.  Nevertheless, I am intrigued by your results.

On a side note, I tend to find an association between sensitive neurodynamic testing on the affected side and PPT. I can also say that PPT really do increase after manual therapy, regardless if it's HVLA thrusts of mobs and the effects I've noticed are not only non specific but often also bilateral even if the therapy was unilateral.  This is purely anecdotal, of course, but my observations on this are quite consistant.

I will welcome any of your thoughts on that!

Cheers!

And my reply:

Thanks for your insightful comments.  Surely you must have been having trouble sleeping if you were reading the paper last night.  It will no doubt cure insomnia :)


Your question raises a very important point, one which I didn't do a very good job addressing in the publication - that is the potential pitfalls of trying to apply results of group means to individual patients.  Remember that the mean is simply that point above and below which scores fall an equal distance.  It's possible, and happens in many cases, that no subject actually falls on the mean.  Yet, it is the mean and distribution of scores around the mean on which we conduct our statistical tests of the null hypothesis - if the null is not true (that is, if a significant difference exists), then we say the means are not the same.  If a sig. difference does not exist, then we say the means are the same, which was the case in this paper.  Did some subjects have PPT lower on their symptomatic side than their non-symptomatic?  Absolutely.  Did some have the opposite pattern (higher on their symptomatic side)?  Yep, that happened too.  In the end, and again this is the problem with focusing only on group means - it all came out in the wash in this case, such that the means were not statistically significantly different.  This of course raises more questions than answers: what is it about those people with PPT values that behave the way we would expect, compared to those people that behave in ways we didn't expect?  Is it something in the CNS?  Is it something to do with fear of pain?  Is it something to do with activity in peripheral nociceptors and the individual's ability to discriminate or perceive mechanical stimuli?  And more importantly, does any of this affect treatment decisions?  Lots more work to be done here.


Interestingly, as we continue to work in this area the results are consistent - on balance, mean PPT are not different side to side, but again that's the mean.  As you mention, on an individual basis PPT values seem to improve with treatment, which is also something we're seeing in a study that's almost complete - PPT values at both the UFT and to a lesser extent the TA are improving over the course of a 4-week PT intervention.  So something is happening, what's really interesting to me right now is what's happening, and how should we be interpreting and responding to these scores.  Lots more to explore in this area for sure.

This is such a critically important issue, and I thank my colleague for allowing me to post our conversation.  There will always be people who fall above the mean when best evidence says they should have fallen below it, and vice versa.  As a clinician and a researcher, one of the things that fascinates me is not only finding out what should happen under normal circumstances, but also trying to figure out what the mechanism is when things don't happen the way I expected - what did I miss?