Why it's important to make a research checklist and even more important to stick to it!

Let's Recap

In last week’s blog post I discussed the trials and tribulations that come along with learning how to write scientific publications. This week I am going to talk about the importance of high-quality data collection, data analysis and dissemination of the results as a manner of showing the effectiveness of your population health initiative. After all, we need to be honest about whether the solution we implement is accomplishing the goals we set out to achieve.

In late 2017 I was getting ready to make a sales pitch to the quality committee of a large health organization. They had invited me to share some pilot data for a population health project we were in implemented at one of their rural primary care clinics. Specifically, they were interested in the impact our program had on lowering blood pressure and HgbA1c. I had been spot-checking results during the 6-month pilot and was very confident in the effect.

My lesson in double checking my research

It was time to determine what data needed to be abstracted from the EMR and how to present it. I collaborated with a research colleague to determine the approach for data analysis and whether this would be publication worthy (I could use another pub on the CV). We exchanged a barrage of emails and had two in-person meetings at which time we determined I had committed an unspeakable error; I neglected to define the parameters for data analysis prior to starting the pilot. ARGH! I was so excited we had the opportunity to implement this pilot for a possible future business contract that I forgot to define the measures of improvement before we started.

After a thorough “talking to” about how disappointed I was in myself, I regrouped and worked with my collaborator to identify a new plan for presenting the data. We decided to go simple with average and percent improvement. Next, we had to decide on a time frame and look back periods. It was a 6-month pilot, so we included chart abstracted data 2 months before the start and 2 months after the end of the 6 months. This would be consistent with how similar data has been reported in the literature. We provided these parameters to one of our students and sent them chart abstracting.

Why I will always implement a checklist

The presentation to the quality committee went very well and ultimately led to a contract for our services. The committee members trusted our results and recommended their executive leadership enter into a business agreement with CHS. While the data was accurate and high quality, it was not robust enough to stand on its own for a manuscript. We ended up including a Return on Investment (ROI) analysis and were able to pair the two together for an eventual publication. In the end, I am glad I made that mistake and was able to learn from it. Now first on my checklist is “Measures of Improvement”.