Critical Appraisal of Research Papers: Proposed Methodology

Introduction

The paper examines and critically appraises the methodological quality of the research and the effectiveness and acceptability of Short Message Services (SMS) for medication adherence. The publication under scrutinization was, “A randomized controlled behavioral intervention trial to improve medication adherence in adult stroke patients with prescription tailored Short -SMS4Stroke study” (Kamal, Shaikh, Paszedha, Azam, Islam, Memon et al., 2015). To summarize, the aim of the parallel group, assessor-blind, randomized controlled trial (RCT) was to examine the effectiveness of mobile technology, specifically SMS services to enhance medication adherence among stroke patients in Pakistan. The theoretical model for the study are the Health Belief Model and Social Cognitive theory, and the findings showed that after two months’ follow-up, the intervention group showed improved adherence than the controls.

For the review of the article, the CONSORT guidelines put forth by Cochrane Collaborative has been used (O’leary, 2013). The essay covers the article portion by portion starting with the critique of the title and abstract followed by the background, research methodology, results, and discussion.

Title and abstract

The title is described aptly with the avoidance of unnecessary jargon and captures the contents of the study in essence as required (Kulkarni, 2013).

The abstract of this study clearly defined by including the necessary information (Andrade, 2011). The results, however, do not provide the significance values that can deem the implications of the findings. Although the total sample size of 200 included it does not specify the number of individuals recruited for intervention and control group. This is important to understand as the 38 lost to follow-up are belongs to intervention or control group. Hence, for this study, although the abstract is concise it lacks certain specific information.

Background and Objectives

In this article, the aims of the study are clearly identified through which the results and discussion hence flow (Deaweb, 2015). On the background of the study, the section should clearly indicate the why the study aims are important and what relevance it holds in daily practice and the rationale for the study (Saiedian, 2007). The background has several structural issues. Firstly, there seems to be no relevant information that backs the question of compliance to medication in stroke patients and its importance. The background starts off with information relating to the disease area that is stroke and its prevalence in Pakistan study setting. However, it directly then goes on to how if adherence to medication were optimal then poor outcomes may be reduced without information on the issue of adherence itself. One way this can be incorporated is to talk briefly of why patients do not adhere, what makes adherence hard and lastly what is the main reason for focusing on adherence. There have been no reasons cited as to the outcomes of non-adherence and need for the study. The authors did not substantiate with supportive references especially when stating the importance of SMS as a culturally acceptable tool with potential behavioral change (Kamal et al., 2015). Further, the author could elaborate whether any studies conducted in Pakistan with the similar kind design as this would enable to identify the gap and quantify the outcome measures (p.2).

Further, there is mention of a “hypothesis” stating that the effectiveness of SMS reminders can be enhanced if used in the health belief model with behavior change theory. Although, there is no elaboration as to how this model improves the effectiveness of the intervention. The author should have elaborated by providing a clear rationale for using the theory to design the intervention. There is a lack of information regarding the justification for the secondary outcome such as blood pressure (BP). The background should have been even more explicit by stating that patient customized and behavioral change messages through SMS reminders for X number of weeks to increase the adherence to stroke medications.

Methods

Trial design

The trial design is an RCT with 1:1 allocation ratio, single centered and single blind. RCT are considered as the “gold standard” when it comes to clinical research to assess the levels of evidence in that particular field of research (Akobeng, 2005; Rajagopalan, Deodurg, & Srikanth, 2013). In this study, the need of RCT was not justified explicitly either in the background or methodology although it aims to measure the effectiveness of SMS health education and reminders for stroke patients (Rajagopalan et al., 2013).

Participants

According to guidelines, several baseline characteristics need to be measured at the time of recruitment. Authors presented the inclusion and exclusion criteria but how the method of recruitment and participation identification whether it was self-selection or by referral or identified consecutively from the Neurology and Stroke Clinics at the tertiary center after their regular appointment or and what are the different approaches used to determine the participants? Secondly, since the author has included the “modified ranking score” as one of the inclusion criteria to recruit participants, it would have been helpful if this was explained (p.2). The study procedure lacks clarity, (p.3) where it is unclear who evaluated the participant eligibility, whether it was the same individual (research supervisor) who did the interview assessed eligibility, which randomized participants and explained the intervention procedure with reference SMS receiving. Although they have mentioned that randomized and assessor are separate, it would have been clearer and precise on their characteristics such as whether they are nurses or lab assistants.

Allocation concealment and blinding

On the ratio of control and intervention the author has chosen 1:1 ratio type. The rationale behind the same however has not been described (Hey & Kimmelman, 2014). The advantage of equal allocation is that it increases the statistical power of a study. (Dumville, Hahn, Miles, & Torgerson, 2006) In this case, as there are no constraints on the availability of interventional material the equal allocation is entirely justified.

Outcomes

The primary outcome measure employed the Morisky Medication Adherence Scale (MMAS) questionnaire to measure medical adherence. But more information on this scale could be presented in an appendix or table along with an interpretation of score what does 0 or 1 mean regarding outcome measures. Further, it is not clear the validation, reliability and clinically significance of the tool. The meaning of the statement on “tool has a sensitivity of 46% and specificity of 60%” was unclear.

The secondary outcome for BP employed the Mindray Datascope Equator and the outcome measure for SMS intervention used a self-reported questionnaire based on Roger’s factors of diffusion of innovations and a self-designed questionnaire from previous literature. Background fails to provide reasoning of why only BP has chosen as a secondary outcome. Authors also measured the patient satisfaction and acceptability as secondary outcomes, and this was measured only to address the feasibility but not assess the effectiveness as stated in their paper (p.4). It is not clear again how far these questionnaires have been used in adherence medical trials and therefore, more information would enable the reader for better interpretation and could attach in an appendix.

Statistical Methods

The sample size calculation (calculated accounted for clinical significance especially a mean difference of 1 with standard deviation (SD) 2 measured through MMAS and whether 2 SD is common for both the groups) has described. But it would have been clear to the reader the rationale behind the 200 whether based on the previous studies or clinical difference of X% increase. Further, the confounding factors for the outcome have not been mentioned at all as it is important as it can add or depreciate the quality of the findings as well as show other associations if any (Attia, 2005).

Results

Participant flow

As per the statement of CONSORT, the participant flow needs to be stated explicitly (Schulz, Altman, & Moher, 2010). In this case, the study has highlighted the flow of participants in Figure 1 clearly.

Recruitment

The study has clearly depicted the data collection that has conducted over a period of two months and illustrated as flow diagram (Figure 1).

Results

The total numbers that have included for analysis are 79 and 83 from the control and 83 for the interventional arm. However, Table 1 presented contradicts Fig 2 when only 79 and 83 considered for final analysis, how come the baseline could be 100 for each. Further, Table 1 author had reported mean and interquartile range but as a reader it was difficult to interpret what does it mean, whether the average of 90 does show the participants are highly educated or uneducated? What was the coding followed to present those findings? Although they report no difference between control and intervention group, it would have been clearer if the exact p value. In regards to mean medication adherence (Table 2), the author reported difference and claimed it significant but without p exact value and rationale for considering the possible confounders. One of the important confounder missing in Table 2 was costs of drugs, which is an important factor for medication adherence but not clear how far these have accounted in the analysis. Although improvement of diastolic shown and it is not clear why only DBP has reported and what about systolic , and not presented either in the background or the study procedure section (p.7).

The section on satisfaction and characteristic of mHealth both did not clearly present regarding whether the patients were satisfied with the intervention or not. Authors have just described the findings but failed to interpret the results in the context of the outcome set. It is also not clear, what is the “tool” mean here. Therefore, the reported numbers are not of any relevance. Further, all of the sudden, mHealth concepts have been introduced by the author, which has not been presented previously in the background, study procedure, and statistical analysis.

Outcomes and estimations

For the process of recording the outcomes, a self-administered tool was used. From this, it can be stated that instances, where self-reported outcomes gave many drawbacks such as recall bias, overestimation by the participant, eagerness to report positively for the sake of the study, and also the errors related to the adherence observations (National Collaborating Centre for Primary Care, 2009). The author also should have described the p value set whether p<.05 or <0.01.

Ancillary analyses

Sensitive analysis and interim analysis have not done, which does not account for the 38 that lost to follow-up and its impact on the findings.

Harms

No adverse effects reported, and no patient has dropped.

Discussion

The study results showed that there is a significant increase in medication adherence, but such statement lacks supportive findings as authors did not possibly support the interpretation of the questionnaire. For example, they had reported baseline 6.6 and after two months MMAS score increased to 7.4 but what it does convey clinically has not been clearly indicated. Further discussion was missing regarding how far the outcome measure contradicted with previous other studies conducted with similar outcome including in Pakistan, or any other developed or developing countries. The clinical implications of the findings are completely missing. Therefore, it was difficult and challenging to conclude whether the intended outcome met. There is no discussion in secondary outcome and challenges faced by the patients while sending the SMS. The limitations and strengths of the study occupied half of the discussion content rather than discussing the findings. Further, there is no discussion why there is such huge drop out (20%); this can attribute to the lack of adequate structure and much missing information presented in the background (du Prel, Röhrig, & Blettner, 2009).

Registration and Funding

Registered at Clinicaltrials.gov (NCT01986023). Funded by Fogarty International Centre, National Institutes of Health and Grand Challenges, Canada.

Conclusion

In conclusion, the research study is moderately designed but one that does not report meaningful results that can help us translate the research into practice. This stems from the fact that the background was not adequately put forth with enough reasoning for the conduct of the present study. The study limitations are mainly, the use of self-reported outcomes, lack of defined and quantified end points, hypothesis and lack of adequate presentation of results as well as the identification of the research gap. The strength of the study is the use of the RCT design with equal allocation and blinding. In Toto, the study is not of the highest quality and the study findings do not necessarily imply translation into daily practice as the validity of the same is questionable.

Researchers to mentor-We write your Assignments & Dissertation

With our team of researchers & Statisticians - Tutors India guarantees your grade & acceptance!

About service

Reference

Akobeng, A. K. (2005). Understanding randomised controlled trials. Archives of Disease in Childhood, 90(8), 840–4..

Andrade, C. (2011). How to write a good abstract for a scientific paper or conference presentation. Indian Journal of Psychiatry, 53(2), 172–175. .

Attia, A. (2005). Bias in RCTs: confounders, selection bias and allocation concealment. Middle East Fertility Society Journal, 10(3), 258–261.

Consort. (2010). The CONSORT Statement. Retrieved January 11, 2016, from

Deaweb. (2015). Get Started Writing the Research Plan. U.S.A.: National Institutes of Health.

Du Prel, J.-B., Röhrig, B., & Blettner, M. (2009). Critical appraisal of scientific articles: part 1 of a series on evaluation of scientific publications. Deutsches Arzteblatt International, 106(7), 100–105. .

Dumville, J. C., Hahn, S., Miles, J. N. V., & Torgerson, D. J. (2006). The use of unequal randomisation ratios in clinical trials: A review. Contemporary Clinical Trials, 27(1), 1–12.

Hey, S. P., & Kimmelman, J. (2014). The questionable use of unequal allocation in confirmatory trials. Neurology, 82(1), 77–79.

Kamal, A. K., Shaikh, Q., Pasha, O., Azam, I., Islam, M., Memon, A. A., … Khoja, S. (2015). A randomized controlled behavioral intervention trial to improve medication adherence in adult stroke patients with prescription tailored Short Messaging Service (SMS)-SMS4Stroke study. BMC Neurology, 15(1), 212.

Kulkarni, S. (2013, October 17). 3 Basic tips on writing a good research paper title. Manuscript Structure. Retrieved from

National Collaborating Centre for Primary Care. (2009). Medicines Adherence: Involving Patients in Decisions About Prescribed Medicines and Supporting Adherence. London, UK: Royal College of General Practitioners. Retrieved from http://www.ncbi.nlm.nih.gov/books/NBK55447/#!po=92.8571.

O'Leary, Z. (2013). The Essential Guide to Doing Your Research Project. Los Angeles, CA: Sage.

Rajagopalan, R., Deodurg, P. M., & Srikanth. (2013). Overview of Randomized Controlled Trials. Asian Journal of Pharmaceutical and Clinical Research, 6(3), 32–38.

Saiedian, H. (2007). Guidelines and Requirements for Writing a Research Paper. In Guidelines for Writing a Research Paper (pp. 1–8). Spring. Retrieved from

Schulz, K. F., Altman, D. G., & Moher, D. (2010). Correspondence CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMC Medicine, 8(18), 1–9.