Introduction: Valid and comprehensive instruments to measure integrated care are required to capture patient experience and improve quality of patient care. This study aimed to validate the Patient Experience of Integrated Care Scale (PEICS), among patients with chronic conditions seen in primary care.
Methods: One hundred and fifty-nine (159) French-speaking adults with at least one chronic condition were recruited in two family medicine clinics in Quebec (Canada) and completed the 17-item PEICS (T1). Fifty (50) participants completed it a second time 2 weeks later (T2). The internal consistency of the scale was assessed using Cronbach’s alpha, the test-retest reliability with the intraclass correlation coefficient (ICC), and concurrent validity using three dimensions of the Continuity of Care from Multiple Clinicians (CC-MC), with Spearman’s rank correlation coefficients.
Results: Cronbach’s alpha for the questionnaire was 0.88 (95% CI: 0.85 to 0.91). The intraclass correlation coefficient was 0.81 (95% CI: 0.64 to 0.90) and Spearman’s rank correlation coefficient with the three dimensions of the CC-MC varied from 0.44 to 0.54.
Conclusions and discussion: The PEICS showed good psychometric properties. This scale could be used in a population with chronic conditions followed in primary care to measure patient experience of integrated care.
A large proportion of the population from industrialised countries suffers from chronic conditions . These conditions usually persist over long periods of time, requiring long-term care from multiple service providers who handle patient needs ranging from physical and medical care to social, economic and psychologic support [2, 3, 4, 5]. Integrated care among all healthcare and social care providers is essential to provide optimal care for these patients .
Integrated care is the management and delivery of coordinated and patient-centered healthcare services [7, 8, 9]. It includes actions aiming to improve coordination, cooperation, continuity, and collaboration across different sectors of the healthcare system to prevent duplication and fragmentation of care while placing patients at the center of their care [10, 11]. Integrated care reduces hospital admissions, enhances adherence to treatment and compliance to medication, and improves quality of care, health-related quality of life, functional health, as well as patient satisfaction [6, 12].
The National Collaboration for Integrated Care and Support in the United Kingdom developed a model of integrated care based on patient experience . This model includes six dimensions: (1) consideration of patient and family needs; (2) communication with the patient and between practitioners; (3) access to information; (4) involvement in decision-making; (5) care planning; and (6) transitions between various health professionals and practitioners.
Valid measures of integrated care may provide useful follow-up indicators of efforts to improve quality of patient care . While a few instruments have been designed to assess patient perception of integrated care [13, 14, 15], they do not cover all dimensions of the National Collaboration for Integrated Care and Support model. To fill this gap, Picker Institute Europe and the University of Oxford were commissioned by the Department of Health and Social Care in the UK to develop a set of items to measure integrated care. These items could be used to help understand success in achieving the goals of the National Collaboration for Integrated Care and Support model,  but these items have not been validated yet.
This study aimed to validate the Patient Experience of Integrated Care Scale (PEICS), based on items proposed by Picker Institute Europe and the University of Oxford, among patients with chronic conditions seen in primary care, more specifically to evaluate its internal consistency, test-retest reliability, and concurrent validity.
To determine how services should be delivered for people with complex conditions, the National Voices in England adopted in 2013 a “narrative” for person-centred coordinated care, i.e. a new definition of integration reported in the form of statements formulated from the position of the patient . Known as the “I” statements, this narrative was built on the experience of patients, healthcare providers, as well as their representative organizations and included 38 statements to illustrate what good integrated care might look like from the patient perspective (Appendix 1) .
In a study carried out by Picker Institute Europe and the University of Oxford where they developed a set of items to measure people experience of integrated care, the “I” statements were used as the framework for development . The study involved a review of evidence, focus groups with patients, a desk review of existing measures, workshops with experts and cognitive testing with patients and healthcare providers. At the end of the process, they suggested a list of 18 statements (“items”) grouped into the six dimensions of integrated care. However, these items have never been validated in a population study.
Based on the dimensions of integrated care model, a consensus was reached among the authors of this paper (CH, MCC and ML) with permission from the first author of the study (JK)  to select the most relevant items (n = 13) to be included in the PEICS. In order to appropriately measure all the dimensions of integrated care, we added items (n = 4) from the initial list of “I” statements, for a total of 17 items. The selected items represented all dimensions of the model of integrated care . The PEICS takes approximately 10 minutes to complete. The scale uses three different five-point Likert scales from 0 to 4 adapted from the original items to fit with the PEICS. The total score is the sum of all questions and ranges from 0 to 68, where a higher score indicates better integrated care. Table 1 shows the complete scale.
The transcultural adaptation process followed the recommendations of Hawkins and Osborne . The process involved four main steps:
French-speaking patients from the waiting room of two primary care clinics in Quebec, Canada were solicited to complete a questionnaire. The first clinic, including 7 practices, was located in a rural region while the second, including 21 practices, was based in an urban region. Participants had to be: over 18 years of age; a Native French speaker; and affected by at least 1 chronic condition such as diabetes, cardiovascular disease, chronic obstructive pulmonary disease, asthma, hypertension, or other. Pregnant women or patients with an unstable acute condition, an uncontrolled psychiatric or cognitive disease were excluded.
Two research assistants asked eligible patients to self-complete the questionnaire a first time (T1) while they were in the waiting room of the primary care clinic. The questionnaire included a sociodemographic questionnaire and the French-language version of the Disease Burden Morbidity Assessment (DBMA) ; the PEICS; and the Continuity of Care from Multiple Clinicians (CC-MC) . The DBMA (21 items) provides a count of the number of chronic conditions and takes into account patients’ appreciation of the limitations arising from the presence of these chronic conditions. The CC-MC (34 items) measures continuity of care from the patient’s perspective. A moderate correlation was therefore expected (concurrent validity) between the PEICS and three dimensions of the CC-MC: Coordination role (5 items), Comprehensive knowledge of patient (4 items), and Team relational continuity (2 items). Two weeks after the first completion, a research assistant contacted the participants by telephone to complete the questionnaire again (T2), by starting with the first T1 respondents until 50 were completed.
This study was approved by the ethics review boards of the Centre intégré universitaire de santé et de services sociaux du Saguenay–Lac-Saint-Jean and the Centre intégré universitaire de santé et de services sociaux de l’Estrie-CHUS (Quebec, Canada). All participants completed and signed an informed consent form.
Incomplete questionnaires were excluded. One questionnaire was removed from the analysis due to the lack of coherence between T1 and T2, where a lot of responses passed from ‘never’ to ‘always’ within two weeks. We described the sample using mean and standard deviation (SD) for continuous variables such as age and integrated care, and proportions for categorical variables such as sex, birthplace, first language, marital status, education, occupation, and family income. The data were analysed using SPSS 24.0.
Psychometric properties of the PEICS were calculated by evaluating the internal consistency using Cronbach’s alpha and the test-retest reliability with the intraclass correlation coefficient (ICC). Because the Kolmogorov-Smirnov test demonstrated non-normality of the distribution, the concurrent validity between the PEICS and the three subscales of the CC-MC was calculated with a Spearman’s rank correlation coefficient.
A minimum sample size of 113 participants was estimated based on detecting a large effect size with α <= .05 and a power of 80% . A total of 159 participants were included in this study, which is higher than expected. For test-retest reliability, a sample size of 50 participants is considered adequate .
A total of 159 participants were included in the study. Figure 1 shows the flow of the patients throughout the study.
Participant characteristics are presented in Table 2.
|Age: mean (SD)||58 (16)|
|Number of conditions – DBMA: mean (SD)||4.1 (2.3)|
|Male: n (%)||53 (33.3)|
|Education completed: n (%) 5 missing|
|Less than high school||36 (23.4)|
|Completed high school||45 (29.2)|
|Occupation: n (%) 6 missing|
|Annual family income: n (%) 6 missing|
|Less than 20,000 CAD||20 (13.1)|
|20,000 to 49,999 CAD||59 (38.5)|
|50,000 CAD or more||74 (48.4)|
|Marital status: n (%) 1 missing|
|Married/living with a partner||85 (53.8)|
|Separated, divorced||29 (18.3)|
Table 3 presents the descriptive statistics of the PEICS as well as its reliability and validity.
|Minimum score||Maximum score||Mean (SD)||Internal consistency: Cronbach alpha (95% CI)||Test-retest reliability: ICC (95% CI)|
|0||68||53 (10)||0.88 (0.85–0.91)||0.81 (0.64–0.90)|
The mean score of the PEICS was 53 out of 68 (SD 10). Cronbach’s alpha for the whole questionnaire was 0.88 (95% CI: 0.85 to 0.91). The intraclass correlation coefficient was 0.81 (95% CI: 0.64 to 0.90). The correlation coefficient between the PEICS and the three subscales of the CC-MC varied from 0.44 to 0.54 (Table 4).
|CC-MC subscale||Concurrent validity: Spearman rank correlation coefficient|
|Coordination role (5 items)||0.49*|
|Comprehensive knowledge of patient (4 items)||0.54*|
|Team relational continuity (2 items)||0.44*|
This first validation study of the PEICS showed a high internal consistency and good temporal stability. A moderate correlation was found between the PEICS and three subscales of the CC-MC. These findings were expected because integrated care and continuity of care are related concepts , both including dimensions such as communication and cooperation between care providers to ensure that care is connected. The robustness of the measure could be attributed to its development based on a strong model of integrated care  and the development of items from a literature review, focus groups with patients, and workshops with experts [8, 16].
To our knowledge, the PEICS is the only instrument designed to evaluate all dimensions of the National Collaboration for Integrated Care and Support’s model in the United Kingdom . Few instruments measure integrated care from the patient’s perspective. The Patient Perceptions of Integrated Care (PPIC) developed by Singer et al. including 36 items showed good internal consistency (Cronbach’s alpha ranged from 0.62–0.80) and discriminant validity (Cronbach alpha coefficients were generally higher than the interscale correlations) . A 46-item instrument designed by Walker et al.  also demonstrated excellent internal consistency for 4 subscales out of 5. However, both instruments are too time-consuming to be used in research and clinical practices. On the other hand, the brief IntegRATE instrument included four items and was designed to assess integrated care from the patient perspective, but has never been validated in a large population (>15 patients) .
The PEICS could be used to assess the experience of patients with chronic conditions with integrated care among healthcare and social care providers. In clinical settings, such evaluation could provide feedback that leads to improvement. This questionnaire could also be used to measure and evaluate integrated care in various research designs.
We applied rigorous linguistic and cultural adaptation methods to the questionnaire with the involvement of the first author of the original items. In addition, our study took place in two different regions of Quebec (Canada), increasing the generalizability of the findings. As the first study assessing the validity of the PEICS, no other study is available for comparison. Moreover, our study may not be applicable to patients on the lower end of the socioeconomic status scale, youth, or people without chronic conditions because very few or none of the participants in our study fell into these categories. The 50 respondents for the follow-up questionnaire were not randomly recruited from among all the participants. A response bias could have been introduced by the use of different methods of completion of the questionnaire at T1 (self-administered in the waiting room) and at T2 (by telephone with a researcher assistant). In addition, several questionnaires (n = 73) were excluded because the patients ran out of time to complete the PEICS in the waiting room before their medical visit.
Future studies should consider concurrent validity with other instruments measuring integrated care. In addition, the factorial structure of the scale has to be confirmed in a larger sample. Sensitivity to change over time should also be further considered as well as validity in samples under-represented in this study.
The PEICS showed good psychometric properties. This scale could be used in a population with chronic conditions followed in primary care to measure patient experience of integrated care, to gather feedback in clinical settings or in a context of research evaluation. Further studies could evaluate its factorial structure, sensitivity to change over time, as well as concurrent validity with other instruments measuring integrated care.
Richard Humphries, Senior Fellow, Policy, Kings Fund, UK.
One anonymous reviewer.
The authors have no competing interests to declare.
Glouberman, S and Mintzberg, H. Managing the care of health and the cure of disease--Part I: Differentiation. Health Care Management Review, 2001; 26(1): 56–69, 87–9. DOI: https://doi.org/10.1097/00004010-200101000-00006
Bodenheimer, T. Coordinating care–a perilous journey through the health care system. The New England Journal of Medicine, 2008; 358(10): 1064–71. DOI: https://doi.org/10.1056/NEJMhpr0706165
Ouwens, M, Wollersheim, H, Hermens, R, Hulscher, M and Grol, R. Integrated care programmes for chronically ill patients: A review of systematic reviews. International Journal for Quality in Health Care, 2005; 17(2): 141–6. DOI: https://doi.org/10.1093/intqhc/mzi016
Singer, SJ, Burgers, J, Friedberg, M, Rosenthal, MB, Leape, L and Schneider, E. Defining and measuring integrated patient care: Promoting the next frontier in health care delivery. Medical Care Reseach and Review, 2011; 68(1): 112–27. DOI: https://doi.org/10.1177/1077558710371485
Kodner, DL and Spreeuwenberg, C. Integrated care: Meaning, logic, applications, and implications–a discussion paper. International Journal of Integrated Care, 2002; 2: e12. DOI: https://doi.org/10.5334/ijic.67
Nolte, E, Knai, C and Mckee, M. Managing chronic conditions: An introduction to the experience in eight countries. In: Nolte, E, Knai, C and Mckee, M (eds.), Managing chronic conditions: exprerience in eight countries, 2008; 1–14. European Union: European Observatory on Health Systems and Policies.
Epping-Jordan, JE, Pruitt, SD, Bengoa, R and Wagner, EH. Improving the quality of health care for chronic conditions. Quality & Safety in Health Care, 2004; 13: 299–305. DOI: https://doi.org/10.1136/qshc.2004.010744
Martinez-Gonzalez, NA, Berchtold, P, Ullman, K, Busato, A and Egger, M. Integrated care programmes for adults with chronic conditions: A meta-review. International Journal for Quality in Health Care, 2014; 26(5): 561–70. DOI: https://doi.org/10.1093/intqhc/mzu071
Singer, SJ, Friedberg, MW, Kiang, MV, Dunn, T and Kuhn, DM. Development and preliminary validation of the Patient Perceptions of Integrated Care survey. Medical Care Reseach and Review, 2013; 70(2): 143–64. DOI: https://doi.org/10.1177/1077558712465654
Elwyn, G, Thompson, R, John, R and Grande, SW. Developing IntegRATE: A fast and frugal patient-reported measure of integration in health care delivery. International Journal of Integrated Care, 2015; 15: e008. DOI: https://doi.org/10.5334/ijic.1597
Walker, KO, Stewart, AL and Grumbach, K. Development of a survey instrument to measure patient experience of integrated care. BMC Health Service Research, 2016; 16: 193. DOI: https://doi.org/10.1186/s12913-016-1437-z
National Voice. The narrative for person-centred coordinated care. [webpage on the internet] [cited 2018 March, 6th] Available from: https://www.nationalvoices.org.uk/publications/our-publications/narrative-person-centred-coordinated-care.
Redding, D. The narrative for person-centred coordinated care. Journal of Integrated Care, 2013; 21(6): 315–25. DOI: https://doi.org/10.1108/JICA-06-2013-0018
Hawkins, M and Osborne, R. Questionnaire translation and cultural adaptation procedure. Melbourne: Centre for Rheumatic Diseases, Department of Medicine, Royal Melbourne Hospital and University of Melbourne; 2007.
Poitras, M-E, Fortin, M, Hudon, C, Haggerty, J and Almirall, J. Validation of the disease burden morbidity assessment by self-report in a French-speaking population. BMC Health Service Research, 2012; 12: 35. DOI: https://doi.org/10.1186/1472-6963-12-35
Haggerty, JL, Roberge, D, Freeman, GK, Beaulieu, C and Breton, M. Validation of a generic measure of continuity of care: When patients encounter several clinicians. Annals of Family Medicine, 2012; 10. DOI: https://doi.org/10.1370/afm.1378
Terwee, CB, Mokkink, LB, Knol, DL, Ostelo, RW, Bouter, LM and de Vet, HC. Rating the methodological quality in systematic reviews of studies on measurement properties: A scoring system for the COSMIN checklist. Quality of Life Research, 2012; 21(4): 651–7. DOI: https://doi.org/10.1007/s11136-011-9960-1
Uijen, AA, Schers, HJ, Schellevis, FG and van den Bosch, WJ. How unique is continuity of care? A review of continuity and related concepts. Family Practice, 2011; 29(3): 264–71. DOI: https://doi.org/10.1093/fampra/cmr104