Start Submission Become a Reviewer

Reading: Exploring Intra and Interorganizational Integration Efforts Involving the Primary Care Secto...

Download

A- A+
Alt. Display

Research & theory

Exploring Intra and Interorganizational Integration Efforts Involving the Primary Care Sector – A Case Study from Ontario

Authors:

Anum Irfan Khan ,

University of Toronto, CA
X close

Jenine K. Harris,

Washington University in St Louis, US
X close

Jan Barnsley,

University of Toronto, CA
X close

Walter Wodchis

University of Toronto, CA
X close

Abstract

Background: The primary care sector is uniquely positioned to lead the coordination of providers and organizations across health and social care sectors. This study explores whether intraorganizational (professional) integration within a primary care team might be related to interorganizational integration between primary care and other community partners involved in caring for complex patients.

Methods: Two care coordination initiatives (Health Links) were selected – one led by a primary care team with a high level of intraorganizational integration as assessed by the Collaborative Practice Assessment Tool (CPAT), and the other led by a primary care team with a low level of intraorganizational integration. A case study design involving a social network approach was used to assess interorganizational integration across six types of relationships including regular contact, perceived level of integration, referrals, information sharing, joint care planning, and shared resources.

Results: Compared to the high-CPAT led case, the low-CPAT led case had higher density (more ties among organizations) in terms of regular contact, integration, and sharing of resources, whereas the opposite was true for the referral, information sharing, and joint care planning networks. Network centralization (extent to which network activity is influenced by one or a group of organizations) was higher for the high-CPAT case compared to the low-CPAT case in the integration, referrals, and joint care planning networks, while the low-CPAT case had higher centralization with regard to regular contact, information sharing, and shared resources.

Conclusion: The interplay between intra and interorganizational integration remains unclear. We found no consistent differences in the patterns of ties across the six types of networks examined between the two cases. Assessing changes in network metrics for different organizations in each case over time, and supplementing network findings through in-depth interviews with network members are key next steps to consider.

How to Cite: Khan AI, Harris JK, Barnsley J, Wodchis W. Exploring Intra and Interorganizational Integration Efforts Involving the Primary Care Sector – A Case Study from Ontario. International Journal of Integrated Care. 2022;22(3):15. DOI: http://doi.org/10.5334/ijic.5541
40
Views
  Published on 08 Sep 2022
 Accepted on 06 Jul 2022            Submitted on 16 May 2020

Introduction

As the prevalence of multimorbidity along with a rapidly aging population continues to present key challenges for health systems [1, 2], adopting an integrated approach to enable collaboration between different levels of the health care system led by the primary care sector is increasingly critical to ensure that limited resources are effectively leveraged to optimize patient and provider outcomes [3, 4, 5, 6, 7]. The coordination of integration efforts within an organization, e.g., different disciplines within a primary care practice (intraorganizational integration), and across organizational boundaries (interorganizational integration) [8, 9, 10] are increasingly needed to more efficiently connect various levels of the health care system i.e., primary care, social supports, home care, and tertiary/acute care. In the context of primary care, intraorganizational integration can be conceptualized as professional integration or interprofessional collaboration within a defined organizational structure, which refers to collaboration between professionals based on shared competences, roles, responsibilities and accountability to deliver a comprehensive and coordinated continuum of care [11, 12]. In contrast, interorganizational integration involves different organizations coordinating efforts to deliver comprehensive care over time, across professions and organizational boundaries to minimize fragmentation and enhance the value of services delivered [5, 12, 13, 14, 15].

Current literature indicates that there is a significant level of overlap in the factors that enable intraorganizational and interorganizational integration [16]. A commitment to a shared set of goals and accountability for outcomes, information technology capacity [17, 18, 19], and various aspects of interpersonal functioning such as trust and communication between team members, and leadership support for collaboration [20, 21, 22] represent common facilitators that enable both intra and interorganizational integration [16]. Differences in professional culture across disciplines and organizations, gaps in role clarity, the absence of shared goals and values, and operational disconnects, i.e., variability in budgetary and planning cycles [23, 24, 25] are among the key challenges organizations face in collaborating effectively both within and across organizational boundaries [24, 26, 27, 10, 28, 16, 29]. While there has been growing emphasis on the adoption of principles for integrated care as part of health system reform in Canada and globally, to date, our understanding of the interplay between intra and interorganizational integration is limited.

In the context of Ontario, an effort to improve the health system’s capacity to better care for complex patient populations, defined as individuals with multiple chronic conditions, functional and/or cognitive impairments, and social vulnerability [30, 31] has resulted in various initiatives to better integrate health and social care for this patient population. This includes the advent of interprofessional primary care teams, as well as formal interorganizational partnerships between networks of providers and organizations across sectors through the Health Links program [32, 33, 34, 35]. The Health Links initiative involved formalizing existing geographic networks of providers including primary care practices, hospitals, long-term care and community agencies, to support interorganizational integration via a shared commitment to coordinate care across sectors for complex high-cost patient populations [36]. But despite strategic and policy efforts to promote integrated care for complex patient populations in Ontario, and emerging evidence around the shared facilitators for intra and interorganizational integration [16], our understanding of the extent of interorganizational integration, and the interplay between intra and interorganizational integration is still in its early stages. Given this gap in knowledge, the objective of this study was to examine whether intraorganizational (professional) integration within a multidisicplinary primary care team is associated with interorganizational integration between the primary care sector and other types of organizations involved in delivering services for complex patient populations, as health and social care systems adapt to changing disease burdens and patient needs.

Methods

Setting and participants

This setting for this study was Ontario (Canada), which has a publicly funded healthcare system that offers universal health care coverage for physician and hospital services. The primary care sector serves as a gateway to the health system, and includes a range of funding arrangements including salaried models, blended fee-for-service and blended capitation. Over the past decade, there has been a transition away from traditional fee-for-service payment models, as part of a systematic shift towards interprofessional models of team-based primary care including Community Health Centres (CHCs) and Family Health Teams (FHTs), in concurrence with a strong emphasis on collaboration between primary care and other providers/organizations in the broader health and social care landscape, which informed the establishment of the Health Links program in 2012 [37]. The objective of the Health Links initiative was to promote coordination between providers and organizations to better serve complex patient populations, defined as patients with four or more chronic conditions, including a focus on individuals living with mental health and addictions, and the frail elderly, to ultimately help reduce healthcare utilization (e.g., emergency department visits, hospitalizations, and readmissions) and improve access to primary and specialist care [32, 34, 36, 38].

By design, all organizations in a Health Link work collaboratively to develop a care plan for the patient and deliver coordinated care with the patient at the center [39]. Each Health Link serves a catchment area with a total population of at least 50,000 people, and involves engagement from at minimum 65% of primary care providers in a given geographic area [32]. At the time of this study, 82 Health Links were operational across Ontario with an estimated 1800 partner organizations from multiple sectors [34].

Each Health Link is led by a lead agency, which could be a primary care practice, hospital, or home and community support agency [32, 40]. The lead agency is required to take ownership of operational, management and agenda-setting aspects for each Health Link [32, 40]. To allow for an in-depth examination of the possible interplay between interprofessional collaboration within primary care practices is associated with interorganizational collaboration, two Health Links led by interprofessional primary care teams that had previously participated in a survey of interprofessional collaboration [41], were selected as case studies. The use of a case study design was selected to allow for an in-depth multi-faceted exploration of how organizations were collaborating within the boundaries of a Health Link [42].

Intraorganizational (professional) integration within each lead agency was measured using the Collaborative Practice Assessment Tool (CPAT) [43]. A recent review of instruments to assess interprofessional collaboration found that the CPAT is considered to be the most appropriate instrument to measure collaborative practice involving providers from a broad range of disciplines, particularly as it relates to chronic disease management [44]. The tool has been used by providers across different types of healthcare settings including primary care teams [43, 45]. The CPAT is comprised of 56 questions across a total of 8 domains, and is scored by averaging all items within each domain, and a total measure of professional integration is generated by adding up the average domain scores [46]. The CPAT was administered to individual providers within each lead agency (interprofessional primary care team) for the two Health Links. Individual provider-level scores are then aggregated to develop a team-level CPAT score. The lowest possible total score on the CPAT is an 8, whereas the highest possible score is a 56.

Interorganizational integration in each Health Link was explored through a social network analysis [15], which is increasingly being used in health services research to evaluate integrated care programs/services [47, 48, 49]. Assessing linkages between organizations using a network lens is grounded in the idea that service systems can be conceptualized as ‘networks’ of interacting organizations, and corresponding network characteristics (i.e., connectivity and flow of information between members in the network) can be inferred from the number and patterns of relationships between those organizations [50, 51, 52].

Case selection

The cases for this study were based on a larger study of 66 primary care teams that completed the CPAT survey [41]. The mean CPAT score across participating practices was 46.6 (sd = 2.47); the small standard deviation revealed a limited spread of scores given the possible range of 0 to 56. Of the 66 primary care teams that participated in the original study, 17 primary care teams were involved in the Health Links program, and ultimately two Health Links led by primary care teams were selected to serve as case studies based on their CPAT scores.

To capture the influence of variability in intraorganizational integration (as assessed by the CPAT scores) on interorganizational integration, the first Health Link selected to serve as a case was led by a primary care team with a low CPAT score [43.74], hereby referred to as the ‘low-CPAT led case’, whereas the second case was led by a primary care team with a high CPAT score [50.01] thus referred to as the ‘high-CPAT led case’. Both Health Links, were led by an interprofessional primary care team (which had a broad range of interdisciplinary providers including family physicians, registered nurses, social workers, and dieticians etc.), and included hospitals, a home and community service agency responsible for coordinating community-based care, a long-term care facility. Overall, the two cases were comparable in terms of the number, types health and social care providers involved, years in operation, geographic area served, and population demographic features (age, medical and social complexity).

Data Collection

A network questionnaire comprised of 6 questions was developed based on by past studies involving interorganizational network analyses, and adapted to reflect the mandate of the Health Links program [53]. The questionnaire sought to explore six types of relationships including: frequency of contact (daily, weekly, monthly, bi-annually, and annually) between organizations perceptions around the level of integration between organizations via a scale ranging from not linked to fully linked (see Table 1) [53, 54, 55], referrals, information sharing, joint care planning, and shared resources. In addition to data collected for the social network analysis, a general question around network member views on the benefits and drawbacks of their involvement in the Health Links program was also included in the network questionnaire (see Appendix 1).

Table 1

Response options for perceived levels of integration between organizations.


LEVEL OF INTEGRATION DESCRIPTION

Not linked We did not work together (to serve patients with multiple chronic conditions) at all and have separate program goals

Communication We shared patient information only when it was advantageous to either or both programs

Cooperation We shared patient information and worked together when an opportunity arose

Coordination We worked side-by-side as separate organizations to achieve common program goals; efforts were coordinated to prevent overlap

Collaboration We worked side-by-side and actively pursued opportunities to work together to support patients with multiple chronic conditions, but did not establish a formal agreement

Partnership We worked together as a formal team with specified responsibilities to achieve common goals (had a Memorandum of Understanding or other formal agreement)

Fully linked We mutually planned and shared staff and/or resources to organize and deliver care for individuals with multimorbidity

Given the central role that the lead agency plays in managing the Health Link, a list of organizations conceived to be a part of the Health Link was then shared with the executive director of the lead agency for each case to get their feedback on the appropriateness of organizations considered for inclusion in the network analysis, and determine which individual from each organization would be best suited to complete the network questionnaire. An electronic link to the network questionnaire was sent to the individual identified as the contact person for each organization in the Health Link. All data was collected between May and July 2018 through the Hosted in Canada Survey Platform. Responses were stored on a password-protected server, which only the research team was able to access. Ethics approval was obtained from the Research Ethics Board at the University of Toronto (Protocol ID 33699).

Data analysis

Data from the Hosted in Canada Survey Platform was imported into UCINET to compute network measures and generate visualizations. Network plots were developed to illustrate the structure and functioning of each case as it relates to the six types of networks that were explored. A detailed description of the network metrics generated is available in Table 2. Missing data were imputed using reconstruction for undirected networks (where there is no specified direction of the tie) [56]. For each relationship between two nodes in an undirected network, when only Organization A and Organization B answered (not both), reconstruction uses the answer of the single node as the relationship value. So, if only Organization A answered the survey and reported a tie with Organization B, the network data would assume that Organization A and Organization B are connected [57].

Table 2

Description of network and node level measures.


MEASURE DESCRIPTION

Density A network-level metric that represents the overall level of connectedness among organizations in the network, based on the proportion of actual links relative to the maximum number of possible links in the network [55]. Density scores range from 0 to 1 – lower scores indicate low levels of connectedness, i.e. a 0 would entail no connections between members in the network, whereas a 1 means that all network members are connected to one another [15].

Degree centralization A network-level measure that depicts how centralized a network is overall, i.e., the extent to which network activity is influenced by one or group of organizations based on the number of connections each network member has with others [54, 60, 61]. Degree centralization ranges from 0 to 1 – a higher value of degree centralization indicates that a small number or group of organizations have a major influence on network activity.

Degree centrality Degree centrality is a node-level measure and typically denotes the number of links each member has with other members in the network – organizations with high degree centrality are considered to be well connected within the network [62]. For directed networks, there are two possible measures of degree centrality: (1) in-degree centrality represents the number of incoming ties to a particular member, and (2) out-degree centrality reflects the number of outgoing ties [63]

The contact measure was dichotomized based on a cut-off at biannual contact, so organizations that had contact with each other more than twice per year were considered linked, while those with less contact than this were considered not linked. In the event of a discrepancy between members in a dyad, the average value was used [53]. The contact network was considered to be undirected in nature, and since missing data was below 30%, reconstruction was used to impute those responses where applicable [58].

Participant views on the perceived level of integration between organizations were dichotomized; organizations were considered to be integrated if they indicated the nature of their relationship as ‘coordination’ or above (coordination, collaboration, partnership, or fully linked) [53]. Mean tie value was used in the event of a discrepancy between two organizations on the perceived level of integration with other network members.

The referral network included both sent and received referrals. Only confirmed ties were analyzed A line between two nodes (organizations) indicates the presence of a confirmed referral relationship; i.e., Organization A indicated they sent a referral to Organization B, and Organization B expressed that they have received a referral from Organization A. Density and degree centralization scores under 0.30 were considered to be low, 0.30–0.50 were seen as moderate, and above 0.50 was deemed high [59].

Results

There were 15 organizations in the low-CPAT led case, and 12 organizations in the high-CPAT led case. Both cases had a very high response rate – 93.3% for the low-CPAT led case (n = 14/15), and 91.7% for the high-CPAT led case (n = 11/12). A 75% response rate is considered sufficient in terms of reliability for network data [64, 65].

Network measures and visualizations

For the network plots in Figures 1, 2, 3, 4, 5, 6, the nodes represent each organization in the network and the color of a node depicts the type of organization, i.e., lead agency, hospital or community agency. The size of individual nodes in each network map indicates degree centrality (number of connections each organization has with other network members) in undirected networks, and in-degree centrality (the number of incoming ties an organization has) in directed networks (referrals, information sharing, joint care planning and shared resources), wherein larger nodes represent organizations that are more prominent in terms of connectivity and play a central role in the network. A summary of the network measures generated across the 6 networks is available in Appendix 2.

Contact

For the contact network, the low-CPAT led case showed a high density score, compared to a moderate level of density in the high-CPAT led case, revealing that 64.8% of linked dyads (pairs of connected organizations) in the low-CPAT led case had regular contact compared to 48.5% in the high-CPAT led case. Node size in Figure 1 represents degree centrality (number of links each member has with other members in the network), and a line connecting two nodes (organizations) indicates regular contact (daily, weekly or monthly basis). Both cases showed a moderate level of degree centralization, but the structural positioning of network members varied considerably across cases. Primary care played a central role in the contact network for the low-CPAT led case, with the lead agency and another primary care practice exhibiting the highest levels of degree centrality, compared to a hospital that had the highest degree centrality in the high-CPAT case.

Network maps depicting regular contact between organizations in the two Health Links
Figure 1 

Regular contact (monthly or more often) between organizations in two Health Links in Ontario – node size depicts degree centrality for each network member (data collected in 2018).

Perceived level of integration

The low-CPAT led case had a moderate density score (0.400) compared to the low-density score observed in the high-CPAT led case (0.242). A total of 40% of possible linked dyads (pairs of nodes/organizations) in the low-CPAT led case indicated being engaged in at minimum a coordinated relationship with other organizations in the network, compared to 24% in the high-CPAT led case. Node size in Figure 2 represents degree centrality, and a line between two nodes indicates that both organizations reported having at minimum a coordinated relationship, i.e., organizations worked side-by-side as separate organizations to achieve common program goals, and efforts were coordinated to prevent overlap. In the low-CPAT led case, the lead agency (a primary care practice), and a hospital had the highest degree centrality, showcasing their prominent positioning in the network. For the high-CPAT led case, a government agency that coordinates home and community care services had the highest degree centrality, revealing its central position in the network.

Network maps depicting the perceived level of integration between organizations in the two Health Links
Figure 2 

Perceived level of integration (at minimum the presence of a coordinated relationship) between organizations across two Health Links in Ontario – node size depicts degree centrality for each network member (data collected in 2018).

Referrals

The density score of the high-CPAT led case revealed a moderate level of connectivity in terms of referral relationships compared to a low level of connectivity in the low-CPAT led case. A total of 37.9% of possible linked dyads (pairs of nodes/organizations) in the high-CPAT led case were involved in referral relationships, compared to only 11.9% in the low-CPAT led case. Referral relationships in both cases were moderately centralized around a few organizations. The arrowhead in Figure 3 indicates the direction of the referral, i.e., whether the referral was being sent or received by any given node/organization (Organization A sent referral → Organization B received referral). Figure 3 reveal several isolates in the low-CPAT led case; 6 of the 15 organizations in this Health Link were not engaged in any type of referral relationships, compared to 1 of 12 organizations in the high-CPAT led case. Node size reflects in-degree centrality (number of incoming referral ties) for each organization.

Network maps depicting referrals between organizations in the two Health Links
Figure 3 

Referral relationships between organizations in two Health Links in Ontario, Canada–node size depicts in-degree centrality for each organization, and the arrowhead reflects the direction of the referral (i.e., being sent or received) (data collected in 2018).

Information sharing

A low density score (0.229) was observed in the low-CPAT led case, indicating limited connectivity in terms of information sharing relationships between network members in this case, compared to the moderate density score (0.500) observed in the high-CPAT led case, as observed in Figure 4. A total of 22.9% of possible linked dyads (pairs of nodes/organizations) in the low-CPAT led case were involved in information sharing relationships, compared to 50% in the high-CPAT led case. The size of each node in Figure 4 represents in-degree centrality (number of incoming information sharing ties), and the arrowhead indicates the direction of information sharing. Information sharing in the low-CPAT led case was centered around the primary care sector; the lead agency (a primary care practice) and another primary care practice had the highest out-degree centrality (outgoing information sharing ties), and a hospital had the highest in-degree centrality. In the high-CPAT led case, the home and community care agency and a hospital had the highest out-degree centrality whereas the same home and community care agency along with the lead agency had the highest in-degree centrality.

Network maps depicting information sharing relationships in the two Health Links
Figure 4 

Information sharing relationships between organizations in two Health Links in Ontario – node size depicts in-degree centrality for each organization, and the arrowhead indicates the direction of information sharing between network members (data collected in 2018).

Joint care planning

Both cases had low density and moderate degree centralization in the joint care planning network. Only 11% of possible linked dyads (pairs of organizations) existed in the low-CPAT led case, compared to 15% in the high-CPAT led case. A line connecting two nodes in Figure 5 indicates that presence of a confirmed joint care planning relationship and node size represents degree centrality for each organization (number of joint care planning relationships with other organizations in the Health Link). Joint care planning in the low-CPAT led case was led by a hospital, which had the highest in-degree and out-degree centrality, compared to the high-CPAT led case, where joint care planning efforts were led by the home and community care agency. Both networks were very centralized as demonstrated by the large central node. Trends in network metrics and node-level measures were consistent when unconfirmed ties were analyzed.

Network maps depicting joint care planning relationships in the two Health Links
Figure 5 

Joint care planning relationships between organizations in two Health Links in Ontario – node size depicts in-degree centrality for each network member (data collected in 2018).

Shared resources

The shared resources network had the lowest density scores across the 4 types of relationships explored through the network analysis. A total of 10% of possible linked dyads (pairs of nodes/organizations) were involved in resource sharing relationships (e.g., providing funding for shared staff members) observed in the low-CPAT led case, compared with only 2.3% in the high-CPAT led case. Node size in Figure 6 represents in-degree centrality (number of links depicting incoming resources) for organizations in each Health Link. The arrowhead indicates the direction of resource sharing, i.e., which organization is providing a given resource and which organization is on the receiving end (e.g., Organization A is sending resources → Organization B receiving resources). The lead agency for the low-CPAT led case had high in-degree and out-degree centrality indicating that it receives and offers resources to other organizations in the Health Link. A total of 12 (out of a possible 15) organizations in the low-CPAT led case were involved in resource sharing relationships, compared to only 6 organizations in the high-CPAT led case were involved in resource sharing relationships, and several isolates are visible in Figure 6.

Network maps depicting resource sharing relationships in the two Health Links
Figure 6 

Resource sharing relationships between organizations in two Health Links in Ontario – node size depicts in-degree centrality for each organization, and the arrowhead indicates the direction of resource transfer between network members (data collected in 2018).

Perceived benefits and drawbacks

Respondents were asked to offer feedback on perceived benefits and drawbacks pertaining to interorganizational integration in each Health Link. Network members may have varying expectations at the outset of a partnership being established [24, 66], so developing a baseline understanding of member perspectives around whether expectations have been met, and what challenges they face, can help facilitate improvements in network structure and functioning [66]. A majority of respondents in both cases indicated that being involved in Health Links, had resulted in or was expected to support improvements in their capacity to better serve patients in their communities, and improve the use of each organization’s services, illustrating that some of the benefits intended in the development of the Health Links initiative have been realized.

There was a clear difference between the two cases with regards to perceived drawbacks. A total of 18% of network members in the high-CPAT led case, stated a loss of control/autonomy over decision-making, and 36% of network members felt that involvement in the Health Link had or is expected to cause strained relations within their organization. In comparison, none of the members from the low-CPAT led case felt that their engagement in these interorganizational partnerships resulted in a loss of control/autonomy over decisions or strained relations within their organization. Only 14% of organizations in the low-CPAT led case believe their involvement in Health Links does or is expected to take too much time and resources, compared to 45% of network members in the high-CPAT led case. A summary of participant feedback on perceived benefits and drawbacks is available in Appendix 3.

Discussion

To date little is known about the extent to which integration efforts within an organization may influence interorganizational integration. The purpose of this study was to explore intra-organizational integration (professional integration) within a primary care team that leads a Health Link, is associated with interorganizational integration between primary care and other organizations involved in delivering services within the Health Link.

For the contact network, the low-CPAT led case had higher density than the high-CPAT led case, indicating that there were more contact ties between organizations in the low-CPAT led case compared to the high-CPAT led case. Greater density in a network signals the presence of more pathways among organizations, that facilitate communication and connectivity between network members, compared to less dense networks [61]. Past research has found that higher density scores are indicative of trust between organizations, greater social capital, and a commitment to engage in achieving shared goals and objectives [67, 68]. Higher network density can also reflect greater homogeneity in the network; individuals or groups are more likely to associate, accept and bond with others that share similar characteristics and/or beliefs [69], which can facilitate a higher likelihood of information exchange and collaboration between network members [70].

Regular contact between organizations in the low-CPAT led case may also explain the higher density score for this Health Link in the integration network; a total of 40% of possible linked dyads (pairs of organizations) in the low-CPAT led case were involved in at minimum coordinated relationships, compared to only 24% in the high-CPAT led case. While both cases had a similar number and mix of organizations, overall, for the low-CPAT case the primary care sector appeared to play a crucial role in connecting organizations that would not otherwise be in contact. In contrast, for the high-CPAT led case, the home and community care service agency, and a hospital had the highest degree centrality, highlighting the less prominent role of the primary care sector in this Health Link.

Mixed trends were observed in terms of network measures that were assessed. The low-CPAT led case appears to have higher density in the contact, shared resources, and perceived level of integration networks, compared to the high-CPAT led case. In contrast, the high-CPAT led case has higher density compared to the low-CPAT led case with regards to sending and receiving referrals, information sharing, and joint care planning. Past studies report associations between regular contact and information exchange or ‘diffusion’ across a network [71, 72]. The low-CPAT led case had high density for contact and moderate density in the integration network and so could be expected to have strong connectivity across the four other types of relationships [71], however our findings indicated otherwise. This could be attributed in part due to strong informal communication between organizations in the low-CPAT case, which can precipitate information sharing and referrals through informal mechanisms (e.g., ad-hoc communication, leveraging existing relationships to call colleagues in another organization to discuss a patient’s needs etc.) [73, 74, 75]. A higher degree of direct contact between network members, may have led organizations in low-CPAT led case to have greater awareness of the types of resources available in the Health Link, and thus directly coordinate care for patients through their frequent contact (possibly resulting in fewer referrals). A recent examination of interagency partnerships between health and social care organizations in the UK also found that coordination between agencies was found to rely more on informal rather than formal relationships [76]. Compared to the low-CPAT led case, the high-CPAT led case had lower density in the contact and integration networks, but higher density in a majority of the relationship networks – illustrating that most of the collaboration between organizations in this case were occurring through formal mechanisms. This finding aligns with past research, which found that lower density in networks corresponds with more formal collaboration or interaction between network members [77].

Degree centralization was similar between cases across all networks, except for the information sharing and referrals networks, wherein the high-CPAT led case had higher degree centralization scores compared to the low-CPAT led case. Greater centralization in a network reflects the presence of ‘hubs’ of network members, who given their central positioning can transmit information to other members in an efficient manner [61]. This could also have contributed in part to the higher density observed in both the information sharing and referrals network for the high-CPAT led case, compared to the low-CPAT led case.

Network metrics reveal that in the high-CPAT led case, organizations appear to be functioning more independently (fewer sharing of resources between members) and have left the coordination of integration between network members to an organization (a home and community care agency) that is not the lead agency (a primary care practice). Whereas in the low-CPAT led case, organizations contact each other more regularly (though fewer referrals are sent between organizations), potentially facilitating a higher perception of interorganizational integration between members in this Health Link, which also features strong involvement by the lead agency (a primary care practice) in many of the key relationships examined.

Past research has shown that the lead agency for a coalition or program typically wields considerable influence in the network based on their structural positioning [53, 71, 72]. Network findings indicate that the lead agency in the low-CPAT led case, had the highest degree centrality for the contact and integration networks, and well as high in-degree and out-degree centrality in all but the one of individual relationship networks examined (information sharing, referrals, and shared resources). Whereas in the high-CPAT led case, the lead agency did not hold the highest in-degree and out-degree centrality in a majority of the networks examined. This variation in structural positioning of the lead agency between the two cases indicates possible differences in the ‘embeddedness’ (assessed by total degree, in-degree and out-degree centrality) each lead agency exhibited in their respective Health Link [78]. Organizations with a low level of embeddedness are considered less prominent in terms of their reputation, less likely to be trusted by other network members, and would typically perceive less overall benefit from their participation in a given network [78]. While direct contact with an influential node can facilitate the adoption of a given behavior or innovation [79], the lead agency in the high-CPAT led case was not prominent in terms of its structural positioning for most of the 6 networks explored, thus potentially restricting the extent to which diffusion of innovation between intraorganizational and interorganizational collaboration, could have occurred.

A majority of network members across cases reported a number of benefits associated with their involvement in the Health Link, including improvements in their capacity to care for patients with multimorbidity, and better use of their organization’s services. However, a greater proportion of network members in the high-CPAT led case felt there were drawbacks to their engagement in Health Links; whereas none of the respondents in the low-CPAT led case reported difficulty in dealing with other members in the network, while 27% of network members in the high-CPAT led case indicated that this had already occurred, and 18% of respondents in the high-CPAT led case (but none of the members in the low-CPAT led case), felt their participation in Health Links had resulted in a loss of autonomy. This contrast between the two cases is reflected in some of the network metrics. Specifically organizations in the low-CPAT led Health Link were more likely to participate in resource sharing relationships, and reported overall a higher degree of perceived integration than those the high-CPAT led Health Link, resulting in perhaps a more effective distribution of roles and responsibilities between network members, in turn reducing possible burnout from their involvement in the Health Links program. The limited influence of the lead agency in the high-CPAT led case may have affected the overall cohesiveness of the Health Link, possibly contributing to lower perceived levels of integration, tension or burn-out among network members [80, 81], and a more negative outlook on the experience of participating in this interorganizational integration initiative.

Strengths

This study offers a unique assessment of how organizations across the health and social care sectors in Ontario are engaged in integrating care across organizational boundaries, which is increasingly critical in improving the capacity of the health system to better respond to changing disease burdens and evolving patient needs [82]. The Health Links program was designed to be a network intervention intended to enable structurally and functionally distinct organizations to collaborate under a shared mandate to reduce fragmentation, minimize redundancies [28, 40], and ultimately improve patient and system-level outcomes [83]. This study offers an in-depth look at how organizations in two Health Links are collaborating to deliver care across organizational boundaries, and provides a starting point to explore the interplay between different types of integration efforts using an innovative network lens. A systematic approach involving in-depth stakeholder consultation was used to determine network boundaries, and a wide range of relationships between organizations were examined to provide a fulsome sense of the mechanisms through which organizations are working together to deliver an integrated continuum of care. Both cases had a high response rate to the network questionnaire; network data included in the analysis included representation from over 90% of organizations in each Health Link. Both confirmed and unconfirmed ties were analyzed to identify consistency in trends across the 6 networks assessed, and trends across network metrics and node-level measures were consistent when only confirmed ties were included in the analysis.

Limitations

Several important limitations must be noted in interpreting these findings. In terms of external validity, given the small number of cases, findings may not be applicable to other jurisdictions or types of interorganizational partnerships. Due to the small sample size and cross-sectional study design, a longitudinal comparison of trends in network density and centrality measures could not be conducted. Given the small number of cases involved, it was not feasible to examine the impact of this interorganizational integration initiative on individual or system-level outcomes for patients served by Health Links. Furthermore, while theoretical overlap in the drivers of intraorganizational and interorganizational integration has been previously explored [16], a measure of interprofessional collaboration was only available for the lead agency in each of the case studies, rather than for every organization in the Health Link, thereby restricting our ability to empirically explore the relationship between intraorganizational and interorganizational integration.

Conclusion and Recommendations

Given the absence of a clear trend across the 6 networks assessed for each case, the interplay between interprofessional and interorganizational integration remains unclear. However both cases indicated perceived improvements in their capacity to care for patients with multiple chronic conditions through access to a wider range of providers and coordination between participating organizations, and better use of their organization’s services through their participation in the Health Links initiative. In terms of network structure the role of the lead agency (a primary care team) plays an important role in influencing network dynamics; and ensuring that the lead agency is set up to model best practices (i.e., regular contact, sharing of resources with other network members etc.) and promote the adoption (or non-adoption) of behaviors for other network members. Results suggest ensuring that the designated lead agency is adequately positioned to influence network functioning, and that organizations with prominent structural positioning (high degree centrality) should be leveraged to disseminate best practices and better optimize for interorganizational collaboration at a provincial scale.

Continuous and on-going interaction between network members plays an important role in building trust as well as social capital, by shaping patterns of interaction over time [67, 84], thus capturing the influence of temporality on network functioning is highly recommended [15, 24, 49, 66]. As such assessing changes in the roles and prominence of different organizations over time can allow providers, policymakers and leadership to adapt resources, structures and/or supports for organizations involved in interorganizational partnerships. And while the use of a network analysis approach to understand the structure of each network is an important first step in exploring how Health Links are operating, a more detailed assessment through interviews or focus groups with organizations in each Health Link, may be helpful to further identify and contextualize the factors that may inform integration efforts in each case. In terms of next steps, incorporating measures of intraorganizational (professional) integration for all network members (instead of the lead agency only), and an evaluation of the impact of network functioning on key patient and system-level outcomes are important next steps to consider.

Additional Files

The additional files for this article can be found as follows:

Appendix 1

Network Analysis Questionnaire. DOI: https://doi.org/10.5334/ijic.5541.s1

Appendix 2

Summary of network measures for two Health Links in Ontario (data collected in 2018). DOI: https://doi.org/10.5334/ijic.5541.s2

Appendix 3

Member perceptions around the benefits and drawbacks of involvement in the Health Links program. DOI: https://doi.org/10.5334/ijic.5541.s3

Acknowledgements

This work was supported by research grants to the Health System Performance Research Network from the Ontario Ministry of Health and Long Term Care (Grant # 06034) and the Ontario SPOR Support Unit, as well as funding from the Canadian Institutes for Health Research (Funding Reference Number TTF-128263). No endorsement by the funding sources is intended or should be inferred. The funding sources for this study had no role in study design, data collection, analysis, or interpretation of data, nor in the writing of the manuscript.

Reviewers

Stefan Essig, MD, PhD, Center for Primary and Community Care, University of Lucerne, Lucerne, Switzerland.

Three anonymous reviewers.

Competing Interests

The authors have no competing interests to declare.

References

  1. Canadian Institute for Health Information. Seniors and the Health Care System: What Is the Impact of Multiple Chronic Conditions? [Internet]; 2011 [cited 2015 May 13]. Available from: https://secure.cihi.ca/estore/productFamily.htm?locale=en&pf=PFC1575. 

  2. Glynn LG, Valderas JM, Healy P, Burke E, Newell J, Gillespie P, et al. The prevalence of multimorbidity in primary care and its effect on health care utilization and cost. Fam Pract [Internet]. 2011 Oct 1 [cited 2017 Apr 13]; 28(5): 516–23. Available from: https://academic.oup.com/fampra/article/28/5/516/822472/The-prevalence-of-multimorbidity-in-primary-care. DOI: https://doi.org/10.1093/fampra/cmr013 

  3. Busse R, Stahl J. Integrated care experiences and outcomes in Germany, the Netherlands, and England. Health Aff (Millwood). 2014 Sep; 33(9): 1549–58. DOI: https://doi.org/10.1377/hlthaff.2014.0419 

  4. Epping-Jordan JE, Pruitt SD, Bengoa R, Wagner EH. Improving the quality of health care for chronic conditions. BMJ Quality & Safety [Internet]. 2004 Aug 1 [cited 2018 Aug 24]; 13(4): 299–305. Available from: https://qualitysafety.bmj.com/content/13/4/299. DOI: https://doi.org/10.1136/qshc.2004.010744 

  5. Kodner DL, Spreeuwenberg C. Integrated care: meaning, logic, applications, and implications – a discussion paper. Int J Integr Care [Internet]. 2002 Nov 14 [cited 2014 Nov 25]; 2. Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1480401/. DOI: https://doi.org/10.5334/ijic.67 

  6. Leijten FRM, Struckmann V, van Ginneken E, Czypionka T, Kraus M, Reiss M, et al. The SELFIE framework for integrated care for multi-morbidity: Development and description. Health Policy [Internet]. 2017 Jun 20; Available from: http://www.sciencedirect.com/science/article/pii/S0168851017301665. 

  7. Nies H. Managing effective partnerships in older people’s services. Health & Social Care in the Community [Internet]. 2006 Sep 1; 14(5): 391–9. Available from: https://onlinelibrary-wiley-com.myaccess.library.utoronto.ca/doi/abs/10.1111/j.1365-2524.2006.00652.x. DOI: https://doi.org/10.1111/j.1365-2524.2006.00652.x 

  8. Ahgren B, Axelsson R. Determinants of integrated health care development: chains of care in Sweden. Journal of Health Planning [Internet]. 2007 Jun [cited 2014 Oct 28]; 22(2): 145–57. Available from: http://ovidsp.ovid.com/ovidweb.cgi?T=JS&CSC=Y&NEWS=N&PAGE=fulltext&D=med5&AN=17623356. DOI: https://doi.org/10.1002/hpm.870 

  9. Ahgren B, Axelsson R. Evaluating integrated health care: a model for measurement. Int J Integr Care [Internet]. 2005 Aug 31 [cited 2016 Jul 12]; 5. Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1395513/. DOI: https://doi.org/10.5334/ijic.134 

  10. Auschra C. Barriers to the Integration of Care in Inter-Organisational Settings: A Literature Review. International Journal of Integrated Care [Internet]. 2018 Jan 16 [cited 2018 Aug 14]; 18(1). Available from: http://www.ijic.org/articles/10.5334/ijic.3068/. DOI: https://doi.org/10.5334/ijic.3068 

  11. Kodner D. All Together Now: A Conceptual Exploration of Integrated Care. Healthcare Quarterly [Internet]. 2009 Oct 16 [cited 2014 Oct 11]; 13(sp): 6–15. Available from: http://www.longwoods.com/content/21091. DOI: https://doi.org/10.12927/hcq.2009.21091 

  12. Valentijn PP, Schepman SM, Opheij W, Bruijnzeels MA. Understanding integrated care: a comprehensive conceptual framework based on the integrative functions of primary care. Int J Integr Care. 2013 Mar; 13: e010. DOI: https://doi.org/10.5334/ijic.886 

  13. Struckmann V, Leijten FRM, van Ginneken E, Kraus M, Reiss M, Spranger A, et al. Relevant models and elements of integrated care for multi-morbidity: Results of a scoping review. Health Policy. 2018; 122(1): 23–35. DOI: https://doi.org/10.1016/j.healthpol.2017.08.008 

  14. van der Heide I, Snoeijs S, Melchiorre M, Quattrini S, Boerma W, Schellevis F, et al. Innovating care for people with multiple chronic conditions in Europe: An overview [Internet]. ICARE4EU; 2015 [cited 2018 May 27]. Available from: https://www.nivel.nl/sites/default/files/bestanden/Rapport-State-of-the-Art-ICARE4EU.pdf. 

  15. Provan KG, Nakama L, Veazie MA, Teufel-Shone NI, Huddleston C. Building Community Capacity Around Chronic Disease Services through a Collaborative Interorganizational Network. Health Educ Behav [Internet]. 2003 Jan 12 [cited 2014 Nov 18]; 30(6): 646–62. Available from: http://heb.sagepub.com/content/30/6/646. DOI: https://doi.org/10.1177/1090198103255366 

  16. Karam M, Brault I, Van Durme T, Macq J. Comparing interprofessional and interorganizational collaboration in healthcare: A systematic review of the qualitative research. International Journal of Nursing Studies [Internet]. 2018 Mar 1; 79: 70–83. Available from: http://www.sciencedirect.com/science/article/pii/S0020748917302559. DOI: https://doi.org/10.1016/j.ijnurstu.2017.11.002 

  17. Lemieux-Charles L, McGuire WL. What Do We Know about Health Care Team Effectiveness? A Review of the Literature. Med Care Res Rev [Internet]. 2006 Jan 6 [cited 2014 Nov 12]; 63(3): 263–300. Available from: http://mcr.sagepub.com/content/63/3/263. DOI: https://doi.org/10.1177/1077558706287003 

  18. Jones N, Thomas P. Inter-organizational Collaboration and Partnerships in Health and Social Care The Role of Social Software. Public Policy and Administration [Internet]. 2007 Jan 7 [cited 2015 May 19]; 22(3): 289–302. Available from: http://ppa.sagepub.com/content/22/3/289. DOI: https://doi.org/10.1177/0952076707078761 

  19. Butler M, Kane RL, McAlpine D, Kathol RG, Fu SS, Hagedorn H, et al. Integration of mental health/substance abuse and primary care. Evid Rep Technol Assess (Full Rep). 2008 Nov; 173: 1–362. 

  20. D’Amour D, Ferrada-Videla M, San Martin Rodriguez L, Beaulieu M-D. The conceptual basis for interprofessional collaboration: Core concepts and theoretical frameworks. Journal of Interprofessional Care [Internet]. 2005 Jan [cited 2014 Oct 23]; 19(s1): 116–31. Available from: http://informahealthcare.com/doi/abs/10.1080/13561820500082529. DOI: https://doi.org/10.1080/13561820500082529 

  21. D’Amour D, Goulet L, Pineault, R, Labadie, J-F, Remondin, M. Comparative study of interorganizational collaboration in four health regions and its effects: the case of perinatal services [Internet]. Canadian Health Services Research. Foundation; 2004 [cited 2015 May 19]. Available from: http://www.cfhi-fcass.ca/Migrated/PDF/ResearchReports/OGC/damour_final_e.pdf. 

  22. Dunlop JM, Holosko MJ. The story behind the story of collaborative networks — relationships do matter! J Health Soc Policy. 2004; 19(3): 1–18. DOI: https://doi.org/10.1300/J045v19n03_01 

  23. Van Eyk H, Baum F. Learning about interagency collaboration: trialling collaborative projects between hospitals and community health services. Health & Social Care in the Community [Internet]. 2002 [cited 2014 Dec 27]; 10(4): 262–9. Available from: http://onlinelibrary.wiley.com/doi/10.1046/j.1365-2524.2002.00369.x/abstract. DOI: https://doi.org/10.1046/j.1365-2524.2002.00369.x 

  24. Ranade W, Hudson B. Conceptual Issues in Inter-Agency Collaboration. Local Government Studies [Internet]. 2003 Aug 1; 29(3): 32–50. Available from: http://resolver.scholarsportal.info/resolve/03003930/v29i0003/32_ciiic.xml. DOI: https://doi.org/10.1080/03003930308559378 

  25. Lawn S, Lloyd A, King A, Sweet L, Gum L. Integration of primary health services: being put together does not mean they will work together. BMC Research Notes [Internet]. 2014 Jan 30 [cited 2015 Jan 3]; 7(1): 66. Available from: http://www.biomedcentral.com/1756-0500/7/66/abstract. DOI: https://doi.org/10.1186/1756-0500-7-66 

  26. Hall P. Interprofessional teamwork: Professional cultures as barriers. Journal of Interprofessional care [Internet]. 2005 [cited 2015 Jun 15]; 19(S1): 188–96. Available from: http://informahealthcare.com/doi/abs/10.1080/13561820500081745. DOI: https://doi.org/10.1080/13561820500081745 

  27. Richardson S, Asthana S. Inter-agency Information Sharing in Health and Social Care Services: The Role of Professional Culture. Br J Soc Work [Internet]. 2006 Jan 6 [cited 2015 Jun 15]; 36(4): 657–69. Available from: http://bjsw.oxfordjournals.org/content/36/4/657. DOI: https://doi.org/10.1093/bjsw/bch257 

  28. Axelsson SB, Axelsson R. From territoriality to altruism in interprofessional collaboration and leadership. J Interprof Care. 2009 Jul; 23(4): 320–30. DOI: https://doi.org/10.1080/13561820902921811 

  29. Ervin NE. Assessing Interagency Collaboration Through Perceptions of Families. Journal of Community Health Nursing [Internet]. 2004; 21(1): 49–60. Available from: http://journals.scholarsportal.info/detailsundefined. DOI: https://doi.org/10.1207/s15327655jchn2101_5 

  30. Goodwin N, Dixon A, Anderson G, Wodchis W. Providing integrated care for older people with complex needs: Lessons from seven international case studies [Internet]; 2011. Available from: http://www.kingsfund.org.uk/sites/files/kf/field/field_publication_file/providing-integrated-care-for-older-people-with-complex-needs-kingsfund-jan14.pdf. 

  31. Kuluski K, Ho JW, Hans PK, Nelson MLA. Community Care for People with Complex Care Needs: Bridging the Gap between Health and Social Care. International Journal of Integrated Care [Internet]. 2017 Jul 21 [cited 2021 Oct 14]; 17(4): 2. Available from: http://www.ijic.org/articles/10.5334/ijic.2944/. DOI: https://doi.org/10.5334/ijic.2944 

  32. Angus H, Greenberg A. Can better care for complex patients transform the health system? Healthc Pap. 2014; 14(2): 9–19. DOI: https://doi.org/10.12927/hcpap.2015.24103 

  33. Huthcinson B, Levesque J-F, Strumpf E, Coyle N. Primary Health Care in Canada: Systems in Motion. Milbank Quarterly. 2011; 89(2): 256–88. DOI: https://doi.org/10.1111/j.1468-0009.2011.00628.x 

  34. Ministry of Health and Long-Term Care. Guide to the Advanced Health Links Model [Internet]; 2015 [cited 2018 Aug 13]. Available from: http://www.health.gov.on.ca/en/pro/programs/transformation/docs/Guide-to-the-Advanced-Health-Links-Model.pdf. 

  35. Rosser WW, Colwill JM, Kasperski J, Wilson L. Progress of Ontario’s Family Health Team Model: A Patient-Centered Medical Home. Ann Fam Med [Internet]. 2011 Jan 3 [cited 2013 Oct 2]; 9(2): 165–71. Available from: http://www.annfammed.org/content/9/2/165. DOI: https://doi.org/10.1370/afm.1228 

  36. Ministry of Health and Long-Term Care. About Health Links [Internet]. news.ontario.ca. 2012 [cited 2017 Dec 8]. Available from: https://news.ontario.ca/mohltc/en/2012/12/about-health-links.html. 

  37. Ministry of Health and Long-Term Care. Bundled Care – Integrated Funding Models [Internet]; 2018 [cited 2018 Dec 28]. Available from: http://www.health.gov.on.ca/en/pro/programs/ecfa/funding/ifm/. 

  38. Wodchis WP, Austin PC, Henry DA. A 3-year study of high-cost users of health care. CMAJ [Internet]. 2015 Jan 1 [cited 2018 Jun 29]; cmaj.150064. Available from: http://www.cmaj.ca/content/early/2016/01/11/cmaj.150064. 

  39. Government of Ontario M of H and L-TC. Healthy Change – Ontario’s Action Plan for Health Care – Public Information – MOHLTC [Internet]. 2015 [cited 2015 May 1]. Available from: http://www.health.gov.on.ca/en/ms/ecfa/healthy_change/healthlinks.aspx. 

  40. Yousefi Nooraie R, Khan S, Gutberg J, Baker GR. A Network Analysis Perspective to Implementation: The Example of Health Links to Promote Coordinated Care. Eval Health Prof [Internet]; 2018 May 2. DOI: https://doi.org/10.1177/0163278718772887 

  41. Khan AI, Barnsley J, Harris JK, Wodchis WP. Examining the extent and factors associated with interprofessional teamwork in primary care settings. Journal of Interprofessional Care [Internet]. 2021 Apr 17 [cited 2021 Apr 18]; 0(0): 1–12. DOI: https://doi.org/10.1080/13561820.2021.1874896 

  42. Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach. BMC Medical Research Methodology [Internet]. 2011 Jun 27 [cited 2014 Dec 17]; 11(1): 100. Available from: http://www.biomedcentral.com/1471-2288/11/100/abstract. DOI: https://doi.org/10.1186/1471-2288-11-100 

  43. Schroder C, Medves J, Paterson M, Byrnes V, Chapman C, O’Riordan A, et al. Development and pilot testing of the collaborative practice assessment tool. Journal of Interprofessional Care [Internet]. 2011 May [cited 2014 Oct 21]; 25(3): 189–95. Available from: http://ovidsp.ovid.com/ovidweb.cgi?T=JS&CSC=Y&NEWS=N&PAGE=fulltext&D=medl&AN=21182434. DOI: https://doi.org/10.3109/13561820.2010.532620 

  44. Bookey-Bassett S, Markle-Reid M, McKey C, Akhtar-Danesh N. A review of instruments to measure interprofessional collaboration for chronic disease management for community-living older adults. J Interprof Care. 2016; 30(2): 201–10. DOI: https://doi.org/10.3109/13561820.2015.1123233 

  45. MacPhee M, Chang L-L, Havaei F, Chou W-S. A Descriptive Account of an Inter-Professional Collaborative Leadership Project. Administrative Sciences [Internet]. 2014 Sep 17 [cited 2016 Feb 25]; 4(3): 373–99. Available from: http://www.mdpi.com/2076-3387/4/3/373. DOI: https://doi.org/10.3390/admsci4030373 

  46. Donnelly C, Shulha L, Klinger D, Letts L. Using program evaluation to support knowledge translation in an interprofessional primary care team: a case study. BMC Family Practice [Internet]. 2016 Oct 6; 17: 142. DOI: https://doi.org/10.1186/s12875-016-0538-4 

  47. Holmesland A-L, Seikkula J, Nilsen Ø, Hopfenbeck M, Erik Arnkil T. Open Dialogues in social networks: professional identity and transdisciplinary collaboration. Int J Integr Care [Internet]. 2010 Sep 16 [cited 2021 Apr 30]; 10. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2948679/. DOI: https://doi.org/10.5334/ijic.564 

  48. Lamontagne M-E. Exploration of the integration of care for persons with a traumatic brain injury using social network analysis methodology. Int J Integr Care [Internet]. 2013 Nov 4 [cited 2014 Nov 18]; 13. Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3821538/. DOI: https://doi.org/10.5334/ijic.1055 

  49. Provan KG, Veazie MA, Teufel-Shone NI, Huddleston C. Network analysis as a tool for assessing and building community capacity for provision of chronic disease services. Health Promot Pract. 2004 Apr; 5(2): 174–81. DOI: https://doi.org/10.1177/1524839903259303 

  50. Blanchet K, James P. How to do (or not to do) … a social network analysis in health systems research. Health Policy Plan [Internet]. 2012 Aug 1 [cited 2018 Aug 28]; 27(5): 438–46. Available from: https://academic-oup-com.myaccess.library.utoronto.ca/heapol/article/27/5/438/752411. DOI: https://doi.org/10.1093/heapol/czr055 

  51. Brass DJ, Galaskiewicz J, Greve HR, Tsai W. Taking stock of networks and organizations: A multilevel perspective. Academy of Management journal Academy of Management [Internet]. 2004 Dec [cited 2018 Aug 28]; 47(6): 795–817. Available from: https://arizona.pure.elsevier.com/en/publications/taking-stock-of-networks-and-organizations-a-multilevel-perspecti. DOI: https://doi.org/10.5465/20159624 

  52. Morrissey JP, Johnsen MC, Calloway MO. Evaluating performance and change in mental health systems serving children and youth: an interorganizational network approach. J Ment Health Adm. 1997; 24(1): 4–22. DOI: https://doi.org/10.1007/BF02790476 

  53. Harris JK, Luke DA, Burke RC, Mueller NB. Seeing the forest and the trees: using network analysis to develop an organizational blueprint of state tobacco control systems. Social Science & Medicine [Internet]. 2008 [cited 2015 Oct 16]; 67(11): 1669–78. Available from: http://www.sciencedirect.com/science/article/pii/S0277953608003602. DOI: https://doi.org/10.1016/j.socscimed.2008.07.013 

  54. Loitz CC, Stearns JA, Fraser SN, Storey K, Spence JC. Network analysis of inter-organizational relationships and policy use among active living organizations in Alberta, Canada. BMC Public Health [Internet]. 2017 Aug 9; 17(1): 649. DOI: https://doi.org/10.1186/s12889-017-4661-5 

  55. Schoen MW, Moreland-Russell S, Prewitt K, Carothers BJ. Social network analysis of public health programs to measure partnership. Soc Sci Med. 2014 Oct 30; 123C: 90–5. DOI: https://doi.org/10.1016/j.socscimed.2014.10.057 

  56. Huisman M, Steglich C. Treatment of non-response in longitudinal network studies. Social networks [Internet]. 2008 [cited 2016 Feb 28]; 30(4): 297–308. Available from: http://www.sciencedirect.com/science/article/pii/S0378873308000270. DOI: https://doi.org/10.1016/j.socnet.2008.04.004 

  57. Huisman M. Imputation of missing network data: Some simple procedures. Journal of Social Structure [Internet]. 2009 [cited 2019 Jan 5]; 10(1). Available from: https://www.rug.nl/research/portal/publications/imputation-of-missing-network-data(ba5942d8-15b0-4540-9ded-2b313585360a)/export.html. 

  58. Huisman M, Krause R. Imputation of missing network data. In: Encyclopedia of Social Network Analysis and Mining. Springer. 2017; 1–10. DOI: https://doi.org/10.1007/978-1-4614-7163-9_394-1 

  59. Valente TW, Palinkas LA, Czaja S, Chu K-H, Brown CH. Social Network Analysis for Program Implementation. PLOS ONE [Internet]. 2015 Jun 25 [cited 2018 Aug 29]; 10(6): e0131712. Available from: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0131712. DOI: https://doi.org/10.1371/journal.pone.0131712 

  60. Petrescu-Prahova M, Belza B, Leith K, Allen P, Coe NB, Anderson LA. Using Social Network Analysis to Assess Mentorship and Collaboration in a Public Health Network. Prev Chronic Dis [Internet]. 2015 Aug 20; 12. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4565512/. DOI: https://doi.org/10.5888/pcd12.150103 

  61. Valente TW, Chou CP, Pentz MA. Community Coalitions as a System: Effects of Network Change on Adoption of Evidence-Based Substance Abuse Prevention. Am J Public Health [Internet]. 2007 May; 97(5): 880–6. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1854884/. DOI: https://doi.org/10.2105/AJPH.2005.063644 

  62. Harris JK, Provan KG, Johnson KJ, Leischow SJ. Drawbacks and benefits associated with inter-organizational collaboration along the discovery-development-delivery continuum: a cancer research network case study. Implement Sci. 2012 Jul 25; 7: 69. DOI: https://doi.org/10.1186/1748-5908-7-69 

  63. Valente TW, Coronges KA, Stevens GD, Cousineau MR. Collaboration and competition in a children’s health initiative coalition: A network analysis. Evaluation and Program Planning [Internet]. 2008 Nov 1; 31(4): 392–402. Available from: http://www.sciencedirect.com/science/article/pii/S0149718908000499. DOI: https://doi.org/10.1016/j.evalprogplan.2008.06.002 

  64. Kossinets G. Effects of missing data in social networks. Social networks [Internet]. 2006 [cited 2015 Apr 4]; 28(3): 247–68. Available from: http://www.sciencedirect.com/science/article/pii/S0378873305000511. DOI: https://doi.org/10.1016/j.socnet.2005.07.002 

  65. Žnidaršič A, Ferligoj A, Doreian P. Non-response in social networks: The impact of different non-response treatments on the stability of blockmodels. Social Networks. 2012 Oct; 34(4): 438–50. DOI: https://doi.org/10.1016/j.socnet.2012.02.002 

  66. Provan KG, Veazie MA, Staten LK, Teufel-Shone NI. The Use of Network Analysis to Strengthen Community Partnerships. Public Administration Review [Internet]. 2005 Sep 1 [cited 2014 Nov 18]; 65(5): 603–13. Available from: http://onlinelibrary.wiley.com/doi/10.1111/j.1540-6210.2005.00487.x/abstract. DOI: https://doi.org/10.1111/j.1540-6210.2005.00487.x 

  67. Dobbels L, Voets J, Marlier M, De Waegeneer E, Willem A. Why network structure and coordination matter: A social network analysis of sport for disadvantaged people. International Review for the Sociology of Sport [Internet]. 2018; 53(5): 572–93. Available from: http://journals.scholarsportal.info/detailsundefined. DOI: https://doi.org/10.1177/1012690216666273 

  68. Lee H-W, Robertson P, Lewis L, Sloane D, Galloway-Gilliam L, Nomachi J. Trust in a Cross-Sectoral Interorganizational Network: An Empirical Investigation of Antecedents. Nonprofit and Voluntary Sector Quarterly [Internet]. 2012; 41(4): 609–31. Available from: http://journals.scholarsportal.info/detailsundefined. DOI: https://doi.org/10.1177/0899764011414435 

  69. Heijmans N, van Lieshout J, Wensing M. Information exchange networks of health care providers and evidence-based cardiovascular risk management: an observational study. Implement Sci [Internet]. 2017 Jan 13; 12. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5237141/. DOI: https://doi.org/10.1186/s13012-016-0532-1 

  70. Mascia D, Di Vincenzo F, Iacopino V, Fantini MP, Cicchetti A. Unfolding similarity in interphysician networks: the impact of institutional and professional homophily. BMC Health Serv Res [Internet]. 2015 Mar 10; 15. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4365537/. DOI: https://doi.org/10.1186/s12913-015-0748-9 

  71. Luke DA, Wald LM, Carothers BJ, Bach LE, Harris JK. Network Influences on Dissemination of Evidence-Based Guidelines in State Tobacco Control Programs. Health Educ Behav [Internet]. 2013 Oct 1; 40(1_suppl): 33S–42S. DOI: https://doi.org/10.1177/1090198113492760 

  72. Mueller N, Krauss M, Luke D. Interorganizational Relationships Within State Tobacco Control Networks: A Social Network Analysis. Prev Chronic Dis [Internet]. 2004 Sep 15; 1(4). Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1277948/. 

  73. Alsamadani R, Hallowell M, Javernick-Will AN. Measuring and modelling safety communication in small work crews in the US using social network analysis. Construction Management & Economics [Internet]. 2013 Jun; 31(6): 568–79. Available from: http://search.ebscohost.com/login.aspx?direct=true&db=buh&AN=90134720&site=ehost-live. DOI: https://doi.org/10.1080/01446193.2012.685486 

  74. Bdeir F, Crawford JW, Hossain L. Informal Networks in Disaster Medicine. Disaster Medicine and Public Health Preparedness [Internet]. 2017 Jun [cited 2020 May 16]; 11(3): 343–54. Available from: https://www.cambridge.org/core/journals/disaster-medicine-and-public-health-preparedness/article/informal-networks-in-disaster-medicine/3B2C90D3D72F568B7DFCF04724F48BC6. DOI: https://doi.org/10.1017/dmp.2016.142 

  75. Johnson JD, Donohue WA, Atkin CK, Johnson S. Differences Between Formal and Informal Communication Channels. The Journal of Business Communication (1973) [Internet]. 1994 Apr 1; 31(2): 111–22. Available from: https://journals.sagepub.com/doi/abs/10.1177/002194369403100202. DOI: https://doi.org/10.1177/002194369403100202 

  76. West RM, House AO, Keen J, Ward VL. Using the structure of social networks to map inter-agency relationships in public health services. Soc Sci Med. 2015 Nov; 145: 107–14. DOI: https://doi.org/10.1016/j.socscimed.2015.10.002 

  77. Singer HH, Kegler MC. Assessing Interorganizational Networks as a Dimension of Community Capacity: Illustrations From a Community Intervention to Prevent Lead Poisoning. Health Educ Behav [Internet]. 2004 Dec 1; 31(6): 808–21. DOI: https://doi.org/10.1177/1090198104264220 

  78. Provan KG, Harvey J, de Zapien JG. Network structure and attitudes toward collaboration in a community partnership for diabetes control on the US-Mexican border. J Health Organ Manag. 2005; 19(6): 504–18. DOI: https://doi.org/10.1108/14777260510629706 

  79. Shaw-Ching Liu B, Madhavan R, Sudharshan D. DiffuNET: The impact of network structure on diffusion of innovation. European Journal of Innovation Management. 2005; 8(2): 240–62. DOI: https://doi.org/10.1108/14601060510594701 

  80. Colvin ML. Mapping the inter-organizational landscape of child maltreatment prevention and service delivery: A network analysis. Children and Youth Services Review [Internet]. 2017; 73(Complete): 352–9. Available from: http://journals.scholarsportal.info/detailsundefined. DOI: https://doi.org/10.1016/j.childyouth.2017.01.003 

  81. Fujimoto K, Valente TW, Pentz MA. Network structural influences on the adoption of evidence-based prevention in communities. J Community Psychol [Internet]. 2009 [cited 2015 Jan 17]; 37(7): 830–45. Available from: http://onlinelibrary.wiley.com/doi/10.1002/jcop.20333/abstract. DOI: https://doi.org/10.1002/jcop.20333 

  82. Rijke M, Struckmann V, Heide I, Hujala A, Barbabella F, Ginneken E, et al. How to improve care for people with multimorbidity in Europe? [Internet]. European Observatory on Health Systems and Policies; 2017 [cited 2018 Dec 24]. (Health systems and policy analysis). Report No.: Policy Brief 23. Available from: http://www.euro.who.int/__data/assets/pdf_file/0004/337585/PB_23.pdf?ua=1. 

  83. Ministry of Health and Long-Term Care. Improving Care for High-Needs Patients [Internet]. news.ontario.ca; 2012 [cited 2018 Aug 30]. Available from: https://news.ontario.ca/mohltc/en/2012/12/improving-care-for-high-needs-patients.html. 

  84. Luque J, Tyson DM, Lee J-H, Gwede C, Vadaparampil S, Noel-Thomas S, et al. Using social network analysis to evaluate community capacity building of a regional Community Cancer Network. Journal of Community Psychology [Internet]. 2010 Jul 1; 38(5): 656–68. Available from: https://onlinelibrary-wiley-com.myaccess.library.utoronto.ca/doi/abs/10.1002/jcop.20386. DOI: https://doi.org/10.1002/jcop.20386