(C) PLOS One
This story was originally published by PLOS One and is unaltered.
. . . . . . . . . .



Mapping clinical interactions in an Australian tertiary hospital emergency department for patients presenting with risk of suicide or self-harm: Network modeling from observational data [1]

['Michael H. Mccullough', 'School Of Computing', 'The Australian National University', 'Acton', 'Act', 'Eccles Institute Of Neuroscience', 'John Curtin School Of Medical Research', 'Michael Small', 'Complex Systems Group', 'Department Of Mathematics']

Date: 2024-01

This study demonstrates a data-driven research framework, new to the best of our knowledge, to assess and reinforce important information pathways that guide clinical decision processes and provide complementary insights for improving clinical practice and operational models in emergency medicine for patients at risk of suicide or self-harm. Our findings suggest that PLNs can play a crucial role in clinical communication, but overreliance on PLNs may pose risks to reliable information flow. Operational models that utilize PLNs may be made more robust to these risks by improving interdisciplinary communication between doctors. Our research framework could also be applied more broadly to investigate service delivery in different healthcare settings or for other medical specialties, patient groups, or demographics.

We deployed 2 observers in a tertiary hospital emergency department during 2018 for a total of 118.5 h to record clinical interactions along patient trajectories for presentations with risk of self-harm or suicide (n = 272 interactions for n = 43 patient trajectories). The study population was reflective of a naturalistic sample of patients presenting to a tertiary emergency department in a metropolitan Australian city. Using the observational data, we constructed a clinical interaction network to model the flow of clinical information at a systems level. Community detection via modularity maximization revealed communities in the network closely aligned with the underlying clinical team structure. The Psychiatric Liaison Nurse (PLN) was identified as the most important agent in the network as quantified by node degree, closeness centrality, and betweenness centrality. Betweenness centrality of the PLN was significantly higher than expected by chance (>95th percentile compared with randomly shuffled networks) and removing the PLN from the network reduced both the global efficiency of the model and the closeness centrality of all doctors. This indicated a potential vulnerability in the system that could negatively impact patient care if the function of the PLN was compromised. We developed an algorithmic strategy to mitigate this risk by targeted strengthening of links between clinical teams using greedy cumulative addition of network edges in the model. Finally, we identified specific interactions along patient trajectories which were most likely to precipitate a psychiatric referral using a machine learning model trained on features from dynamically constructed clinical interaction networks. The main limitation of this study is the use of nonclinical information only (i.e., modeling is based on timing of interactions and agents involved, but not the content or quantity of information transferred during interactions).

Reliable assessment of suicide and self-harm risk in emergency medicine is critical for effective intervention and treatment of patients affected by mental health disorders. Teams of clinicians face the challenge of rapidly integrating medical history, wide-ranging psychosocial factors, and real-time patient observations to inform diagnosis, treatment, and referral decisions. Patient outcomes therefore depend on the reliable flow of information through networks of clinical staff and information systems. This study aimed to develop a quantitative data-driven research framework for the analysis of information flow in emergency healthcare settings to evaluate clinical practice and operational models for emergency psychiatric care.

Funding: MM, MS, BJ and SH received funding for this research from The University of Western Australia Young Lives Matter Foundation (UWA YLM) which was a cross-disciplinary research initiative active from 2018 to 2021. UWA YLM supported research at The University of Western Australia through internal funding schemes ( https://www.research.uwa.edu.au/ ). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

We addressed these issues by developing a data-driven research framework to analyze patient trajectories using approaches from network science and machine learning. Specifically, the objective of this investigation was to build systems-level network models that map patient trajectories and clinical information flow for presentations with suspected risk of suicide or self-harm. Using these models, we sought to determine whether aspects of clinical practice or operational structure in the ED could affect the flow of clinical information in ways that could negatively impact patient outcomes, and how potential risks could be mitigated. Furthermore, we explored patient trajectories as dynamic networks to understand the processes of clinical decision-making and referrals as they occur in practice. By quantifying the characteristics of patient trajectories in the ED and the patterns of communication and interaction that drive them, this study aims to provide a research framework to evaluate clinical practice and operational models to guide process improvement and enhance treatment outcomes for patients with suicide or self-harm risk.

While clinical management of suicide-risk patients in emergency settings has been prioritized in many national and international suicide prevention strategies [ 20 ] existing research into urgent emergency care models for mental health patients is limited, with a lack of evidence-based models in the literature [ 12 ]. Notably, patient journeys through EDs for mental health presentations are not well understood [ 21 ] and a systematic review of research related to patient experiences in emergency care after self-harm found that there were only a few studies that attempted to describe the care pathway and interactions with medical professionals in detail [ 20 ]. Multiple studies also report considerable inconsistencies and discrepancies in the clinical practice guidelines and service delivery models for emergency mental health care in the United States of America, the United Kingdom, Australia, and New Zealand [ 11 , 22 – 26 ] including a systematic review by Bernert and colleagues [ 27 ]. As a result, there have been calls for further research into methods for monitoring and evaluating the implementation of clinical practice guidelines to improve patient experience and treatment outcomes [ 11 , 20 , 28 ].

Mental health crisis presentations account for 4% to 10% of ED presentations [ 12 ] and are growing in number [ 13 – 16 ]. This increases strain on EDs [ 13 ] and impacts patient flow because mental health presentations typically take more time to assess and staff often report feeling ill-equipped to care for these patients [ 12 ]. Further, EDs are widely understood to be challenging environments for patients affected by mental health issues for reasons including long wait times, noise, lack of privacy, harsh lighting, and negative attitudes of staff [ 17 , 18 ]. These and other factors result in a predominantly negative experience of acute care settings for mental health patients [ 19 ]. This is particularly problematic for patients affected by suicidal behaviors because negative experiences of treatment may increase self-harm risk [ 20 ].

It has been estimated that as many as 77% of individuals who die by suicide will have made contact with a primary care provider in the year prior to their death [ 4 , 5 ], and up to 10% to 20% will have visited an emergency department (ED) within 1 to 2 months prior [ 2 , 6 , 7 ]. EDs are an important and often primary point of access for mental health support services [ 8 – 10 ] and therefore provide an opportunity for suicide-risk screening and prevention [ 2 ]. However, the population of individuals affected by suicidal behaviors is highly heterogeneous and poses significant challenges for risk assessment and clinical management, especially in emergency settings [ 11 ].

Suicide is a major global public health issue causing over 700,000 deaths per year, often with far-reaching impacts on families and communities that can persist well-beyond each individual tragedy [ 1 , 2 ]. In addition, the prevalence of suicide and suicidal ideation creates considerable economic burden for society (estimated at over US$90 billion in the United States of America alone in 2013 [ 3 ]) and has been linked to increasing healthcare costs [ 2 ].

Methods

Ethics statement This study was approved as a Quality Activity by the Government of Western Australia Department of Health North Metropolitan Health Service and received Ethics Approval in accordance with the Human Research Ethics Committee Protocol for Quality Activities. Consent for study participation was obtained verbally. In preparation for the study, the observation team was carefully selected to include a medical practitioner in psychiatry who was familiar with the working environment and experienced in the sensitive assessment of patients presenting with suicidal ideation in the emergency setting. A consultation process took place between the research team and emergency physicians in the department. An understanding was reached whereby any incident of concern that was uncovered during data collection would be conveyed to the departmental leadership in real time.

Data collection Data were collected at Sir Charles Gairdner Hospital, which is a tertiary hospital in the city of Perth, Western Australia. A team of 2 observers worked simultaneously to record instances of clinical interactions for patients presenting with risk of suicide or self-harm. The observation team comprised a Psychiatry Registrar and a postdoctoral research scientist. Potential participants were assessed for inclusion in the study upon presentation to the ED triage when clerked into the Emergency Department Information System (EDIS). Patients were considered suitable for inclusion if suicidal ideation or self-harm was listed in EDIS as a presenting complaint. Patients were consented for participation in this study prospectively as they met criteria for inclusion. Consent was sought at the earliest possible opportunity during which no active clinical interaction or assessment was taking place. Participant information approved by the Research Ethics Committee Chair was conferred to the participants by way of a discussion, and their verbal consent was obtained prior to each patient trajectory being included. Participants were informed that no identifying data about themselves, their condition or their treatment would be recorded as part of the study design. Participants were made aware that participation in the study, or refusal thereof would have no bearing on the clinical care they would receive and that they could revoke this consent at any time. Participants who were intoxicated on entering the ED were retrospectively consented once no longer intoxicated. Observers collected data for each study participant by recording instances of clinical interactions along the patient’s trajectory through the ED. This included the mode of presentation, date and time of each interaction, the agents involved in each interaction (e.g., patient, associates of the patient, clinical staff, support and community health services, or emergency services), patient records or information systems accessed during the interaction, and actions resulting from an interaction such as referrals or discharge. Data were recorded in an event log developed specifically for this study (Table 1). The team of observers combined and cross-checked observations in the event log regularly during data collection to reduce the likelihood of data errors. PPT PowerPoint slide

PNG larger image

TIFF original image Download: Table 1. Example event log: Data were collected in the form of an event log as illustrated by this table. Shown here are partial trajectories for 2 patients. Each row corresponds to an observed interaction, listed in chronological order. For each patient, we recorded the method of presentation. For each subsequent interaction, we recorded the date and time of the interaction, the agents involved (i.e., Patient, Triage Nurse, Consultant), and any information systems accessed during the interaction (i.e., reading from or writing to the Patient File, accessing digital patient record databases, completing risk assessment forms). We also recorded actions that took place as a result of interactions (i.e., referrals or discharge). https://doi.org/10.1371/journal.pmed.1004241.t001 Efforts were made to protect patient confidentiality as far as possible. The data collected were strictly nonclinical information only. Date information in the data was altered to minimize the possibility of retrospective data linkage. Interactions between patients and physicians were observed from a sufficient distance such that clinical conversations could not be heard. The researchers were never present in a confined environment that would have compromised physician–patient privilege. There was minimal impact on routine clinical processes. In instances where it was not clear if an interaction pertained to a study participant (e.g., a phone call or discussion between clinical staff), or involved access of one or more information systems (e.g., use of a computer terminal), clinical staff were approached at an appropriate time and briefly interviewed to establish the relevant details. Observations were taken for a total of 101 h in the ED over a period of approximately 1 month spanning July and August in the year 2018, including 40 h during day shifts, 40 h during night shifts, and 21 additional hours of observations during day shifts specifically focused around the ED Psychiatry team. Observation hours were distributed between weekdays and weekends and were generally undertaken in 8-h shifts. A total of n = 213 interactions were observed from n = 36 patient trajectories in the ED. Further observations were undertaken at the Sir Charles Gairdner Hospital Mental Health Observational Area (MHOA). The MHOA is an 8-bed short-stay psychiatric unit located physically adjacent to the ED and staffed by a multidisciplinary team. Its model of care emphasizes acute presentations that are expected to resolve within a 24- to 72-h duration of admission, or for patients whose disposition to a traditional acute psychiatric unit can be confirmed during that time. The MHOA receives patients from the ED, its sole admission pathway. ED Psychiatry and MHOA staff operate under the same clinical governance for all mental health staff on the campus. The MHOA was observed for a total of 17.5 h over 5 weekdays. A total of n = 59 interactions were observed from n = 7 patient trajectories in the MHOA. The study population was reflective of a naturalistic sample of patients presenting to a tertiary ED in a metropolitan Australian city. No demographic details were captured in keeping with the study design. We refer readers to a study by Dragovic and colleagues [29] for baseline population characteristics of mental health presentations to tertiary hospitals in the same district operated by the Western Australia Department of Health North Metropolitan Health Service for the period of 25 January to 1 May 2019 (within 1 year after data collection for the present study).

Data analysis and software Data analysis were performed in Python using the packages Numpy [30], Pandas [31], NetworkX [32], Scikit-Learn [33], and Imbalanced-Learn [34]. Figures and data visualizations were prepared using Matplotlib [35] and NetworkX.

Patient trajectory network and basic trajectory statistics To map possible trajectories from the point of presentation to referral or discharge, we constructed a patient trajectory network using only the observational data. We defined nodes in the network representation of patient trajectories as the clinical team treating the patient, modes of presentation, and modes of discharge. An unweighted and directed edge was assigned between a pair nodes if we observed at least 1 instance of that transition in our data (i.e., a patient being moved from the ED to the Observation Ward). The combined data from all ED and MHOA observations were used to construct the patient trajectory network. When computing the presentation type, trajectory time, interactions, and clinical staff types, we excluded patients that were observed only in the MHOA. We excluded this data because the MHOA served a different function than the other areas of the ED, providing a specialized short-stay environment for observation of at-risk mental health patients. This exclusion was also applied in the subsequent analysis of the clinical interaction network.

Clinical interaction network To investigate the flow of clinical information in the ED, we constructed a network model of the interactions between agents (i.e., the patient, doctors, or nurses) and clinical information systems observed along the combined set of patient trajectories. Minimal assumptions were imposed only to ensure that nonrealistic interactions were not erroneously included in the network (e.g., a hard-copy Patient File cannot directly interact with a digital records database). The network model was otherwise constructed entirely from the observational data. We then estimated the importance of individual agents and the overall efficiency of the system with respect to information flow based on quantitative measures of connectivity patterns in the network model, detailed as follows. We define the clinical interaction network as comprising a set of nodes , where each node i corresponds to one of the total possible agents or information systems. The network is represented by an n by n adjacency matrix A. Elements of A are given by a i,j ∈{0,1}, where a i,j = 1 implies an unweighted bidirectional edge between nodes i and j. An edge was assigned between nodes i and j if and only if an interaction between the corresponding pair of agents or information systems occurred at least once in the combined data from all patients. The network edges can therefore be interpreted as communication channels for clinical information that would inform patient diagnosis and treatment. The operational policy of a hospital imparts intrinsic structure in the network that is not directly reflected in the event log data. For example, a recorded interaction may have involved the Patient, a Nurse, and the Patient File (which the Nurse may have read or appended information to). However, a Patient File is never accessed directly by the Patient. Therefore, edges between nodes corresponding to the Patient and the Patient File are considered forbidden and are excluded from the clinical interaction network by definition. In this study, forbidden edges included those (a) between clinical information systems; (b) between clinical information systems and agents that were not clinical staff at the hospital; (c) between the Psychiatric Services Online Information System (PSOLIS) or ED Psychiatry Handover Document (EDYHO), which are clinical information systems specific to psychiatry, and any agents that were not part of the Emergency Psychiatry team. We rendered the visualization of the clinical interaction network using the spring_layout function from NetworkX [32] that produces a force-directed graph layout. To detect community structure in the network, we applied a greedy modularity maximization algorithm [36]. To estimate the importance of agents in the network, node centrality measures were computed based on the definitions by Newman [37], as briefly summarized here in Eqs (1) to (3). The degree centrality of a node is the number of edges connected to that node. The degree centrality of node i was computed as follows: (1) Closeness centrality is the inverse of the average distance from a given node to all other nodes in the network. The closeness centrality of node i was computed as follows: (2) where d i,j is the length of the shortest path on the network between nodes i and j. The betweenness centrality of a node measures how often that node forms part of a path between other pairs of nodes. The clinical interaction network models the flow of information between agents. In this context, high betweenness would suggest that a node is important for passing information between other agents or different communities in the network. If a node with high betweenness becomes compromised this is likely to adversely impact the flow of information through the network more than for a node with low betweenness. The betweenness centrality of node i was computed as follows: (3) where σ(j,k|i) equals the number of shortest paths between nodes j and k which pass through node i, and σ(j,k) equals the total number of shortest paths between nodes j and k. This particular variant of betweenness centrality is described in [38]. We further normalized b i by the total number of possible paths through node i [39].

Network vulnerability analysis We applied a random shuffle algorithm to assess the degree to which the structure in our observed network was due to inherent structure in the data rather than randomness. The principle is that we generated an ensemble of networks which appeared similar to our clinical interaction network (they have the same number of nodes, node degrees, etc.) but were otherwise random. We then sought to answer the question of whether the observed clinical interaction network was different from random—and if so, how? Specifically, random shuffling of the clinical interaction network was performed using a connected double-edge swap algorithm [40] to preserve local and global degree structure. This algorithm begins by randomly selecting 2 pairs of nodes (i,j) and (v,u) such that the nodes within each pair are connected (i.e., a (i,j) = a (v,u) = 1). The edges are then swapped so that the network has 2 new connected node pairs (i,v) and (j,u). This swap is only performed if: (a) the edges for new node pairs (i,v) and (j,u) did not already exist in the network; and (b) the network remains connected after the swap. If these conditions are not met, the edges for these 2 node pairs are left unchanged and the algorithm proceeds to attempt a swap with a different randomly selected set of node pairs. For this study, we imposed a further constraint that edges could only be swapped if the resulting network remained free of forbidden edges as defined for the clinical interaction network (henceforth referred to as the constrained connected double-edge swap algorithm). We generated 1,000 shuffled networks from independent sequences of random edge swaps to assess the likelihood of the observed network configuration. For each shuffle, we attempted 20,000 connected double-edge swaps, of which approximately 1,900 swaps were successful on average. We assessed network vulnerability based on changes in closeness centrality and global efficiency [41] when a potentially vulnerable node was removed. Global efficiency measures how efficiently information propagates on a network. Assuming that efficiency of information flow between a pair of nodes i and j is inversely proportional to the shortest path between them d i,j , the global efficiency is the average over all node pairs, computed as follows: (4) To investigate strategies to mitigate against the adverse effects of a compromised node, we developed a simple greedy algorithm for the targeted addition of edges to increase global efficiency, as follows: (a) remove the compromised node from the network; (b) for each edge in a set of candidate edges, add the edge to the network which maximizes the increase in global efficiency; (c) repeat the previous step until all edges from the candidate set are added to the network. If a tie is encountered with respect to the increase in global efficiency, then this greedy algorithm is no longer guaranteed to find an optimal sequence for the addition of edges. Therefore, once a tie occurred we tested all possible permutations for the sequence of the remaining edges that had not yet been added. This allowed us to enumerate the complete set of optimal solutions. For our data and the specific set of candidate edges investigated in this study, there was a subset of edges that were tied with respect to their contributions to global efficiency, regardless of the order in which they were added to the network.

Machine learning for predicting clinical referrals To assess which agents and interactions were likely to precipitate a referral to the Emergency Psychiatry team, we trained a machine learning model to predict the referral point based on the evolving state of clinical interaction networks along individual patient trajectories. We call this the referral prediction model. The model was investigated to identify the features of the network (i.e., nodes and edges) that were most predictive of referral. We used machine learning for this task as a way of extracting structural information from the underlying data, independent of our own application driven bias. The machine learning algorithm was agnostic to our knowledge of the system and was simply applied to extract significant structural patterns from the data. The referral prediction model was trained on data from trajectories for n = 20 patients who were referred from the Emergency Medical team to the Emergency Psychiatry team during the period of observation. A dynamic clinical interaction network A p,t was iteratively constructed along the trajectory for each patient p and observation number t starting with an empty network A p,0 . For each interaction along the trajectory, we added nodes and edges for the corresponding agents if they did not already exist in the network. This process continued up to and including the interaction which precipitated referral. The final state of A p,t along the trajectory was assigned a positive class label y p,t = 1, delineating the referral point. All other states of A p,t were assigned the class label y p,t = 0. Each state of A p,t was mapped to a 1-dimensional binary feature vector: (5) where v i is a Boolean variable representing the existence of node i in the network at time t for patient p, and a i,j is the Boolean variable representing the existence of an edge between nodes i and j. The indices i and j correspond to those for the complete interaction network A. The features X p,t and labels y p,t were then used to train a Bernoulli naive Bayes classifier with Laplace smoothing [42] to predict the referral point based on the dynamic network state. This machine learning algorithm was selected for the referral prediction model because it takes binary feature vectors as input, which matched the structure of our dynamic network state vector X p,t , and has an intuitive probabilistic interpretation. Briefly, this machine learning algorithm estimates the probability P(y|X) for class y and multivariate input feature vector X, under the assumption that each input feature is independent for a given class label. For example, in the context of our application the model assumes that the involvement of one agent or interaction in a patient trajectory occurred independently of any other. This simplifying assumption makes the estimation of P(y|X) more tractable in the case of high-dimensional data and a limited sample size, which would otherwise require a very large sample of observations to accurately estimate probabilities for the vast number of possible input vectors (up to different possibilities for our dynamic network state vector X p,t ). For these reasons, the Bernoulli naive Bayes classifier is well-suited for the psychiatric patient trajectory data gathered in this study, which are intrinsically high-dimensional and highly variable. We hypothesized that some features of X p,t would be more predictive than others and that it might be informative to identify the agents or interactions corresponding to these features. To investigate this, we used permutation feature importance [43] that quantifies the contribution of each feature in the model by measuring the change in a scoring metric when the data for that feature are randomly permuted. We used balanced accuracy [33] as the scoring metric because class labels are highly imbalanced—positive class labels (referral) typically only occur once in a patient trajectory through the ED and only account for 23% of the data (20 out of n = 87 interactions in X p,t for our data). The balanced accuracy scoring metric is given by: (6) where TP, FP, TN, and FN are the number of true positives, false positives, true negatives, and false negatives in the test data, respectively. We estimated the permutation feature importance for 10,000 randomly re-sampled 80:20 train/test splits of the data. Data were grouped such that observations from any given patient trajectory could not be split between the training and test sets. To avoid bias in the model due to highly imbalanced class labels, we used random over-sampling of the minority class to balance the data in each training split [34].

[END]
---
[1] Url: https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1004241

Published and (C) by PLOS One
Content appears here under this condition or license: Creative Commons - Attribution BY 4.0.

via Magical.Fish Gopher News Feeds:
gopher://magical.fish/1/feeds/news/plosone/

You are viewing proxied material from magical.fish. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.