July 12, 2021 Dr. Nazrul Islam

1

July 12, 2021

Dr. Nazrul IslamResearch EditorThe BMJ BMA House, Tavistock SquareLondon, WC1H 9JR, UK

RE: Resubmission of BMJ-2021-064099 entitled "Effect of Post discharge after surgery Virtual Care with Remote Automated Monitoring technology versus standard care: The PVC-RAM-1 randomised controlled trial"

Dear Dr. Islam:

Thank you for inviting us to resubmit our paper. Below is an itemized summary of the editors’ and reviewers’ comments, followed by our responses. To facilitate your review, in this response letter all manuscript text is italicised and additions to the manuscript are highlighted in red. We are submitting two versions of our paper and accompanying documents (i.e., versions with track changes and clean versions), unless no change was made to a document.

As per your request, we had centres obtain 6-month data on days alive at home. While obtaining these data a few additional outcomes at 30 days after randomization were identified. These events did not materially affect the results at 30 days after randomization. Due to installation of an upgraded electronic medical record system at one site, an unanticipated time lag occurred in recording of surgeon, family physician, and specialist in-person or virtual clinic visits at 30 days after randomization. The site investigator recognized this issue while obtaining the 6-month data. The updated data resulted in an 8% absolute change in the proportion of patients who had a surgeon, family physician, and specialist in-person or virtual clinic visit at 30 days after randomization; however, there remained no difference between the two treatment groups for this outcome.

Editors’ comments:

1. The trial protocol specifies that the primary and secondary outcomes would be evaluated at 6-months. In that regard, the trial is, in principle, not yet completed. Please add the 6-month outcomes evaluation to the manuscript.

Response: When Dr. Devereaux spoke with Dr. Islam, he explained that there is a lag-time in obtaining the 6-month administrative data, whereby we will not obtain these data until sometime in the fall of 2021. Subsequent to this conversation you stated the following in an email on April 6, 2021. “I have discussed the issue re: the 6-month outcome with two of our senior editors who specialise in the RCTs. They suggested that we can be flexible around the secondary outcomes but we would need to have the 6-month data for the primary outcome.” Based on this correspondence, we have now

2

obtained the 6-month data for the outcome days alive at home (i.e., the 30-day primary outcome) through active follow-up and the associated outcomes that make up this outcome (e.g., acute-hospital care). In the Results Section, we have added the following text on page 14, paragraph 2.

Table 9, Supplement 3 reports the effects of virtual care with RAM on tertiary 6-month outcomes. There was no impact on days alive at home at 6 months.

In the Online-Only Supplement 3, we have added the following table on page 38.

3

Table 9: Effects of virtual care and remote automated monitoring on tertiary 6-month outcomes

Outcome Virtual-caregroup

(N=451)

Standard-caregroup

(N=454)

Relative risk*(95% CI)

Absolute difference†

% (95% CI)

P Value#

Days alive at home – mean (± SD)Acute-hospital care – no. (%) Hospital re-admission – no. (%)Emergency department visit – no. (%)Urgent-care centre visit – no. (%)All-cause hospital days (median [IQR])Death – no. (%)

176.7 (25.5)170 (37.7)101 (22.4)151 (33.5)

7 (1.6)0 (0-2.0)17 (3.8)

176.7 (26.1)189 (41.6)107 (23.6)166 (36.6)

13 (2.9)0 (0-2.0)18 (4.0)

1.00 (0.98-1.02)0.91 (0.78-1.07)0.97 (0.76-1.22)0.92 (0.77-1.10)0.54 (0.21-1.35)0.89 (0.60-1.33)0.95 (0.51-1.80)

0.0 (-1.7-1.7) ^

3.9 (-2.5-10.3)1.2 (-4.3-6.7)3.1 (-3.1-9.3)1.3 (-0.6-3.2)0.4 (0.2-0.6) ^

0.2 (-2.3-2.7)

0.590.230.670.330.180.580.88

CI = confidence interval; IQR = interquartile range; no. = number; SD = standard deviation; % = percentage* Relative risks and 95% confidence intervals were obtained from Modified Poisson model† Absolute differences and 95% confidence intervals were calculated from the crude proportions. ^ Absolute rate differences and 95% confidence intervals were determined based on a Normal Approximation to Poisson.# P values are from Wilcoxon, Student’s t and Chi-square test

4

2. Editors highlighted that the comparison group, "standard-care group" in a Canadian context, may not be easily appreciated by our international medical audience. Therefore, please add a brief section on some of the salient features of the standard care for international comparison.

Response: In the Methods Section, we have added the following text on page 7, paragraph 3.

Canada has a universal public payment system that covers the cost of hospital and physician services, which alleviates cost as a barrier to these services post-discharge after surgery. In Canada, standard care for most patients after non-elective surgery would include seeing a healthcare provider within 30 days of hospital discharge. Prior to this visit, the onus is on the patient to connect with their surgeon should questions arise related to the appropriate use of medications or symptoms or signs of potential complications.

3. Patients interacted with the nurses daily on days 1-15, and then every other day on days 16-30. It sounds a very labour-intensive intervention. Could the authors comment on the feasibility of such rigorous interaction with patients from the healthcare resources perspective. Also, please add the schedule of interaction from 31st day onward with your report on the 6-month outcomes.

Response: When Dr. Devereaux spoke with Dr. Islam, he explained that the economic analyses require the administrative data, which will not be available until the fall of 2021. In the Discussion Section, we have added the following text on page 17, paragraph 3.

Our study provides proof of concept that virtual care with RAM can improve outcomes after discharge following non-elective surgery. Further trials are needed to improve the efficiency (e.g., not all patients need to interact with a nurse on days 1-15 and every other day from days 16-30 after hospital discharge) and cost effectiveness of virtual care with RAM in this setting.

In the Online-Only Supplement 3, we have added the following text on page 20, paragraph 2.

At 6 months after randomization, study personnel contacted patients to obtain data on days alive at home.

4. Please add the reason why some cells in Table 2 were marked as NA or NR.

Response: In Table 2, we now report the absolute difference for all outcomes and have removed the NAs from the table. Regarding NR we have expanded our explanation for NR at the bottom of Table 2.

NR = not reported, because there were too few events to produce a stable relative risk estimate based on a modified Poisson regression;

5

5. Could the authors comment on patients' perception on the intervention.

Response: Our group has conducted many perioperative trials. This is the only trial in which many patients told us that we needed to make this intervention available for future patients having surgery. Based on this experience, we decided to conduct a separate qualitative substudy to capture data on patients’ perceptions of virtual care with RAM. We have conducted semi-structured intervention exit interviews with 21 patients and 20 involved clinicians across study sites. Patient and family interview questions focused on the recovery experience through PVC-RAM-1, compared to having experienced surgical recovery in the past without virtual care with RAM. All interviews are being transcribed. We will undertake inductive thematic analyses to code and analyze these data, with NVivo used to facilitate analysis. We expect to complete these analyses in the early autumn of 2021. Therefore, we are not able to report data on patients’ perception of the intervention, at this time. When these data are analysed and the accompanying paper is written, we will submit the results of this sub-study to a peer-reviewed journal. If the BMJ would be interested in these data, we would appreciate the opportunity to submit this work to your journal for consideration of publication.

6. There were major changes in the protocol, which is transparent. Because the primary outcome was also changed, how the authors calculate the sample size a priori. Moreover, the sample size calculation included in the final manuscript reads as if the days chosen (e.g., 29.81 days) were made-up to match with the sample size. The Editors wondered how these time-points (such as 29.81) were precisely chosen. Please clarify if these were calculated post-hoc.


PVC-RAM-1 was designed to randomise 900 patients, Appendix 8, Supplement 3.

In the Online-Only Supplement 3, we have added the following text on page 21.

APPENDIX 8. Sample sizeWhen we initially designed PVC-RAM-1, the sample size was prospectively

based upon the original primary outcome of acute-hospital care at 30-days after randomisation. We determined that enrollment of 900 patients would give the trial 98% power to detect a relative risk of 0.60 in the virtual care with remote automated monitoring (RAM) group, at a two-sided alpha level of 0.05, on the assumption that the rate of acute-hospital care in the standard-care group would be 25%.

When we changed the primary outcome to days alive at home, we undertook analyses to determine if our sample size of 900 patients remained adequate. Using data from an international, 40,000 patient, prospective, cohort study that our group undertook (i.e., the VISION Study),1 we estimated that patients in the control group would have on average 29.60 days alive at home, of 31 potential days. We then

6

calculated that if, on average, virtual care with RAM resulted in 29.81 days alive at home, we would have 89% power based on a sample size of 450 patients in each study group. An additional 0.21 days alive at home (i.e., the difference between the two study groups) in the virtual care with RAM group corresponds to an additional day alive and out of hospital for each 5 patients assigned to virtual care with RAM, which we viewed as clinically relevant. For other possible estimates of days alive at home in the control group (i.e., 29.40, 29.50, 29.60), for absolute increases of 0.21 to 0.30 days alive at home in the intervention group, we had 89%-99% power. Our calculations were based on comparing the means of two independent Poisson distributions, using the relevant subroutine in PASS v13.0 software.

7. Please add more details on sample size calculation including the software/analytic code for transparency and reproducibility.

Response: In the Online-Only Supplement 3, we have added the following text on page 21.

Our calculations were based on comparing the means of two independent Poisson distributions, using the relevant subroutine in PASS v13.0 software.

8. Could you please add a section on the possibility of, and any potential challenges in, implementing the intervention outside the research context (within and outside of Canada/North America/Europe, resource-poor settings).

Response: In the Discussion Section, we have added the following text on page 17, paragraph 4.

Centres in high-income countries could implement our virtual care with RAM intervention. Key issues for centres to consider before implementing our intervention include: ensuring an adequate supply of dedicated and committed nurses and physicians to ensure 24 hours a day patient support; procuring reliable and reusable virtual care and RAM technology (e.g., the Cloud DX technology we used in this study); establishing if the patient population resides in areas with cellular coverage; and ensuring adequate funding. Although some may question whether virtual care with RAM is viable in patients being discharged after surgery in low-income countries, given that the dominant cost of this intervention is personnel costs, which are often more affordable in lower-income countries, and that many low-income countries have extensive cellular coverage, it is possible that low-income countries could leapfrog past high-income countries in the use of this technology. More research, including research in low-income countries, is needed to inform the potential and cost effectiveness of virtual care with RAM in patients being discharged after surgery.

9. Our statistician had the following comments: The authors have used a Poisson model, and modeled days alive at home out of 30 days post-surgery procedure as if it is a count variable. It will not have a Poisson distribution (it is curtailed at day 30). A more suitable alternative will be to look at the inverse (days not alive or not at home), but

7

even that will have a zero loading. He suggests that the authors consult with relevant statistical expert(s) to deal with such models.

Response: We have consulted with statistical experts. They acknowledge the complexities of the primary outcome of “days alive at home” (Y) and also of its inverse “days not alive or not at home” (W=31-Y). Each of these have their limitations:

Outcome LimitationsY = days alive at home Bounded at 31

Highly left-skewed Excess count at 31

W = 31-Y = days not alive or not at home Highly right-skewed Excess count at 0

We were advised by our statistical experts that:

The primary outcome Y should not be modeled by the regular Poisson model. We were already aware of this limitation when the study Statistical Analysis Plan (SAP) was developed. Therefore, the SAP stated that the outcome would be modeled with the “modified Poisson” regression model as proposed by Zou (2004) A modified Poisson regression approach to prospective studies with binary data, American Journal of Epidemiology, 159:702–706. This approach models an individual’s repeated binary outcome of ‘day alive at home’ (yes/no) in a generalized estimating equation (GEE) setup, and uses a robust variance estimate known as the ‘sandwich estimator’ to correctly estimate the standard error of the estimated relative risk. Our reported relative risks and corresponding 95% confidence intervals used this approach.

Additionally, we opted for statistical significance testing using the Wilcoxon-Mann-Whitney nonparametric test for the primary outcome Y. Both approaches provided consistent results; i.e., non-significance of the primary outcome Y.

As suggested by the reviewer, to ensure results were consistent with our stated primary outcome Y = ‘days alive at home’, we had already considered the inverse outcome W = ‘days not alive or not at home’ = 31-Y. Given that W is highly skewed to the right and has an excess of zeroes, our statistical experts suggested we consider the negative binomial regression model and the zero-inflated negative binomial (ZINB). These models have limitations – the negative binomial does not account for the excess of zeroes, while the ZINB conceptualizes the extra zeroes occurring due to a different process from the process that produces the non-zero counts; in our study, these processes cannot be viewed as different. We nevertheless examined these other models to verify consistency, and also used the Modified Poisson model for the inverse outcome W. Our results on the significance of the intervention effect across all these models, crude and adjusted models, were entirely consistent, as shown in the table below:

8

Crude model – only intervention

Adjusted model – covariates*

Beta Coef.

RR & 95%CI

p-value

Beta Coef.

RR & 95%CI

p-value

Outcome = Y = Days alive at homeModified Poisson 0.0053

1.01(0.99-1.02) 0.54 0.0054

1.01(0.99-1.02) 0.53

Negative Binomial 0.0053

1.01(0.98-1.03) 0.67 0.0055

1.01(0.98-1.03) 0.65

ZINB 0.00531.01

(0.98-1.03) 0.66 0.00541.01

(0.98-1.03) 0.66Outcome W = Days not in health care or deadModified Poisson -0.114

0.89(0.62-1.29) 0.54 -0.1107

0.90(0.61-.30) 0.56

Negative Binomial -0.114

0.89(0.63-1.26) 0.51 -0.1289

0.88(0.62-1.24) 0.46

ZINB -0.1140.89

(0.63-1.26) 0.51 -0.12890.88

(0.62-1.24) 0.46

* age, male, active cancer, requires assistance for living, bleeding, pulmonary embolism, infection, cardiac surgery

Note that it may seem odd in the table above that the RR for Y is ~1.01 while the one for the “inverse” W=31-Y is ~0.89 and not the reciprocal of the RR for Y. The reason is based on the underlying probability distributions of the random variables:(a) Both of the count variables Y and W are based on counting Bernoulli random

variables ~ Bern(p), where p=prob of event. In our case, the event is ‘day alive at home’.

(b) The counts are limited to the range of 0 to 31, so an easy calculation of the probability that the count is k events, is given by the binomial distribution ~Bin(31,k). Note that the modified Poisson is based on the binomial where the observations are correlated, hence the GEE approach; the regular binomial would consider each day’s probability of alive and at home to be independent, which they are not in our case.

(c) Now, let px = prob of event given covariate x. If x is the intervention indicator variable, let px1 = prob of event in the intervention arm and px2 = prob of event in the control arm. Then the relative risk for outcome Y is given by

.𝑅𝑅𝑌 =𝑝𝑥1𝑝𝑥2

(d) Now, since W=31-Y, when Y=k, W=31-k. Thus, the relative risk for outcome W is given by

9

. 𝑅𝑅𝑊 = 31 ― 𝑌 =1 ― 𝑝𝑥11 ― 𝑝𝑥2

(e) The assumption that RRW = 1/RRY is mathematically incorrect. We do note that if px1 and px2 were close to 0.5, these relative risks may be close to being the reciprocal of each other. But we are close to 1, since ‘days alive at home’ was ~29-30 out of 31!

(f) From our data: Common event ‘days alive at home’: We had ~29.7 days out of 31 px1=0.958 Very small effect in intervention arm: We had ~29.6 days out of 31 px2=0.954

So with effects similar to ours:

These are very close to the estimates we get with all the models for the two formulations of the primary outcome.

In summary, after having explored the various alternatives as suggested by the reviewer and after consulting with statistical experts, we prefer to present the results as they are in the manuscript - the primary outcome of ‘days alive at home’ using the modified Poisson regression approach, for the following reasons:

i. Model preferred: modified Poisson regression model

o The modified Poisson regression model provides correct estimates of the standard error of the estimated relative risks to compute correct confidence intervals.

o The negative binomial model and the ZINB model have limitations (but we are reassured by the fact that they provide similar results to the modified Poisson models).

o The modified Poisson regression was specified in the study Statistical Analysis Plan, which was finalized before the study results were unblinded to the investigators.

ii. Outcome preferred: days alive at homeo Both outcome formulations (‘days alive at home’ and ‘days not alive or not

at home’) have limitationso The primary outcome of ‘days alive at home’ was pre-specified:

Was the basis for the sample size calculation of the study Was specified in the study protocol Was specified in the protocol registered in ClinicalTrials.gov, no.

NCT04344665

px1 px2𝑅𝑅𝑌 =

𝑝𝑥1𝑝𝑥2 𝑅𝑅𝑊 = 31 ― 𝑌 =

1 ― 𝑝𝑥11 ― 𝑝𝑥2

0.960 0.955 1.005236 0.8888890.958 0.954 1.004193 0.913043

10

Was specified in the study Statistical Analysis Plan, which was finalized before the study results were unblinded to the investigators

Was pre-specified in our publication of the study’s methodology – McGillion et al (2021) Post Discharge after Surgery Virtual Care with Remote Automated Monitoring Technology (PVC-RAM): protocol for a randomized controlled trial, Canadian Medical Association Journal OPEN, doi:10.9778/cmajo.20200176.

o Finally, the main result – non-significance of the primary outcome - does not change with the reframing of outcome to ‘days not alive and not at home’

10. It was not clear whether the patients were blinded or not. Please add more descriptions along with any justifications in the revised manuscript.

Response: In the Abstract Section, we clarified our statement regarding blinding on page 2.

Patients, healthcare providers, and data collectors were aware of patients’ group allocations.

In the Methods Section, we have added the following text on page 6, paragraph 2.

Due to the nature of the intervention and follow-up procedures, patients, healthcare providers, and data collectors were aware of patients’ group allocations.

11. In your response please provide, point by point, your replies to the comments made by the reviewers and the editors, explaining how you have dealt with them in the paper.

Response: We have complied with your instructions.

Reviewer One’s Comments

12. Is the article important? I believe this article is very important. The quantity of cancelled operations is alarming (https://www.england.nhs.uk/statistics/wp-content/uploads/sites/2/2020/02/QMCO-Statistical-Commentary-Q3-2019-20-4jaJ8.pdf) and especially so among cancer patients (https://www.independent.co.uk/news/health/coronavirus-nottingham-cancer-operations-cancelled-outbreaks-b1373423.html), so any solution that can contribute to dealing with the backlog has to be taken seriously. This article provides evidence that decisions to cancel operations because hospital after care cannot be provided should be revisited. In addition to this point, it begs the question whether the default setting of discharge should be early discharge home with remote monitoring should become TAU, thus releasing ward staff to better use. Do patients recover from surgery regardless of nursing care or because of it? This question should be urgently considered by key

https://urldefense.com/v3/__https:/www.england.nhs.uk/statistics/wp-content/uploads/sites/2/2020/02/QMCO-Statistical-Commentary-Q3-2019-20-4jaJ8.pdf__;!!Eh6q0Q!nC6MiLjt6Q18W4mvRsWUCCOwdVRFSWdl-RdEPKR7c0A7zDNK3oqy4r3jdIT2tjOsCpY$

https://urldefense.com/v3/__https:/www.england.nhs.uk/statistics/wp-content/uploads/sites/2/2020/02/QMCO-Statistical-Commentary-Q3-2019-20-4jaJ8.pdf__;!!Eh6q0Q!nC6MiLjt6Q18W4mvRsWUCCOwdVRFSWdl-RdEPKR7c0A7zDNK3oqy4r3jdIT2tjOsCpY$

https://urldefense.com/v3/__https:/www.independent.co.uk/news/health/coronavirus-nottingham-cancer-operations-cancelled-outbreaks-b1373423.html__;!!Eh6q0Q!nC6MiLjt6Q18W4mvRsWUCCOwdVRFSWdl-RdEPKR7c0A7zDNK3oqy4r3jdIT2De0XMDU$

https://urldefense.com/v3/__https:/www.independent.co.uk/news/health/coronavirus-nottingham-cancer-operations-cancelled-outbreaks-b1373423.html__;!!Eh6q0Q!nC6MiLjt6Q18W4mvRsWUCCOwdVRFSWdl-RdEPKR7c0A7zDNK3oqy4r3jdIT2De0XMDU$

11

stakeholders in the light of this evidence.

Response: We appreciate the reviewer’s comments. In the Discussion Section, we have added the following text on page 18, paragraph 2.

If physicians and patients knew immediately after surgery that patients were randomised to receive virtual care with RAM after discharge, this knowledge could have facilitated earlier hospital discharges, compared to patients randomised to standard care. Because we randomised patients after the most responsible physician had decided to discharge the patient home, we were not able to inform this issue.

13. Will it help our readers to make better decisions?It ought to. Patients should be informed by this article to ask if they could be better off at home with remote monitoring instead of lengthy inpatient stays. Hospital managers should be looking to ask if there could be a better use of resources. Surgeons should be asking why are their patients on wards exposed to nosocomial infections when they could equally be managed at home amidst their own flora. Infection control specialists should be asking if costs could be reduced by early discharge. Commissioners should be looking at this paper with particular interest. The NIHR should be considering funding a trial in this country with a view for NICE to make new recommendations.

Response: We appreciate the reviewer’s comments.

14. Will the article add enough to existing knowledge?I believe this paper will add substantially to existing knowledge and should open the way for further trials. Nosocomial infections pre-Covid have long been identified as a major source of healthcare expenditure and patient distress (https://pubmed.ncbi.nlm.nih.gov/10993621/. and. https://pubmed.ncbi.nlm.nih.gov/14750066/. and https://emedicine.medscape.com/article/967022-overview) and there is no reason why patients should be exposed to the risk of nosocomial infection, especially in this period of Covid-19 unnecessarily. (https://www.hsj.co.uk/patient-safety/covid-infections-caught-in-hospital-up-by-half-in-a-week/7029320.article). This paper demonstrates an important case for investment in remote monitoring.

Response: We appreciate the reviewer’s perspective.

15. Does the article read well and make sense? Does it have a clear message?The authors are to be complimented in writing an accessible paper. The language style needs by virtue of it being an academic paper to be written at a level above the average reading age, but that apart, it still isn’t a difficult paper to read and follow.


16. Originality — does the work add enough to what is already in the published literature? If so, what does it add? Please cite relevant references to support your

https://urldefense.com/v3/__https:/pubmed.ncbi.nlm.nih.gov/10993621/__;!!Eh6q0Q!nC6MiLjt6Q18W4mvRsWUCCOwdVRFSWdl-RdEPKR7c0A7zDNK3oqy4r3jdIT2jkFsM2k$

https://urldefense.com/v3/__https:/pubmed.ncbi.nlm.nih.gov/14750066/__;!!Eh6q0Q!nC6MiLjt6Q18W4mvRsWUCCOwdVRFSWdl-RdEPKR7c0A7zDNK3oqy4r3jdIT2jUwZWOA$

https://urldefense.com/v3/__https:/emedicine.medscape.com/article/967022-overview__;!!Eh6q0Q!nC6MiLjt6Q18W4mvRsWUCCOwdVRFSWdl-RdEPKR7c0A7zDNK3oqy4r3jdIT2AZ6OL2Y$

https://urldefense.com/v3/__https:/www.hsj.co.uk/patient-safety/covid-infections-caught-in-hospital-up-by-half-in-a-week/7029320.article__;!!Eh6q0Q!nC6MiLjt6Q18W4mvRsWUCCOwdVRFSWdl-RdEPKR7c0A7zDNK3oqy4r3jdIT2mL8cud8$

https://urldefense.com/v3/__https:/www.hsj.co.uk/patient-safety/covid-infections-caught-in-hospital-up-by-half-in-a-week/7029320.article__;!!Eh6q0Q!nC6MiLjt6Q18W4mvRsWUCCOwdVRFSWdl-RdEPKR7c0A7zDNK3oqy4r3jdIT2mL8cud8$

12

comments on originality.There is little evidence on this topic, and as such this is refreshingly original. A small number of papers have been published to highlight the potential of using mobile applications for remote monitoring (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5224949/) and (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7728408/), but this study goes further and makes new ground.

Response: We appreciate the reviewer pointing out a study of which we were unaware. In the Discussion Section, we have added the following text on page 14, paragraph 4.

An observational study of 20 patients discharged after esophagectomy demonstrated patients’ use of virtual care with RAM after discharge was feasible and well received by all patients.8

17. Importance of the work to general readers — does this work matter to clinicians, researchers, policymakers, educators, or patients? Will it help our readers to make better decisions and, if so, how? Is a general medical journal the right place for it?This study has relevance to all readers listed above for reasons already stated. The BMJ is exactly the right place for this paper to be published, but moreover, I hope this paper will be picked up by responsible journalists beyond the BMJ and given a wider audience. Of particular interest is that those discharged home with remote monitoring experienced better pain control than those who remained in hospital.

Response: We appreciate the reviewer’s comments

18. Scientific reliability:I am not qualified to make a judgement on the science.

Response: We respect the reviewer’s perspective.

19. Research question — clearly defined and appropriately answered?The research question, “Among adults discharged after non-elective surgery, does virtual care with RAM increase days alive at home during the first 30 days after randomisation, compared to standard care?” Was clearly defined and comprehensively answered.

Response: We appreciate the reviewer’s feedback.

20. Overall design of study — appropriate and adequate to answer the research question?The study was a blinded multicentre RCT of 903 adults. For 30 days post-discharge, the intervention cohort were given the necessary equipment to record vital signs and wound state to interact with nursing staff. The TAU cohort stayed in hospital receiving standard care. The primary outcome measure was days alive, once home, the

https://urldefense.com/v3/__https:/www.ncbi.nlm.nih.gov/pmc/articles/PMC5224949/__;!!Eh6q0Q!nC6MiLjt6Q18W4mvRsWUCCOwdVRFSWdl-RdEPKR7c0A7zDNK3oqy4r3jdIT2jeA_XCQ$

https://urldefense.com/v3/__https:/www.ncbi.nlm.nih.gov/pmc/articles/PMC7728408/__;!!Eh6q0Q!nC6MiLjt6Q18W4mvRsWUCCOwdVRFSWdl-RdEPKR7c0A7zDNK3oqy4r3jdIT2U1A73P0$

13

secondary outcomes included assessment of inpatient care, titration and monitoring of medication, and pain scores.

Response: Patients in the standard care group were not kept in the hospital receiving standard care after randomisation. To ensure there is no ambiguity on this point in the Methods Section, we have added the following text on page 5, paragraph 3.

Eligible patients were ≥40 years of age, had undergone inpatient non-elective surgery and the most responsible physician had decided to discharge the patient home or patients were within 24 hours after discharge home without having obtained acute-hospital care since discharge, and provided informed consent to participate.

In the Discussion Section, we have added the following text on page 18, paragraph 2.


21. Participants — adequately described, their conditions defined, inclusion and exclusion criteria described? How representative were they of patients whom this evidence might affect?“ Eligible patients were ≥40 years of age, had undergone inpatient non-elective surgery…excluded patients who: 1. were discharged to rehabilitation or convalescent care for >7 days; 2. were unable to communicate with research staff, complete study surveys, or undertake an interview using a tablet computer due to a cognitive, language, visual, or hearing impairment; or 3. resided in an area without cellular coverage.” There is no ambiguity here. Participants were randomised at the point the surgeon decided they were able to be discharged home.


22. Methods — adequately described? Main outcome measure clear? Is the study fully reported in line with the appropriate reporting statement or checklist (these are all collected and regularly updated at http://www.equator-network.org/)?Was the study ethical (this may go beyond simply whether the study was approved by an ethics committee or IRB)?This was a multi centre trial and each centre obtained ethical approval. There is no reason to suggest the study was unethical for any reason. All participants recruited had capacity to consent surgery, and it is to be expected that all participants had capacity to consent to be included in the trial.


https://urldefense.com/v3/__http:/www.equator-network.org/__;!!Eh6q0Q!nC6MiLjt6Q18W4mvRsWUCCOwdVRFSWdl-RdEPKR7c0A7zDNK3oqy4r3jdIT2pqNqP8g$

14

23. Results — answer the research question? Credible? Well presented?There was minimal difference between the two groups, suggesting that patients need not necessarily remain in hospital post surgery if after care could be provided remotely.It should be noted that, “More patients in the virtual-care group compared to the standard-care group had a medication error detected (134 patients [29.7%] versus 25 patients [5.5%]” , which could be serious if translated to a wider cohort. Balanced against this is the observation that “Fewer patients in the virtual care group compared to the standard-care group reported pain at 7, 15, and 30 days after randomisation”, which is a point that will be of special interest to a wide number of academics and clinicians from different disciplines.

Response: In the Methods Section, we make the following statement on page 8, paragraph 1.

We hypothesized that we would detect more medication errors and corrections in the virtual-care group compared to the standard-care group, and a priori stated we would interpret this as an improvement in care.


Drug errors after hospital discharge post-surgery proved common (i.e., 29.7% of virtual-care patients and these patients had a mean of 2.1 drug errors). Virtual care demonstrated large absolute benefits in detecting (24.2%) and correcting medication errors (24.4%). Detection and correction of drug errors have the potential to improve both short and long-term health.

24. Interpretation and conclusions — warranted by and sufficiently derived from/focused on the data? Discussed in the light of previous evidence? Message clear?The interpretation of data is compelling. The data illustrates that under set conditions, early discharge with remote monitoring is safe. The data does indicate that there is a case to understand why those discharged home early with remote monitoring experienced more drug errors than those kept in hospital. “(Detection of drug omission was more common in the virtual-care group (82 patients [18.2%]) compared to the standard-care group (16 patients [3.5%])”). Communication is always a cause for concern. Hospital staff assume patients understand all they think they have told patients. Patients assume they have understood all they think they have been told. But it is interesting to ask how it is there were the drug errors made by nurses (or prescribers or dispensers) looking after patients in hospital.

Response: In the Results Section, we have added the following text on page 12, paragraph 2.

Table 4, Supplement 3 reports the most responsible person for drug errors and the reason for drug errors. Patients were responsible for 77.6% of the drug errors, and the most common reasons for their drug errors included an intentional decision (46.1%), mistake (22.7%), forgot (11.3%), and financial barrier (8.6%). Physicians and nurses

15

were responsible for 18.5% of the drug errors, and the most common reasons for their drug errors included failure to communicate clear instructions on what medications should and should not be taken at home (54.1%), failure to write a prescription for a new medication (34.4%), and failure to write a prescription to discontinue a medication (6.6%). Pharmacists were responsible for 3.6% of the drug errors and this was always due to a failure to provide the medication as prescribed.


16

Table 4. Most responsible person for drug error and reason for drug error

Virtual-caregroup

(N=451)

Standard-caregroup

(N=454)

P Value

Patients with a drug error – no. (%) 134 (29.7) 25 (5.5) <0.001

Most responsible person for drug error* - no. patient physician/nurse pharmacist unknown

218 5891

38330

Primary reason for drug error made by patient – no. intentional patient decision mistake forgot financial barrier did not fill prescription for non-financial reasons intolerance/side effect unknown

96 55202112104

22 391120

Primary reason for drug error made by physician/nurse – no. failure to communicate clearly what medications patients should or should not take at home failure to write prescription for new medication failure to write prescription to discontinue medication unknown

321943

1200

Primary reason for drug error made by pharmacist – no. did not provide medication as prescribed 9 3

no. = number; % = percentage* Some patients had multiple medication errors. Therefore, sums of most responsible person for error and primary reason for error surpass total number of patients with a medication error.

17

25. Patient and Public involvementThere were four members of the public who reviewed the symptom survey for clarity and ease of use. There is no evidence they were involved in trial design, analysis of data or contributed to the paper.


A panel of four patient partners reviewed the daily symptom survey for clarity and perceived ease of use. Given rules on social distancing and limitations to in-person meetings, all feedback was provided via email. Patients were not involved in the trial design or analyses and did not contribute to the paper.

26. References — up to date and relevant? Any glaring omissions?The oldest reference was 2204, the most recent was 2021. There appear to no omissions, ambitious, or inappropriate references used.


27. Abstract/summary/key messages/what this paper adds — reflect accurately what the paper says?The abstract is clear and reflects well the body of the study paper


28. ∙ Documents in the supplemental files eg checklists for reporting statements eg CONSORT, PRISMA, and STROBE (see http://www.equator-network.org for other examples and for extensions to existing statements); and the protocol for an RCT. Do these properly match what is in the manuscript? Do they contain information that should be better reported in the manuscript, or raise questions about the work?The patient flow chart makes interesting reading for a profile of those who weren’t included in the study. Were the NIHR minded to fund a study on this topic, potential research teams would be wise to reflect on this.


29. The baseline characteristics and general data also is interesting. While I can’t discern how many were kept in hospital or were part of the intervention cohort, 8.2% were discharged with a drain and 4.9% with a stoma. It does beg the question were these part of the intervention or TAU cohort?

Response: Patients in the standard care group were not kept in the hospital receiving standard care after randomization. To ensure there is no ambiguity on this point in the Methods Section, we have added the following text on page 5, paragraph 3.

https://urldefense.com/v3/__http:/www.equator-network.org__;!!Eh6q0Q!nC6MiLjt6Q18W4mvRsWUCCOwdVRFSWdl-RdEPKR7c0A7zDNK3oqy4r3jdIT2QNUZy_Y$

18

Eligible patients were ≥40 years of age, had undergone inpatient non-elective surgery and the most responsible physician had decided to discharge the patient home or patients were within 24 hours after discharge home without having obtained acute-hospital care since discharge, and provided informed consent to participate.



30. Table 3 illustrates well how successfully the remote monitoring worked.The trial protocol is as interesting as the paper detailing the study results, not least because it provides extra details about Cloud DX.The online only supplement gives more detail of the interactions between the nursing staff and the patients, and this demonstrates how timely this system worked, as well as particular details of how the study progressed. Sceptics of remote monitoring would do well to study this well.


Reviewer Two’s Comments

31. Comments:Thank you for the opportunity to review this manuscript - it is interesting and I enjoyed reading it.

A multi-centre, randomised controlled trial of remote care with remote monitoring after non-elective surgery is presented.

The topic of telemedicine and remote monitoring after surgery is timely and important. There is a prevailing trend towards personalised and digital innovation in surgery for patient benefit. Most existing evidence for telemedicine and remote monitoring is observational and focuses on assessments of proof-of-concept or feasibility, so the present report adds novelty and represents progress.

The manuscript is well written and transparently reported. It was registered prospectively (NCT04344665; 14th April 2020). There were several changes to the study design (including a change in primary outcome), which are described clearly.


19

32. I am concerned about the selected population. From the report, it is not clear what types of non-elective surgery were included (i.e. major abdominal, orthopaedic, thoracic). The impression is that the study population is a mix of all of the above. Within abdominal surgery alone, this could include anything from a laparoscopic appendicectomy to an open AAA repair, which have very different implications for recovery. Unfortunately, the stratification does not account for this (planned strata: cardiac and non-cardiac surgery), which makes it difficult to draw clinically meaningful take home messages.


Table 1, Supplement 3 reports the subtypes of surgery patients underwent, which proved similar between groups.


20

Table 1. Subtypes of surgery patients underwent

Characteristics Virtual-care group

(N=451)

Standard-care group

(N=454)

Type of surgery* – no. (%) Non-cardiac General other intra-abdominal complex visceral resection partial or total colectomy or stomach surgery cholecystectomy incarcerated hernia, perforated appendectomy or small bowel resection major head and neck resection for non-thyroid tumor other Urology/gynecology radical prostatectomy radical hysterectomy nephrectomy (partial or complete) bilateral salpingo-oopherectomy cystectomy transurethral resection of bladder tumor nephrostomy/ureteric stent/ileal conduit transurethral prostatectomy penectomy/vulvectomy cystoscopy cytoreductive other Orthopedic major hip open reduction internal fixation (excludes hip) knee surgery pelvic surgery ankle surgery internal fixation of femur spine surgery tumor resection knee arthroplasty lower leg amputation shoulder surgery above knee amputation(s) other neurosurgery spine surgery craniotomy shunt surgery vascular peripheral vascular reconstruction endovascular abdominal aortic aneurysm repair

366 (81.2)146 (32.4)38 (8.4)35 (7.8)33 (7.3)28 (6.2)33 (7.3)8 (1.8)4 (0.9)

81 (18.0)24 (5.3)23 (5.1)13 (2.9)10 (2.2)10 (2.2)4 (0.9)2 (0.4)2 (0.4)1 (0.2)2 (0.4)0 (0.0)12 (2.7)62 (13.7)13 (2.9)7 (1.6)5 (1.1)1 (0.2)2 (0.4)4 (0.9)3 (0.7)3 (0.7)1 (0.2)3 (0.7)2 (0.4)1 (0.2)9 (2.0)30 (6.7)23 (5.1)7 (1.6)0 (0.0)22 (4.9)12 (2.7)2 (0.4)

366 (80.6)130 (28.6)24 (5.3)26 (5.7)32 (7.0)33 (7.3)27 (5.9)5 (1.1)2 (0.4)

91 (20.0)16 (3.5)20 (4.4)19 (4.2)13 (2.9)10 (2.2)7 (1.5)6 (1.3)5 (1.1)5 (1.1)4 (0.9)2 (0.4)10 (2.2)68 (15.0)17 (3.7)5 (1.1)3 (0.7)5 (1.1)4 (0.9)1 (0.2)3 (0.7)2 (0.4)3 (0.7)1 (0.2)2 (0.4)1 (0.2)5 (1.1)31 (6.8)19 (4.2)11 (2.4)2 (0.4)25 (5.5)14 (3.1)5 (1.1)

21

extracranial cerebrovascular surgery aorto-iliac reconstruction thoracic aorta reconstruction thoracic lobectomy wedge resection thoracotomy other plastic major plastic minor plastic other Cardiac coronary artery bypass grafting on pump off pump valve aortic mitral other aortic atherectomy other

4 (0.9)3 (0.7)1 (0.2)23 (5.1)13 (2.9)6 (1.3)4 (0.9)8 (1.8)10 (2.2)7 (1.6)4 (0.9)10 (2.2)

89 (19.7)69 (15.3)69 (15.3)0 (0.0)28 (6.2)19 (4.2)10 (2.2)0 (0.0)12 (2.7)5 (1.1)7 (1.6)

3 (0.7)2 (0.4)1 (0.2)17 (3.7)12 (2.6)3 (0.7)1 (0.2)5 (1.1)6 (1.3)6 (1.3)0 (0.0)15 (3.3)

89 (19.6)75 (16.5)74 (16.3)1 (0.2)19 (4.2)13 (2.9)5 (1.1)2 (0.4)6 (1.3)6 (1.3)7 (1.5)

no. = number; % = percentage* Some patients had more than one type of surgery or multiple surgeries within the same subtype. Therefore, sums of subtypes of surgery and surgical procedures surpass total number of patients.


Although baseline variables, including the subtypes of surgery, appear balanced between the two treatment groups, we cannot exclude the possibility of a baseline imbalance of prognosis in this moderate-sized RCT.

33. The intervention leans towards being complex. It involves a telemedicine video format (as opposed to standard follow up), daily consultations (as opposed to standard 1-2 consultations within 30 days), and remote monitoring of body parameters (as opposed to in-hospital monitoring alone). In essence, the intervention changes multiple components of care and it would be good to see some justification as to how these changes interact and whether the intervention is stable. For instance, is it strictly necessary for video consultations to take place daily, or could this be rationalised? What are the unintended consequences i.e. is it be possible that the intervention could lead to some excess re-admissions due to false negatives related to the monitoring. All of these are important considerations for clinical, economic, and logistical implementation in the

22

future.



34. The choice of primary outcome seems problematic I’m afraid. The revised primary outcome is a composite of readmission outcomes and mortality, which is reported according to number of days alive and at home. It is important to consider to what extent this is meaningful to patients (i.e. were the components of re-admission and mortality informed by public engagement?). Also, as described in the outcome definition table, it is possible for this outcome to be affected by post-randomisation non-clinical/logistical delays in discharge, which would seem to be a reasonable limitation.

Response: In the Discussion Section under the Limitations subheading, we have added the following text on page18, paragraph 2.

We did not ascertain if patients viewed days alive at home as an important outcome.

35. The approach to follow up requires some further consideration. Due to unstratified variation in the type of surgery and unstandardised discharge criteria, patients’ stage of recovery at the point of randomisation will be different. This is problematic for the primary outcome (readmission/mortality) as the setting in which a complication is experienced (i.e. in hospital or at home) may be partly determined by how enthusiastically the patient was discharged by the clinical team.

Response: As highlighted in our response to comment 32, the subtypes of surgeries were similar between the two treatment groups. In Table 1, we have also added data regarding the timing of hospital discharge relative to randomisation, and this was similar between the two groups.

23

Table 1. Baseline characteristics and surgical details

Characteristics Virtual-care group

(N=451)

Standard-care group

(N=454)

Age (years, mean [± SD]) 63.2 ± 10.4 62.9 ± 11.2

Sex (men) – no. (%) 259 (57.4) 236 (52.0)

History before randomisation – no. (%) hypertension active cancer* coronary artery disease diabetes smoked within 28 days before surgery obstructive sleep apnea myocardial infarction atrial fibrillation chronic pain chronic obstructive pulmonary disease peripheral arterial disease stroke congestive heart failure transient ischemic attack deep venous thrombosis pulmonary embolism needing assistance with activities of daily living COVID-19 infection

241 (53.4)153 (33.9)100 (22.2)98 (21.7)88 (19.5)74 (16.4)56 (12.4)54 (12.0)52 (11.5)41 (9.1)22 (4.9)22 (4.9)20 (4.4)18 (4.0)11 (2.4)10 (2.2)9 (2.0)0 (0)

264 (58.1)156 (34.4)107 (23.6)96 (21.1)88 (19.4)75 (16.5)60 (13.2)60 (13.2)50 (11.0)39 (8.6)32 (7.0)17 (3.7)17 (3.7)15 (3.3)8 (1.8)9 (2.0)7 (1.5)1 (0.2)

Type of surgery† – no. (%) Non-cardiac†

general urology/gynecology orthopedic neurosurgery vascular thoracic plastic other Cardiac†

coronary artery bypass grafting valve aortic other

366 (81.2)146 (32.4)81 (18.0)62 (13.7)30 (6.7)22 (4.8)23 (5.1)10 (2.2)10 (2.2)89 (19.7)69 (15.3)28 (6.2)12 (2.7)12 (2.7)

366 (80.6)130 (28.6)91 (20.0)68 (15.0)31 (6.8)25 (5.5)17 (3.7)6 (1.3)15 (3.3)89 (19.6)75 (16.5)19 (4.2)6 (1.3)13 (2.9)

Timing of surgery – no. (%) semi-urgent urgent emergent

241 (53.4)178 (39.5)32 (7.1)

273 (60.1)142 (31.3)39 (8.6)

24

Same-day surgery – no. (%) 28 (6.2) 42 (9.3)

Surgical approach† – no. (%) open minimally invasive endoscopic/endovascular

341 (75.6)63 (14.0)76 (16.9)

338 (74.4)68 (15.0)71 (15.6)

Anesthesia† – no. (%) general neuraxial regional block local

435 (96.5)53 (11.8)22 (4.9)10 (2.2)

436 (96.0)64 (14.1)15 (3.3)11 (2.4)

New diagnoses from initiation of surgery until randomisation – no. (%) bleeding myocardial injury after non-cardiac surgery infection delirium

29 (6.4)24 (6.6)11 (2.4)5 (1.1)

29 (6.4)18 (4.9)11 (2.4)4 (0.9)

Laboratory measurements before randomisation hemoglobin (g/L, median [IQR]) creatinine (umol/L, median [IQR])

108 (94-124)69 (58-85)

110 (95-124)71 (58-88)

Present at time of hospital discharge – no. (%) surgical drain stoma

37 (8.2)22 (4.9)

19 (4.2)17 (3.7)

Timing of hospital discharge relative to randomisation patients randomised before hospital discharge – no. (%) time from randomisation to discharge (days, median [IQR]) patients randomised within 24 hours after hospital discharge – no. (%)

358 (79.4)0.08 (0.04-0.17)

93 (20.6)

361 (79.5)0.08 (0.04-0.17)

93 (20.5)

IQR = interquartile range; no. = number; SD = standard deviation; % = percentage* Defined as a patient with a diagnosis of cancer who was receiving or has received active treatment for their cancer (e.g., chemo, radiation, or surgery) within the previous 6 months; however, it does not apply to patients with non-melanoma skin cancers. † Some patients had more than one type of surgery, surgical approach, or anesthesia. Therefore, sums of subtypes of surgery, surgical approach, and anesthesia surpass total number of patients.

25

36. Whilst the timing of this study (during the COVID-19 pandemic) should not detract from genuine clinical take home messages, the generalisability of the results to non-COVID-19 practice should be justified more clearly. As we know, surgical practice and outcomes changed substantially during the pandemic. A good example of this is the management of appendicitis, where (in most places) only the sickest of patients may managed surgically. It would be useful to report a summary of the usual standard of care (i.e. enhanced recovery/discharge protocols) at participating sites and how these changed during the pandemic. This will provide a clearer context in which to interpret the results.

Response: In the Discussion Section under the Limitations subheading, we have added the following text on page 18, paragraph 2.

We do not have documentation on how the usual standards of care (e.g., discharge protocols) changed at participating centres during the COVID-19 pandemic.

37. The discussion of previous literature seems superficial. There is a small but growing body of evidence relating to telemedicine and remote monitoring. This is well summarised in a review by Williams and colleagues (PMID: 29963556). It would be useful to provide a more detailed description of earlier work to contextualise the current results.


A systematic review that evaluated virtual care in the recovery of surgical patients after hospital discharge demonstrated that investigators have thus far conducted only small observational studies with a high-risk of bias; the 3 randomized controlled trials (RCTs) included a total of only 153 patients.9 Although the findings of this review require cautious interpretation, the studies suggest the acceptability of virtual care by patients and physicians, the potential to save patients time and money related to avoiding travel to clinics and missing work, and providing hospital clinic space for new patients.

38. The number of patients excluded due to surgeons’ refusal (n=564) is notable. It would be useful to provide some discussion of equipoise and the possible challenges of implementing the intervention in practice.According to the results, almost 10% of patients were not compliant to the intervention. Again, it would be useful to provide some discussion about the feasibility of the intervention in practice.


Among eligible patients for PVC-RAM-1, approximately 18% refused to participate, and 18% of surgeons did not agree to have patients participate. Moreover, only 9% of patients permanently discontinued using virtual care and RAM before completing the

26

trial. Although our study demonstrated that the majority of patients and surgeons were agreeable to the trial and compliant with the intervention, further research is needed to establish what barriers exist for patients and surgeons regarding virtual care with RAM after surgery and participation in clinical trials.

39. The references seem appropriate, but the discussion and context of existing evidence should be expanded. The figures are clear and self-explanatory.

Response: We appreciate the reviewer’s feedback. As per our response to this reviewer’s prior comment (i.e., comment 37), we have expanded our discussion of the existing evidence.

Reviewer Three’s Comments

40. This study randomized patients after hospital discharge of a emergent-urgent-semiurgent surgery to either usual post op care vs remote virtual care in 8 centers. Virtual care included vital signs measurements and was conducted initially by a RN, and if deemed necessary by a physician. Patients allocated to virtual care experienced less need to acute-emergency care and re-hospitalization, had more detection of medication errors and experienced less pain in the first 30 days.

The authors should be congratulated by such a huge effort during the Covid-19 pandemics and this study should be published after some issues are resolved.


41. Total number of patients screened was 10 times higher than patients actually randomized to PVC-RAM. Please comment how this could limit the conclusions of this study to have widespread application.

Response: Although we screened 9433 patients, most patients were excluded because they did not fulfill our eligibility criteria (e.g., 3032 were <40 years of age, and 2590 had surgery not considered semi-urgent, urgent, emergent, or reflected traditional same-day surgery with low likelihood of acute care post discharge). Of the 3211 patients who met our eligibility criteria, we randomized 905 patients (i.e., 28.2%). This inclusion rate is consistent with other perioperative clinical trials (e.g., N Engl J Med. 2014; 370: 1504-1513; Lancet. 2018; 391: 2325-2334).

42. Please clarify what does it mean "usual care". Would that include personal return of the patients to the hospital? I do understand that "usual care" might vary considerably among the 8 centers, but the authors should describe succinctly the general characteristics.


27

In Canada, standard care for most patients after non-elective surgery would include seeing a healthcare provider within 30 days of hospital discharge. Prior to this visit, the onus is on the patient to connect with their surgeon should questions arise related to the appropriate use of medications or symptoms or signs of potential complications.

43. Was the data of the patients who discontinued using the tablet included in the Results? Or was it removed? Did this patients immediately returned to the "usual care" of each center?


Patients were analysed in the groups to which they were randomised, regardless of compliance.

44. The virtual care group patients received daily conversations in the initial 15 days. How frequently did the usual care group patients attended a consultation (please provide a mean and SD)? One explanation to the reduced use of acute hospital care in the virtual care group could be a more intense and continuous check of medical issues by the RNs and a more efficient correction of medication use. Please comment on that, if you think it makes sense.


Usual post-discharge follow-up was consistent for both trial groups: an in-person or virtual follow-up visit with a non-study surgeon, family physician, or specialist occurred in 76.9% of patients in the standard-care group and 77.2% of patients in the virtual-care group.

We agree with the reviewer that the more intense check of medical issues by the nurses working virtually and more efficient correction of problems may reduce acute hospital care. In the Discussion Section, we state the following on page 16, paragraph 3.

It is only credible to expect virtual care with RAM to impact outcomes if these interventions identify problems and lead to changes in management.

45. Table 1 shows the baseline characteristics from both groups. It is very complete and adequate. But the comparisons should include a test of statistical significancy for each characteristic.

Response: Given we already know any differences that exist in baseline characteristics are due to chance (i.e., randomization), we did not report any P values for the comparison of baseline characteristic across treatment groups. We believe this is consistent with the BMJ’s policy. As such we have not added P values to Table 1. If

28

you want us to add these P values please let us know and we will add the P values to Table 1.

46. The authors did not mention the costs of the tablets and the center for virtual care. Please describe how the devices were handled and returned to the hospitals. Please describe the scenario in which the RN and physician delivered the virtual care. Was it from the hospitals or even from home? Was it the same RN that followed each particular patient every day or was there a larger team involved? In that case, how was transfer of information performed?

Response: The data to understand the full economics related to the intervention requires the administrative data, which will not be available until the fall of 2021.

In the Methods Section, we have added the following text on page 7, paragraph 2.

Appendix 3, Supplement 3 reports further details regarding how nurses and physicians delivered virtual care and how devices were returned. In the Online-Only Supplement 3, we have added the following table on page 6.

APPENDIX 3. Additional details regarding patient training, how nurses and physicians delivered virtual care, and how devices were returned

The initial patient training on the use of the tablet and devices took approximately 35 to 40 minutes. Whenever possible, patients’ family members were encouraged to participate.

The registered nurses and physicians delivered care from the hospital sites. At each site, nurses were stationed in specially designated virtual care spaces, outfitted with workstations for the nurses. The nurses worked in scheduled teams in order to facilitate nursing coverage 24 hours a day, 7 days per week. Nurse-to-nurse handover of patient care in the trial was orchestrated similar to how ward nurses transfer patient accountability. The nurses worked in 8 to 12 hours shifts, with the nurses on each shift giving verbal report on their patients to the next oncoming nurse. The nurses also completed a standardized, written nurse-to-nurse transfer of accountability report, which summarized key patient issues. The on-call perioperative physicians were connected with the patients and nurses through the Cloud DX connected health Zoom interface; they could log in to the system through the Cloud DX secure remote access portal.

Hospital-to-home handling and processing of the Cloud DX Connected Health kits was as follows. Patients decided if they would personally deliver the kit back to the hospital or if they wanted it couriered back to the hospital. If they wanted it couriered, the patient was given a pre-paid courier slip. At the end of the 30-day intervention period, patients either called the courier to facilitate kit pick up at their home, or they brought the kit to the nearest courier depot for delivery back to the hospital. Patients and families were oriented to these procedures at the start of their participation in the trial. Once kits were returned, study personnel would clean them according to procedures approved by hospital site infection control, take inventory to ensure all

29

components were accounted for and working, and repackage the kit for the next trial patient.

47. From the 13 secondary outcomes, the authors could find a benefit of virtual care in 5 of them. Please briefly comment on the neutral results of the other 8 outcomes.

Response: In the Discussion Section, we include the following text on page 15, paragraph 2.

Virtual care and RAM did not significantly affect days alive at home, but raised the possibility of a reduction in acute-hospital care (22.0% in the virtual-care group versus 27.3% in the standard-care group; relative risk, 0.80; 95% CI, 0.64-1.01), brief acute-hospital care (13.7% versus 18.1%, relative risk, 0.75; 95% CI, 0.56-1.02), hospital re-admission (9.5% versus 12.8%; relative risk, 0.77; 95% CI, 0.53-1.11), and emergency department visit (19.7% versus 24.4%; relative risk, 0.81; 95% CI, 0.64-1.04).

48. Regarding the tertiary objectives, the Plan of Investigation includes the 6 month effect of virtual care on several outcomes. In the body of the text it is not clear if these were also neutral. Please clarify.


Table 9, Supplement 3 reports the effects of virtual care with RAM on tertiary 6-month outcomes. There was no impact on days alive at home at 6 months.


30

Table 9: Effects of virtual care and remote automated monitoring on tertiary 6-month outcomes

Outcome Virtual-caregroup

(N=451)

Standard-caregroup

(N=454)

Relative risk*(95% CI)

Absolute difference†

% (95% CI)

P Value#

Days alive at home – mean (± SD)Acute-hospital care – no. (%) Hospital re-admission – no. (%)Emergency department visit – no. (%)Urgent-care centre visit – no. (%)All-cause hospital days (median [IQR])Death – no. (%)

176.7 (25.5)170 (37.7)101 (22.4)151 (33.5)

7 (1.6)0 (0-2.0)17 (3.8)

176.7 (26.1)189 (41.6)107 (23.6)166 (36.6)

13 (2.9)0 (0-2.0)18 (4.0)

1.00 (0.98-1.02)0.91 (0.78-1.07)0.97 (0.76-1.22)0.92 (0.77-1.10)0.54 (0.21-1.35)0.89 (0.60-1.33)0.95 (0.51-1.80)

0.0 (-1.7-1.7) ^

3.9 (-2.5-10.3)1.2 (-4.3-6.7)3.1 (-3.1-9.3)1.3 (-0.6-3.2)0.4 (0.2-0.6) ^

0.2 (-2.3-2.7)

0.590.230.670.330.180.580.88

CI = confidence interval; IQR = interquartile range; no. = number; SD = standard deviation; % = percentage* Relative risks and 95% confidence intervals were obtained from Modified Poisson model† Absolute differences and 95% confidence intervals were calculated from the crude proportions. ^ Absolute rate differences and 95% confidence intervals were determined based on a Normal Approximation to Poisson.# P values are from Wilcoxon, Student’s t and Chi-square test

31

49. Regarding the teaching of the patients in the use of the tablet and virtual devices, could the authors describe how time consuming was that? Did the teaching include family members?

Response: In the Online-Only Supplement 3, we have added the following on page 6.

The initial patient training on use of the tablet and devices took approximately 35 to 40 minutes. Whenever possible, patients’ family members were encouraged to participate.

Reviewer Four’s Comments

50. This is an interesting manuscript reporting the outcomes of a large multicentre RCT comparing the impact of remote virtual monitoring following discharge after surgery compared with standard care. This is particularly relevant in a COVID setting when healthcare resources are under pressure. The primary outcome was days alive at home. This was changed from acute hospital care due to an appreciation of competing outcomes. There were no differences in the primary outcome, but more medication errors were detected in the intervention group and patients in this group reported less post operative pain at prespecified time points.

The study is well designed and robustly conducted. The risk of bias is low.


51. The choice of primary outcome (days alive at home) is interesting. It is clearly relevant in the broadest sense, but given the range of surgical procedures eligible for inclusion, it raises the issue of study power given that the 30 day mortality for some of these procedures is low. The original primary outcome would seem more relevant and meaningful, despite the authors' concerns. The relevance and definition of 'medication errors' given that this includes mainly omission of doses (mainly of analgesia) is also interesting as it is unclear whether this may have impacted patient outcomes. Better compliance is likely to explain the reduced pain scores in the intervention group. The authors did not include an assessment of quality of life as this would also have been relevant in this setting.

Response: In the Discussion Section under the Limitations subheading, we have added the following text on page 19, paragraph 1.

We did not assess the impact of the intervention on quality of life.

32

52. Almost 10% of patients in the intervention group discontinued the intervention before 30 days. This is a fairly high percentage and may indicate that patients found compliance arduous. On this basis were there any plans to modify the intervention or to undertake qualitative work to explore patients' experiences?

Response: We disagree that the discontinuation rate was high; however, we do agree further research is needed to improve the intervention. In the Discussion Section, we have added the following text on page 17, paragraph 3.

Further trials are needed to improve the efficiency (e.g., not all patients need to interact with a nurse on days 1-15 and every other day from days 16-30 after hospital discharge) and cost effectiveness of virtual care with RAM in this setting. 53. There was PPI, but it would be helpful to have some elaboration on the way in which the patients shaped the study, especially with regard to the choice of primary outcome and design of the intervention.


A panel of four patient partners reviewed the daily symptom survey for clarity and perceived ease of use. Given rules on social distancing and limitations to in-person meetings, all feedback was provided via email. Patients were not involved in the trial design or analyses and did not contribute to the paper.

54. The intervention was more effective at reducing unplanned acute presentations in centres with high escalation of care but had much smaller impact in centres that did not escalate as commonly. Reasons for this should be explored.

Response: In the Results Section, we state the following text on page 13, paragraph 3.

The results of the escalation of care varied across centres. For example, the mean change in medication per patient in the virtual-care group was 1.3 in the highest, 0.7 in the intermediate, and 0.3 in the lowest escalation of care centres.

In the Discussion Section, we state the following text on page 16, paragraph 3.

It is only credible to expect virtual care with RAM to impact outcomes if these interventions identify problems and lead to changes in management.

55. The results have raised a number of questions and the authors next steps regarding potentially refining or further evaluating the intervention would add to the manuscript. Response: In the Discussion Section, we have added the following text on page 17, paragraph 3.

33

Further trials are needed to improve the efficiency (e.g., not all patients need to interact with a nurse on days 1-15 and every other day from days 16-30 after hospital discharge) and cost effectiveness of virtual care with RAM in this setting.

Reviewer Five’s Comments

56. This is a moderately large and well conducted RCT evaluating the use of virtual care post-discharge with standard post discharge care. This is an area worth investigating as it offers potential to leverage technological aids to improve care or convenience. The current situation with the COVID pandemic has given it addition importance and currency.

Broadly speaking, the trial has demonstrated equivalence between the two approaches. There was no difference in primary outcome - days alive at home - nor in most secondary or tertiary outcomes. The exception being the detection/correction of drug errors and pain reporting which were improved in the virtual care group.

Days alive at home is an appropriate primary outcome measure but it seems unlikely that it would have ever been able to demonstrate superiority of the virtual approach given how close the groups were to all 30 days being "alive at home".

Response: We respect the reviewer’s perspectives.

57. I have some questions in three areas:Trial design/intervention.There are some areas of the intervention that are not clear, for example how long did the training for patients randomised to the virtual group take.

Response: In the Online-Only Supplement 3, we have added the following on page 6.

The initial patient training on the use of the tablet and devices took approximately 35 to 40 minutes. Whenever possible, patients’ family members were encouraged to participate.

58. It is also not clear what "standard" care consisted of and what variation between centres might have been.


Canada has a universal public payment system that covers the cost of hospital and physician services, which alleviates cost as a barrier to patients obtaining these services post-discharge after surgery. In Canada, standard care for most patients after non-elective surgery would include seeing a healthcare provider within 30 days of hospital discharge. Prior to this visit, the onus is on the patient to connect with their

34

surgeon should questions arise related to the appropriate use of medications or symptoms or signs of potential complications.

59. Why was block randomisation used not individual randomisation? Blocks varied in size but what is an indicative size of a block?

Response: Block randomisation is commonly used in multicenter trials to ensure that randomisation is balanced within participating centres. Blocks sizes varied from 2-6 patients.

60. The virtual follow up is very intensive, does this make a fair comparator to standard care?

Response: Patients in the intervention group interacted with nurses and physicians more frequently than in the standard-care group due to the nature of the intervention; however, patients had the same follow-up for outcomes.

61. With a standardised approach why were there such huge variations in escalation between centres?

Response: In the Discussion Section, we state the following on page 17, paragraph 2.

Although we established predetermined thresholds for biophysical measurement in which nurses were to escalate care to a physician, nurses or physicians could adjust the frequency of biophysical measurements and parameters for alerts. Moreover, nurses decided if they had concerns about patients’ health that required a physician’s attention.

62. Clinical relevance of the outcomes.Statistical differences were observed in drug error detection, chiefly errors of omission and dose. However it is not clear if there was any clinical significance to these errors.The use of acetaminophen was different between groups, this seems likely to have had a bearing on pain scores so the two outcomes are probably largely measuring the same thing aren't they?

Response: In the Online-Only Supplement 3, we report the impact of medication errors on page 29; however, the medications errors we identified were overwhelmingly correctly immediately after detection in the virtual-care group. It is not possible to know what impact these errors would have had if they were never detected or detected at a later time.

In the Discussion Section, we have added the following statement on page 16, paragraph 2.

Detection and correction of drug errors have the potential to improve both short and long-term health.

35


Our finding that demonstrated a substantial increase in acetaminophen usage at 30-days after randomisation in the virtual-care group (i.e., absolute difference 25.2%), suggests that healthcare providers can, through virtual care, increase the use of this well tolerated drug and substantially improve pain after hospital discharge following surgery.

63. Cost effectiveness.The virtual follow up arm was very intensive with daily and then every other day interactions with a nurse, plus any escalation related interactions. Is the cost of this justified by the cost of this intervention? The trial may be measuring a Hawthorn effect of more intensive contact post discharge, for example in reported pain levels.Might there be a threshold of intensity of follow up that would achieve equivalence with standard follow up at lower cost?

Response: The economic analyses require the administrative data, which will not be available until the fall of 2021. After obtaining these data, we will subsequently publish the economic results. Regarding a possible Hawthorn effect, we also demonstrated reductions in moderate to severe pain and not just overall pain.



64. I think the paper deserves to be published but I would like to see more discussion of the above points.


Reviewer Six’s Comments 65. Very interesting study. In my opinion undoubtedly worthy of publication. For a long time, great attention has been paid to virtual care systems, with the aim of reducing as much as possible the hospitalization of patients with consequent reduction of health care costs, but without compromising the health care quality. Research in this field has received an enormous boost in the last year. During the Covid 19 pandemic, the reduction of hospitalization time, went hand in hand with the desire of surgical patients to reduce as much as possible the duration of their hospital stay and subsequent

36

hospital visits after discharge. In addition, it was coupled with the need to have a large number of beds available for the hospitalization of Covid patients.Many virtual care systems have been proposed by the industry for the post-discharge monitoring of surgical patients. In the study proposed by the Authors, the virtual care monitoring after discharge was performed with a virtual system, named RAM (remote automated monitoring). The results reported are very encouraging. However, as noted by the Authors themselves, these results and their reproducibility, are obviously related to the rigorous training of the team.


66. The only observation I could make is related to the cost. The cost of virtual system RAM is not reported, but I believe it is not inconsiderable. Moreover, the types of surgery considered in the study are reported in a very general way. For example, I do not believe that such close post-surgical monitoring is required after all types of general surgery operation. I think that such close monitoring (and the associated costs) should be reserved for particular categories of patients and after certain types of surgical intervention at higher risk of postoperative complications. In other cases, simple telephone availability might be sufficient. I think it is important to be careful not to run the risk of over-treatment, with an unjustified increase in health care costs without real benefits in outcomes.

Response: We agree with the reviewer’s comments. As stated above, we will obtain the necessary data to undertake the economic analyses in the fall of 2021.



Thank you for giving us the opportunity to revise our manuscript. We believe that the editors’ and reviewers’ comments have improved our manuscript. All members of the writing committee have seen and approved the final manuscript submitted.

PJ Devereaux Michael McGillion Joel Parlow

Flavia Borges Maura Marcucci Michael Jacka

Anthony Adili Manoj Lalu Carley Ouellette

37

Marissa Bird Sandra Ofori Pavel Roshanov

Ameen Patel Homer Yang Susan O’Leary

Vikas Tandon Gavin Hamilton Marko Mrkobrada

David Conen Valerie Harvey Jennifer Lounsbury

Rajibul Mian Shrikant Bangdiwala Ramiro Arellano

Ted Scott Gordon Guyatt Peggy Gao

Michelle Graham Rahima Nenshi Alan Forster

Mahesh Nagappa Kelsea Levesque Kristen Marosi

Sultan Chaudhry Shariq Haider Lesly Deuchar

Brandi LeBlanc Colin McCartney Emil Schemitsch

Jessica Vincent Shirley Pettit Deborah DuMerton

Angela Paulin Marko Simunovic David Williams

Samantha Halman John Harlock Ralph Meyer

Dylan Taylor Harsha Shanthanna Christopher Schlachta

Neil Parry David Pichora Haroon Yousuf

Elizabeth Peter Andre Lamy Jeremy Petch

Husein Moloo Herman Sehmbi Melissa Waggott

Jessica Shelley Emilie Belley-Cote

July 12, 2021 Dr. Nazrul Islam

Documents

Transcript of July 12, 2021 Dr. Nazrul Islam