WHO Verbal Autopsy Reference Group

Reliable and comparable data on the levels and causes of mortality are cornerstones for building a solid evidence base for health policy, planning, monitoring and evaluation. Ideally, causes of death are reported by a physician using the standard medical certificate of cause of death. However, in some settings, no physician sees the dead in order to formulate and report the medically certified cause of death. For such settings, verbal autopsy (VA) can be used to obtain the most probable cause of death for such cases.

In VA, the cause of a death is ascertained based on an interview with the next of kin or other caregivers.  The interview is done using a standardized questionnaire that elicits information on signs, symptoms, medical history and circumstances preceding death.  The cause of death, or the sequence of causes that led to death, are assigned based on the data collected using the VA questionnaire and any other available information.  Data collected using the VA questionnaire are interpreted by applying the clinical opinion of physicians or by a computer algorithm to determine the cause of death.

The primary objective of VA is to describe the pattern of causes of death that occur outside of medical care at the population level.  As such, VA has become an essential public health tool for obtaining a reasonable estimation of cause specific mortality fractions for deaths without medical certification of the cause of death.

The latest release of the WHO VA standard questionnaire can be found on the official WHO VA standards webpage. This VA Reference Group website is intended as a supplemental resource for VA users to access relevant additional tools and resources that support the implementation of VA (e.g. training materials).

For additional information, please contact the WHO Verbal Autopsy Reference Group Chairs, Dr. Erin Nichols (igd1@cdc.gov) or Dr. Daniel Cobos Muñoz (daniel.cobos@swisstph.ch).

National governments, international agencies, other public health decision-makers and researchers want VA data to inform burden of disease estimation and program evaluation; cause of death estimates fit for this purpose must meet high accuracy standards and be comparable over time and across countries.  This demand requires the standardization of methods of collecting causes of death, assessing their quality, and combining data from different sources for analysis and presentation; and this standardization, in turn, requires international collaboration of all relevant disciplines.

A WHO VA working group was established at the first meeting of the WHO Reference Group on Global Health Statistics, 9–10 December 2013. Led by Daniel Chandramohan of the London School of Hygiene and Tropical Medicine, who was the first chair of the group, it brought together experts in this field to support WHO in the development of WHO VA standards. The working group was re-constituted as a WHO VA Reference Group, which convened for the first time in Seoul, South Korea at the annual meeting of the WHO Family of International Classifications (WHO-FIC) Network in October 2018. As VA is increasingly becoming part of routine mortality data collections systems, the Reference Group supports and advises WHO regarding:

• Development and maintenance of WHO VA standards;
• Refinement of the WHO VA instrument with evidence from the field;
• Standards and recommendations for VA training and implementation;
• Advancement of methods and tools for assigning causes of death from VA interviews;
• Use of the causes of death data determined by VA;
• Use of the VA causes of death data as a complement to medically certified cause of death data;
• Quality assessment and assurance of VA data;
• Methods for comparing and evaluating VA results, including causes of death distribution estimated by different VA instruments.

The group further addresses data sharing, the development of standardized data sets, methods for validation of VA analysis, integration of VA in existing information systems, and reference definitions for causes of death of interest to ensure consistency with ICD classification rules. A copy of the terms of reference for the group is available here.

Ohio 2018
As the collection of VA data is rapidly increasing, there is a need to develop further tools and guidance regarding the analysis of VA raw data for the purpose of quality control. To meet this need the WHO VA Reference Group, together with the Bloomberg Philanthropies Data for Health Initiative and Ohio State University, hosted a workshop on 05 -16 November 2018 in Columbus, USA. More information and presentations for the training of trainers.

2018 WHO-FIC VA Reference Group Meeting
The WHO VA Reference Group (formerly Working Group) convened for the first time at the WHO Family of International Classifications (WHO-FIC) Network annual meeting in Seoul, South Korea on October 24, 2018. The WHO-FIC Network, a network of WHO Collaborating Centres, NGOs, and selected experts, was established in 1970 to support WHO’s work on international classifications. The principal role of the WHO-FIC Network is to promote the implementation, use, maintenance, development, and updating of the WHO reference health classifications, which now officially includes verbal autopsy. 

Technical Meeting on Mortality Data Analysis with Verbal Autopsy, Washington DC, 2018
Together with the WHO VA Reference Group and the Bloomberg Philanthropies Data for Health Initiative, the U.S. CDC’s National Center for Health Statistics hosted a technical workshop on 12-14 September 2018 to generate a draft set of principles, considerations, and recommendations about how countries can: 1) yield high quality statistics from VA data and 2) integrate VA results and other sources of mortality data into their mortality data processes. Meeting materials and a summary of outcomes are available. More information

Singapore ODK and GitHub 2018
To share experiences and build global capacity in focus countries of the Bloomberg Philanthropies Data for Health Initiative and beyond on the use of the Open Data Kit (ODK) platform for electronic data collection for VA and the use of the GitHub platform for questionnaire management, the Initiative together with Swiss TPH organised two workshops on 28 May - 01 June and 11-15 June 2018 in Singapore. More information

Singapore openVA Pipeline 2018
To facilitate the automatic determination of the COD from VA and to automate the processing of that data from the data collection platform (Open Data Kit) to DHIS2, the openVA Pipeline was developed. To gather developers and users, partners in the Bloomberg Philanthropies Data for Health Initiative and experts from Ohio State University, organised a meeting on 23-25 May 2018 in Singapore. More information

Accra 2017
Implementing VA within Civil Registration and Vital Statistics (CRVS) systems is a major undertaking, and global experience on how to achieve such integration and obtain COD data from community death as part of routine processes is rapidly accumulating in Bloomberg Philanthropies Data for Health Initiative countries and beyond. To share experiences and discuss emerging international best practice, partners in the Initiative together with other relevant global stakeholders convened a meeting of global “VA in CRVS” practitioners on 11-14 December 2017 in Accra, Ghana. More information

Training Materials

• Verbal autopsy standards: VA field interviewer manual for the 2016 WHO VA instrument
• Verbal autopsy standards: Manual for the training of interviewers on the use of the 2016 WHO VA instrument
• Training curriculum and agenda for the training of VA interviewers
• Verbal autopsy standards: Manual for the training of master trainers and supervisors on the use of the 2016 WHO VA instrument
• Training curriculum and agenda for the training of VA master trainers and supervisors
• Checklist of preparations required before the start of training for the training of trainer

The training materials are available here: https://drive.google.com/drive/folders/1LvfHFQ2rl2jQgtTsLgceGYUauLYf8Bfn

Cause of Death Assignment

openVA

• Purpose: openVA is a software that runs automated VA coding algorithms and is available as a package for the R statistical software.  The primary motivation is to help users compare results across different algorithms using a single tool. The software is freely available and open source, so users have access to the code and can see exactly how the program works.
• Components: The available algorithms include InSilicoVA, InterVA4, InterVA5, and Naïve Bayes Classifier, and Tariff.  Tariff2 will be available in the near future.
• URL:  https://openva.net
• GitHub URL: https://github.com/verbal-autopsy-software/openVA
• R CRAN URL: https://cran.r-project.org/package=openVA
• openVA vignette URL: https://cran.r-project.org/web/packages/openVA/vignettes/openVA-vignette.html

openVA pipeline

• Purpose: The openVA pipeline is an open source software that automates the processing of VA data by downloading records, assigning a cause of death, and posting the results to a DHIS 2 server.  Data and results are also stored locally on the computer in an encrypted SQLite database, referred to as the transfer database.  The openVA pipeline can be set to run automatically and is particularly useful when a large amount of data is being collected on a continual basis.
• Data flow:  The openVA pipeline performs 3 general tasks: (1) downloads new records from an ODK Aggregate server; (2) runs openVA to assign causes of death with an algorithm specified by the user; and (3) uploads the VA data with assigned causes and metadata to a DHIS server that has the VA module
• Software implementation:  The openVA pipeline is written in the Python programming language and depends on several other programming languages and tools, including: ODK Briefcase, R, Java, and SQLite.
• GitHub URL:  https://github.com/verbal-autopsy-software/openva_pipeline
• Python Package Index (PyPI) URL:  https://pypi.org/project/openva-pipeline/
• Documentation URL: https://openva-pipeline.readthedocs.io/en/latest/

Algorithms for assigning causes of death to verbal autopsy data

• The InterVA algorithms combine the reported symptoms with probabilities (produced by medical experts) of how likely the observed symptoms are if a particular cause is responsible for the death.  The cause with the greatest propensity (above a given threshold) is assigned to the death – if none of the propensities exceed the threshold, then the death has an “indeterminate” cause.  The distribution of deaths across the data serves as an estimate of the cause-specific mortality fraction (CSMF), or the percent of all deaths that are due to each cause. http://www.interva.net/
• InSilicoVA combines both the presence and absence of each symptom with probabilities of how likely it is to observe each symptom if a particular cause is responsible for the death.  This algorithm also includes a statistical model for the CSMF, which is linked to the causes assigned to each individual in the data to ensure the two estimates are consistent.  InSilicoVA also provides estimates of uncertainty for all of the outputs, e.g., individual cause assignments and the CSMF. https://cran.r-project.org/web/packages/InSilicoVA/index.html
• The Naïve Bayes Classifier algorithm combines both the presence and absence of each symptoms (similar to InSilicoVA), but uses the distribution of assigned causes in the data to estimate the CSMF (similar to InterVA). https://cran.r-project.org/web/packages/nbc4va/vignettes/nbc4vavignette.html
• The Tariff algorithm starts by calculating tariffs that measure how informative each symptom is for each cause using the symptom-cause patterns observed in the PHMRC gold standard VA validation study dataset. Tariffs are then summed across all positive responses, and the cause with the highest ranked Tariff Score is assigned to the death.  The distribution of deaths across the data is used to estimate the CSMF.
• Tarrif2 is implemented in the SmartVA Analyze software and implements the Tariff algorithm with several modifications: (1) tariffs that are less reliable are removed; (2) the preparation of data from the open-ended narratives is standardized and the associated tariffs are validated by physician review; (3) implausible causes and unlikely symptom-cause associations in the PHMRC gold standard validation study dataset were adjusted or removed; and (4) deaths (in the PHMRC dataset) that lack enough information for assigning a cause of death are removed. https://github.com/ihmeuw/SmartVA-Analyze; https://www.healthdata.org/data-tools-practices/verbal-autopsy

Implementation Support  

• openVA Pipeline 

• Integrating community-based verbal autopsy into civil registration and vital statistics (CRVS): system-level considerations
  Don de Savigny, Ian Riley, Daniel Chandramohan, Frank Odhiambo, Erin Nichols, Sam Notzon, Carla AbouZahr, Raj Mitra, Daniel Cobos Muñoz, Sonja Firth, Nicolas Maire, Osman Sankoh, Gay Bronson, Philip Setel, Peter Byass, Robert Jakob, Ties Boerma & Alan D. Lopez. (2017) Global Health Action, 10:1, DOI: 10.1080/16549716.2017.1272882

• VA Costing and Budgeting Tool
  The VA costing tool will help you estimate the resources required to implement VA as part of the routine CRVS system. It is a Microsoft Excel file that will guide the team during the data collection or analysis. It provides the total financial and economic cost of implementation as well as unit cost (e.g. cost per VA) relevant for policy making.
  It is a flexible tool that can be adapted to different implementation scenarios. It also has a modelling function to project cost over time according to user specified assumptions. Download the tool here. For more information contact daniel.cobos@swisstph.ch.

• VA Sample Size Calculator
  The purpose of this package is to assist countries with scale up and rollout planning for the application of verbal autopsy (VA) as a function of a national Civil Registration and Vital Statistics (CRVS) system. Users will include those tasked with designing and managing the national CRVS VA system, supported by a governing body such as a National Mortality Committee of the National CRVS Committee.
  In addition to discussing strategies and principles, this guidance package proposes approaches to conventional cluster sampling methods and provides the statistical rationale, logic, mathematical formulations, and a worked example, for: i) calculating the required number of clusters needed in a VA cluster sample design; and ii) drawing the needed clusters from a national sampling frame. Download the tool here. For more information contact daniel.cobos@swisstph.ch and d.desavigny@swisstph.ch.

Use of Verbal Autopsy Data

• Guidelines for Interpreting Verbal Autopsy Data
  Guidelines developed by a technical working group of the Bloomberg Philanthropies Data for Health (D4H) initiative that provide five steps for users of VA to follow to help them interpret and present their VA data, and thus improve the VA’s utility for public health decision-making.

• Guidance on the Analysis of Verbal Autopsy (VA) and Medical Certificate of Cause of Death (MCCD) Data (under review)
  This document provides guidance on how to aggregate mortality data from multiple sources and to integrate the results into national statistical processes. While such guidance is expected to evolve as more countries gain additional experience in putting mortality data to use, this initial guidance aims to demonstrate the utility of VA data alongside MCCD data, moving countries towards using VA data now. The guidance was developed by a technical working group of the Bloomberg Philanthropies Data for Health (D4H) initiative and is currently under review by the WHO VA Reference Group.

Governance and Legal Aspects

• Principles and Recommendations for a Vital Statistics Systems, Revision 3
  This document of the United Nations Statistics Division presents a set of principles and recommendations and provides guidance on establishing a functioning system for collecting, processing and disseminating vital statistics; improving sources of vital statistics, primarily the functioning of the civil registration system and its components; and the role of complementary sources of vital statistics, such as population censuses, household surveys and public-health records. The version referenced here includes highlights of points relevant to verbal autopsy implementation.

• Guidelines  on the Legislative Framework for Civil Registration, Vital Statistics and Identity Management (draft available for review)
  These Guidelines offer a tool for developing a strong rights-based legal framework for civil registration, vital statistics and identity management (CRVSID) systems. Prepared by the United Nations Statistics Division and Vital Strategies, with financial support and technical input from the Bloomberg Data for Health Initiative,these Guidelines complement the Principles and Recommendations for a Vital Statistics System, Revision 3 and the Handbooks on Civil Registration and Vital Statistics Systems. The version referenced here includes highlights of points relevant to verbal autopsy implementation.

Needs and demands for standardization led to the development and publication of the WHO 2007 VA standard tools, which many researchers adopted and adapted to accommodate local needs and context. To support the compilation of national mortality statistics and serve the needs of countries’ CRVS systems, WHO developed a VA instrument targeted for routine use, designed for all age groups - including maternal and perinatal deaths; the latest release, the WHO 2016 VA instrument, is fully compatible with publicly available automated analytical software for assigning the cause of death (SmartVA, InterVA, InSilicoVA).

The systematic application of the 2016 version of the WHO VA instrument will facilitate consistency and cross-comparability of VA-derived mortality data in the context of routine mortality surveillance. To support the maintenance and update process of the WHO VA standards and optimize the field experience for interviewers and respondents, it is important that users report issues encountered with the application of the VA instrument. Issues with the instrument should be submitted to the Public GitHub platform of the 2016 WHO VA instrument—https://github.com/SwissTPH/WHO_VA_2016– which contains a list of submitted issues and related feedback. To report an issue via the Public GitHub platform, please send an email to who.va.issues@swisstph.ch. To submit other input or inquiries, including  issues involving private and confidential information, translations of the WHO VA instrument, and other VA implementation support materials, please email WHO directly at verbalautopsy@who.int.

The instrument will be reviewed in 2020, based on users’ feedback and evidence from the field. Users are asked to share the information collected in their VAs in a way that allows assessment of validity and feasibility of the questions in the field. Please request additional information by sending an email to verbalautopsy@who.int.

The latest release of the WHO VA standard questionnaire can be found on the official WHO VA standards webpage (https://www.who.int/standards/classifications/other-classifications/verbal-autopsy-standards-ascertaining-and-attributing-causes-of-death-tool).

As there is anticipation of widespread community deaths associated with coronavirus disease 2019 (COVID-19), VA can fill a critical gap in measuring the mortality from COVID-19 for deaths which occur outside of a healthcare setting. WHO has issued international guidelines for medical certification of cause of death and classification according to the International Classification of Diseases (ICD) (ICD mortality coding) of COVID-19 as a cause of death (WHO Guidance), which are applicable when a physician is available to complete the medical certificate of cause of death (MCCD).

Guidance has also emerged for the use of rapid mortality surveillance for total, all-cause mortality, where counts of deaths, stratified by age, sex, and location of death, can be used – even in the absence of cause of death information – to support epidemic awareness in low-resource settings (https://www.who.int/publications/i/item/revealing-the-toll-of-covid-19). Also see a recent webinar on this topic: https://vimeo.com/user13782109/review/411057982/c348da3aa5, slides

The WHO VA Reference Group is reviewing the emerging evidence on the COVID-19 pandemic to inform the use of VA in the context of the pandemic. A set of questions to identify COVID-19 deaths has been included in version 1.5.3 of the electronic (ODK) 2016 WHO VA instrument, and guidance on the new questions has been added in the VA Field Interviewer Manual). Additionally, considerations for the use of VA in the context of COVID-19 have been added to the WHO VA standards manual. These updated documents are available on the WHO VA standards website: https://www.who.int/standards/classifications/other-classifications/verbal-autopsy-standards-ascertaining-and-attributing-causes-of-death-tool.
 

Link to recordings

Link to recordings

Link to recordings

Link to recordings

Prior to the establishment of the VA-COP, a group connected via a monthly call to understand how VA information is processed, to share information and updates about tools to support VA implementation, and to discuss needs and opportunities to improve the IT tools used to support VA implementation. Recordings of these calls are available here: Recorded calls