Analytics in VigiLyze

Analytical methods developed by the UMC Research team are integrated in VigiLyze.

The primary research tools used for signal detection in VigiBase – for assessing, ranking and verification - are described in the Research & Scientific Development pages, for which there are links at the bottom of this page. On this page you’ll find the primary statistical concepts and methods associated with VigiLyze.

Your questions answered

The following concepts and methods underlie the results that VigiLyze will deliver for your search enquiry.

What is the IC value?

IC stands for 'Information Component' which is an indicator value for disproportionate reporting when using the method for disproportionality analysis developed by UMC. As with the EBGM, PRR or ROR, this is a way to relate observed and expected values to find drug-adverse effect combinations that have been reported more often than one would expect. These are statistical methods used when mining data in large data sets such as VigiBase. (EBGM = Empirical Bayes Geometric Mean, PRR = Proportional Reporting Ratio, ROR = Reporting Odds Ratio)

IC = log2 ((Nobserved + 0.5)/(Nexpected + 0.5))

where Nexpected = (Ndrug * Neffect) / Ntotal

Nexpected: the number of case reports expected for the drug-effect combination

Nobserved: the actual number of case reports for the drug-effect combination

Ndrug: the number of case reports for the drug, regardless of effects

Neffect: the number of case reports for the effect, regardless of drug

Refs: Bate A, Lindquist M, Edwards IR, Olsson S, Orre R, Lansner A, De Freitas RM. A Bayesian neural network method for adverse drug reaction signal generation. European Journal of Clinical Pharmacology, 1998, 54(4):315-321.

Norén GN, Hopstadius J, Bate A. Shrinkage observed-to-expected ratios for robust and transparent large-scale pattern discovery. Statistical Methods in Medical Research, 2013, 22(1):57-69.

What is IC025?

IC025 is the lower end of a 95% credibility interval for the Information Component. A positive IC025 value is the traditional threshold used in statistical signal detection at UMC. IC025 is also used as one component of the disproportionality parameter in the vigiRank method.

What is IC0005?

The lower endpoint of a 99.9% credibility interval for the Information Component, IC0005 is used to support analysis of subgroup-specific associations between substances and effects analogously to how IC025 is used for general analysis of substance-effect associations: a positive value for IC0005 suggests, but does not prove, a causal relation between the substance and the reaction in the subgroup under consideration. The reason that analysis of subgroup-specific associations requires a wider credibility interval than standard analysis is that many more potential associations are investigated; for each drug-reaction pair, one IC value is computed for each age group, for each sex, for each country, and for other variables. This decreases the risk of detecting spurious positive associations.

Ref: Hopstadius J, Norén GN. Robust discovery of local patterns: subsets and stratification in adverse drug reaction surveillance. Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium, 2012, 265-274.

What is the significance of Ncomp09?

When given in the VigiLyze Combinations list, Ncomp09 is a count of the number of ICSRs in a case series for a drug-adverse effect combination that have a vigiGrade completeness score of 0.9 or higher. When the vigiRank method was developed, a number of different parameters were evaluated for their predictive ability to find true emerging signals. One parameter shown to have predictive value was high completeness of the ICSRs in the case series, and the threshold that provided the best predictive performance in this context was a vigiGrade completeness score of 0.9 or higher.

What is Nfatal?

The number of ICSRs in a case series for a drug-adverse effect combination that had a fatal outcome. Information from several parts of the ICSR is analysed to determine the fatal outcome of the case: terms that indicate fatality (e.g. ‘completed suicide’); date of death; cause of death or performed autopsy; outcome is fatal; seriousness criterion is death, among others. The processed ‘fatal’ value does not define a causal relationship between the patient’s death and the drug or effect.

What is Nserious?

The number of ICSRs in a case series for a drug-adverse effect combination that are considered serious according to any of several different criteria; marked ‘serious’ by the sender; one or more seriousness criteria given; processed as fatal = Y by the criteria described earlier. Terms are not considered.

** References can be found under Scientific Publications.

What is vigiMatch?

An automatic algorithm for detection of suspected duplicates, vigiMatch, is available by default in VigiLyze from 2017-01-11. For suspected duplicates, only the report with the highest vigiGrade completeness score is shown in the ICSR list and used in the calculations for the statistics and data mining views, thus minimizing efforts spent on statistically disproportionate drug-ADR combinations boosted by the presence of suspected duplicates. If a report has suspected duplicates, this is indicated by a double arrow in the ICSR list. Clicking this followed by “Show suspected duplicates” at the bottom of the view presents a complete list of reports

It is important to note that vigiMatch does not have perfect performance, and some suspected duplicates will be “false positives”, i.e. reports that are not true duplicates, but have been marked as such by vigiMatch. In our experience (see also the evaluation in Tregunno et al) reports that are not true duplicates but otherwise related are particularly problematic in this respect. This is because vigiMatch compares two extreme hypotheses: i) that the reports are of completely different origins and refer to independent cases, and ii) that the reports describe the same case. Pairs of reports that fall somewhere in between these extremes are more likely to result in false positives. Examples include reports of separate adverse events in the same patient and reports for different patients extracted from the same scientific paper. Reports from mass vaccination events (for example of a school class) or from studies may generate many false positives, because they tend to match on key information such as the date of onset, country of origin, vaccine and (sometimes) adverse event – especially if the drug and/or adverse event is rare.

Conversely, there will also be “false negatives”, i.e. true duplicates that have not been highlighted by the algorithm. A specific challenge here is lack of information. Many reports do not have enough information to allow for a reliable match even with a report that matches perfectly.

vigiMatch uses a statistical model that scores pairs of reports, taking into account the amount of matching and mismatching information:

The scores received for each individual field are summed together to obtain a total score for the report pair. For a report pair to be automatically flagged as suspected duplicates, the score needs to reach above a certain threshold based on the expected number of duplicates in VigiBase. The vigiMatch method for detecting suspected duplicates in VigiBase has been used routinely in UMC signal detection since 2014. 

Refs:
Norén GN, Orre R, Bate A, Edwards IR. Duplicate detection in adverse drug reaction surveillance. Data Mining Knowledge Discovovery 2007;2007(14):305–28. 
Tregunno PM, Fink DB, Fernandez-Fernandez C, Lázaro-Bengoa E, Norén GN. Performance of Probabilistic Method to Detect Duplicate Individual Case Safety Reports. Drug Safety, 2014, 37(4):249-258.

Please contact us if you need additional information

Read more

Last modified on: April 20, 2018
.