01
Name the research question and design before judging the conclusion.
RESEARCH LITERACY / VNS STUDIES
A study headline can tell you whether a result was positive. It cannot tell you whether the method was reproducible, the comparator credible, the effect precise, or the finding relevant to another device. This guide shows what to check before you carry a result forward.
The practical answer
01
Name the research question and design before judging the conclusion.
02
Reconstruct the exact population, device, site, protocol and comparator rather than relying on the acronym VNS or taVNS.
03
Read effect estimates, confidence intervals, missing data and harms—not only p values or the word significant.
04
Keep target engagement, physiological markers, meaningful outcomes and product-specific evidence as separate layers.
12-STEP READING FRAMEWORK
Use the same sequence for a randomized trial, mechanistic paper, safety study or systematic review. Some checks carry different weight by design, but none should disappear.
01
02
03
04
05
06
07
08
09
10
11
12
1 / START WITH THE QUESTION
Begin with the research question, not the abstract conclusion. A mechanistic study can show that a pathway or marker changed. A feasibility study can show that a procedure was workable. A safety study can describe adverse events at a tested exposure. A randomized trial can estimate a between-group effect. Those answers are useful, but they are not interchangeable.
| Design | What it can help answer | What to inspect | What it cannot establish alone |
|---|---|---|---|
| Randomized parallel trial | Whether outcomes differ between assigned groups | Allocation, concealment, blinding, prespecified outcomes, missing data | Universal effectiveness or relevance to another protocol |
| Crossover trial | Within-person differences between conditions | Sequence, washout, carryover, period effects, blinding | Long-term effects without suitable follow-up |
| Mechanistic or neuroimaging study | Whether stimulation changes a proposed pathway or physiological signal | Timing, site, control, signal definition, multiplicity | Clinical or everyday benefit |
| Safety or tolerability study | Which events occurred at a tested exposure | Collection method, denominators, severity, causality, discontinuation | Efficacy or safety outside the tested population and dose |
| Systematic review or meta-analysis | What a defined body of studies shows together | Search, eligibility, risk of bias, compatibility, missing evidence | Quality beyond the included studies or product equivalence |
2 / RECONSTRUCT WHAT WAS TESTED
The label taVNS is not a complete intervention description. Readers need the participant group, exact device, electrode, anatomical site, side of stimulation, waveform and total exposure. Farmer and colleagues proposed tVNS-specific minimum reporting standards because incomplete method descriptions prevent replication and responsible comparison.
| Domain | Examples | Why it changes interpretation |
|---|---|---|
| Anatomy and contact | Neck or ear site, side, electrode geometry, skin preparation | Changes current paths, sensation and the assumed neural target |
| Electrical parameters | Frequency, pulse width, waveform, amplitude or titration rule | Changes delivered charge, comfort and physiological response |
| Exposure | On/off cycle, session duration, schedule, total sessions | Changes cumulative dose and the time window for benefits or harms |
| Delivery and fidelity | Operator, app, training, adherence, actual settings | Shows whether participants received the planned intervention |
| Co-interventions | Medication, therapy, usual care, lifestyle changes | Can create or obscure differences between groups |
3 / WHAT WAS THE COUNTERFACTUAL?
A controlled study estimates a difference between conditions. The interpretation therefore depends on what the comparison group experienced. In electrical stimulation research, sham may imitate contact, sound, attention or sensation, but it can also produce somatic or physiological effects. ‘Sham-controlled’ is not enough detail by itself.
CREDIBILITY
Compare sensations, device appearance, session contact and guesses about allocation. Blinding can fail even when the protocol says double-blind.
BIOLOGICAL ACTIVITY
Low intensity, another ear site or brief stimulation may be less active, but biological inactivity should not simply be assumed.
EQUAL ATTENTION
Differences in staff attention, app guidance, expectation or usual care can contribute to observed outcomes.
ANALYSIS
Allocation guesses can be informative, but they are influenced by both sensations and whether a participant improved.
4 / READ THE NUMBER, NOT ONLY THE LABEL
Definition
The outcome intended to answer the main research question, ideally prespecified with a defined measure, time point and analysis.
Definition
The measured difference, ratio or change used to describe the magnitude and direction of the observed effect.
Definition
A range showing the precision of the estimate under the statistical model. A wide interval can include materially different conclusions.
A p value addresses compatibility with a statistical model under a null hypothesis. It does not tell you whether an effect is large, useful, precise, unbiased or relevant to a new product. Read the estimate, interval, units, baseline risk and outcome scale together.
| Layer | Question | Common error |
|---|---|---|
| Outcome | Was this primary, secondary, exploratory or post hoc? | Promoting an exploratory signal to the main conclusion |
| Magnitude | How large was the absolute and relative difference? | Using only percentage change or a p value |
| Precision | What range of effects remains compatible with the data? | Treating a wide interval as a settled estimate |
| Meaning | Would the difference matter to participants or decisions? | Equating statistical significance with practical importance |
5 / KEEP THE EVIDENCE LAYERS SEPARATE
Target engagement asks whether an intervention affected its predicted target. A biomarker or imaging response may support that question. A clinical or lived-experience outcome asks whether participants improved in a meaningful way. One layer can strengthen a mechanistic argument without proving the next layer.
DELIVERY
Electrical output, contact or perceived sensation can confirm delivery conditions, but not selective neural engagement.
TARGET
A prespecified, credible measure should show the predicted change in a molecular, neural, physiological or behavioral target.
MEDIATION
A stronger causal chain tests whether change in the target is associated with or mediates change in the meaningful outcome.
OUTCOME
The outcome should be suitable, prespecified, measured at an appropriate time and interpreted with effect size and uncertainty.
6 / LOOK FOR WHAT COULD DISTORT THE RESULT
Risk of bias is not a general impression of whether a paper looks scientific. It asks whether features of the design, conduct, analysis or reporting could systematically move the result away from the truth. RoB 2 organizes this for randomized trials across randomisation, deviations from intended interventions, missing outcomes, outcome measurement and selection of the reported result.
7 / MOVE FROM ONE PAPER TO THE BODY OF EVIDENCE
A systematic review uses explicit methods to identify, select and appraise relevant studies. A meta-analysis statistically combines compatible effect estimates. A review may use narrative synthesis instead when studies are too different. Neither label guarantees that the underlying evidence is complete, unbiased or directly applicable.
| Concept | What it describes | What to inspect |
|---|---|---|
| Systematic review | A transparent method for finding and synthesising a defined body of evidence | Question, search, eligibility, duplicate processes, risk of bias, missing evidence |
| Meta-analysis | A statistical summary of compatible study results | Effect measure, model, heterogeneity, weighting, sensitivity analyses, small-study effects |
| Certainty of evidence | Confidence in an effect estimate across a body of evidence for a defined outcome | Risk of bias, inconsistency, indirectness, imprecision and publication bias |
| Recommendation | A judgment that also considers benefits, harms, values, resources and context | Who made it, for which population, and with what strength and certainty |
8 / DECIDE WHAT THE STUDY CAN TRAVEL TO
The last step is not ‘Was the study positive?’ It is ‘What exactly can this result support?’ Applicability depends on the match between participants, intervention, comparator, outcome, setting and follow-up. Product transfer adds another layer: hardware, placement, protocol, intended use and regulatory context should be relevant to the new device and claim.
Worked examples
Use the same reading framework while keeping the design-specific lesson visible.
The study examined brain responses during electrical stimulation at an external-ear location associated with auricular vagal pathways.
Reading lesson
Read it for pathway and target-engagement context. Do not relabel an imaging response as proof of symptom improvement or of another device’s effect.
The review synthesised adverse-event reporting across human taVNS studies and assessed the wider safety literature.
Reading lesson
Read the pooled safety picture together with protocol variation, inconsistent event collection, study exclusions and the tested exposure window.
The review assessed taVNS studies measuring heart rate variability and highlighted variation in protocols and HRV methods.
Reading lesson
Read HRV as a physiological outcome with methodological limits. It is not automatically target engagement, clinical benefit or a product claim.
The review combined a small clinical evidence base on taVNS and insomnia-related outcomes.
Reading lesson
Read pooled findings together with certainty of evidence, study quality, protocol compatibility and the distinction between the tested devices and a consumer wellness product.
PRACTICAL APPRAISAL
Record what is reported, what remains unclear, and whether each limitation changes the claim you are willing to repeat.
Check 01
What exact question was the study designed to answer?
Check 02
Who was studied, where, and under which eligibility rules?
Check 03
Which device, anatomical site, electrode and delivery method were used?
Check 04
What frequency, pulse width, waveform, intensity, duty cycle and session exposure were delivered?
Check 05
What did the control group receive, and could it have been biologically active?
Check 06
How were groups assigned, concealed and kept unaware of allocation?
Check 07
Which outcome was primary, how was it measured, and when?
Check 08
How large was the difference, and how precise was the estimate?
Check 09
Who entered the analysis, who dropped out, and how were missing data handled?
Check 10
How were adverse events defined, collected, compared and linked to discontinuation?
Check 11
Was the study registered, were plans accessible, and who funded or influenced it?
Check 12
Does this exact device, protocol, population and intended use match the new claim?
Frequently asked questions
No. Randomisation can reduce confounding, but reliability still depends on allocation concealment, blinding where feasible, protocol fidelity, outcome measurement, missing data, analysis choices, sample size and selective reporting.
No. A p value does not show effect size, precision, practical importance, bias or applicability. Read the estimate and confidence interval, the prespecified outcome, the comparison used and the wider evidence.
No. A meta-analysis is a statistical synthesis. Its value depends on the review methods, risk of bias, compatibility of studies, missing evidence, model choices and certainty of evidence for each outcome.
Yes. Electrical stimulation at low intensity or another site can create sensation and may influence somatic or physiological pathways. The sham must be described and its credibility and possible activity considered.
No. HRV and fMRI can provide physiological or mechanistic information. They do not automatically establish target engagement, symptom improvement, clinical importance or a product-specific benefit.
Only if a defensible product-specific bridge exists. The exact device, electrode, placement, protocol, population, intended use and outcome must be relevant. General field research remains educational context unless that bridge is documented.
Source basis
Hopewell S, Chan A-W, Collins GS, et al.
BMJ
Current reporting standard for randomised trials, including registration, protocols, harms, missing data, intervention delivery and limitations.
DOI: 10.1136/bmj-2024-081123
Hopewell S, Chan A-W, Collins GS, et al.
BMJ
Explains why complete reporting is necessary before readers can judge the reliability and validity of a trial.
DOI: 10.1136/bmj-2024-081124
Page MJ, McKenzie JE, Bossuyt PM, et al.
BMJ / PRISMA Executive
Reporting framework for systematic reviews, including search, selection, synthesis and transparent flow of included evidence.
DOI: 10.1136/bmj.n71
Higgins JPT, Thomas J, Chandler J, et al., editors
Cochrane
Methods reference for effect measures, heterogeneity, missing evidence, meta-analysis and interpretation of intervention evidence.
Open sourceSterne JAC, Savović J, Page MJ, et al.
BMJ / Risk of Bias tools
Structured assessment of bias arising from randomisation, deviations, missing outcomes, outcome measurement and selective reporting.
DOI: 10.1136/bmj.l4898
GRADE Working Group
GRADE Working Group
Living official resource for judging confidence in an effect estimate across a body of evidence; chapters are being added and unreplaced sections still refer readers to the original GRADE Handbook.
Open sourceFarmer AD, Strzelczyk A, Finisguerra A, et al.
Frontiers in Human Neuroscience
tVNS-specific reporting recommendations for anatomy, device, electrodes, parameters, controls, participants, outcomes and safety.
DOI: 10.3389/fnhum.2020.568051
PMID: 33854421
Hoffmann TC, Glasziou PP, Boutron I, et al.
BMJ
Shows why intervention and comparator details must be complete enough for replication, including dose, delivery, tailoring and fidelity.
DOI: 10.1136/bmj.g1687
PMID: 24609605
National Institute of Mental Health
National Institute of Mental Health
Defines target engagement as verification that an intervention affected its predicted target and keeps that question separate from clinical outcomes.
Open sourceContinue reading
Start with the wider Neuvago evidence layer and its boundaries.
Open research hubApply the framework to individual VNS, taVNS, HRV, safety and sleep papers.
Browse studiesPlace one paper inside the broader implanted and non-invasive VNS field.
Explore VNS researchReview tVNS and taVNS protocol variables, controls and target-engagement limits.
Explore tVNS researchConnect study interpretation with outer-ear anatomy, electrodes, sham and placement.
Explore auricular researchRead adverse-event evidence with exposure, exclusions and reporting quality visible.
Review safety researchUse the glossary definition before interpreting trial labels.
Open definitionDistinguish a transparent review method from a general narrative overview.
Open definitionUnderstand pooling, heterogeneity and why precision can be misleading.
Open definitionSeparate confidence in a body of evidence from the design label of one paper.
Open definitionReview why a control condition can affect both blinding and physiology.
Open definitionKeep target verification separate from clinical and wellness outcomes.
Open definitionResearch principle
The goal is not to distrust every study. It is to identify the exact question, method, estimate and boundary before evidence is carried into another context.