A 54-Study Phishing Review Is a Reason to Ask Better Questions, Not Rank Careless People

The tempting version of a phishing research story is a ranking: which people are easiest to fool, which personality predicts a mistake, and which training makes the problem disappear. That version offers a tidy explanation. It also asks evidence to do something that a collection of studies may not support: predict a particular person’s behavior in a particular situation.

A 2026 review by Mark Spicer and Jason R. C. Nurse provides a useful occasion to slow that reasoning down. The author’s published summary says the review included 54 studies and considered individual factors, environmental factors, and the gap between intentions and behavior. It distinguishes stronger agreement from contested, null, or thin evidence. This article uses that summary to frame questions, not to invent a factor-by-factor ranking from a full paper we could not inspect.

Start with the unit of evidence

A study is not a vote cast by an equally informed person. One experiment might examine whether participants recognize a suspicious message in a survey. Another might observe a response to an unexpected email during ordinary work. Both concern phishing, but their outcomes are not interchangeable. Recognizing a screenshot is different from deciding whether to comply while a customer is waiting.

Before asking what the research proves, ask what was actually measured. Was the outcome a click, a submitted password, an approved permission, a report, or an answer to a hypothetical question? Was it measured once or repeatedly? Did participants know that security was the subject? These are not academic decorations. They determine which operational decision the result can reasonably inform.

The word susceptibility can conceal these differences. A team might use it to mean opening a message, while a reader imagines losing control of an account. A careful report defines the behavior before attaching a percentage or a label. Our earlier discussion of why click rates need context explains the measurement problem. Here the additional question is how several different studies can be combined without erasing those distinctions.

A practical evidence-reading table

This table is an editorial reading aid, not the review’s scoring system. Its purpose is to keep interpretation attached to the evidence available. If a news summary cannot answer these questions, the appropriate response is narrower language, not a confident guess about what the missing methods probably showed.

Imagine two inboxes, not two kinds of person

Consider an illustrative comparison. A purchasing coordinator receives an unexpected invoice request during a month-end reconciliation. A designer receives the same request while working on an unrelated project. The document may contain identical wording, but it fits one person’s immediate work much better. That does not establish which person is more careful, intelligent, or technically capable.

Now change the environment rather than the message. The coordinator has a known supplier directory, a backup reviewer, and authority to delay payment. In a second version, the directory is outdated, the reviewer is unavailable, and delays are criticized. A knowledge quiz would not capture all of those differences. A useful investigation would ask which routes were available when the decision had to be made.

This is consistent with the purpose of the NIST Phish Scale, which helps practitioners rate human detection difficulty for phishing emails. It is not a personality test. Treating message difficulty as part of the measurement makes comparisons more intelligible; it does not produce a complete model of every attack or guarantee a particular outcome.

Three levels of explanation deserve three different questions

What information did the person have?

Reconstruct the available view rather than the investigator’s later view. A mobile notification may show less context than a desktop message. A legitimate contact may have introduced the conversation earlier. A suspicious attachment may have arrived within a real project. These possibilities need evidence; they should not become excuses invented after the fact. But leaving them out also creates a distorted account.

A useful interview question is, “What made this request fit the work you were doing?” It invites a description of cues. “Why did you ignore the signs?” assumes the signs were visible and their meaning obvious. The first question can expose a process dependency. The second can produce an apology while leaving that dependency untouched.

What action was realistically available?

Knowing a verification rule does not mean someone has the means to follow it. Was there a working contact route? Could the task be saved and resumed? Did a supervisor accept a delay? Did the system offer an alternative to granting broad access? A policy that requires a resource nobody can reach is not equivalent to a usable control.

For a small team, a single concrete change may be more useful than another long awareness presentation: record the backup approver in the payment workflow, for example. That is an editorial proposal to test locally, not a finding that this specific intervention was proven by the 54-study review. Distinguishing recommendations from reported findings keeps the analysis honest.

What happened after the first action?

A click does not describe the whole event. A browser may block a destination. A person may stop at a permission request. A colleague may report a similar message. A team may contain exposure before a payment leaves. Record these stages separately, because a system can improve at one stage while remaining weak at another.

The reverse is also possible: fewer clicks can coexist with a more damaging exception process. A confident dashboard therefore needs a narrative about the controls around the measured behavior. Readers should be skeptical when one number stands in for an entire security program, especially if no one explains what happened after the number was recorded.

What to do when the full study is unavailable

Abstracts and author summaries are useful discovery tools. They are not substitutes for every table, inclusion rule, and limitation in a paper. If only a summary is available, describe what the authors say at that level. Do not supply missing effect sizes, invent a hierarchy of factors, or imply that all included studies point in the same direction.

For this review, the summary supports discussing the authors’ multi-factor framing. It does not let this article tell you whether any specific age group, personality trait, or training format had a particular evidence verdict. Readers seeking those details should consult the published review. Access restrictions are a limit on our claim, not an invitation to fill in a plausible result.

Turn a research discussion into a small investigation

A team can begin with one recurring decision, such as handling an unexpected request to share a folder. Write down the normal route, the exception route, the evidence visible to the requester, and who can stop the action. Then ask a few people who perform the task to describe their actual process. This is a local workflow review, not a psychological assessment.

Keep the questions specific. Can a staff member identify the data owner? Does the permission screen say who receives access? Is there an easy way to request a narrower scope? What happens if the owner is away? Testing these questions may reveal a missing dependency that an annual knowledge score cannot describe. Document findings without ranking individuals as weak links.

Choose one change, state what improvement would look like, and check for side effects. A new approval step might reduce accidental sharing but create a backlog that encourages unofficial workarounds. A shorter route might help reporting but remove context needed for triage. The goal is not to maximize friction or eliminate every decision. It is to make the appropriate decision feasible.

The conclusion should remain smaller than the people

Research can help explain patterns. It should not turn a person into a permanent category after one event. The more useful conclusion is conditional: under these circumstances, this task produced this behavior, and these surrounding controls deserve examination. That formulation leaves room for uncertainty and gives a team something it can change.

Read the evidence alongside our analysis of exposure versus breach statistics. Both problems begin when an accurate observation is made to answer a different question. Better security reporting keeps the observation, the interpretation, and the proposed action separate.