Without Transparency, Threat Monitoring Tools Are Setting Up Users to Fail

As Open Measures’ chief of staff, I regularly speak with research analysts and threat monitoring teams who use our platform to ask them about their experiences, how it compares to other tools they’ve used, and if there are things we can do to make their work easier. In nearly all of these conversations, professionals share their frustrations that the research tools they rely on lack comprehensive coverage of the sources they need to monitor.

For years, digital threat detection tools were designed to watch for activity on major platforms and flag harmful content when it surfaced at high volumes. But this approach has been rendered obsolete by evolutions in the online landscape, as threat actors have spread out across many niche communities and alternative platforms.

Despite contrary claims, the influence of these relatively fringe spaces has not diminished in recent years; in many ways, their impacts are as pronounced as ever. Even as mainstream platforms have seen a sizable uptick in toxic content after curbing their enforcement programs, alternative platforms continue to serve as vital launchpads for violent extremism, online abuse, cybercriminal activity, disinformation campaigns, and harmful narratives.

But the inherent limits of many tools prevent analysts from seeing the entire ecosystem. Many teams have told us that they’ve tried to address these coverage gaps by using multiple tools at once: a process they describe as time-consuming, imperfect, and emotionally taxing. This piecemeal approach drastically slows their response time, results in them missing important signals, and contributes to analyst burnout.

Teams that broaden their monitoring to include alternative platforms are also forced to contend with reviewing the massive amounts of data they can introduce into their analysis. This concern has become especially pertinent in recent years, as a growing number of threat actors adopt artificial intelligence tools to intensify the impacts of their activities.

Some online monitoring platforms have tried to address these challenges by offering AI-driven features capable of compressing millions of data points into short reports for analysts to review. While the summaries these tools generate can appear comprehensive, few are fully transparent in how they produce their findings or the limitations they possess, making it difficult for analysts to fully trust their outputs.

Most AI-driven analysis tools available today bake in assumptions or make decisions on behalf of the analysts using them that can’t be altered, and the reports they generate often obscure coverage gaps and lack citations for the data points used to form their conclusions. The fixed presets in black-box tools often miss low-volume signals that precede escalations and low-profile threat actors, and they can prove to be wildly inconsistent in their flagging, interpretation, and classification of potential threats – ultimately hurting analysts more than they help them.

Large language models (LLMs) and similar technologies can offer significant time-savings to analysts and threat detection teams, especially those who use tools that include data for alternative platforms and niche communities. But for those tools to be useful, they must be customizable and show the hidden scratchwork behind the reports they generate. Even the best software cannot provide a substitute for human oversight and expert decision-making.

At Open Measures, we recently introduced an LLM-powered tool (Research Assistant) into our platform that seeks to address these gaps with a “human-in-the-lead” design philosophy, where users maintain control over how it gathers data and receive outputs that are clearly explained and auditable. Instead of trying to replicate our users’ expertise and intuition, we’ve aimed to create a tool that empowers it.

The analysts and safety teams I speak with keep asking for the same things: a fuller field of vision across today’s threat landscape and transparent tools that can help them process all that extra data. As the digital ecosystem expands and places additional strain on monitoring efforts, it’s critical that we take those requests to heart and trust that the professionals know best.