SMAT is now Open Measures! Read more about our rebrand here.
About
The web needs cooperative, transparent tools to uncover online disinformation and extremism.
Open Measures is built to help journalists, researchers, and social enterprises investigate harmful online activity such as extremism and disinformation. In turn, our open source and enterprise user communities make feature and data requests to help guide our tooling.
We are opposed to the threats presented by dangerous online users, malicious state actors, and conspiracies aimed at causing mass harm. We want to help our users respond to the next generation of threats by providing an open and accessible way to conduct threat intelligence.
“…their purpose-built platform and set of APIs provide for a comprehensive solution to effectively identifying and addressing disinformation on social networks. Open Measures enables our team to quickly detect false or misleading narratives & extremist content. I’ve tried a number of tools in my career; without Open Measures, harmful online activity would be significantly harder to locate.”
– Tim Chambers, Principal at Dewey Square Group
“The Open Measures platform has been by far the easiest to use and most customisable system I’ve ever used for Telegram. It’s extremely fast, easy to add channels with back-crawling, and allows you to filter by any data obtainable from the API. Just as importantly, it’s completely transparent in how it works with no black-box processing or pseudo-AI.”
– Pierre Vaux, Investigator at Centre for Information Resilience
Previous
Next
Contact
Have a question that isn't covered in the FAQ? Drop us a line by clicking the link!
How do you ensure data coverage is robust and comprehensive?
Some of our crawlers iterate by spidering, some through users and others through predictable post IDs. All of these subtleties , as well as other strategies, determine how comprehensive the data is. That said, there are occasional gaps in data and sometimes web hosts change the formats of their APIs so we are constantly updating our methods. In some cases, like with Telegram, we crawl everything from select channels which means we lean on our community to help us build out our library. Got a channel you’d like to see us crawl? Reach out at [email protected] with your request.
How can I validate results?
If you make a request to our API or our Search tool, you can then seek out the original post on the platform. You can do this either by the URL we store in the individual items we crawl or by searching other available collections. In some cases, the platform may have deleted the post. In those instances we encourage you to check for archives on archive.org or archive.today.
Our datasets have also been independently verified in multiple articles by research teams including this one at New America. Open Measures has also been a part of many scholarly conference papers exploring our data. The first introduced our early model and the second published our first open-source Parler dataset.
How do you manage historical data?
We crawl the full history of data, starting at the time the crawler was initially deployed. If the data was there when we began crawling then we have it indexed.
Do you open-source your crawlers?
For the security and sustainability of our platform, we do not open-source crawlers but we do open-source as much as possible. All open-source code is available via our GitLab project.
Data Access
Do you make any endpoints available publicly?
Yes! We have several public endpoints available for querying our data store, including the /content endpoint which returns raw data. It is worth noting that our public endpoints are rate limited at 39 requests per day. You can read more about our public endpoints in the API section of our documentation.
I need higher frequency than the public endpoints or API tooling allows. What are my options?
We support multiple options for teams in need of high volumes of data. We offer packages that include our Pro and Enterprise platforms as well as direct access to our data stores. If you are interested in learning more about how your team can get more robust tooling, fill out the following form.
Partnerships
How long does it take to implement Open Measures?
The implementation timeline is up to you! Most partners take anywhere from a few days to a month to fully implement depending on the complexity of their needs and in-house setup. Our team is there for you every step of the way. We begin every partnership with onboarding sessions and offer ongoing support to ensure you continue to have the resources you need for success.
Who do you partner with?
We partner with a range of organizations and individuals that are trying to better understand, and respond to, emerging online threats of disinformation and hate. Those organizations include leading non-profits in the hate monitoring space, campaign organizations, researchers, news and press desks, and other private research teams.
What if I don't work for an organization that would be a good partner fit but I still want to contribute to the open-source movement?
We’re proud partners with Open Collective, our fiscal host. Contributions from the community make our dedicated work in the open source ecosystem possible. You can make a one time or recurring contribution here.
