Databases

Quick word about searching for data:

The important when searching for an information, is where to find it.

I hope this short list helps to clarify what is the purpose of each publisher of information (website, entity, publisher…), and the type of dataset you can find.

It’s also important to keep in mind the thematic of the data,the perspective of the analysis (financial, law, technical…) and to understand the problematic behind a data collection or agglomeration (for example what problem people are trying to solve and how these data can help?).

disclaimer: Most additions to the list are quick notes from study cases. Feel free to add any resources usefull to you and your team.

Each line tells if MISP has an import module related to it. If not, and you would like to see the integration, you can post an issue directly on the [MISP repository] (https://github.com/MISP/MISP/issues)

misp logo

Database list (last update 01/30/2021)

Name link last-view label MISP module
CDTC data https://www.ctdatacollaborative.org/… 02/12/2021 Human trafficking  
Data.gov Australia https://data.gov.au/… 02/12/2021 Human trafficking  
Data.gov US https://catalog.data.gov/ 02/12/2021 Human trafficking, child labor  
European Data Portal, Justice, Legal System and Public safety datasets https://www.europeandataportal.eu 01/30/2021 justice, public safety  
FBI Cyber Most Wanted https://www.fbi.gov/… 01/30/2021 law enforcement  
FBI Records, The Vault https://vault.fbi.gov 01/30/2021 law enforcement  
Export.gov, export information by industry (US) https://legacy.export.gov… 01/30/2021 export  
GIJN METHODE : explore data, work with data (from GIJN Global Investigative Journalism Network Helpdesk) https://helpdesk.gijn.org/… 01/30/2021 journalism  
Global Terrorism Database https://www.start.umd.edu/… 02/20/2021 Terrorism  
Harvard Dataverse, Human trafficking https://dataverse.harvard.edu/… 02/12/2021 Human trafficking  
ICIJ Offshore Leaks (International Consortium Investigative Journalists) https://offshoreleaks.icij.org/ 01/30/2021 leak  
International Aid Transparency Initiative https://www.iatiregistry.org/… 01/30/2021    
Interpol Notices https://www.interpol.int/en/How-we-work/Notices/About-Notices 01/30/2021 law enforcement  
Investigative Reporters and Editors Resources https://www.ire.org/resources/… 01/30/2021    
International Trade Administration, Consolidated Screening List (US) https://www.trade.gov/… 01/30/2021 trade  
Migration data portal https://migrationdataportal.org/… 02/12/2021 Human trafficking  
ONUDC DATAONUDC https://dataunodc.un.org/ 29/03/2021 crime, drugs  
OCCRP Research Desk (Organized Crime and Corruption Reporting Project) - OCCRP ID https://id.occrp.org/databases/ 01/30/2021 corruption, crime  
OCCRP Aleph https://aleph.occrp.org/ 01/30/2021 corruption, crime  
Open-data Portals https://opendatainception.io/ 01/30/2021    
ProPublica Data Store, Criminal Justice Datasets https://www.propublica.org/datastore/… 01/30/2021 justice, crime  
Respect international Counter trafficking data collaborative https://respect.international/counter-trafficking-data-collaborative/ 02/12/2021 Human trafficking  
State of California Department of Justice, Databreach list https://oag.ca.gov/privacy/databreach/list 01/30/2021 databreach  
UCR FBI https://ucr.fbi.gov/hate-crime/ 02/12/2021 Hate crime  
Unesco Digital Library https://unesdoc.unesco.org 01/30/2021    
World Bank data catalog https://datacatalog.worldbank.org/collections 01/30/2021    

unsorted

Normalization

Name link last-view label MISP
ONUDC ICCS reports https://www.unodc.org/unodc/en/data-and-analysis/… 02/13/21 NOMENCLATURE OBJECT TEMPLATE OR TAXONOMY
ICD-11 International Classification of diseases https://icd.who.int/en 02/13/2021 cause of death