The Humanitarian Data Exchange (HDX) is an open platform for sharing data across crises and organisations. Launched in July 2014, the goal of HDX is to make humanitarian data easy to find and use for analysis. Our growing collection of datasets has been accessed by users in over 200 countries and territories. Watch this video to learn more.
HDX is managed by OCHA's Centre for Humanitarian Data, which is located in The Hague. OCHA is part of the United Nations Secretariat and is responsible for bringing together humanitarian actors to ensure a coherent response to emergencies. The HDX team includes OCHA staff and a number of consultants who are based in North America, Europe and Africa.
For the latest pieces about HDX, click here. A selection is below:
- UN Radio - Saving lives with data: how the UN is developing digital tools for improved humanitarian aid
- Capacity4Dev.eu - Q&A: The challenge of open data in humanitarian response
- AcademyHealth - Sixth Annual Health Data Liberator Award Honors Work to Increase Use and Impact of Humanitarian Data
- Devex - Opinion: Humanitarian world is full of data myths. Here are the most popular
- Center for Data Innovation - 5 Q’s for Sarah Telford, Chief of Data Services at UNOCHA
- Aidpreneur - Terms of Reference Podcast, with guest Sarah Telford
- Devex - Solving the data conundrum: How to leverage tech and 'big data' for impact
- Thomson Reuters Foundation - From displacement to death, UN data innovation aims to boost aid response
- Huffington Post - Open data platform lets aid groups respond more efficiently to crises
- UN News - UN Agencies boost partnership on visualization of food security data for Yemen
- The Guardian - Data exchange helps humanitarians act fast and effectively
- Fast Company - How the candy crush of data is saving lives in Nepal
- Forbes - UN deploys new tech to make relief faster in Nepal
- The Independent - Open Data: What is it and why are people so scared of it?
To find our logo files and other materials, please click here.
- data about the context in which a humanitarian crisis is occurring (e.g., baseline/development data, damage assessments, geospatial data)
- data about the people affected by the crisis and their needs
- data about the response by organisations and people seeking to help those who need assistance.
- Contact data contributors to ask for more information about their data.
- Request access to the underlying data for metadata only entries (our HDX Connect feature).
- Join organisations to share data or to access private data, depending on your role within the organisation, e.g. an admin, editor, or member of the organisation (see more below).
- Request to create a new organisation and if approved, share data publicly or privately.
- Add data visualizations as showcase items alongside your organisations datasets.
- Follow the latest changes to data.
You can also see a timeline of how often an individual dataset has been downloaded on each dataset page. The download timeline is located on the left side of a dataset page, just beside the dataset description. Downloads for a dataset are counted as the total number of downloads of any resource in a dataset, with repeated downloads of the same resource by the same user being counted a maximum of once per day.
There is a delay, usually less than one day, between when a user views a page or downloads a resource on HDX and when the activity is visible in these graphs and figures.
- Administrators can add, edit and delete datasets belonging to the organisation and accept or refuse new member requests.
- Editors can add, edit and delete datasets belonging to the organisation but cannot manage membership.
- Members can view the organisation's private datasets, but cannot add new datasets or manage membership.
The user who requests the creation of an organisation is assigned an administrator role. That person can invite other HDX users into their organisation and assign them one of the three roles above, or registered users on HDX can request membership from the organisation's administrator(s).
Organisation admins can invite new members, remove existing members or change their roles from the 'Members' tab on organisation page.
Registered users can also initiate a request to join your organisation during the signup process or later on from your organisation page(if you want to disable this option, read the question below'I am an organisation admin. I don't want anyone to request membership and want to manually add/remove members.').Membership requests are sent to your email and also added as a notification on HDX. If you can confirm that the user works for your organisation (ie. by using a company directory) or is in your trusted network, then you may approve the request. If you cannot verify who the user is, you should decline the request. Please do not approve membership requests for people outside your organisation or working group. For full details on managing members, please read this document. Please be aware that anyone added to your organisation on HDX can view the organisation's private datasets.
If you have previously created an organisation and no longer see it on the site, this is because you have not yet shared a public dataset. Once you share a dataset, your organisation will become active and visible on the site. For details on how to upload a dataset, see "How do I add a dataset?".
The resource can be stored on HDX or as a remote resource at another URL. Quick Charts will be generated from the first resource with HXL tags in the list of a dataset's resources. The system will try to generate up to three charts based on the HXL tags, and these can be changed to best tell the story in your data. You can edit each Quick Chart's title, axis labels, and description. Don't forget to save the changes so they become the default view that users see when viewing your dataset. Here's a good example to get you started.
Learn more about HXL and HDX Tools in the section below.
Learn how to do this by taking a quick look at these slides.
- HDX via dataset pages (The "Validate with Data Check" option will appear under "More" button under HXL-tagged resources)
- HDX Tools, for datasets that exist outside of HDX. For this option, you should not use Data Check to process personal or otherwise sensitive data.
You can access both versions of Data Check without being a registered user of HDX. For instructions on how to use Data Check, review these walkthrough slides.
Data Check uses a generic schema that detects many kinds of common errors like possible spelling mistakes or atypical numeric values, but in some cases, an organisation will want to validate against its own more-specific rules. In that case, you can write your own, custom HXL schema and validate using the HXL Proxy (Data Check's backend engine) directly. Information is available on these pages in the HXL Proxy wiki: HXL schemas, Validation page, and Validation service.
A PDF file is not data. If you have a data visualization in PDF format, you can add it as a showcase item on the dataset page. If you wish to share documents, graphics, or other types of humanitarian information that are not related to the data you are sharing, please visit our companion sites ReliefWeb and HumanitarianResponse. A resource, such as a readme file, could also contain documentation that helps users to understand the dataset.
If you have data that is substantially different, like a different type of assessment or data about a different province, we recommend creating a separate dataset.
For resources: by default, the resource name is the name of the uploaded file. However, you can change this if needed to make it more clear to users.
For zipped shapefiles: we recommend the filename be name_of_the_file.shp.zip. However, the system does not require this construction.
The preview feature will continue to work when there are multiple geodata resources in a single dataset (i.e., one HDX dataset with many resources attached). The layers icon in the top-right corner of the map enables users to switch between geodata layers. Here is an example.
Entering a search term causes HDX to look for matching terms in the titles, descriptions, locations and tags of a dataset. The resulting list of items can be further refined using the filter options on the left side of the search result. You can filter by location, tag, organisation, license and format as well as filtering for some special classes of datasets (like datasets with HXL tags or datasets with Quick Charts) in the 'featured' filters.
Data Grid is not available for all countries. Here is an example.
This metadata field indicates how often you expect the data in your dataset to be updated. It should reflect the frequency with which you believe your data will change. This can be different from how often you check your data. It includes values like "Every day" and "Every year" as well as the following:
- Live - for datasets where updates are continuous and ongoing
- As needed - for datasets with an unpredictable, widely varying update frequency
- Never - for datasets with data that will never be changed
The HDX team manually reviews every dataset uploaded to the platform as part of a standard quality assurance (QA) process. This process exists to ensure compliance with the HDX Terms of Service, which prohibit the sharing of personal data. It also serves as a means to check different quality criteria, including the completeness of metadata, the relevance of the data to humanitarian action, and the integrity of the data file(s).
If an issue is found, the resource(s) requiring additional review will be temporarily unavailable for download and marked as 'under review' in the dataset page on the public HDX interface.
HDX is currently adding features to visualise HXL-tagged data. To learn more about HXL and who's currently using it, visit the HXL standard site.
- Quick Charts - Automatically generate embeddable, live data charts, graphs and key figures from your spreadsheet.
- HXL Tag Assist - See HXL hashtags in action and add them to your own spreadsheet.
- Data Check - Data cleaning for humanitarian data, automatically detects and highlights common errors including validation against CODs and other vocabularies.
You can find all HDX Tools through tools.humdata.org. The tools will work with data that is stored on HDX, the cloud or local machines. The only requirement is that the data includes HXL hashtags.
- The first resource in your dataset (stored on HDX or remotely) must have HXL hashtags.
- That dataset must have the HDX category tag 'HXL' (not to be confused with the actual HXL hashtags).
For more details you can view these walkthrough.
For more details you can view these walkthrough slides.
- Non-Sensitive - This includes datasets containing country statistics, roadmaps, weather data and other data with no foreseeable risk associated with sharing.
- Uncertain Sensitivity - For this data, sensitivity depends on a number of factors, including other datasets collected in the same context, what technology is or could be used to extract insights, and the local context from which the data is collected or which will be impacted by use of the data.
- Sensitive - This includes any dataset containing personal data of affected populations or aid workers. Datasets containing demographically identifiable information (DII) or community identifiable information (CII) that can put affected populations or aid workers at risk, are also considered sensitive data. Depending on context, satellite imagery can also fall into this third category of sensitivity.
The Working Draft of the OCHA Data Responsibility Guidelines ('the Guidelines') helps staff better assess and manage the sensitivity of the data they handle in different crisis contexts. We recommend that HDX users familiarize themselves with the Guidelines.
Different data can have different levels of sensitivity depending on the context. For example, locations of medical facilities in conflict settings can expose patients and staff to risk of attacks, whereas the same facility location data would likely not be considered sensitive in a natural disaster setting.
Recognizing this complexity, the Guidelines include an Information and Data Sensitivity Classification model to help colleagues assess and manage sensitivity in a standardized way.
For microdata (survey and needs-assessment data), you can manage the sensitivity level by applying a Statistical Disclosure Control (SDC) process. There are several tools available online to do SDC - we use sdcMicro.
The Centre has developed a Guidance Note on Statistical Disclosure Control that outlines the steps involved in the SDC process, potential applications for its use, case studies and key actions for humanitarian data practitioners to take when managing sensitive microdata.