Over the past few months we’ve been streamlining and optimizing the process of Unigraph generation and export.
In that period only a selected group of early adopters and Open Data enthusiasts enjoyed access to the complete Unigraph datadump. We’re thankful for their commitment and feedback and happy to open a new and exciting chapter of making the world’s knowledge ubiquitous:
Today, the entire knowledge stored in Unigraph becomes available for anyone to download and explore! All 1.5B+ claims connecting objects from the world we live in.
We reached an important milestone that makes it possible and viable to move to a more frequent update cycle: a Unigraph generation from scratch (which will hardly be necessary, as we add or remove only the differences reported by the sources) and export now happens in 4 hours! A remarkable feat given the size and the inconsistency of the data. It takes ~2 hours to combine, clean and reconcile the data from the different sources that make Unigraph and another 2 for the .json export. Depending on your connection, there might be a new dump available before you’ve finished your download. We’ll be providing a new dump on a weekly basis, even more frequently, if a new major source of information has been added.
In less than a month Unigraph grew from 16 to 25GB compressed and from 167GB+ uncompressed to a whopping 275GB! We’ve added many new sources of information and with the ones in progress we expect to double in size in a month.
As sources update and new ones are added, the frequent datadumps will reflect these changes, interconnecting deeper the nodes in the graph and providing an ever more complete picture of the complex world we live in.
Download the dump and start building amazing applications and services with it!
Reach out on twitter, share what you’ve created, discovered and which sources of information you would like to see as part of Unigraph.
Subscribe for updates and be the first to benefit from the new developments and sources additions.
The data made available in the dump will always be a few days behind what the Unigraph API exposes. As the dump continues to grow in size, so will your associated infrastructure costs. Consider using the Streaming and REST-like APIs instead. Contact us for a quote!