Breaking the 1 million barrier

I finished loading the updated National Register of Historic Places database into Freebase last week. In addition to containing the latest data released by the National Park Service, combined with the latest Wikipedia articles, this run created new topics where Freebase didn't have existing ones. You may remember that the initial run focused solely on reconciling existing Freebase topics.

Freebase should now have a complete copy of all National Register of Historic Places entries which are of International, National, or State significance. The Local significance listings still used the old strategy of only reconciling existing topics.

Below is a summary of the before and after counts. We picked up 4,535 entries which had either been added to Wikipedia, added to the Register, or both. On top of that we created another 20,553 entries, bringing the grand total to over 35,000 listings.

	Starting Count	Existing Topics Reconciled	New Topics Created	Ending Count
International	0	1	10	11
National	2010	699	4386	7095
State	2423	1121	16065	19609
Local	5978	2627	92	8690
TOTAL	10434	4535	20553	35518

Each topic contains a fair amount of information, so the entire load amount to about 750,000 "facts" (or "triples" in RDF-speak), bring the total number of facts that I've written to Freebase to over 1.1M. Unfortunately, their "tallybot" which does the nightly updating of totals has been broken for a while, so I'm only getting credited with a paltry 300K.

The one remaining loose end is to try and do a better job of reconciling the architects/builders and what the Park Service calls "significant people" associated with the listing. This will require human vetting of a queue of tasks, so it'll require some additional infrastructure to be put in place before I can set people loose on working on it.

Freebase Hack Day - Sat. July 11 San Francisco

Just two weeks until the Freebase Hack Day that Metaweb is running at their San Francisco headquarters. It's free and will feature unconference style discussions/presentations as well as general hacking. Read more about the goings on in this blog post. Although it's free, space is limited, so you'll need to register with Eventbrite (when it comes back up from its upgrade).

This is the second Hack Day they've held and I'll be in attendance for this one, so if you're going, give me shout.

Freebase, open government, and enumerations

I'm preparing a short series of articles about Freebase, but Raymond Yee had a question about something I was working on over the weekend, so here's a quick hint to help him along.

What he calls "keys" are called "enumerated properties" in the Freebase documentation and there's an article on how to set them up. Unfortunately, the schema editor was broken when I was working on the National Register of Historic Places database schema, so I had to resort to reverse engineering things from the Explore view (accessible by pressing F8 on any page and scrolling to the bottom of the page) and then modifying the schema's property type by hand using their MQL query language. You can see the end result in the schema where item_number is typed as an enumeration.

There's also a good article on how to create a URL template that I used successfully to link to the original application submissions. For the Congressional Bioguide, it can be used to link back to the original biography.

Coincidentally and independently from Raymond's project, I was actually working on loading up all the Congressional Bioguide ID's last weekend because they are used in the XML form of legislation on THOMAS, which is run by the Library of Congress. I decided to take a slight detour to write a little name parser and Freebase name queryer in Python, so haven't actually gotten around to loading the IDs yet. One of the biggest problems in working with Freebase is reliably resolving personal names. They typically only have the main name that was used as the Wikipedia article name. There's really no telling what name form the article's editors will have chosen and even though the full name and some aliases are often identified in the opening sentence of the article, Freebase doesn't import this information from Wikipedia.