Project Real-Realator

     So I have come up with an idea of a new start-up web company. Don't want to share the general idea with the world for the sake if keeping it my idea. But I will document the adventure along the way. Thus far... I have siphoned a massive data set from the US Census Bureau. Rough estimate of total unique items in my DB should be in the millions. My dubbed my Database name Dragonoff (Like the Russian Sniper Rifle). I have lots of tables in which I need to tie together. Getting the primary key to tie all the columns together was a real pain in the ass. Turns out, the US Census Bureau data is highly unorganized. Thus far, I think I have a good starting point for my project. There will always be the chance to add in more data to get  more statistical analysis but I will cap it off here.
     I have also registered with zillow to use their API. They use a simple HTTP GET. As long as you pass along your Zillow ID, you could get almost anything. Will use this API to get very low level neighborhood data.
     So my next step is to create views for every city in the US. To do this, I will simply use all primary keys from all tables with a  keystone from a "Primary Table" if you will. Hopefully, you could give it a zip code and it will bring back all related table information. From there, I may want to see how Hibernate/PersistenceManager/StraightSQL performence measures up. I will probably just use Seams Persistence Manager. It will make for some rapid prototyping. Down the road, hopefully I have to deal with the headack of scaling, I will research it more in depth.
     Once views are complete, I will infer new data per row by running all the data through different algorithms. I will create another table with this inferred data. Once all cities have completed the inference process, I will tie it together into a single view again.
     Once the views are totally complete, I can than display them via HTTP. Once I get there, I will figure out what to do then.