If you are a New England based geospatial professional you have probably visited or downloaded data from the Map and Geographic Information Center (MAGIC) at the University of Connecticut’sHomer Babbidge Library. MAGIC just recently announced their plans for the summer and I am looking forward to seeing what data and tools they roll out next. MAGIC was one of the first map libraries to offer GIS data over the web and since it has always been a place of innovation.
I was fortunate to have worked at MAGIC from the fall of 2004 through the spring of 2009. As a grad student I was able to work on a number of really interesting projects that expanded my technical skills more than any class ever could (I was one of the last people at MAGIC to program in Perl:( ). Many GIS professionals in New England and beyond can trace their roots back to MAGIC, whether it was working for Pat, Bill, Sarah, or Michael.
I get to do fun things with GIS, like develop models that handle and process millions of records. This model is used to clean a number of spatial variables for input into another analytical model. This awesome monstrosity contains:
205 million input xy points read from SQL tables that are processed into over 500,000 polygons
500,000 polygons merged and manipulated into about 200,000 polygons
Over 20 different spatial processes (merge, clip, field calculations, projections, spatial stats, etc…)
Database driven model, using a mix of SQL tables, SDE datasets, and File Geodatabases
Four custom python scripts, some of which are repeated several times
The output? Two text files, which are generated from python scripts.
Output used in another set of spatial models
The model took about a week to build and calibrate. The model itself runs in about 30 hours on a four core, 64 bit machine and produces close 20GBs of data from about 4GBs of input data.
Anyone can build a mapping website (including me!), but a true GIScience geek lives on this stuff. I love spatial analysis.
Ignite Spatial: Boston 3 is this week! The talks have been announced and can be found on ISpatial Boston’s website (just scroll down the page a little bit to get to them).
It was also announced that the venue has changed from the SCVNGR offices to the IBM Research offices at 1 Rogers Street in Cambridge. Some tickets have opened up as well. Check out the waiting list to see if there are any left!
There appears to be a good variety of talks from a number of different perspectives, which is always a good thing for these types of events. I’m looking forward to Adena Schutzberg’s talk. I’ve seen her talk a couple times now and she always has some really interesting to say. I really like conferences/meetings like this, where you have people from a variety of backgrounds sharing their experiences with spatial analysis and GIS. By listening to their unique point of view, it may challenge your own, or give you an idea on how you can improve what you do.
It is an exciting time for the Boston geospatial community!
We all know what happened late last week in northeastern Japan. There have been a tremendous amount of causalities and people have lost everything. A number of agencies are predicting that this event could be the most costly in history, and we all know that it will take many years for this region and country to recover.
It is also during times like these that geospatial professionals are called upon provide any number of spatial services. Now the hard part. Where is the data? How do we get to it and make sense of it? A number of data sources are becoming available, many at a high resolution or very current. These critical datasets are being generated using a number of different technologies, from satellite imagery to social networking, providing relevant information on-the-fly. It is really amazing to see this information become available so quickly after such a colossal event.
The following websites provide just a glimpse into the amount of data that is becoming available. By no means is this list complete or authoritative.
One of their most interesting 2010 Census Google Map Mash-Ups on their site displays dual-linked Google Maps, with one map displaying the 2010 Connecticut census results by town and the other the 2000 Connecticut census results by town. The dual-linked maps are synchronized, meaning that movement one map happen on the other. The user also has the ability to search for their town, or download the data for their own use. The user can also click on a town and view a table of census data to see how population has changed across Connecticut. I’m looking forward to what other mash-ups they roll out over the next few days.
Take a look at their site. MAGIC has been around for a long time, and they have always been innovators in their field.
and…quick disclaimer, I helped write the code for the underlying mash-up. That’s it, they did the rest and they did a great job.