ArcGIS – Benjamin Spaulding

January 25, 2012

Problem of the Day – ArcGIS Basemaps in ArcMap

I’ve been a big fan of the Esri basemaps for a while now. I use them in web applications all the time. Within web-apps they are fast, and when used correctly, they are an effective reference layer.

Lately I’ve been running into problems using these basemaps within ArcMap. As you know, you can add these basemaps to an ArcMap view through an ArcGIS Server connection. However, whenever I add any of the basemaps to my map, ArcMap starts to chug, and chug badly.

I know there are a couple issues that will cause any basemap provided through an ArcGIS Server connection to bring ArcGIS to a crawl. The Esri basemaps are all provided in a web Mercator projection and if you are trying to reproject the layer on-the-fly it will slow down the draw rate within the map. Even when I have all my data sets projected as web Mercator I still have drawing and performance issues. In my experiences the initial load of the basemap usually draws quickly, but whenever a pan or zoom event or I start to toggle layers ArcMap will crawl.

What really worries me about this problem is that my fully patched and up-to-date copy of ArcGIS will often lock up or crash when these basemaps are being used in the application. I only have this problem with the basemaps from ArcGIS. I’ve never have this problem with a WMS layer. Perhaps these data sets shouldn’t be used with ArcGIS itself and maybe I’m using these basemaps incorrectly?

Have you experienced this problem? Do you have any remedies to fix this issue (besides not using the layers)?

The next problem I have had with these layers is something I can’t figure out. On a pretty consistent basis I see the inconsistent extent error (see image below). I will get this error when I add one of these basemap layers into the map, whether it is the first layer into a new project or the last layer added. The projection of the view does not matter and sometimes the layer will draw with a blur.

Let me know what your experiences have been and any potential fixes you may know. If I can come to a solution or someone sends me a good one I’ll post it here.

September 14, 2011

The Query Layer Query

The Problem

I am a big fan of the query layer in ArcGIS 10. I routinely create and manipulate spatial data that lives in the geography data type in SQL Server. The query layer tool has opened up a whole new set of analysis and visualization possibilities for a number of large data sets I work with. Overall, I am a big fan…

But, I’ve run into a problem. Using the query layer tool I couldn’t connect to a number of SQL Server databases. When I would try to connect to certain databases I would get this cryptic error.

What Does this Mean?

What is going on? Is this an actual error? I am missing some font pack therefore displaying the characters I see on screen?

With this useless error I started to trouble shoot. I knew I had SQL Server databases that worked correctly and some that generated this error. The only difference between the working and non-working databases was the length of the database name. As it turns out the length of the database file name that is allowed by the query layer is limited to 31 characters. Anything greater than 31 characters will generate this error. This is not listed in the query layer documentation.

What the Heck!

In my job I work with dozens and dozens of databases that have a prescribed naming convention. Sometimes, these SQL Server database names can get long, but no where near the SQL Server character limit. Within the Esri software family there are described limits to database names. When using ArcSDE the name of the database is limited to 31 character, which is understandable.

However, if a user adds a SQL Server database to the “Add OLE DB Connection” tool in ArcCatalog the database name can be more than 31 characters, but if the user adds a SQL Server database with a name greater than 31 characters to the query layer tool it will error out. What a bummer.

What’s Next

I posted this question in the Esri forums and the response I got from the Esri rep was accurate, but this limit is a pain to those want to use this tool. In the online documentation for the query layer the text states “Query layers allow both spatial and nonspatial information stored in a DBMS to be easily integrated into GIS projects within ArcMap.” I agree with this statement 100%, unless my database name has more than 31 characters…

By limiting the length of the database name Esri is saying that I cannot take full advantage of the tools provided by SQL Server. Also, if I can only use 31 characters, please document it in the query layer documentation. Either this is a bug, an undocumented limitation that needs to be addressed, or it is documented somewhere and I can’t find it (which is a real possibility). I am thinking that this problem is probably a 32 bit issue, in fact a couple of my coworkers are thinking the same thing. I’m using a 32 bit software in a 64 bit world!

As a geospatial database admin (GeoDBA for short), I need to make a decision that will impact the workflow for my organization. I have business requirements that limit me from renaming these business critical databases, but I have new business requirements that are greatly benefited by the query layer functionality. Do I rename databases (and modify the tools that access this information) or do I wait for Esri to fix the problem? Moving forward I’ll try to limit the number of characters in my databases but I’m using a SQL Server database, not an SDE database. Why is Esri limiting what I can do with that piece of software? Unfortunately, it may be faster for me to rebuild everything than it would be to get a solution from Redlands.

I’ll submit this problem to Esri tech support soon.

And…I am running ArcMap 10 with SP 2 using SQL Server 2008 R2 that is fully patched and updated. The error described has been repeated on several machines within my organization.

Until next time GIS geeks!

May 31, 2011

Is this email really from Jack Dangermond?

I am very fortunate to be attending the Esri User conference in mid-July. At my job I use Esri software in a significant portion of my everyday workflow. At the conference I am looking forward to meeting up with 10,000 other geo-geeks to talk GIScience, spatial database optimization, and the future of the “GIS guy” .

Like every other conference I have ever been to I received a pre-conference survey from Esri, asking a number of standard questions about how I use the software, what I would like to get out of the conference, and a number of other topics. One of my favorite parts of the survey was where they asked for general comments. I commented that ArcGIS (all products) should have been fully 64 bit with v10. If you didn’t know, ArcGIS 10 works on a 64 bit machine but will only use two cores and up to four gigabytes of RAM for certain operations. Now, those specs, are…so 2002.

Due to this amazing limitation I use Spatial SQL in Microsoft SQL Server 2008 R2 for a number of large and complex geo-processes. Why so? Well, SQL Server 2008 R2 is fully 64 bit, supports geo-datatypes, has a growing number of spatial applications, and will use all of the processing power of a multi-core, 64 bit machine.

Esri followed-up my pre-conference survey with a nice email to let me know that ArcGIS Server 10.1 will be fully 64 bit, but the email I got was sent to me from no other than “Jack Dangermond“. Check it out:

Now, I totally don’t believe that Jack Dangermond would have the time to email me a response to a pre-conference survey. But it was nice that someone at Esri took the time to read my response to the pre-conference survey and send me a follow-up email. Now, Mr. Dangermond, if you really did email me (and wouldn’t it be a trip if he were reading my blog, too?) email me again and let’s set up a lunch at the user conference. Have your people call my people…

April 7, 2011

Keyboard Shortcuts in ArcGIS 10

I’m always looking for ways to make my workflow more efficient, from creating scripts to automate a process or trying to simplify models so that they are both effective and efficient.

What about making the day-to-day use of ArcGIS faster? Many of us create our own toolbars or custom buttons to make workflow faster, but what about using keyboard shortcuts? If you do a lot of programming, or spend a lot time using any piece of software you want to know the available keyboard shortcuts.

We use keyboard shortcuts all the time from ctrl+c, ctrl+v, ctrl+s and many, many others. What about keyboard shortcuts in our favorite program, ArcGIS? Yes, they exist, and many of them are pretty helpful. The basic ones for copy, paste, and save all exist, and there are many more including shortcuts for toggling all the layers in the table of contents, working in a table, or editing features.

Where can one learn all about these shortcuts? Well, Esri has published a document on this exact topic that I find very useful. There are a number of keyboard shortcuts and a number of other general tips that will benefit the novice or expert user.

What about tips and shortcuts for other tools that GIS users access all the time? Well, there are number of tips for the field calculator, however, this tipsheet is a little old, but still relevant.

Know of any shortcut lists? Post a link the comments section. I’m sure someone will find it helpful.

March 21, 2011

Cancelling a Geoprocessing Tool in ArcGIS 10

My colleagues at work have just recently been upgraded to ArcGIS 10 and one of the most common questions they have is how to cancel a geoprocessing tool. For those of you running ArcGIS 10, you know that Esri has changed the way geoprocessing runs. No longer does the user have to view the old-school geoprocessing window that displays the ever popular progress bar. With this window now gone the user may be unfamiliar with how to cancel a process, because the cancel button is no longer on screen.

Yes, the geoprocessing window still exists in ArcGIS 10 for a few processes, including if the user is running a model that they created. But for a number of tools, including many of those that are standard in the ArcGIS Toolbox, the progress of the tool is monitored in the results window. A lot of information is available in the results window, and in this window the user can cancel the progress of the tool. The following will walk a user through this process.

First, the user needs to add the results window to their view. This is easy to do. The user will select Results from the Geoprocessing menu.

Now the user will run any process their heart desires. The progress bar now appears along the bottom of the program. The best part of this new feature is that the user can continue to work in the program and any data sets that are being used in the geoprocessing is locked, as not to corrupt it during the use of a tool.

Now the user can monitor the progress of the geoprocessing tool through the results window, and if needed, can cancel the tool. To cancel the process the user will open the results window and open the current session.

The user will then right click on the tool being run, as indicated by the clock icon next to the process, and select cancel.

There it is. The tool is canceled and any locks on the data are removed. There are a couple of caveats to this new feature. Sometimes there is a lag between the time you start a tool and when the progress bar starts, or there may be a lag between then the progress bar finishes and when the tool actually completes. I’ve noticed in my working environment that the lag is greater when processing large data sets. The only problem I’ve run into is if I forget about the running process and I try close ArcGIS. A notification will appear on screen about the running tool.

Well, that is it. Pretty simple.

March 10, 2011

More on ArcGIS “Sucks”

On February 24, 2011 I posted a blog asking for opinions on ArcGIS 10, mainly due to all the bad reviews I see and hear. Since that post the hits on this site have spiked, and not because of my awesome content regarding information about earning an advanced degree in GIS or learning about different uses of geographic information on the web. Most people have found their way to my site through “Googling” something related to their feelings about ArcGIS 10. Thanks to Google Analytics I have extracted the following keywords that have driven traffic to my site over the past two weeks.

arcgis 10 sucks
arcgis 10 comments
“arcgis 10 sucks”
arcgis 10 is awful
arcgis 10 issues
arcgis 10 never opens
arcgis 10 problem
arcgis 10 reviews
arcgis stinks
arcgis sucks
arcmap 10 sucks
arcgis10 review 2011
does arcgis10 suck
esri arc 10 ratings
i hate gis – (haha, I love this one. I once had a friend in grad school who lived by this mantra)
problems with arcgis 10
problems with arcmap 10
reviews arcgis 10

This is crazy! What drives people to go to Google and type in one of the previous keywords? I want to know! Yes, there are problems with ArcGIS 10, but I have found much more success than frustration since I installed it this past June. For example, I have built a number large and complex models and python scripts that handle millions (yes, millions) of points, that perform analysis on tens of thousands of polygons, and create multi-gigabyte output files that run perfectly ever time. I have a model running right now and the results will be ready when I get to work. I’ve had no major problems with raster analysis, map creation, or data sharing. Are there bumps in the road? Sure, but a comparable number of bumps to the other software that I use…

Now, do I have some kick ass machines that run the processes? Yes. Do I have an ArcInfo level licenses? No, I’m using ArcView. But the point is that I am able to do everything I want to do with the software. Are people having issues with software configurations, hardware limitations, or user error (it’s never user error by the way…).

I want to know what problems others are running into so that I can avoid those mistakes, because I hate downtime and I love results.

Have you had a problem? Leave a comment. I’d love to know what problems people are having. You never know, someone else may have had the same problem, or they know of a solution.

February 25, 2011

In Defense of the WMS

Earlier this week James Fee, celebrity GIS blogger (seriously, how many other GIS bloggers can you name), posted a blog about the death of the WMS. He was commenting on a blog from Sophia Parafina, who was discussing the difficulties of working with the WMS URL. Surprisingly enough, this isn’t the first time someone has shared their “love” of the WMS standard.

If you have programmed using a WMS you probably have become frustrated when trying to understand what each component of the URL means, the differences between the different versions, and how different GIS server software handles the WMS. I don’t think the WMS is dead, but its use may need to be reevaluated. For example, the WMS servers that I take advantage of tend to have few datasets (less than 30) served from them, and I only use raster datasets, such as historical maps, or aerial photography, in my applications. I try to avoid large WMS servers with 100s of layers, and I rarely use vector from them, because I don’t like wait times with large WMS servers, and vector data renders poorly.

I like to use WMS data from a number of different sources as base maps for use in Google map mash-ups. I’ve found that using the WMS for this purpose equals fast response times in an easy to use interface. Now, what about decoding the URL? Well, it took a couple of hours of trial and error but a few lines of javascript can read the WMS URL pretty easily. I didn’t create the code myself, but like any programmer, I definitely had help from a number of different online sources.

Here are a few Google Maps V3 API examples running WMS data:

ArcGIS Server WMS in Google Maps

Open Layers WMS in Google Maps

Cadcorp WMS Google Maps

Microsoft Research Maps in Google Maps

Multiple WMS Service in a Single Site

Large Aerial Photography Datasets

That’s all for today! Hooray weekend!

p.s., For my next post I’ll try to have a couple of pictures. People hate reading.

February 24, 2011

Does everybody hate ArcGIS 10?

We have all been there. You are in the middle of some process using ArcGIS 10, trying to finish a project for a deadline, and one of two things happen: the hour glass starts to run and never stops, or your ArcGIS screen just disappears and you get the “oh no, something broke” window to report an error (which you never do).

AAAAHHHH!

Since ArcGIS 10’s debut back in June I’ve read and heard a number of opinions (many times unsolicited) about the latest release and subsequent service pack. As you can imagine some reviews are glowing, and some are not so great. When I do a quick Google search for “ArcGIS 10 Review” very few results appear that aren’t from Esri themselves.

Now when I do a Google search for “esri arcgis 10 stinks” or “esri arcgis 10 sucks” the results are much more, let’s say, robust. Three message boards really stuck out and can be found here, here, and here.

When reading though these message boards you must read them with a grain of salt. First of all, people will complain about anything and everything. Second, many complaints have nothing to do with the software itself, but with user error or machine problems.

I also believe that many users are intimidated by GIS software and when something goes wrong they may get the urge to post insane comments with poor grammar because they are frustrated with something that may be out of their control. I can understand where some of these people are coming from, but ArcGIS really isn’t a beginner’s software. I’ve been using Ersi’s software for over eight years and I learn something new all the time.

Yes, ArcGIS 10 has its share of problems. Yes, ArcGIS 10 probably isn’t the best piece of software for beginners or light users of GIS. I do believe if you are a high end user working with large datasets or you need to perform some serious GIS analysis ArcGIS is a strong tool that I would recommend.

It could be worse for Esri, they could be SQL Server, Oracle, or Visual Studio.

And for the record, I like ArcGIS 10, especially ArcGIS Server 10.

Well, that is my two cents. What has your experience with ArcGIS 10 been like? Did you have to revert back to 9.3.1? Do you post insane comments on GIS message boards when you are having a tough GIS day? If so let me know.