Map Usability: What's great about this is that its a discussion about the usability of maps in the public sphere not a secret report for some company. IMHO there is a great need for much more of this work, we're woefully unaware of how to make interactive maps usable. It has lots in common with a current MSc project I'm supervising that I discussed last week looking at placemark clustering.
Simple Testing: Patrick discusses results from videoing users and eye tracking. These tools definitely help analyse and communicate the issues that arise from testing but you can still find out a lot without these tools via a technique is called Hallway testing (talk aloud procedure). It takes about half a day and needs no eye tracking or video recording.
Testing 'Doing' not Gathering Opinions: Hallway testing is pretty quick given the amount of information you get from them but its quicker still to gather feedback by questionnaires. Questionnaire feedback gives you some useful insights into your map system but it is by no way a complete picture. This was bought home to me this week in a reference I found about users searching for targets in overview* maps. The experiment tested search tasks with and without an overview map. Users were very positive about the value of overview maps but when the speed and accuracy of the searching were analysed using the overview it turned out that their performance didn't improve. Questionnaire data can be misleading, to really find out the truth you have to observe (and measure if you can) users trying to complete tasks with you map based tool.
*An overview is a small map in the corner of a web page showing the view from a higher altitude and usually marking the current view.