So, you’re out test driving survey software, or the demos of an online survey programming company. Everything you see looks wonderful! These guys have all the latest bells and whistles. All the fancy stylings, animations, drag-n-drop features you can dream of, and all these features look and function in stunning fashion... for you. But how do they look to the guy answering the survey on a mobile device? Can the lady programming away on her Ubuntu machine see these featuers? If not, what does she see? What about that trendy couple surfing the Web on their matching MacBook Air notebooks at the coffee shop?

If you don’t know, you need to ask. We don’t live in a Windows/IE-only world anymore. Firefox has a huge userbase. Chrome is gaining ground quickly. Apple is picking up market share like gangbusters. More and more people are doing their Web surfing on their mobile phones. You need to have confidence that your survey provider is thinking of these things. People who use the Web on uncommon machines represent a very important segment of the online population. Quite often, they think very differently than your average Joe IE/Dell user does. You want to be sure to give them every opportunity to express their opinions right along with Joe Dell.

At W³ we test every survey on multiple browsers operating on multiple platforms. At the time of this writing, here is our current lineup of test platforms:

Windows

• IE
• Firefox
• Chrome
• Opera

Mac OS X

• Safari
• Firefox
• Camino
• Chrome
• Opera

Ubuntu

• Firefox

iPhone OS

• Safari

Everything in our “standard” surveys passes our tests on these browsers with flying colors. From time-to-time a client will make a special request, which we will gladly accommodate. When we add these special features to a survey, they are thoroughly tested in all the browsers listed above, and the client receives a report that details which browsers can use the fancy features, and which ones can’t. Most importantly, we let them know what will happen in the browsers of those who do not pass, and how badly it hinders the functionality of the survey for those folks.

I strongly encourage you to continue seeking out the latest, greatest bells and whistles. At the same time, I caution you to hold your programmer accountable for knowing how they will work with every browser you suspect might be used to participate in your survey.

It's pretty common practice to force respondents to answer some questions, and leave others unforced. When you do this, there is one very important question to ask yourself: Can the respondents tell which ones are forced and which ones are optional? If they can not, the question becomes: What will your respondents do if they assume a question is forced?

If a respondent has already been prompted to answer a skipped question or two, and they get to an un-forced question, chances are they are going to assume that question is forced too. Unless your survey wording communicates that the question is optional, you've given them no reason to believe that they won't be prompted, should they skip the optional question. If they don't know the answer, or have no opinion on the matter, the respondent is likely to randomly select an answer, leaving you with useless data.

Sadly, there is no "one size fits all" solution to this dilemma.

The (seemingly) obvious solution is to communicate to the respondents which questions are not forced, simply by adding the word "optional" to the question somewhere. The problem with this is that you're going to end up with a lot of people skipping the optional questions, even if they have valid opinions on them, simply to get through the survey quicker. This is especially problematic if you have a lot of consecutive optional questions.

More often than not, a good way to go is to force every question, but be certain that your questionnaire is well-constructed. In other words, don't ask them to rate the hamburgers at XYZ Restaurant, unless you've already determined that the respondent has eaten a hamburger at XYZ Restaurant.

Strategically-placed "don't know / no opinion / does not apply" options can also help you get the most accurate data.

Most surveys are too long, and suffer from fatigue and/or race-through bias. So it's a good idea to look at every question in your survey with great scrutiny. If you're okay with respondents not answering this question, do you really need this question at all?

In a nutshell, it's very important to evaluate each and every "optional" question situation with from a fresh perspective, and use the method that works best for each instance.

I love Grid/matrix questions! They are such a great time-saver when compared to asking respondents to rate each item as a separate question on a separate page. But do your respondents know what answers they are giving? Maybe... Maybe not!

Let's look at a lengthy grid:

Doesn't look too bad, eh? The problem lies in grids that get really long, like the one above. A respondent stopping by, who is answering your questions from a computer with a windowed browser (common among Mac users) or a low-resolution display setting might see this when they scroll down to answer the questions that appear lower on the grid:



Clearly the problem here is that the respondents can't see the column headers, therefore are marking radio buttons without knowing what answers they are giving.

One possible solution is to repeat your column headers at the bottom of the matrix, like so:



Better yet, why not toss one in the middle too?:



Repeat the scale throughout the grid as many times as you need to in order to ensure that respondents can see what answers they are giving.

Many survey applications offer the ability to allow respondents to start an interview, quit in the middle, then come back and pick up right where they left off. Recently I participated in a survey that presented this the wrong way.

The survey I completed actually had a "Quit" button on every page. It looked a little bit like this:

I couldn't believe my eyes. Our industry is facing all-time low response rates, and here I was, being presented with an advertisement to quit the survey on every single page.

Would you train a telephone interviewer to ask respondents "would you like to quit now and finish this later" after every question? Of course not. Your dropout rates would be through the roof.

Using a suspend/resume feature is a great idea in most online survey situations. Mention this option in the invitation, but do not remind them on every page that it's okay to stop the interview. Instead, program your survey in such a way that if a respondent quits by any means (such as closing their browser window) and returns to the survey at a later time, they're automatically directed to the spot in the questionnaire where they left off.

For many, many years, I recommended very specific survey navigation button placement:

The "Next" button was always in the bottom-right corner of the browser window, while the survey "Back" button was always in the bottom-left corner.

My reasoning:

• I've read several studies about banner placement, and the bottom-right corner freqently achieved the best click-through rates.
• Because of all the vertical scrolling required to read Web pages, it is very natural for a respondent's cursor to gravitate toward the bottom-right corner of the window.
• These positions seemed to emulate the natural flow of how we read traditional media. To advance to the next page of a book or magazine, you grab the page on the right. To return to the previous page, you grab the page on the left.

When I set up these guidelines, 17" monitors were considered large, very few people had widescreen displays, and 1024*726 was a high, rare resolution setting. While the same reasoning is still applicable today, the instruments on which our surveys are displayed has changed.

Nowadays it is not uncommon for a respondent to stop by and attept to complete your survey using a 24" widescreen monitor set to 1920*1200. Using my old recommendations, and this configuration for comparison, the radio buttons for a Yes/No question would be approximately 15" away from the "Next" button in a maximized browser window. This is way too far to require a respondent to move their mouse between pages.

In keeping up with the spirit of my previous suggestions, but adjusting for newer, wider displays, I now recommend centering your buttons at the bottom of the page, with the "Back" button on the left, and the "Next" button on the right.