JTE v2n1 - Why I Didn't Respond to Your Questionnaire

EDITORIALS

Why I Didn't Respond to Your Questionnaire

John V. Gallagher

One role I must play, as a scholar, is to actively participate in the research activities of other technology educators. Generally, this means responding to research questionnaires to which I feel qualified to contribute. Either I will complete the research instrument, or, if the content is out of my area of competence, return it with a note stating the reason for not completing it.

However, your questionnaire falls into neither of the above categories. It contains a number of serious defects which threaten its reliability, validity, and generalizability. This places me in the dilemma of whether to spend time responding to a clearly defective survey instrument. I didn't respond to yours for one or more of the following reasons. I'm sorry.

Reason 1. You failed to review the literature and available data bases. You are asking for information on technology education that is readily available or published recently.

Discussion: A researcher owes the respondent the courtesy of using a systematic process of research to obtain information to answer the research questions/hypotheses. The use of data collection instruments should be the last resort to obtain information because it is unavailable elsewhere.

Reason 2. You failed to field test your data collection instrument and revise it. Your instrument has vague instructions. Your terms are not defined. I started to respond but became frustrated because the lack of internal consistency and mutual exclusivity of the variables confused me. You sent me an instrument which has a sloppy format, confusing page layout, misspelled words, incorrect grammar, etc. You didn't tell me what the limits of the study are so I feel that I will never finish your instrument.

Discussion: How can a respondent give reliable and valid information on an instrument with these and other defects? Too often, the researcher skips the step of conducting a field test. A multiple step field test and revision cycle will eliminate most defects in the survey instrument. The researcher should conduct a final field test with a small sample of members of the population to be surveyed, make corrections, and only then send the final version of the instrument.

Reason 3. You timed your data collection effort poorly. I received your instrument three days after the date you wanted me to respond so my input cannot be included in your research. Or, you gave me only a week to respond and your instrument arrived during my midterm grading week or at the end of the semester when I was grading term papers and final examinations. Or, your instrument arrived during winter break (or summer vacation) when I was away and the due date passed before I returned.

Discussion: Make it convenient for the respondent. Give the respondent sufficient time to complete the instrument. Make allowances for delayed mail, holidays, conventions, or academic year events when your respondents are from the academic community. Print follow-up copies of the instrument well in advance so they may be mailed to non-respondents weeks before the return date. Budget your study so you can use FIRST CLASS MAIL for all data collection activities both TO and FROM the respondents.

Reason 4. You failed to honestly identify yourself. Who are you and why should I spend my valuable time to give you information?

Discussion: Researchers in technology education need to identify their sponsoring organization and the function they perform in the organization. If the sponsoring organization is generally unknown to technology education respondents, then a paragraph explaining its roles and purposes is needed. Graduate students conducting technology education thesis or dissertation research should identify their status in the cover letter accompanying the instrument. Graduate advisors should add a signed statement to the cover letter stating that the instrument is part of an approved thesis or dissertation, that the advisor reviewed and approved the instrument, and that he or she requests respondent cooperation. A copy machine facsimile of the advisor's signature is appropriate but the researcher should personally sign each cover letter.

Reason 5. You failed to justify the research. Your assertion that the research will make "a valuable contribution to technology education" doesn't motivate me to spend my valuable time responding.

Discussion: In the cover letter, provide a purpose statement and a statement of need briefly describing how the research findings will fill a gap in the body of knowledge of technology education. Describe how the researcher, the respondent, or others can use the research results.

Reason 6. You didn't promise me an abstract of the results if I request it. What am I going to get out of my time spent responding?

Discussion: Provide the respondent with a place to check on the instrument to request an abstract of the results of the research or a separate postcard to request an abstract if respondent confidentiality is necessary. The respondent spends time reflecting on the items of the data collection instrument. The respondent needs to grow from the research and often wonders how he or she contributed to the results. An abstract will allow the respondent to compare the results with his or her own views and learn from the research.

Reason 7. You failed to say please and thank you.

Discussion: Researchers sometimes get so involved with their research procedure that they fail to attend to common courtesies. Make your thank you active in voice, personal, in the first and second person, direct, and brief.

SUMMARY

Experienced survey researchers will find nothing new here, yet we continue to receive poorly designed and conducted surveys in the mail. This threatens the integrity of our discipline, because this causes us to wonder whether survey research data is valid and reliable. Poor instrument design also leads to low instrument returns, further threatening the generalizability of the findings.

Technology educators conducting mailed surveys face a difficult challenge in obtaining a representative response rate. The request to complete an instrument imposes upon the valuable time of the respondent. The past history of poorly designed research instruments places a negative bias on the process. I cannot emphasize enough the importance of screening a prototype research instrument through a multiple-cycle field test and revision process to eliminate threats to validity and reliability and to make the instrument "respondent friendly."


John Gallagher is Associate Professor, Department of Technology, Glassboro State College, Glassboro, New Jersey.

Permission is given to copy any article or graphic provided credit is given and the copies are not intended for sale.

Journal of Technology Education Volume 2, Number 1 Fall 1990