Blogs

Blog 303-Unlearning the Routine, Understanding the Reasoning: A Graduate Researcher’s Curiosity on Social Science Measurement

In this blog, Aseemudheen reflects on his experiences with research methods and tools in agricultural extension, highlighting the importance of understanding not just how to use them, but why they are used.

CONTEXT

“The variable was measured using a scale developed by X (year), with suitable modifications.”
Many postgraduate researchers will recognise this sentence from theses and journal articles. It sounds methodologically appropriate: a published scale has been cited and modifications have been made. But what exactly have we modified? The wording, response categories, language, or the meaning of the construct itself? And when we say a scale is validated, what has actually been validated, and for whom?

As a doctoral researcher in agricultural extension, I have begun to question these familiar practices, not because established methods are necessarily flawed, but because their routine use can overshadow the reasoning behind them. Does methodological rigour begin with selecting a statistical technique or with understanding what we intend to measure?

FROM CONSTRUCTS TO QUESTIONS

Extension research frequently investigates attitude, motivation, entrepreneurial ability, information-seeking behaviour, and environmental orientation. Since these are not directly observable, we infer them from responses, behaviours or indicators. That involves a chain of connected decisions: defining the construct, identifying its dimensions, developing items, selecting response categories, deciding how to score, and interpreting the scores.

Consider a study on farmers’ environmental orientation. Statements about soil health, biodiversity, pesticide use, and sustainability may all seem relevant, but do they measure the same attribute? Some may reflect attitudes, others knowledge or actual practices. That they are related does not justify combining them into a single scale.

The terms matter here. A Likert-type item is an individual statement with ordered response categories, whereas a Likert scale combines multiple items intended to represent a common construct. An index, by contrast, may combine distinct indicators representing a broader condition without assuming they are interchangeable manifestations of one latent attribute. The distinction has practical consequences: for such an index, there is no reason to expect the indicators to correlate strongly, so a low Cronbach’s alpha would not signal a problem.

Who then decides which items belong in a measure, and on what grounds? Sivakumar and Sulaiman (2015) stress selecting variables carefully and specifying how they are measured. A simple decision matrix, comparing candidate variables against theoretical relevance, empirical support, contextual importance, and feasibility, can record why indicators were included, excluded, or modified. That record makes our choices transparent and defensible.

ARE FIVE RESPONSE CATEGORIES ALWAYS ENOUGH?

The five-point format from strongly disagree to strongly agree is widely used, yet familiarity does not establish suitability. The choice between four, five, or seven categories may depend on the construct, respondents’ ability to distinguish between options, and the setting in which the questionnaire is administered.

The midpoint is another challenge. “Neither agree nor disagree” may signal neutrality, uncertainty, indifference, or ambivalence, which are not equivalent. Removing it does not resolve the ambiguity either; respondents who hold no position may be pushed toward one. Likert-type responses are also ordinal and assigning numerical codes does not establish equal distances between categories. Composite scores are sometimes analysed as approximately continuous, but that decision requires justification.

WHEN THE QUESTION TRAVELS: FROM ENGLISH TO THE INTERVIEW SCHEDULE

These choices matter even more in agricultural extension, where questionnaires are often translated into regional languages and administered orally. Translating the words does not necessarily preserve the meaning. A question intended to capture a farmer’s willingness to experiment with new practices might be heard as asking whether the farmer can afford the financial risk.

Language is not merely a medium for the question; it is part of the measurement. A sound command of English helps in distinguishing closely related concepts, but linguistic sophistication should not be confused with complicated wording. The real skill lies in expressing a concept precisely, in language respondents can meaningfully understand.

The International Test Commission (2017) emphasises linguistic and cultural adaptation rather than word-for-word translation. Cognitive interviewing in northern India, in a health survey context, has likewise shown that respondents can interpret questions and response categories differently from what researchers intended. Asking farmers to explain how they understand a question and arrive at an answer is a practical way to catch confusing terms and unintended readings before revising and pilot-testing. The aim is not a grammatically correct questionnaire, but one that communicates the intended construct.

FRAMING: A LESSON FROM BEHAVIOURAL ECONOMICS

My interest in these issues grew partly from AESA Blog 105, Rethinking Policy from a Behavioural Angle: A Case of Nudge, which shows how the presentation of choices can shape decisions. Questionnaires are not policy nudges, but the same logic applies: framing can be a source of measurement error.

Consider two statements: “Reducing pesticide use is important for protecting the environment” and “Reducing pesticide use could increase the risk of crop loss on my farm.” Both may be relevant to farmers’ views, but they foreground different considerations. Neither is automatically inappropriate; the question is whether the wording measures the intended construct and whether its framing could steer responses. Presentation matters beyond wording as well,  show that spacing, position and other visual features of survey questions can affect how respondents interpret them.

A PUBLISHED SCALE IS NOT A CERTIFICATE

A published instrument carries authority through its citations, prior use, and reported psychometric properties. Yet these should not remove the need to examine its suitability for a new study. AESA Blog 206, What Makes for Rational Extension Conversations?, discusses appeals to authority and popularity; the same caution applies to measurement. An instrument should be accepted for the evidence and reasoning behind it, not because it is widely adopted.

Cronbach’s alpha is often reported as proof that a scale is sound, but reliability does not establish validity. Items can be internally consistent and still miss the intended construct. Factor analysis can show how items cluster, but it cannot tell us whether they adequately represent the concept or whether respondents understood them as intended. Other tools address those questions: expert judgement, Item–Objective Congruence (IOC), Lawshe’s Content Validity Ratio (CVR), Q-sort methodology, and ANOVA combined with expert ratings can all assess item relevance and content adequacy, while cognitive interviewing explores respondents’ understanding. Each addresses a different aspect of measurement, and the choice depends on the evidence the study needs. The Standards for Educational and Psychological Testing similarly stress evidence and theory supporting the interpretation and use of scores for a particular purpose.

Validity, then, is not a permanent certificate attached to an instrument. When a scale is translated, modified, or used with a different population, we need to ask whether the evidence still supports our intended interpretation.

Two AESA publications are good starting points for scholars who want to understand not only how established methods are used but why they are chosen: Extension Research in India: Current Status and Future Strategies, which discusses research design, variable selection and methodological rigour, and the Manual on Good Practices in Extension Research & Evaluation, which covers instrument development and assessment.

UNLEARNING THE ROUTINE, WITHOUT ABANDONING THE METHOD

I am not suggesting that researchers abandon Likert-type items, reject published scales, or develop a new instrument for every study. Established methods carry accumulated knowledge and tested procedures. The invitation is simpler: do not allow familiarity to replace justification.

Measurement decisions, from choosing variables and items to wording, response categories and analysis, shape how farmers’ experiences are represented in research and how findings inform extension interventions and policy. Perhaps better measurement begins not with abandoning familiar methods, but with understanding them more deeply and being willing to question, learn, and refine our practice.

Acknowledgement

Some ideas begin with a question, but they grow when the right people encourage us to pursue it. I am sincerely grateful to my Chairman of my Advisory Committee Dr. M. Ramasubramanian, Senior Scientist & Head, ICAR – KVK, Virudhunagar, whose nudge to explore emerging methodological innovations encouraged me to look beyond routine research practices and turn my curiosity into this blog. I owe a special thanks to Dr. C. Karthikeyan, Professor and Head (DAE&RS, TNAU) whose Research Methodology classes during my Ph.D. coursework helped me discover an unexpected fondness for a subject that can often feel dry, but becomes fascinating when we begin to question the why behind the how. My heartfelt gratitude also goes to Dr. D. Alagu Niranjan (Research and Knowledge Management Officer, CRISP) for guiding us through the AESA Volunteer Programme, helping us find direction in our contributions and encouraging me to bring this reflection to life as a product of AVP Level 2.

Mohamed Aseemudheen M is a Ph.D. scholar in Agricultural Extension Education at Tamil Nadu Agricultural University, Coimbatore and a Level 2 volunteer under the AESA Volunteer Programme. He can be reached at aseem1264@gmail.com.

TO DOWNLOAD AS PDF CLICK HERE