Understanding Research (8 page)

Moving on now to another loaded term for many: method/s. The characterization of method offered by two social epidemiologists below, who also know how to bake, encapsulates this second dimension to the ‘how’ of research designs for most of us. Namely that methods are

Exactly which unwritten and written ‘rules and procedures’ are considered to be the tried and true ways to accomplish the task is where paths diverge once again; philosophers speak of ‘philosophical research’ (Radder 2006), linguists may speak of ‘semiotic methods’ (see Scolari 2009), political scientists may speak of ‘critical realism’ (Burnham et al. 2004), feminists of ‘gender mainstreaming’ (True and Ackerly 2010).

The point of knowing which recipe you are following and to make what sort of dish, to extend this analogy, is a first-base distinction in terms of method/s. A second-base one, to continue the baseball analogy, is that by learning how to do things a certain way we are also learning to know things a certain way. Immediately, this takes us into the terrain of what counts as the best way to
follow
a particular recipe; leaving aside for the moment questions of why this recipe (see section above). For instance, two key criteria for
empirical
researchers, two criteria for assessing the mettle of any research design and the
inferences
its results rest upon, are
replicability
and
transparency
.

1. Whether someone else – another student, colleague, or lay person – could undertake another research project along the same lines, using and being able to access the main sources of information accordingly (written texts, similar sorts of people, raw materials).
   
2. Could someone else consult your ‘data set’ or array of evidence, go to where you conducted your fieldwork, locate your source literature, if not to compare their conclusions against your own then to double-check the source on which your findings rest?
   
3. Unlike investigative journalists, and depending on the ethical terrain at stake in the project, academic researchers do have to reveal their sources.
   ¹¹
   

For less rules-and-procedures bound ways of conducting research, these general criteria hold as well. For most though, to speak of method is about how you went about things; I went here to talk to whomever, I accessed this space with permission from whom, I took part in an internet discussion group with permission or prior knowledge, those ideas from others I cite can be found on such and such a page, this many respondents did my survey. However, the term also entails more complex criteria, and issues about the strengths and weaknesses – and appropriateness – of the chosen method/s given the stated aims and objectives of the project and the research question.

This is why method/s are not innocent in the sense of being neutral. One size does not fit all because all methods (recipes) can be applied and used in various ways. They also arise from various layers of understanding of the processes, and outcomes which they are laying out; e.g. various sampling techniques, questionnaire formats, experimental parameters, dependent versus independent variables, primary versus secondary sources, software analytical tools, on-the-ground participant-observation versus – or alongside – online (web-based) ones.

In addition, certain broad categories of methods are distinctive, ‘brand names’ in their own right; for example,
regression analysis, semiotics, psychoanalysis
. Their defining role in particular debates, lines of intellectual allegiance and professional qualifications also means that opting to use some particular sorts of methods for the ‘data-gathering’ pertinent to your project brings certain conceptual vocabularies, authors, and expectations with them. In these cases, your nominal method speaks for itself even though nuances reside within debates generic to these approaches.

Nonetheless, this second side to the ‘how’ question is actually not that mysterious; it involves us taking the time and space to provide simple, though not mundane explanations of what exactly we intend to do (proposals) or what we did do (afterwards); advantages and disadvantages included. What these techniques, tools, or combinations will provide in terms of ‘facts’, ‘data’, ‘insights’, or ‘experiences’ and what they cannot do are the baselines for any project’s claims and achievements.

But what is the distinction between your method/s you are employing, your theory – conceptual framework – and the need then to talk about
methodology
? At times, little point in that the latter is used synonymously with methods; both refer to a ‘description of the methods or procedures used in some activity’ (Sloman 1977: 387). Then there are those instances where methodology is pronounced as ‘merely as a more impressive-sounding synonym for method’ (ibid.: 388), its often unintentional use as a synonym for theorizing notwithstanding. But there is more to it than this.

This term also refers to a particular sort of undertaking, an ‘investigation of the aims, concepts, and principles of reasoning of some discipline, and the relationships between its sub-disciplines’ (ibid.: 388). In this wider sense, methodology can also be an object of study, an academic discipline in itself. There are theorists and philosophers whose specialization is methodology. Moreover, every discipline generates its own set of methodological conundrums, in turn those who specialize in asking each other and working researchers awkward questions about a research practice, particularly those that become standard procedures or ways of talking about the ‘right’ and ‘correct’ procedures in any domain as if they were beyond question (see Oakes and Kaufman 2006: 7
passim
).

Moses and Knutsen distinguish between the two ‘m’s’ in their likening methodology to the toolbox and respective methods to the tools in the box (Moses and Knutsen 2007: 4–7); different tools need different sorts of toolboxes. Creswell opts for the expression ‘strategies of inquiry’ instead, which may help those who are not into DIY (Creswell 2009: 11), when he distinguishes between methodology as
strategies
and the particular techniques –
methods
– used to conduct the research . . .

How do these nuances actually pan out in general practice? In the day-to-day grind of getting a research project done do we need to be so concerned about such analytical distinctions? Whatever the response, you need to engage at some level of methodological explication, including the pros and cons of this chosen approach; sooner rather than later.

Here are some general things to aim for even before you know exactly how you want to go about your investigation:

• First, take note that a discussion about ‘methodology’ entails more than a description of particular ‘methods or procedures’ or literature review-like exegesis of generalities (e.g. ‘participant-observation’). The practical specifics need to be there, as well as make sense for your statement of purpose, if not encompass an understanding of your own worldview and why this approach is suitable for this project.
  
• Then, relax; laying out the specific data-gathering techniques (methods) you plan to use, and then actually do employ is relatively straightforward writing; unless of course you do not have a clear idea about the ‘rules and procedures’ (Oakes and Kaufman 2006) you are employing or believe, erroneously I would argue, that there are none worth mentioning.
  
• In formal research proposals, and dissertation formats, you need to deal with these aspects in relatively simple language. A straightforward description of how exactly the data was gathered and under what conditions – method in the strictest sense – is indispensable information; indeed obligatory if others are to be able to assess the findings, duplicate the research, adapting or countermanding it accordingly. As mundane as it is, the method/s section, wherever its place in the proposal or final report, tells a reader a lot about the working premises you are using to investigate or argue your case.
  ¹²
  
• What lifts the more descriptive aspects of any ‘method chapter’ or ‘methodology’ discussion up a level, is how effectively you discuss the implications your chosen approach has for your research question, eventual findings, and conclusions.
  

How do these rules of thumb play out within, and across the quantitative–qualitative divide, particularly in the early decision-making and planning stages?

For those working in the quantitative tradition, and even those working with qualitative data and relying on an understanding of
scientific reasoning
(see Chapter 3), the above privileging of methodological discussions, with a big M, is less evident in either a research proposal or final report. The following provisos may well apply to your project if not expectations arising from your disciplinary setting:

• When working with an approach where the
  empirical data
  gathered, or produced are through forms of (in)direct observation (see Chapter ?????? section), the rules and procedures for carrying out these observations constitute method in the stricter sense.
  
• For more quantitative sorts of projects, in the written research plan or report itself there is seldom reflection on broad, meta-level methodological debates; for example, around the relationship between the observed and the observer.
  
• Unless these might relate to problems with (mis-)measurement; for example, human error, or the statistical instruments used, or considerations of how a researcher may need to minimize the impact of his or her presence (e.g. in experimental set-ups). Then these would be discussed in the context of how they need to be taken into account; for example, how they may influence the results and then any inferences drawn from them.
  
• This is different from seeing such discussions as integral to the aims and objectives of the project. In these cases, specific and more general considerations may actually comprise the research question. Methodological questions, as outlined above, can also be research topics. That said, these too need to be rendered as a particular research question; one that is of a different order and register than in quantitative empirical projects.
  

There are several bottlenecks, however, no matter which way you look at things. To recapitulate, research proposals and the reports that ensue often stand or fall on what peers, supervisors, and examiners, make of the practical side of things; how the data was gathered, and then analysed (see Gray 2009: 57–8). Before considering ways of dealing with a number of recurring bottlenecks, let’s look at them more closely:

• Critics of approaches that keep all broader methodological discussions at arm’s length see this as ‘uncritical’; a convenient way of compartmentalizing intractable issues which, if addressed would indeed raise questions about the very rationale behind any chosen method; for example, why embark on a double-blind experiment, or conduct a large-scale survey when human behaviour is so notoriously unpredictable?
  
• Reference to the data-gathering or analysis tool, for example, the swathe of statistical and content-analysis software packages currently available, does not constitute an adequate presentation of your methodology in the round.
  
• Misperceptions and mutual misunderstandings aside, certain qualitative approaches do baulk at having to present method is such matter-of-fact ways.
  
• Conversely, others have little truck with research reports that do not deem it necessary to provide any indication of the procedures used to get results, or sense of the various sorts of sources upon which the conclusions are being drawn; for example, when a primary source (first-hand, or ‘raw data’) is usually privileged above a secondary one (second/third-hand); or the significance of choosing to undertake a survey with a highly selective sample when making generalizations.
  
• Whatever the larger conversations may be, these silences can hold you up in getting to grips with the practicalities that apply to your research project (e.g. your questionnaire questions, the sorts of focus groups you intend to set up, the archival sources you need to consult, or communities you plan to observe). As fascinating as these meta-methodological debates are:
  
  • At the planning stage not coming to terms with the admittedly mundane ways in which you are going to proceed can lead later to scrambling around in methods texts to identify said method at the last minute, with no time to understand discussion about their respective strengths and weaknesses.
    
  • Worse still, you may find yourself being lost for words when asked (in a Ph.D. defence for instance) ‘So, tell me, what
    was
    your method?’ When there is lack of engagement with ‘method’ in practical terms to speak of, pages of meta-level methodological discussion make little sense.
    
• That said, unless you are very clear about your intentions, avoidance here can result in two sorts of bifurcations in the final report:
  
  • one in which the theoretical and empirical sections diverge completely, seldom referring to each other and related to different sorts of questions;
    
  • in other cases, the description of the procedures used and then discussion of any findings assumes that issues broached in the preceding theoretical exegesis (often in the form of the ‘literature review’) are done and dusted; it may well
    be that the findings being presented actually support a different set of issues that those presented.
    

Either way, not having an idea about what position to take and why leaves you open to not only tough but also fundamental criticism about the very point of your work; the relevance of the data-gathering approach you opted for, and by implication the quality of the findings and conclusions made. These need not come from less sympathetic audiences from the ‘other side’, or hostile external reviewers. They may well be valid points raised from within your own scholarly circle-of-choice or affiliation.

Next: how to deal with these headaches, especially early on when positioned on either side of the divide?

• One rule of thumb, in the
  qualitative tradition
  at least, is that presenting any chosen method/s requires you to present its corresponding theoretical underpinnings; what this approach brings to bear on the research question in particular. However, recall that the converse is not always the case; much theoretical work, philosophical explorations for instance, can be achieved without a separate section on ‘method’; unless the latter is the object of inquiry.
  
• Similarly for research requiring quantitative techniques. A researcher is concerned about making an original contribution in a chosen area of study. There are different ways to make clear how that contribution relates to the data-gathering approach you want to take, which requires you to engage with the literature on methods in this field. For example, if your research proposes to make an advance in the study of voting behaviour or media effects, you would be remiss to ignore quantitative studies given that the quantitative approach dominates in these areas. However, this does not mean that innovative studies on citizens and political participation, media effects, or audience response cannot be conducted using qualitative methods (see Franklin 2010, Gunter 2000).
  

In short, some baseline rules apply to most of us when setting out:

1. do not presuppose that a method explains all, resolves the larger intellectual puzzles for you;
   
2. nor that a theoretical or methodological exegesis is a substitute for resolving the nitty-gritty details of how you will gather and then make sense of your material for you;
   
3. nor that multiplying methods will cover all your bases; sometimes less is more in this respect;
   
4. even if you have just completed some sort of methods instruction, or read a book about methods, avoid nominating your method first; more on this pitfall in due course.