Module 3: Designing your Research Project

Module 2 introduced you to the challenges and opportunities of non-doctrinal research. The readings, and the Sage site illustrate an incredible variety of non-doctrinal methods. Later we will discuss some qualitative, quantitative, and mixed methods in more detail but for now we will concentrate on the design of a research project.

Methods should be tailored to the research problem

There is an aphorism that “for someone with a hammer, every problem is a nail”, which in research is demonstrate by someone having a method, then looking for a problem to investigate using that method. A common example in law and policy research is a student deciding that they should use a stakeholder survey before considering the alternatives. Surveys are often costly, slow, difficult and do not always yield useful intelligence. Research should be tailored after you have decided the specific questions that you are trying to answer, and methods should be considered after the researcher has ‘unpacked’ what the research needs to answer; and considered which methods will be simple and efficient to use to provide reliable answers.

Using a survey (then analyzing the narrative constituted by the responses) is not a surprising choice. Discourse is central to doctrinal law and jurisprudence, and of politics and other social studies, so a bias towards using narrative as data is understandable. Some of the papers you have read have also assumed that researchers will use this approach. However, this ignores (for example) visual methods, dynamic systems modelling, experiments, meta-analysis of prior research, co-creation with stakeholders, and many other contemporary approaches.

Using surveys requires ethics approval, selecting and recruiting a suitable sample, interviews, or mail or face-to-face survey forms, recording and coding, data analysis, and managing the logistics of the work; and all of this takes time and effort. Interpretation can require sophisticated techniques, possibly using specialist software. Your surveys may not yield useful data, which will consume resources for little benefit. Depending on the research question and how it is framed, alternatives to surveys include meta-analysis of data that other researchers have gathered; using expert informant interviews to obtain insights based on their accumulated knowledge of the issues; observational methods; systems dynamic or economic modelling; and many other approaches.

Investing care in deciding what methods will be useful and feasible can avoid wasted effort and frustration, and lead to more useful results.

Design guidance

There are guides for researchers about designing and setting up projects. Sage Research Methods provides a 10 step “Project Planner”, with connections to relevant readings (http://methods.sagepub.com.ezproxy.une.edu.au/project-planner); and other books and guides (even You Tube clips) provide guidance. However, the differences between disciplines, between research issues, and the researchers’ situation (resources, training, time etc.) mean that not all advice will be appropriate for all situations.

Researchers should think carefully about research design (and about the possible impacts of the design on the outcomes of the research). For example, if you use in-depth interviews with a small sample of experts to explore complex issues, it is unlikely that statistical analysis will be useful. Trying to follow principles for gathering statistical data will be of limited relevance, and may complicate the work. It is worthwhile to discuss proposed methods with research supervisors and experienced researchers and mentors – good advice about research design can save you pain later on.

Unpacking research questions and methods for complex law and policy research questions can pose unique challenges. These are not discussed in the general methods literature. The complicating factors that affect research design include: the number and variety of issues, their complexity, the importance of subjective considerations (e.g. beliefs and values), and the dynamics of the social transactions. Later we will deal with some other challenges.

In Module 1, we considered three examples of law and policy questions to illustrate the challenges of law and policy research design. These were rules to: (1) protect vulnerable female migrant workers; (2) ensure that endangered species are protected; and (3) implement cost recovery for public legal and administrative services. Each topic involves legal doctrinal issues, administrative and legal processes, and socio-cultural and economic variables. Different analysis is needed for each sub-issue, and each method requires specific data, and data-gathering methods. Failing to address any sub-issue could reduce the validity of the research.

Such difficulties are more likely with applied research. For example, the capability of implementation agencies and community groups and the capacity of those affected, may be important practical concerns that are less relevant with theoretical investigations. Interest-bargaining and politics may also need to be considered in applied research. It may even be necessary to consider performance monitoring, evaluation and continuing improvement mechanisms, when designing institutional arrangements for implementation.

What questions can actually be investigated, and what methods can be used, is however limited by what is feasible. A common challenge in law and policy research is balancing the need for comprehensiveness and the limits of what is feasible. The video that follows considers this challenge.

Designing your project

Having watched the video, you should begin the design of your research project. I suggest this be done in 4 stages. You will probably have to revisit each stage a few times as your ideas evolve. It may be useful to use a spreadsheet to document your design, to facilitate communications with mentors, colleagues or your supervisor as you develop your plan. Naturally, thorough background research of the literature and the views of well-informed people will make it more likely that this process will be productive.

Step 1: decide the central research question/issue.

Consider:

1.   What is the issue/problem/opportunity or challenge that you intend to address?

2.   Why is this an important question to investigate (and for whom is it potentially significant)?
a.   Practical application and impact?
b.   Theoretical importance?
c.   The value of developing the methods and data?
d.   Other reason?

3.  What makes you think that this research might be feasible?
a.   Sources of data?
b.   Accessible interest or stakeholders?
c.   Other reason?

Step 2: Decide what sub-questions/topics you need to address to deal with your main question?

1. In essence this requires that you unpack your main question to find what issues are embedded within it. Consider
a.   causes,
b.   effects,
c.   institutions,
d.   drivers
e.   etc.

Often this will ‘explode’ the research into many issues/questions.

2.Identify the most significant sub-issues (bearing in mind what might be feasible given your time and resources) that you intend to research. Note the sub-issues that you do not intend to investigate so that you can disclose these gaps, or point to possible hypotheses based on the literature (for future investigations), when writing up your research. In your research publications you might include a discussion based on the literature or other information, of the issues that you were not able to research and disclose why it was necessary to exclude these.

Step 3: for each sub-question you intend to investigate, consider what type of analysis of what type of evidence might provide meaningful insights.

a.   Investigate possible methods by considering previous studies, books and papers on methods, or advice from supervisors and experts.
b.   Consider the likely feasibility of those analyses for you,
i.   The analysis method that is required;
ii.  The data that is required;
iii.  Your capabilities and resources.
c.   Decide what method of analysis you propose to use;
d.   What data will be needed to use that method; and
e.   Decide what methods of evidence gathering you intend to use.

Step 4: Develop a staged and costed plan for the research. The following are complexities that researchers can easily overlook.

1. Administrative processes – e.g. the process and time for research approval within your institution, ethics approval, travel approval and booking, or to access third party data. Consider the possibility of bureaucratic delays.
2. Data access – delays, approvals and costs of accessing, and the time to convert data into the form needed for the analysis (e.g. coding, data entry).
3. Recruiting and managing sources, including experts, stakeholders and research subjects, and corporate collaborators.
4. Software you may need, software training, and the time required to become competent.
5. Third party support, such as data processing, graphics, editing and reviewing.
6. Allowance for possible personal or institutional commitments, or possible interruptions.

These steps should provide a summary research plan, as the basis for a methods chapter or a grant application, or to explain your research.