Module 4: Learning from Research Practitioners

The short answer to the question “what method should I use?” is that this choice depends on what precise questions you need to answer, your skills and training and interests, and what time and resources you have to carry out the research, and the constraints under which you are operating. To make a sensible choice you need to know what methods might be useful for what purposes, and what is involved in each.  The explosion of methods has made this complicated.

Researchers can find information about possible methods through the internet (e.g. the Sage Research Methods site, or bibliographic sources such as Scopus or Web of Sciences) or library searches (in physical libraries or using online search tools such as Google Scholar). Non-doctrinal methods underpin many publications in law and policy journals, though few explain methods in detail. There is a small but increasing literature on non-doctrinal methods in journals such as the Journal of Empirical Legal Studies (Wiley) or Law and Social Enquiry (American Bar Association). Various books discuss non-doctrinal methods and provide examples, including:

• McConville, Mike, ed. Research methods for law. Edinburgh University Press, 2017.
• Cane, Peter and Kritzer, Herbert eds. The Oxford Handbook of Empirical Legal Research Oxford University Press 2010.
• Frans L. Leeuw and Hans Schmeets Empirical Legal Research, Edward Elgar 2016
• Epstein, Lee and Martin, Andrew D. An Introduction to Empirical Legal Research, Oxford University Press 2014
• Hoecke, M. (Ed.). (2011). Methodologies of legal research: Which Kind of Method for What Kind of discipline? Methodologies of Legal Research. Oxford UK: HART Publishing.

It is difficult (if not impossible) to adequately summarise the growing body of non-doctrinal methods. Each method has unique norms and practice requirements, and there are many descriptions in books, articles and websites. Most discussion of qualitative, quantitative or mixed methods does not give a lot of insight into how and where to use various methods. To categorise methods such as ‘photo-voice’, ‘auto-ethnography’, or ‘systems-dynamic modelling’, modern bibliometrics or observational methods as qualitative or quantitative does not help explain how to use them, nor where they would be useful.

Though books and videos can outline the purpose and process of particular methods, accessing lessons from first-hand experience is difficult. Below I provide links to short videos  of researchers experienced in various methods for law and policy investigations, highlighting what they have learned. It only deals with a few methods, but it should give researchers a feel about different approaches and some of the issues that can arise in practice.

Each interviewee was asked to consider the following questions:

Introduction: experience with non-doctrinal investigations

• What law or policy question did you set out to answer?
• What sub-questions did you need to answer, to tackle this central question?
• Can you describe one or more of the methods you used?
• What evidence did you need to apply the method?
• How did you go about gathering the evidence that you needed?
• What challenges did this involve, and how did you address them?
• From your experience, what lessons can you pass on

These videos highlight that a researcher should be clear about the precise question they are trying to answer, as this will help them focus on a method that can answer that question. Selecting a method that is compatible with your abilities and interests, and learning it property, can also make the task easier. Most researchers make mistakes, but they learn from them. This is why people with more experience can provide advice that you cannot find in a book.

From the many approaches I and my colleagues in our research centre there are five highlighted below that may be relevant to other non-doctrinal law and policy researchers, and I point to publications that illustrate their use.

1. Using a strategic logic in policy research.  Some of the discourse about research methods in law assumes polarities between (1) ‘scientific’ and ‘non-scientific’, and (2) ‘doctrinal’ and ‘non-doctrinal’ investigations. However, in practice many important non-doctrinal law and policy investigations require methods that challenge these assumed polarities. Many investigations inevitably draw conclusions based on inference from inadequate evidence, assumptions or untested hypotheses, and must combine doctrinal, philosophical and empirical elements; and applied research often has to deal with pragmatic challenges such as resourcing of implementation, and (sometimes irrational) political dynamics. Strategy is a long-proven ‘architecture’ to structure applied law and policy investigations. Within that architecture, multiple methods are often used to deal with the diverse issues that are involved. See Martin, P., & Craig, D. (2015). Accelerating the Evolution of Environmental Law Through Continuous Learning from Applied Experience. In P. Martin & A. Kennedy (Eds.), Implementing Environmental Law (pp. 27–49). Waikato: Edward Elgar Publishing. https://doi.org/10.4337/9781783479313.00006
2. Legal futurism. There is a time-lag, often years, between a law or policy issue being identified, analysis of the issues, deciding how to address them, implementing new instruments, and an effect on the issue. All this time, change is happening – to the stakeholders and actors involved, to socio-economic systems, and to the governance system. The nature of the problem and the possible solutions will have evolved. Different futurism methods can be used in law and policy research to make it more likely that eventual innovations will fit with the future context, at the time when they are applied. See Low Choy, D., Serrao-neumann, S., Schuch, G., & Martin, P. (2017). Scenario Planning for Institutional Improvements to Support Citizen Action in Invasive Animal Management. Invasive Animals Cooperative Research Centre Technical Report. Canberra ACT. https://community.invasives.com.au/wp-content/uploads/sites/4/2020/07/170508_pdf_Scenario-Planning-report-IA-CRC.pdf
3. Objective evaluation of instruments. Under the scientific method knowledge is systematically built on empirical foundations. A similar logic applies for continuous improvement in social methods such as management, policy and government – objective evaluation of existing systems, to identify where they might be improved, fuels reliable improvement. See Martin, P., Boer, B., & Slobodian, L. (2016). Framework for Assessing and Improving Law for Sustainability A Legal Component of a Natural Resource Governance Framework. Bon, Germany. https://portals.iucn.org/library/node/46120
4. Systems thinking about institutions. Systems thinking can improve understanding of the dynamics of interconnected things such as cyclical water flows, or the interaction of people and organisations in society. This can provide comprehensive explanations of the how many elements shape what happens in society, or highlight the many things that should be managed to maximise the effectiveness of proposed interventions. See Martin, P., & Gunningham, N. (2011). Leading Reform of Natural Resource Management Law: Core Principles. Environmental and Planning Law Journal, 28(3), 137–158 https://www.researchgate.net/publication/287882393_Leading_reform_of_natural_resource_management_law_Core_principles
   and Martin, P., & Reid, M. (2019). The socio-ecological systems approach. In P. Martin, T. Alter, D. Hine, & T. Howard (Eds.), Community-based Control of Invasive species (pp. 28–47). Canberra ACT: CSIRO Publishing.
5. Engaged research methods. Doctrinal and jurisprudential research is generally carried out with little involvement of the actors who will be affected by the regulation, and without much attention to the practicalities that will be encountered with implementation. Engaged research aims to involve those who will be affected by changes to institutions, including legal instruments, to help understand the issues and to design more effective instruments. See Martin, P., Alter, T., Hine, D. W., Low Choy, D., & Curtis, P. (2019). Managing outcome-focused, engaged “human” research. In P. Martin, T. Alter, D. Hine, & T. Howard (Eds.), Community-based Control of Invasive species (pp. 141–161). Canberra ACT: CSIRO Publishing.