What does it really mean to be methodologically rigorous?

One of the reasons I chose to get my doctorate in political science at UC Berkeley is because our department is known for being “methodologically plural,” meaning that multiple methods are embraced and taught: statistical analysis, game theory, survey analysis, case studies, comparative-historical analysis, and others.

I came into the program somewhat skeptical about the idea of social “science” – I wanted to study comparative politics, and this seemed to be the place to do it. But I learned something simple and profound about the scientific ideal: it’s about logic, consistency, clarity, and transparency. The ideal is that you make your methods of data collection and analysis clear enough that someone else could use your data, re-run the analysis, and get the same results. In practice, this meant thinking a lot about case selection, about the potential sources of error, and about the tools of data analysis.

What I took away was the idea that rigor is about making explicit what many take for granted: where did you get your information, how did you analyze it, how else could you have analyzed it, and how do your results follow from your analysis? With so much focus on data and metrics in the nonprofit sector and philanthropy, it’s important to remember that simple idea: rigor is not an elaborate technique or a fancy spreadsheet – it’s about honesty, with yourself and your audience, about the limitations, and the possibilities, of your work. If we can message that more effectively, it may be easier for some folks to get on the metrics bandwagon, and for the public at large to trust in the results of our work.


Tags: , ,

Leave a Reply