Sustainable Development’s Big Data Revolution: Top-down, Bottom-up or Middle-out?

In the concluding chapter of ‘Bottom-up’ Approaches in Governance and Adaptation for Sustainable Development: Case Studies from India and Bangladesh we introduce as a possible alternative to the quintessential “top-down or bottom-up” conundrum in sustainable development the idea of “qualitative big data”–the amassing of huge volumes of case studies and other types of unstructured text-based data–organized around a “middle-out” information management structure. Below are some excerpts from the chapter and at the end of the post is a PDF of the entire chapter.

We know that a data revolution is unfolding, allowing us to see more clearly than ever where we are and where we need to go, and to ensure that everyone is counted in. We know that creative initiatives across the world are pioneering new models of sustainable production and consumption that can be replicated. We know that governance at both the national and international levels can be reformed to more efficiently serve twenty-first century realities. And we know that today our world is host to the first truly globalized, interconnected and highly mobilized civil society, ready and able to serve as a participant, joint steward and powerful engine of change and transformation. (United Nations General Assembly 2014: 7)

This statement by the UN General Assembly reflects an understanding of the scale of the sustainable development challenge we face and contends that given this scale, volumes of data as well as the best possible technologies for big data management and analysis must be brought to bear. We conclude this volume by suggesting that the scale of the challenge, and perhaps more importantly the depth of the solutions required, call not just for the quantitative big data implied in the UN General Assembly’s statement, but also for “qualitative big data.” By “qualitative big data”, we emphasize the local and grassroots level knowledge of the stakeholders – knowledge without a priori structure that tends to be documented in ways, such as through case studies, that produce “unstructured” data. Case studies, such as the research presented here in this volume, allow us to capture the important depth and richness of cultural contexts that must be understood for sustainable development “solutions” to work at the local level. If harnessed at a large enough scale, this depth and richness derived from specific case studies can meaningfully inform policy, economic, technological, social and other innovations across the infinitely diverse communities and cultures endeavoring to achieve sustainable development.

Adapting case studies for a “middle-out” bridge between top-down and bottom-up
Given the scale and speed of global climate change and its attendant socioecological disruptions, there is an urgent need to begin identifying successful sustainable development strategies across multiple social, geographical and temporal scales. This volume makes the case that neither top-down nor bottom-up approaches alone have the ability to produce the needed insight. And yet, given the demands of climate adaptation, community-based sustainability experiments–often little more than tinkering or muddling in livelihood and survival strategies–are occurring on a daily basis around the world. The case study approach represents one possible method for beginning to tap into the knowledge being produced through these experiments. Furthermore, the information technology now exists to facilitate compilation of qualitative reports of community-based sustainability experiments into a vast corpus with the potential to be mined both by communities and individual actors as well as high-level institutions and decision-makers within the sustainable development apparatus.

Figure 1 depicts the relationship between the state and global actors currently controlling the sustainable development apparatus and the community actors engaged in iterative, and typically closed, processes of experimentation to achieve local sustainable development goals. The “middle-out” approach does not call for new institutions bridging these extremes. Middle-out refers instead to a system of information flow wherein lessons learned in the previously isolated and closed-loop community-based experiments is aggregated and organized. With access to this information, top-down decision-makers can more effectively support strategies relevant to local contexts and community-based actors, who previously relied on ad hoc networks to learning anecdotally from one another, can more systematically learn directly from one another.

SEVeN middle out figure

In short, the middle out approach relies on aggregation of case studies and other reports of community-based sustainability experiments to develop a knowledge base for community-based sustainability experimentation processes that can be leveraged by a wide range of decision-makers and actors at multiple levels. The “middle-out” platform will create a space for both top- and bottom-level stakeholders to combine the power of the descriptive richness of qualitative case studies with the analytical power of natural language processing and other information science tools. This strategy holds the potential to bring to bear substantial yet untapped knowledge in sustainable development strategies. The virtual “middle-out” space is possible because of present rapid diffusion and access of information and communication technologies in developing countries like India and Bangladesh. Brabham (2009) has identified that “the medium of the Web enables us to harness collective intellect among a population in ways face-to face planning meetings cannot.” (p. 243) The task of creating a structured and user friendly “middle-out” platform is no doubt challenging as it involves vast amounts of unstructured data. However, we believe that if successful, this platform can open up a plethora of rich local level knowledge which can be utilized from local to national and global level stakeholders.

The middle-out approach begins with the assumption, explained above, that small experiments in climate adaptation and sustainable development are producing extensive yet underutilized knowledge vital to the social learning required for societies to transition to a sustainable future. It draws on the “small experiment framework,” an approach to behavior change developed by psychologists (Irvine and Kaplan 2001; Kaplan 1996) that is premised on the notion that the human tendency to “muddle through” problem situations can be modified into a form of “adaptive muddling” that can result in more rapid social learning and change (De Young and Kaplan 2012). In “muddling through,” humans constantly take small steps but without straying far from the results of past changes. The problem is that “[m]uddling is a process characterized by…a tendency to compromise, and an avoidance of significant bold or visionary steps” (De Young and Kaplan 2012: 290). Key to adaptive muddling is that it emphasizes not small steps but small experiments. It offers a way of simultaneously exploring several possible solutions,” according to De Young, “thus avoiding the sluggishness that plagues one-solution-at-a-time approaches. People are empowered to apply local or personal knowledge to a situation. Different people applying different knowledge to the same situation creates a variety of potential solutions” (1999: 602).

Middle-out approach: Beyond participatory development
Participation, empowerment, and partnership are the key concepts related to development discourse. Participation ‘by the people’ is defined not just as a basic need, but as a fundamental human right which is a tool for empowerment (Cornwall 2002, 2003). According to the World Bank, participatory development is “a process through which stakeholders influence and share control over development initiatives, and the decisions and resources which affect them.” (World Bank 1996). The overarching objective of participatory development is to involve people and communities actively in identifying problems, formulating plans and implementing decisions over their lives (DFID 2003). In this process, all the stakeholders have a critical role in the success of the project (Chopra et al. 1990, Mansuri and Rao 2004). A stakeholder is “any individual, community, group or organization with an interest in the outcome of a programme, either as a result of being affected by it positively or negatively or by being able to influence the activity in a positive or negative way” (DFID 2003: 2.1). Yet nowhere in the participatory development literature is there discussion of how to aggregate dispersed and isolated knowledge that exists in communities and individuals throughout the world so that participation is not just a process but also a crucial mechanism for accessing and applying networked knowledge. The middle-out approach we describe here fills this gap.
The middle-out approach presumes participatory processes as a starting point, then proposes tools for aggregating the cumulative knowledge of communities to be accessed through participatory processes thus making them more effective. If scaled up appropriately, the middle-out platform can create a significant amount of data which can be utilized across communities. As such, the middle-out approach enhances participatory development. We introduce the middle-out approach in this concluding chapter with the hope of provoking readers into considering the possibilities, within this new conceptual model, for ICTs to provide tools that address the gaps between top-down and bottom-up approaches captured in the preceding chapters. As a conceptual model, we do not have details of actual implementation to share with readers. A small pilot project aggregating case study and other bottom-up knowledge in single country would be a logical next step for testing proof of concept.

In a modest way, this volume points to the potential value of building a middle-out approach by aggregating case studies across bottom-up and top-down approaches to sustainable development. Scaled up 100 or even 1,000 times from the 14 case studies compiled here, new insights will emerge and new strategies for locally relevant and contextualized sustainable development strategies will follow. Local innovators and leaders could have access to extensive and systematic case study-derived knowledge to adapt and apply to local experiments while top-down institutions can more sensitively and strategically place resources to facilitate local strategies. The decentralized structure and horizontal rather than vertical linkages will provide peer-to-peer communication and diffusion of knowledge with the potential to overcome imbalances of power, economic inequality, cultural barriers, and other social factors that prevent the dissemination, flow and implementation of sustainability solutions. Governance for sustainable development in the anthropocene will require nothing less.


Brabham, D. C. (2009). Crowdsourcing the public participation process for planning projects. Planning Theory, 8(3), 242-262.

De Young, R. (2014). Some behavioral aspects of energy descent: how a biophysical psychology might help people transition through the lean times ahead. Frontiers in psychology, Vol. 5, Article 1255: 1-16.

De Young, R. 1999. “Tragedy of the Commons.” In D. E. Alexander and R. W. Fairbridge (Eds.) Encyclopedia of Environmental Science. (Pp. 601-602) Hingham, MA: Kluwer Academic Publishers.

De Young, R. and S. Kaplan. 2012. “Adaptive muddling.” In R. De Young and T. Princen (Eds.) The Localization Reader: Adapting to the Coming Downshift. (Pp. 287-298) Cambridge, MA: MIT Press.

Irvine, K. N., & Kaplan, S. 2001. Coping with change: The small experiment as a strategic approach to environmental sustainability. Environmental Management, 28(6), 713-725.

Kaplan R. (1996). “The small experiment: achieving more with less.” In Public and Private Places, (Pp. 170–174), Nasar J. L., Brown B. B., editors. Edmond, OK: Environmental Design Research Association.

Kaplan, S., and Kaplan, R. 2003. Health, supportive environments, and the reasonable person model. American Journal of Public Health 93: 1484–1489. doi: 10.2105/AJPH.93.9.1484

Kaplan, S., and Kaplan, R. 2009. Creating a larger role for environmental psychology: The Reasonable Person Model as an integrative framework. Journal of Environmental Psychology 29: 329–339. doi: 10.1016/j.jenvp.2008.10.005

Seligman M. E. P., Railton P., Baumeister R. F., & Sripada C. 2013. Navigating into the future or driven by the past. Perspectives in Psychological Science 119–141. 10.1177/1745691612474317

United Nations General Assembly. 2014. The road to dignity by 2030: ending poverty, transforming all lives and protecting the planet A/69/700, December 4. Available at: [accessed 29 Jan 2016]

Click for full text PDF of “Neither ‘Top-down’ nor ‘Bottom-up’: A ‘Middle-out’ Alternative to Sustainable Development,” pp. 315-325 in ‘Bottom-up’ Approaches in Governance and Adaptation for Sustainable Development: Case Studies from India and Bangladesh, Swarnakar, P., Zavestoski, S., & Pattnaik, B. K. (Eds.), SAGE Publications, New Delhi (2017). 

The ‘Middle-out’ Approach to Climate Change

It’s fashionable today to push for bottom-up approaches for addressing all major problems. However, authors of a new book argue for a middle-out approach that takes advantages of learnings and experience of leadership from the top and blend it with ground realities. Here’s a peek into the idea:

Given the scale and speed of global climate change and its attendant socio-ecological disruptions, there is an urgent need to begin identifying successful sustainable development strategies across multiple social, geographical and temporal scales. In this scenario neither top-down nor do bottom-up approaches alone have the ability to produce the needed insight. One possible solution to this puzzle is “middle-out” approach.

This is featured as cover story of leading sustainability magazine Sustainability Next:

SEVeN’s Origins

The Sustainability Experimentation Venture Network (SEVeN) emerged out of work Stephen Zavestoski and Pradip Swarnakar did in organizing a conference on “Environment, Technology and Sustainable Development” in 2014 in Gwalior, Madhya Pradesh, India. Papers received for the conference revealed a rich body of case study-derived knowledge that was being underutilized. Conversations with conference participants led us to realize the potential in forming a knowledge network into which researchers, practitioners and lay people could feed examples of small-scale, community-based efforts to engage in experimentation to achieve sustainability.

The vision was shaped by a sense that case studies could be aggregated to advance the field of sustainability transitions research. Much of the existing work on sustainability transitions examines top-down, policy- and technology-driven approaches to sustainability experiments. From this orientation, “big data” means large quantitative data sets structured and populated based on assumptions made from within the dominant institutional perspectives on sustainable development.

We suspect that bottom-up responses to resource shortages, climate change, and other environmental changes–and even local knowledge emerging out of everyday livelihood strategies–can contribute importantly to a vast database of unstructured qualitative data that can be approached from a wide range of perspectives and needs to aggregate and disseminate knowledge that can inform sustainable development strategies at all levels. This is the promise of SEVeN.