New Journal Article

design, design research, Informatics, knowledge production, knowledge tools, philosophy of science, research

Terrific (not-so-new) news! She Ji: The Journal of Design, Economics, and Innovation published an article I wrote with Erik Stolterman about whether knowledge claims could be a useful way to distinguish research communities from one another. Here is the abstract:

While much has been written about designerly knowledge and designerly ways of knowing in the professions, less has been written about the production and presentation of knowledge in the design discipline. In the present paper, we examine the possibility that knowledge claims might be an effective way to distinguish the design discipline from other disciplines. We compare the kinds of knowledge claims made in journal publications from the natural sciences, social sciences, and design. And we find that natural and social science publications tend to make singular knowledge claims of similar kinds whereas design publications often contain multiple knowledge claims of different kinds. We raise possible explanations for this pattern and its implications for design research.

… and a link to the article, which is freely available for download. I’d welcome any comments or feedback. This is part of a broader project investigating the transfer and interplay of knowledge in research communities.

what should theory mean?

design, design research, design theory, HCI, hci research, HCI/d, Human-Computer Interaction, karl popper, philosophy of science, theory, theory building, theory development, theory-practice gap

I’ve been collecting (and modifying and losing) thoughts about theory and the different ways it has been discussed and debated in academia for some time now. Recently I started organizing a collection of readings on/about theory according to different concepts like: theory development, theory-practice relationship, theory-artifact relationship, etc.

What’s really striking about all these readings is the way in which authors talk about what theory means before ultimately choosing a meaning to work with. There is almost universal agreement that theory is a complex word that can mean lots of things to lots of different people who work in different places and think about different things.

But how do we make a choice between different meanings and do we examine the consequences for our choices? Why do we not write about the choice when we write about the multiple meanings of theory?

Contemplating these questions would bring a great deal of clarity to an ambiguous discourse (esp. in HCI where researchers have since the early days of the field wrestled with theoretical adequacy, the relationship between artifacts and theory, etc.)

the future of hci research

HCI, hci research, knowledge production, knowledge tools, peer review, philosophy of science, quality control, science, scientific method

My PhD advisor, Erik Stolterman, recently penned a blog post called hci research and the problems with the scientific method. It’s a good post and you should read it. It was motivated by a New Yorker article written a few years ago about the ‘decline effect’ in science, which refers to the phenomenon wherein at-first positive results of scientific experimentation decline over time. I’m simplifying here, but you can read more about it in the New Yorker piece itself or the Wikipedia page on the decline effect.

In any event, this article led Erik to speculate about the implications of this ‘problem’ with the scientific method for hci research, a field grappling with a dilemma that he describes as “[stemming] from a shared view that HCI is not really a scientific enterprise while at the same time scientific research is still valued and rewarded.” He then speculates about how hci will develop in relation to science and expresses optimism about where the field will go, and here is where my thinking picks up the thread.

His last paragraph makes me think about the possibility of greater diffusion of the field and what that means for something like ‘quality control,’ in other words, “the criteria to assess the quality of the work and the teams that carry it out,” in hci research. I’m using Gibbens et al.’s definition of quality control from their book ‘the new production of knowledge’.

Anyway, when he writes that hci research could move in the direction of science ‘or’ the humanities ‘or’ design, I wonder whether that means that one of those paths will be dominant. For instance, if hci moves towards a more scientific tradition, that could mean that the humanistic and design traditions become ‘lesser research traditions’ within hci research. And if one of the traditions becomes more dominant, then quality control seems like it might be less of an issue since science and the humanities, at least, have rich histories and rigorous ways of evaluating research contributions. This might not be so true of design research.

But of course there doesn’t have to be a dominant path, and maybe a dominant path is impossible in hci research. Say hci continues to diffuse towards each of the traditions and maybe even, as he mentions, towards some new ones. That there ceases to be a dominant approach probably means that quality control becomes more context-dependent, temporary, and fluid, descriptors I’m borrowing (again) from Gibbens et al.

With respect to the dilemma he describes, denying that hci is a scientific enterprise while simultaneously valuing and rewarding scientific research, i think the possible “diffuse” future encourages a resolution characterized by valuing and rewarding scientific research, humanistic research, design research, and maybe to spend more time thinking about the relationship between these different traditions.

Knowledge Claims in Design Research

design, design research, design theory, design thinking, HCI/d, karl popper, philosophy, philosophy of science, theory, Uncategorized

One of my summer research projects has to do with knowledge claims in design research publications. The question stems from an interest in understanding the similarities and differences between how knowledge claims are formulated in design research publications as opposed to natural and/or social scientific research. One would expect there to be some overlap as there is scientific work done in design research. In fact, there are lots of different kinds of researchers publishing in design journals so one might expect to see a variety of kinds of knowledge claims (claims with different ontological and epistemological underpinnings) being made in a given journal or conference. But what (if anything) sets a knowledge claim in design research apart from knowledge claims in the natural and social sciences? A preliminary observation is that within some articles there are a variety of different kinds of knowledge claims that have conflicting epistemological and ontological underpinnings. How do we account for this variety? And what are its implications for publishing in design research?

Summer Research Projects

design, design research, design thinking, HCI, HCI/d, Interaction Design, philosophy, philosophy of science, science, theory, writing

The academic year is over, and after having taken a few days off, today is the day I launch into several really exciting and interesting projects.

Following are summaries of each major project. Hopefully, writing about each of them on the blog throughout the summer will provide (1) a modicum of accountability to the anonymous readers who may be checking in with me and (2) an opportunity to play a bit with the ideas in a public forum. This latter provision stems from a perceived need to achieve a greater comfort level with playing with ideas “out in the open” instead of waiting to share them in a more final, polished format.

For each project, I provide the “framing question,” and some amplification.

  1. What form do knowledge claims take in design research? This project is a comparative analysis of three different kinds of research publications: design publications, natural science publications, and social science publications. Random samples of each type (e.g. papers from Design Studies, Nature, and the American Sociological Review) have been preliminarily analyzed in order to gain a deeper understanding of the practice of making knowledge claims in design research by comparing a sample of its claims against other kinds. This is part of a larger project aimed at explicating the practice of design research.
  2. How do design researchers cite Donald Schon? Schon is probably one of the most frequent citations in design research. When researchers cite Schon, how are they citing his work? Are they demonstrating their familiarity with the “canon,” or does the citation have broader implications for their research questions or analysis? This paper is a first step towards a broader project of understanding the practice of design research. This project is being done in collaboration with Laureline Chiapello, from the University of Montréal.
  3. How is theory used in ToCHI publications? A similar study currently in review for publication in Design Studies asks how design researchers use theory in their texts. We distinguish between theory use in a written texts and theory use in a research project. We make no claims about how theory functions in the broader research project–only about how it functions in the text. This paper asks the same question, but focuses on publications in a leading HCI journal: ToCHI. **Another paper follows this same pattern in analyzing papers published as CHI, the leading HCI conference.**
  4. What possible (presents and) futures does the design research community imagine for itself? One way to approach this question is to inquire as to how we characterize it now. When I say “design research,” I mean “research through design,” as opposed to the scientific, humanistic, and historical (among other) kinds of research being conducted in the field. I use discourse analytic techniques to explicate a lengthy discussion on the PHD-Design listserv in order to suss out the generative metaphors that writers use to (1) characterize the current state of design and (2) imply (particular) possible futures.
  5. How have design theories progressively deepened our understanding of designing? Building off of a current project that adopts methodological falsification as an analytical lens, this project departs from Lakatos’s notion of sophisticated falsification and asks how a sub-set of design theories (theories about design) have deepened our understanding of design. In Lakatos’s words, referring to scientific theories, “… a theory is ‘acceptable’ or ‘scientific’ only if it has corroborated excess empirical content over its predecessor (or rival), that is, only if it leads to the discovery of novel facts.” (Lakatos and Musgrave, 1970) Using CK theory, the FBS framework, Figural Complexity, and Bounded Rationality, we ask what “novel facts” each theory provides.
  6. Can there be scientific theories that do not scientize design? This question served as the foundation for a paper recently presented at the European Academy of Design conference in Paris. It uses Karl Popper’s criteria for scientific theories as an analytical framework to argue that there can be scientific theories of designing and, importantly, that these theories do not “scientize” the design process, which is to say that they do not provide us with an understanding of designing as a scientific activity. We are in the midst of overhauling the paper in order to submit it for publication in a leading engineering design journal.

Aside from preparing for my qualifying exams in the fall, this list describes how I’ll be spending my summer days (save for a week off in August before the semester begins). Now the only thing left to do is get started…

Knowledge Contributions | Discourse Analysis

design, design research, design theory, knowledge tools, philosophy of science, science, theory

Borrowing from a personal email exchange, in my final paper I’d like to look at “how publications in a journal like Nature talk about their knowledge contributions and how publications in a journal like Design Studies talk about their knowledge contributions.” The interest stems from a parallel research project I’m doing about theoretical vs schematic knowledge contributions.

One of the things driving this topic is the seemingly perpetual comparison (stacking up?) that seems to happen between science and scientific knowledge and other disciplines and other types of knowledge in the academy. I’m not really taking issue with the comparison. Of course, one way disciplines come to achieve “self” knowledge is by comparing themselves to others. It’s important for design to compare itself with science, art, sociology, psychology, medicine, etc. in order to map the intersections and divergences. Divergences. That’s a key for me right now.

Wooffitt writes, in the section on externalizing devices, “How can we use language to establish that something is ‘out there’, and that it has an independent objective existence?” And in the context of the chapter (and elsewhere) it’s quite apparent why we would want to do that. It enhances our credibility in casual conversations and in publishing academic research. I pulled the following from a document that serves both as part of my final project research for this class and as a part of an outside research project:

After skimming five [design research] articles, there seems to be a balance between the use of the language of “truth” and that of the language of “utility” with regard to the knowledge contribution. I looked both at abstracts and conclusion sections as primary sites in the texts for talking about the knowledge contribution(s)…

There are lots of instances of what could be construed as scientific buzz words [words that lend themselves to the idea of objective existence] like “experiment,” “results,” and “hypothesis.” These words are of course flexible and they are not necessarily connotative of [empirical] science. However, when these words are used primarily by authors in talking about their “observations” or “experiments” it seems fairly obvious that, in context, they are imbued with the values of empirical science. In other words, they are tools for affirming the objectivity of the claims being made.

There’s no 1:1 correlation between the language of truth as I describe it in the above paragraphs and the language of utility. And actually the question I’m grappling with right now is whether it’s possible to warrant a claim to utility without appealing to the language of truth. For instance, one of the texts I read contains the following quote:

“We have repeatedly observed that with descriptions of biological phenomena, designers tend to rely on non-analogous associations over analogical reasoning… Our work suggests that specific mapping instructions and problem-independent scenario mapping could enable designers to better focus on analogical reasoning.”

The “observations” in the first sentence seem to me to be an example of language use that establishes that “something is out there” and not “from the researchers themselves.” This is what I’m calling the language of truth. The second sentence is interesting, too, because it contains elements of the language of truth. The blue text imbues “our work,” which is not the same thing as “us,” with the authority of more objective truth to warrant the utility claim highlighted in green. Note that the scientific language in the first sentence reaffirms the “objective truth value” of the blue text. It creates some space between the scientists (who are biased and flawed) from their work, which, so long as it followed a scientific process, transcends their biases and flaws.

Currently, I’m curious to know how these claims can be made in different ways. And so this is where I think something like Jorgensen’s analytical tools can come in handy. Something like substitution, perhaps? How do the words “repeatedly observed” effect the claim? How does the claim “read” differently with words cut and/or swapped out?

With descriptions of biological phenomena, designers tend to rely on non-analogous associations over analogical reasoning… We suggest that specific mapping instructions and problem-independent scenario mapping could enable designers to better focus on analogical reasoning.”

What’s lost? What’s gained? There is certainly objectivity lost. But could it also be argued that “designerliness” is gained? Maybe. Maybe not. I don’t know yet. It is an agenda-laden question, too. I ask it as someone currently trying to understand what makes a designerly knowledge contribution different from a scientific one and, ultimately, probably, to argue for making more the former in design research publications in order to build design as a discipline.

European Academy of Design

design, design research, design theory, interaction design, karl popper, philosophy of science, science, theory

I’m really excited to have the opportunity to travel to Paris this coming April to present a paper written with Erik Stolterman at the 2015 EAD conference. Our paper is part of the design theory / design philosophy track, which you can read more about here. The paper abstract follows:

This paper asks, Can there be scientific theories of design that do not scientize design? And it answers in the affirmative. Not only can there be scientific theories of design that do not scientize design but also that a scientific lens can potentially reveal important aspects of the design process. We apply Karl Popper’s criteria for the scientific status of a theory to four seminal theories of the design process: Bounded Rationality, FBS Framework, Figural Complexity, and C-K Theory. We demonstrate that (1) some theories about design can be construed as scientific in Popper’s terms, and that (2) these theories do not “scientize” the design process.

I will post a version of the paper to my academia.edu page for those interested in reading it.