This chapter discusses the investigative and analytical traditions of this thesis and how they were applied during the various stages of research that captures. As this research considers workplace settings its primary focus it is to be expected that the framing of the thesis, the analytical heritage, and practical application of methods all draw from established realms that centre the performance of work (and the implications thereof).
First the chapter considers the thesis’ place within the tradition of Workplace Studies. Setting out the characteristics of a workplace study; it outlines how these are useful and appropriate for the thesis’ focus and outlines how the subject matter and setting of the research within Digital Civics and HCI make it a natural fit for this framing. After establishing this I then turn to outline the thesis’ Orientations to Analysis wherein the analytical methods I described and justified. Alongside outlining my chosen analytical tools I spend some time along with some background to illustrate how the marriage of these frameworks is both appropriate and complementary given the subject of my investigations. After the investigation has been grounded in its traditions the chapter turns to the pragmatic and details how the research was actually enacted. I first present an Overview and Timeline of the research and delineate how each phase of the study was enacted and contributed to the investigation.
Finally, I end this chapter by recounting a Description of Methods used to perform fieldwork, design my interventions, and analyse data collected. These are situated within the traditions I outline in previous sections.
This section situates the thesis within the tradition of a workplace study. Workplace Studies are a form of research that concern themselves with how workplace activities are organised and, in particular, the roles in which technologies play in assisting workers organising mundane activities and collaborative tasks (Heath et al., 2000).
Workplace studies came to be established within HCI and closely related fields such as Computer Supported Co-operative Work (CSCW) and Information Systems (IS) as a result of these areas of study moving beyond the scope of examining a single user utilising a single interface to consider group settings, as well as important revelations by Suchman that more consideration needed to be paid to the nature of interactions as situated in settings and context (Suchman, 1987, 1995). Kuutti and Bannon discuss a “turn to practice” within the scope of HCI and related fields, that encompasses this shift in focus from laboratory studies to studying and designing for real-world practices as they occur (Kuutti & Bannon, 2014). Schmidt writes of the critical role workplace studies have in dismantling supposedly common-sense notions of cooperative work by uncovering how it is routinely accomplished (Schmidt, 2000).
Within CSCW and HCI Workplace Studies have been used to inform systems design at various stages of design research. Plowman et al note three phases of design where workplace studies have been used: Initial Research and Implications; Design and Change Phase; and the Evaluation and Development Phase (Plowman et al., 1995). The research encapsulated in this thesis covers all three of these phases situating it firmly within the tradition of Workplace Studies through its methods and narrative.
Workplace Studies have close ties with ethnography, particularly the analytical framing and studies of work practice (Nilsson, 2005). The value of findings from studying work places is core to one of the goals of this thesis to understand how financial practices and Transparency obligations of a charity manifest in daily workplace practices so that they may inform design. Heath et al demonstrate how a workplace study may be used to derive implications for systems that support work practice with their analysis of dealers in a London securities house (Heath et al., 1994). Through analysis of the systematic way that dealers organise and co-produce their trading they elicit how systems may be better designed to support this work such as “Pen-based” systems to capture gestures that make the actions of others obvious and visible. In addition to providing the implications for specific workplaces Heath et al demonstrate the generalisability of their findings; highlighting the broader moves towards seamlessness between individual and collaborative activities through systems that enable cooperative editing. This thesis works within this tradition both practically and theoretically with the first phase of research embodying the Initial Research and Implications phase described by Plowman. In Chapter 3 I describe the performance of an ethnographic study of work practice which elicited initial findings and implications for the design of systems produced and evaluated later in the research. These “Implications for Design” (Dourish, 2006) address the research’s request for empirical data on what is done (and how) to produce Transparency in a setting.
The Design and Change phase of a workplace study is concerned with the production of prototypes or change in working practices (Plowman et al., 1995), and this is detailed further in Chapter 5.
Lastly, the Evaluation and Deployment phase of a workplace study is presented in Chapter 6 of the thesis. In this phase I take the prototype systems that were implemented in the previous phase (and the design of which was informed by the first phase) and evaluate them over an extended period. This sits comfortably in the tradition of previous cases such as Sanderson’s case study of the implementation of a video conferencing system (Sanderson, 1992), Bowers’ work within the UK Central Government (Bowers, 1994), and Rogers’ evaluation of a multi-user system in a London workplace through field visits (Rogers, 1994).
This section describes how I went about collecting data for this thesis and describes my analytical orientations to the field site(s) and the data that I collected.
The goal of this research was to understand the ways in which Transparency and Accountability are “done” in charitable organisations with specific regard to the role which digital technologies may play in facilitating this, and to provide these understandings as insights that may be used for design workers and researchers in this space. Following the tradition of a Workplace Study, I needed to understand the setting, then design for it or intervene some way, and then evaluate my designs (Plowman et al., 1995). In order to understand the setting I employed fieldwork techniques that were oriented towards producing materials for design (Randall et al., 2007; Crabtree et al., 2012).
Fieldwork for Design (Randall et al., 2007) situates fieldwork and ethnographic materials as being valuable to design. It can be used to establish a corpus of data from which, through analysis, one can derive materials that are used to inform design (Randall et al., 2007, p.147). This has bearing not only on the immediate research presented in this thesis, but adds to a broader corpus of data about particular settings to support researchers and designers understanding the similarities and differences in them. This material can then be used to inform requirements when designing through the analytical process of structuring the data collected through fieldwork (Randall et al., 2007, p.148).
There are many different ways one can orient the performance of fieldwork when collecting data and performing analysis (Randall et al., 2007; Crabtree et al., 2012, 2009). My particular orientations to the analysis of a field setting were heavily inspired by Crabtree et al.’s Doing Design Ethnography (Crabtree et al., 2012) which provided a practical set of instructions for orienting oneself to the fieldsite and collecting fieldwork data. This manifested chiefly as an orientation to the ‘work practices’ of the settings so that they may be described and used as material for design (Crabtree et al., 2012). Here the term ‘work practices’ is used to refer to the methodical ways that tasks and work are accomplished at the fieldsite [(Button, 2012; Randall et al., 2007; Crabtree et al., 2012). This type of study has a history within HCI and CSCW venues, with notable examples being Suchman’s seminal Plans and Situated Actions (Suchman, 1987) and Harper’s Inside the IMF (Harper, 2009).
One of the key ways that I attuned myself to the work practices of the setting was to develop a Vulgar Competence in these matters (Crabtree et al., 2012). Crabtree et al. write that developing a Vulgar Competence involves attending to the practical actions and reasoning that members of a setting are employing and identifying the methodical ways that they accomplished their work practice so that it is seen by the researcher in the same way as the other members of the setting (Crabtree et al., 2012). As described in Section 3.5 and later in Chapter 4 this was accomplished by myself through participation and immersion in the everyday activities of my main research participants during the fieldwork stage of the research.
Developing a Vulgar Competence of a particular setting’s work was beneficial to the research in several ways. First, by developing Vulgar Competence in a setting I as a researcher-cum-designer developed an intimate understanding of its work which may be used to develop accounts and inform design (Crabtree et al., 2012); since the goal of this research was to actually design in this space as well as provide long-term design requirements to inform future work this is not only appropriate but imperative. Secondly, my fieldwork was oriented to is predicated on members’ engagement in work practices that makes their actions account-able to others in a setting by making them observable and understandable to all who care to look (Crabtree et al., 2012; Button et al., 2015). In summary, developing a Vulgar Comptence in the setting was beneficial to the research because it allowed me to understand the mundane acts of producing Transparency and Accountability in a charity; since the topic of this research concerns Transparency and Accountability, albeit in the grander sense of the terms, this makes it appropriate as an orientation to analysis as this orientation mirrors the subject of my attention as a designer and a researcher.
It is important as a researcher conducting this kind of qualitative, fieldwork-driven, research to acknowledge myself and my own positioning as to my orientations towards the data and the analysis (Anderson, 1991; Sultana, 2007; Delamont, 2009).
Academically and professionally I sit within a technological tradition. My undergraduate degree was in Computing Science and I entered the Digital Civics programme immediately following its completion. HCI is a multi/inter/trans-disciplinary space, and while I have attempted to shed my techno-solutionist tendancies; it must be acknowledged that HCI and CSCW as spaces are geared towards producing materials for designing — and it is largely for designing new interactive computing systems. One need only look at the tradition of ‘Implications for Design’ prominent at our targeted venues to see the pervasiveness of this (Dourish, 2006). My reflexivity here then presents itself as entering the fieldsite and being oriented towards collecting materials that were useful for design. This should, hopefully, be self-evident in the nature of this thesis. I did not enter to perform a sociological or anthropological study and make no claims to this; however it may be that the materials presented in this thesis have value beyond that which is useful for design and that, in my HCI/design tradition, I’ve not understood the full dimension of. Working in charities is an inherently political space (Hansmann, 1980; Feis-Bryce, 2015), especially during the era of UK austerity. Due to my choice of research partners I was often confronted daily with the realities of the economic imbalance between: myself as an academic whose lab could pay for taxis to fieldsites, webservers, and design materials; and the charities who were often struggling for grant income.
This economic imbalace affected how I thought about the work of designing. This is most evident in Chapter 5 and Chapter 7 in reflections on how design may be conducted in these settings. It also oriented me somewhat to pay attention to particular realities of work practices in charities, as it lined up with the way which I view the world socially and politically. Although it does not bear relevance to the analytical claims I make in this thesis, it must be said that my reality a dedicated Marxist-Leninist and literal card-carrying member of the Communist Party of Britain1 sensitised me in particular ways to the plight of workers in the charities and the economic relationships they held with others who operate within the charity ecosystem. Again it must be stressed that this thesis only presents data that was collected directly at fieldsites and through interviews, and makes no claim of particular theoretical frameworks. The data was analysed according to the steps of organising the ethnogaphic records into a narrative as described by Crabtree et al (Crabtree et al., 2012), however I was sensitised to the political economy of work and how it is alternatively seen as being ‘productive’ or ‘unproductive’ depending on the lens it is viewed through.
This research began in late February 2016 when I reached out to the Patchwork Project (Patchwork) as potential participants (the reasoning for approaching this group in particular is given in Chapter 4). Following a brief meeting with them at a restaurant in Newcastle I began fieldwork the following week. The research then ended in late August 2018, culminating full-circle in a meal at the same restaurant (and on the same table) as it began. The work in-between these two dates consisted of several “phases” of research within the context of the workplace study that addressed the material needs of the project: an initial in-depth phase of fieldwork to understand how the interactional work of Transparency and Accountability was organised within a charity; a phase of iterative user-centred design to produce responses to the initial findings; early and expanded deployments which involved multiple organisations to test initial assumptions in my design response and to potentially bring further insight into the design space from organisations which operated differently to Patchwork; and finally some additional evaluation designed to bring in perspectives of other workers within the ecosystem such as accountants and funders as well as gather field data on some final iterative improvements made to the systems. It should also be acknowledged here that while the definition of ‘Charities’ I gave in Section 2.2.1 remains important to encompass the variety of Third Sector Organisations in the world, that this research was therefore performed with small front-line organisations and participants in the UK Charity Sector.
In practice, the “phases” of research followed on naturally from each other and are not as cleanly delineated as Figure 3.1 implies. As the research progressed partners were added and my understanding of each facet of the research grew, the research and I needed to accommodate this growth despite perhaps having ostensibly “passed that phase of the research” previously. This was most prominent in the relationship between the “design phase” of the research and the “evaluation phase”. For example sometimes the addition of raw exposure through more time spent at Patchwork during deployments meant I partook in a conversation or observed something that lead to a new understanding of their work practice. Or a conversation with another partner organisation (e.g. Older People’s Charity) lent an important critique of the work so far. This is perfectly normal for evaluation, but notably lead to further iteration on tools and systems to incorporate the new knowledge. To do otherwise seemed unnatural and, frankly, unethical given the collaborative nature of the setting and the nature of each partner’s work. By this I mean that my presence in these organisations meant that my actions had an effect on them and their ability to delivery front-line work; and I viewed the purpose of this research (and Digital Civics more generally) as having the ultimate purpose of benefiting their ability to operate. I felt that not iterating on designs to respond to new insight would thus constitute a breach of my integrity both as a collaborator who was taking up the time of the organisations and as a researcher who was in genuine attempt to progress knowledge and practice.
As such the discussion of each phase of research in turn here denotes the dominant focus of the research as it progressed; but it should be acknowledged that in any given phase the activities of a previous phase continued. Field notes were always taken with a focus on work practice and the manifestation of Transparency work, and small bouts of user-centred design were performed to fix bugs or add features when needed.
Phase 1: Initial Fieldwork ran from February 2016 to late summer of the same year at Patchwork. The purpose of this dedicated block of fieldwork was to orient myself to the work practices of the field-site. This initially involved looking explicitly at the different ways Transparency and Accountability manifested themselves as everyday practices through: characterising the performance of work related to Youth Work and management; and creating extra work from perceived or legal obligations for more formal forms of Transparency and Accountability. This phase of work started with weekly field visits to Patchwork, which overtime became either more frequent or less frequent as my schedule was intertwined with that of the organisation and different fieldwork methods were employed. Often I would visit several times a week in order to work on the accounts with the workers, volunteer, and take part in a team meeting. Sometimes, such as during the summer, I may not have visited during a given week. As noted, a variety of standard fieldwork methods were employed during this phase to develop praxeological accounts through techniques such as field notes, interviews, and participating in the work of the organisation itself. As the research progressed I became integrated into the daily life of the charity (and they became integrated into my daily life as a PhD student). The nature of my field notes thus shifted from “all-encompassing” to focused on specific phenomena, as I relied less on them to describe broadly how the workers at Patchwork organised their social world.
Phase 2: User-Centred Design (UCD) was the main characteristic of the next phase of research as focus evolved to investigating, designing, and implementing potential interventions. Fieldwork began to shift into UCD in late August 2016 with the completion of a design workshop focused around future technologies. In September the field visits took a short pause mandated by both Patchwork’s need to deliver the final part of their summer programme, and Open Lab’s need to deliver CHI papers. This afforded me the space to reflect on the findings of the fieldwork to date which resulted in the analysis presented in Chapter 3 of this thesis. The work’s rhythm resumed in October with design work consisting of early prototyping and regular design crits (Goldschmidt et al., 2010) worked into field visits and discussions and captured through field notes and requirements documents where appropriate. Design discussions maintained a vision of the larger system, but focused on different components in turn. October to December mostly focused on producing designs and implementations for a mobile application which became Accounting Scrapbook. After the winter break, development on what became Rosemary Accounts began and design was directed towards this away from Accounting Scrapbook. Throughout the whole process, discussions around the needs of the data and tools fed into the development of the Qualitative Accounting data standard, although it should be noted that Patchwork were much more interested in how the tools could produce and process information rather than standards development required for the design. This phase of research did not have a tangible drive by either myself or Patchwork to put our designs to use, simply develop them, whereas later stages of the research contained small pockets of iterative design and development while the tools were (supposedly) in use. In late April Patchwork and I were getting ready to begin phasing in use of the tools as additional partners were added to the research in the form of Community Project Gateshead and Older People’s Charity (GOPA) (both organisations have been pseudonymised for reporting, as they did not request de-anonymisation). Early discussions with these partners revealed new needs which were acted upon through some further design and implementation work and extended this phase of research until the end of May to account for their needs, improve the overall quality of the tools, and encourage their participation during deployments
Phase 3: Early Deployments lasted a total of four months from the beginning of June until the end of September. The deployments themselves were mostly unshepherded, in that I did not instruct the participants to use the tools in a particular manner other than providing technical support on their use when requested. At the outset of the early deployments all participants expressed their enthusiasm at using the tools they’d seen develop, so my intention was to try and understand how these tools could be appropriated by workers to support their existing work as it pertained to collecting and presenting information. I did “check in” on each of the participant organisations throughout the deployment, although the nature of these was different depending on my relationship to each one. “Checking in” on Patchwork was integrated into my visits there, where I could observe the use and non-use of the tools and casually chat or interview the workers as the deployment went on. As my relationship with GOPA and Community Project Gateshead was not as strongly developed; my regular visits had a distinctly more formal feeling. These were performed either bi-weekly or monthly depending on the schedules of myself and the workers there. Occasionally some lightweight design work was performed to fix a bug or tweak a feature to encourage or facilitate use. Despite this and for a variety of reasons this phase of the research did not see a lot of engagement from any participants regarding the deployed tools (even at Patchwork). This is analysed and reflected on in detail in the discussion in Chapter 06.
Phase 4: Expanded Deployments became necessary due to the poor uptake of the systems that had characterised the previous phase of the research. A year on from the original phase of design and development I renewed focus on making Accounting Scrapbook and Rosemary Accounts better integrate into the daily practices of my partners. Community Project Gateshead unfortunately withdrew from the research as the worker who had been my primary contact left the charity and, in lieu of the limited engagement so far, the organisation felt they couldn’t commit to maintaining our relationship. Patchwork and GOPA, however, agreed to commit to a more structured effort to use the technologies with a view of iteratively improving them as we continued the deployment. The deployments were characterised by being slow and requiring several different attempts to encourage engagement such as “weekly tasks” designed for participants to walk themselves through different features of the system which eventually became walkthrough sessions lead by myself. As such these structured deployments lasted a long time; from October 2017 until April 2018. It should be noted that the timescales for this phase of the research were extended due to the rise of mental health issues affecting my work during 2018. When both Patchwork and GOPA had completed the initial set of structured tasks I sought engagement from other actors within the sector such as funders and accountants. Several individuals were happy to give me their time and they assisted the evaluation of the tools by participating in interviews during the summer.
Phase 5: Additional Evaluation was performed in the final months of the research. This was, in my eyes, designed to compensate for what I perceived as a lack of proper engagement with the tools and to discuss with participants findings that had arisen during my conversations with funders and accountants. Through lessons learned from both earlier deployments, a short “challenge” was issued to participants to try and capture a “week in the life” of their organisation using the features of Accounting Scrapbook and Rosemary Accounts as much as possible. The engagement with this led, somewhat ironically, to the discovery of several technical issues in the systems which meant that the “week” turned into several weeks as progress was halted and began again several times. Following this dedicated use of the tools and reflection thereof through discussion, an exit interview was performed with each organisation to discuss the purpose and implementation of the research itself and the final state of our designs.
This section outlines the practical methods which I utilised to undertake this research and describes their appropriateness both within the context of the Workplace Study tradition that my research continues and the analytical approaches that it takes.
As described earlier I desired to develop a Vulgar Competence of the setting in order to understand how its members account for and produce the social order (Garfinkel, 1967; Crabtree et al., 2012). To accomplish this I performed extensive fieldwork within Patchwork, the rough shape and duration of which I outlined in the previous section. I wish to describe now the specific tools and techniques I used during the performance of this fieldwork.
The foundation of my fieldwork was extensive site visits at Patchwork, initially performed weekly but then changing frequency as I grew more involved with the organisation and our work rhythm became intertwined. Similarly, the types of activities I participated in developed in scope. In ethnographic terminology my participation in these activities may be characterised as Active Participation (Schwartz & Schwartz, 1955). During site visits I would: visit and work on the allotment alongside staff and the beneficiaries of the project (ie young people); assist staff in producing the budget and accounts; producing an annual report; help plan and deliver activities for group sessions unless inappropriate; report to trustees2; and many others too numerous to list specifically. As outlined by Crabtree et al in Doing Design Ethnography these activities served to develop my vulgar competence in the work of Patchwork but they also allowed me to gain acceptance in the setting (Crabtree et al., 2012). As I provided in input of labour through volunteering and assisting with preparatory work, and I demonstrated my commitment to seeing their work through their eyes, the workers at Patchwork began to see me as a member of the setting as well and this allowed me to have more frank and genuine discussions with them.
The extent of my participation at Patchwork allowed me to collect a variety of data and assemble a clear ethnographic record throughout. The tools and techniques I used here are standard to ethnographic enquiry, and should not need too much outlining. A lot of data was collected through the use of field notes and a fieldwork diary – which I populated with questions, observations, and diagrams to support my analysis (Crabtree et al., 2012, p.79) . An example of this is diagram of the Sequential Order of Work (ibid, p. 105) that I drafted in my notebook and then reproduced digitally for inclusion in Chapter 4. I was also able to perform individual or group interviews (ibid, p.80), which were useful to get an overview or to drill down into the work of something. In some cases these interviews were recorded whereas some of them were what I describe as “in-situ” ie they manifested as an in-the-moment questioning of a concept or some practical action being performed by someone there. Where not recorded these interviews were incorporated into my data corpus via my fieldwork diary. The fieldnotes and fieldwork diary, as well as interview transcripts, were used to create praxeological accounts of action and vignettes for presentation in this thesis and derived publications. These allowed for the conveyance of the local Accountability and situated action that were important both to the overall research and the design process that followed the initial fieldwork (ibid, pp. 122-130).
As the fieldwork process continued into the summer of 2016 discussions at Patchwork began to slowly and naturally turn to what design interventions may manifest as a result of the initial fieldwork I had performed. To support these conversations and create explicit room for them I also performed three “workshop” activities inspired by the concept of Futures Workshops (Jungk & Müllert, 1996). A Futures Workshop consists of three phases: critiquing the current state of the way things are done; a fantasy phase wherein participants come up with grand ideas to respond to problems; and finally an implementation phase where these fantasies are brought back towards the pragmatic in terms of what may be accomplished (ibid). As part of the research, these activities mark a transition from the purely investigative phase of fieldwork to one that was directly working with members of the setting to inform design, and therefore they may be conceptualised as an investigation into the social order of the setting and how technologies may support this work.
While Jungk describes a Futures workshop as a single workshop (ibid) the pragmatics of doing work with Patchwork necessitated that the three phases of the workshop were split across three months from June to September 2016. The reason for this is that Patchwork were delivering their Summer program during this time, which left no time for full day workshop. This change in pace allowed me to reflect on the conversation that was had during each workshop, as well as design materials and activities to be used as conversation pieces during the next one.
During the first workshop, the participants in Patchwork were guided in producing an artefact which mapped the flows of information and interactions with technologies. My questions and their answers served to check my understanding of their situated work that I had gathered from fieldwork to that point and also also to question things that were not clear to me yet. Hearing the workers at Patchwork reason out loud together about their work practice as a whole, rather than discrete portions of it, also helped illuminate the inter-connectivity of the setting’s interactional work.
In the second workshop, I produced a series of short Design Fictions (Hales, 2013) that were tailored to deliberately contrast or caricature the perceived values, behaviours, and norms of the setting and members. My intent here was to cause a reaction and make explicit the normal social order, and as such they may be considered as derived from Garfinkle’s “breaching experiments” – where the researcher disrupts the routine production of daily life in order to make this reasoning visible (Garfinkel, 1967). Crabtree writes of breaching experiments in technology design that they may be used to “provoke” (literally call forth) practice and that while they may be disruptive this is not necessarily the case (Crabtree, 2004). My Design Fictions were intended to be a little disruptive as I wished Patchwork to subsequently rally against the dystopian futures I detailed and instead proffer alternative designs that would be more closely aligned with their practice. In this sense they also touch on the notion of Provotypes; where a prototype is designed to provoke discussions around contemporary and desired future practice (Boer & Donovan, 2012). Provotypes also draw on Dialectics (outlined above) where the contradictions that give rise to practice are highlighted and then new practice may be considered; bridging investigation and design (ibid). While my samples of short Design Fiction may not be a true provotype (nothing was designed and deployed at this stage), they embodied this dialectical goal of assisting me in unpicking the contradictions in work practice and how this is made Transparent and Accountable; and ultimately lead to insights for design work. In the second half of this workshop Patchwork were asked to write their own design fiction and elaborate on what a theoretical pie-in-the-sky technology may look like although this didn’t occur on the day as planned (discussed further in Chapter 05).
The third and final workshop involved grounding the design insights from the first two workshops into what may achieved pragmatically. Because a month had passed since the previous workshop, a short design challenge was issued to the group to create Magic Machines (Andersen, 2013) that supported their organisation’s work in becoming Transparent and Accountable, and what the work practice was surrounding this. The results from these were then discussed as a group to unpick what desired work practice may be and how we may get closer to this given contemporary technologies and the scope of the research.
One of the goals of this research was to actually design and subsequently deploy technologies for use within the setting to later evaluate them and another was to understand how technologies may be designed in Third Sector Organisations. As noted in the earlier overview this phase is characterised by User-Centred Design (UCD) methods.
My use of UCD methods emerged in response to the pragmatics of designing in the research space. Daily life at Patchwork was (and remains to this day) very busy and they have a pressing need to deliver their services to beneficiaries and respond to their needs. Therefore, despite the best will in the world, Patchwork has limited capacity to sit down with me and co-design systems in the name of participation. Indeed it became clear when the design phase of my research was starting that they had little interest in designing systems together as they felt that it was my role to do the design and implementation work. I discuss this in greater detail in Chapter 5.
My response to this challenge was to involve Patchwork as much as I could during the design process. Following the completion of the three workshops outlined above the data from these fed into the larger corpus garnered during fieldwork and analysed to produce high-level implications for design, as well as indications for specific design elements to incorporate for a deployment here. Attention was paid specifically to the interactional work of the design ie the workflow of systems designed were derived from Patchwork’s model of work, and then these assumptions were tested during this design and later evaluation phases both inside and outside of Patchwork. Inside of Patchwork my weekly field visits continued ahead of my volunteering sessions and deeper integration into the workplace such as attending extra-curricular activities with the workers (e.g. Fell walking, allotments, etc) meant that I was served plenty of opportunities to check my designs and assumptions with the workers there. The most used technique here was the use of the Design Crit (Goldschmidt et al., 2010); weekly I would present Patchwork with ideas, sketches, wireframes, prototypes etc and we would discuss the design. Key notes from these were recorded into my notebook or, occasionally, a requirements document. I would make changes or otherwise progress the design before returning the next week for more of the same.
It is that design rhythm and workflow that leads me to call this phase of work User-Centred Design as opposed to Participatory Design or Co-Design. The Scandinavian co-operative design movement arose out of the concern of workers having technologies negatively influence their working practices (Schuler & Namioka, 1993) which was certainly an initial concern shared by myself and Patchwork. It could be argued that there are elements of Participatory Design present in the research. The way that I participated and integrated myself into Patchwork, as well as my fieldwork’s analytical focus on work practice and my genuine concern for the workers and organisation meant that gradually their concerns became my concerns. Therefore it could be said I was facilitating worker’s design of technologies or that participation was somehow “configured” (Vines et al., 2013) in the research as I part of my work there was to design technologies for which which I systematically and enthusiastically sought feedback and approval on. I reflect on this more critically in Chapter 05
Nevertheless, I do question how truly participatory the act of design may be in this context, or indeed needs to be given the nature of the organisation. My feelings on this are expanded on later in the thesis, but the core value embedded in this research is to support the Third Sector through my research and technical skills. Patchwork didn’t expect me to become involved in everything in the business and in fact explicitly noted areas where it would be inappropriate and similarly felt that it was inappropriate to apply themselves to an area that was clearly my wheelhouse. As an (initially) external technical expert I fit the bill as someone who was suited to the design and implementation of digital technologies and for them the act of participation was that I was there and contributing at all.
Late in the UCD phase of research I expanded the scope to include involvement with two other organisations – Community Project Gateshead and Older People’s Charity (GOPA). I was introduced to Community Project Gateshead as they were visiting Open Lab due to a collaboration they had with another researcher there, and they seemed interested in my work. Since I desired to deploy the designs that Patchwork and I had worked on I followed up with a meeting to which they invited the manager of GOPA. Initially this meeting was to introduce them to the technologies and garner their interest in participating in evaluating them, however the discussion raised new needs that required addressing. I followed up with several design crits with Community Project Gateshead and GOPA to account for this and improve the overall quality of the tools. These iterations were fed back to and checked with Patchwork as well, although the organisations never showed interest in meeting together.
After the design phase was complete the tools were deployed for evaluation in order to understand their appropriateness and to further illuminate the design space for future work. As described in an earlier section, the design phase involved instances of evaluation through the use of crit sessions with participants. After this, evaluation took place across three distinct phases where each had particular methods attached.
The first phase of evaluation began with early deployments that were, as noted, totally unshepherded and used observation and some small interviews to understand the worker’s interactions with the technology. After a short instructional session at each of the three participating organisations (Patchwork, Community Project Gateshead, and GOPA), the technologies were effectively “left” with my partners in that apps were installed on their phones and they knew of the existence of Rosemary accounts and had outlined to me a rough plan for their use. At Patchwork my regular visits across the week provided opportunities to witness use (and non-use) of the technology, and in addition to this I did small interviews in place of the design crits that we’d normally have. These lasted no longer than five minutes each and were not audio recorded – instead making their way to my fieldwork diary. I did not have similar levels of access to GOPA and Community Project Gateshead since I had not integrated myself in the same way as I did with Patchwork. In these cases I performed regular site visits at each organisation which varied from bi-weekly to monthly depending on our schedules. Here I did not get the chance to observe the technologies “in the wild”, as it were, but casually interviewed participants about how they were finding the technology. This also presented an opportunity for them to ask questions of me about the pragmatics of using the technology. Again these were not recorded in order to put participants at ease (they were often embarrassed about their non-use of the technology) but similarly integrated these findings through my fieldnotes.
The second phase of evaluation, which I’ve termed “Expanded Deployments”, was instigated after four months when it became apparent that there was poor uptake of the technology at all organisations – which I could only account for at Patchwork initially. After some wrangling of the remaining participants (Community Project Gateshead dropped out), I redesigned some elements of the technologies to make it a little smoother to integrate into daily life and we renewed a commitment to evaluate the technologies with a more structured evaluation. There was an understanding across the participants that the research here would be slower as I considered the daily pressures at the charities. The structure took the form of “weekly tasks” designed to walk participants through the use of various features of the system, with the intention of interviewing participants at monthly intervals after they’d completed three-to-four such tasks. After a few of these interviews revealed that, similar to previous attempts, there was similar lack of engagement I turned to the use of think-aloud co-operative evaluation methods (P. C. Wright & Monk, 1991a, 1991b) with audio-recorded in-situ interviews. These allowed me to sit with the participants and engage them while they used the system, as well as create explicit space for engaging with the systems within each Patchwork and GOPA.
During this phase of expanded deployments I also approached other participant groups that orbited around the sector as I desired their input on how useful the systems could be for their work. I interviewed several accountants and funders to gain first-impressions of the technologies and discuss future possibilities. These additional perspectives began to reveal what the “other side” of the interactions with the systems may look like given further consideration and development.
To tie off the evaluation a last “week in the life” deployment was performed. This used a similar unshepherded deployment method as used in the first phase, with the understanding that since workers were now familiar with the system and only had to commit to a week’s use that uptake would be more natural. There were a few technical issues which ended up restarting the deployment multiple times, however this was largely a success. During this week in the life of I didn’t perform any observations but instead interviewed individual participants about the system at the end of the evaluation. Finally, I performed two group interviews; one with each Patchwork and GOPA after the culmination of the research. We discussed the original aims of the research and reflected on its performance and challenges, as well as what future work in the space may look like. This was audio-recorded and transcribed.
The research presented in this thesis involved the study of human activity through the methods outlined above. This requires some reflection on research ethics within the context of this project. In this section I first present the formal ethical approval process I underwent at Newcastle University, and then explicate on issues such as measures taken at field sites to ensure the safety of my participants and informants as well my own safety.
This research was concerned with how organisations made themselves transparent and accountable to their stakeholders and as such the ethical approval process for this research was relatively straightforward, owing to the nature of the questions I was asking and who my desired informants were to be i.e. the workers and administrative staff within charities, and some of their stakeholders.
The ethical approval for this research was completed in three stages, at the beginning of each calendar year and intended to cover the research activities for that year. This roughly correllated to getting permission for: fieldwork activity and use of questionnaires with partner organisations; design workshops and design activity with partner organisations; and the evaluation of deployed designs with partner organisations. Since I was not studying children or animals in any way, or people who would be vulnerable, this did not require a particularly strenuous ethics approval process. Newcastle University provides an online form (Newcastle University, 2021) which I used for the ethicals approval and to flag high risk areas during the initial application.
The only high risk area that I was required to highlight was that the research involved the use of Human Participants in a Non-Clinical Setting through use of some of the research methods I was to employ (observations, focus groups, etc). This section expanded to provide a checklist of areas that Newcastle University considered high risk activities. None of these areas were applicable to myself, as I didn’t require access to vulnerable groups and would be working and studying the staff within the organisations I partnered with.
When I later began volunteering at Patchwork, a frontline youth-work charity which is discussed in detail within Chapter 4, they requested that I begin volunteering with them. This was for two reasons: first, it would give me first-hand experience of the daily work of the setting and increase the face-time I spent with the staff; and secondly, volunteering would ensure that I was contributing back to the organisation and not simply “using” Patchwork as an interesting case study for resarch. I readily agreed to this but was concerned that this would invalidate my original ethics approval since the volunteering put me in direct contact with young people aged 8 and above.
I discussed this with my supervisor the next day who assured me that as long as Patchwork had some safety procedures in place, and that I didn’t generate any research material from the young people I was involved with, that the original framing of my ethics approval wouldn’t change. This was because the staff at Patchwork (and Patchwork itself) remained my focus of study, and I wasn’t going to be interviewing any of the young people (neither were Patchwork acting as a gatekeeper). This decision was sense-checked with other PhD supervisors at the time who agreed that as long as Patchwork had safety measures in place, that it was fine to rely on these as efficient safeguards. I discuss these safeguards in greather depths in the next section.
This section explicates the ways in which the everyday performance of the research maintained the safety of both participants and researchers, and ensured the consent of all participants at each stage of the research. I first discuss broad practices that apply to all participants, research partners, and settings. Then, given the extended and deep nature of my involvement with Patchwork in the research (and given the nature of their work with young people), I focus on the safety measures taken within this context.
Throughout the entire research process there were several key measures that were undertaken in order to ensure the safety and informed consent of all participants. Participants were presented with consent forms and information sheets for each activity of the research. This included getting explicit consent for: fieldwork; design activity; group interviews (where members of the group hadn’t already signed a consent form); individual interviews for partners involved at later stages of the research; and evaluation of the designs produced. All participants were anonymised (via pseudonyms) for reporting in the research, although several members of The Patchwork Project requested to be de-anonymised for reporting in this thesis and subsequent research materials3.
As noted in the previous section; one of my key research partners, The Patchwork Project (Patchwork), requested that I begin volunteering with them in order to ensure that I was contributing back to the organisation as I studied them. This placed me in direct contact with young people and as such I was concerned about the ethics of this. As noted, it was discussed with my supervisory team and other academics with Open Lab at the time and found to be no ethical issue as long as I did not study the young people and conducted myself appropriately and under the assumption that Patchwork would put safety measures in place. Patchwork made me undertake a “DBS Check” via the Disclosure and Barring Service in the UK, which involves checking my criminal record to ensure that I was suitable for working with children (UK Government, 2021). This is something that Patchwork require of all staff, volunteers, and trustees as part of working within Patchwork. It is expected that these are refreshed regularly, with Patchwork requiring a minimum of 4 years between checks. As well as the DBS check I was required to undertake an accredited child safeguarding course and attain a certificate in Awareness of Child Abuse and Neglect (Virtual College, 2021). Again, this is something that Patchwork require of all staff, volunteers, and trustees and is generally renewed at the same time as the DBS check. Further to this, both myself and Patchwork undertook mundane good practice measures to ensure the safety of myself and the young people. I was always one of multiple staff or volunteers present when engaging with young people, and Patchwork took measures to ensure I did not participate in groups with particular demographics that may be perceived to be inappropriate for me. For example, as a large male researcher in my 20s it was simply understood as part of best practice that I would not be engaging in group activities with teenage women and girls.
In terms of risks to myself, there were no considerable risks that arose through the direct performance of my research activities. The majority of my field sites were within the organisational offices or buildings. On paper there were tangenital risks associated with the geographic location of my main research partners (The Patchwork Project) due to the socio-economic status of their local area and their service users. There were no incidents during my research there that made me concerned for my safety or the safety of others. Unfortunately, I did need to log one safety incident during the later stages of my PhD work. One day I was returning from a fieldwork site via taxi / hire car and was subject to unwanted sexual advances by the driver at the conclusion of my ride. I reported the incident around two weeks after it occurred and it was dealt with swiftly by the Open Lab staff and I took further measures to ensure that it did not repeat and that I was safe4.
This section contains my personal reflections on the ethics of performing this research. Further critical reflections are given in Chapter 7 regarding responsibilities of ‘Ethical Responsiveness’ (Durrant & Kirk, 2018) when discussing the contributions of this thesis. Some of these themes are also discussed in more detail in Chapter 5 when discussing the performance of the design work, particularly around the role of the fieldworker in the setting and group (Fuller, 1999).
It is safe to say that my involvement with Patchwork, my primary research partners has grown beyond that of a formal research arrangement. My relationship with the team there and the organisation itself continues to this day in both formal and social capacities, well beyond the conclusion of the research contained in this thesis. Since 2018, I have continued in my capacity as a Trustee of the organisation and regularly see the staff for social events. During my research, though, I increasingly conceived of myself as holding a dual-role as both a member of Patchwork and being that of a research student. This presented itself as dilemmas between feeling that I was “Going Native”5 (Fuller, 1999; Kanuha, 2000) at times when I was engaged in social activities that felt distinctly “extra-curricular” in nature.
Joseph and Donnelly report on engaging in drinking activities as part of engaging in the group, contrasting the notion that the academics in the field should only be conceived of as “sober data collectors” (Joseph & Donnelly, 2012), and note that when participation in these activities diminish their informants are often less happy to speak with them. Their conclusions are that researchers should feel open to discussing activities such as drinking “on the job” (Joseph & Donnelly, 2012) as a normal part of the research setting and provide practical advice for verifying findings that were initially made during social drinking sessions. I do not drink alcohol so I didn’t get drunk with my participants; however I fully immersed myself, to the degree that I could, in the social calendar of Patchwork. This often involved evenings, weekends, and even holidays with the staff as if I was one of their own. This resulted in many “adventures” arising from fieldwork which Liebling and Stanko note are the type of fare that traditionally remains private or exchanged behind closed doors rather than published in an ethnographic account (Liebling & Stanko, 2001). For my part; the focused nature of my inquiry meant that there was often a clear delineation between “research activity” and “social activity”, and so these sessions mostly served me with opportunities to ingratiate myself into the group and get to know my research partners so that I could better understand them later. There were, however, cases where this line was blurred such as when I was climbing mountains with Patchwork in southern Spain. This was ostensibly staff training and socialising for Patchwork staff; however, when walking across a ridge there, conversation naturally turned to the progression of my research and the design of the applications we were building. This echoes Goodwin’s realisation that they were “capitalizing on [their] insider status” (Goodwin et al., 2003, p.571) and I was worried that I was exploiting the trust of my partners in this relaxed setting. As my engagement with Patchwork continued, however, this worry was alleviated as they made it clear to me that they were keenly aware of my original purpose in the organisation but had participated to the point where I was conceived as one of them; and that research and design was part of my role there. This is elaborated on in Chapter 5.
There were several times, however, during my research that Patchwork reported to me that academics (particularly my research institution within Newcastle University) held a poor reputation in the charity sector in Newcastle. This was, reportedly, because they were perceived to: swoop in; promise resources and technologies that the organisation badly needed; not bother to engage the workers or service users properly enough to understand their issues; and leave once their papers or research had concluded6. This meant that there were several times where I felt the need to “defend” or otherwise gatekeep my research partners from other researchers and academics to prevent these practices occurring, and to ensure Patchwork did not think I would “open the door” to more researchers looking to exploit the setting. There were also other times when the economic divide between my research institution and my charity partners became all too apparent: such as when Patchwork asked about what financial resources I had available to me at the University, and whether it was conceivable that a grant be made to them for their participation in the research. These issues were raised and discussed in a workshop that myself and some colleagues ran at CHI 2018 entitled “Untold Stories” (Strohmayer et al., 2018) wherein we reflected critically on the academy’s relationship with charity partners. It was conversations such as these, as well as reflections on design practice, which lead me down the path of conceiving of ‘Vanguard Design’ as discussed in Chapter 5 and Chapter 7, to address the fact that researchers can often have the ability to capitalise on their insider status (Goodwin et al., 2003) and to try and tilt the balance back in favour of more equal participation and being answerable to my participants (Durrant & Kirk, 2018).
This chapter has discussed the framing, analytical inheritance, and methods used to perform the research in this thesis. I began with establishing the thesis as situated in the tradition of a Workplace Study in HCI and design, before illuminating the analytical traditions that result from this tradition as well as the research space.
With this instituted I then mark out the practical performance of the research. First I outline the timeline of research and set out phases inquiry that gave rise to the use of particular methods. Next I discuss in detail the methods used for fieldwork, design, and analysis of data before finally providing a summary of the research ethics involved in this research including critical reflections on some issues I encountered.
As I have now provided a scrutiny of the investigative traditions and practical applications of these in the research this thesis will now provide a detailed account of the first phase of this; a fieldwork case study of work practice.
Andersen, K. (2013) 'Making magic machines', in 10th European Academy of Design Conference. [Online]. 2013
Anderson, J.M. (1991) Reflexivity in fieldwork: Toward a ferninist epistemology. Image: The Journal of Nursing Scholarship. 23 (2), 115–118.
Boer, L. & Donovan, J. (2012) 'Provotypes for participatory innovation', in Proceedings of the designing interactive systems conference. [Online]. 2012 pp. 388–397.
Bowers, J. (1994) 'The work to make a network work: Studying CSCW in action', in Proceedings of the 1994 ACM conference on Computer supported cooperative work. [Online]. 1994 pp. 287–298.
Crabtree, A. (2004) 'Design in the absence of practice: Breaching experiments', in Proceedings of the 5th conference on Designing interactive systems: Processes, practices, methods, and techniques. [Online]. 2004 pp. 59–68.
Crabtree, A., Rodden, T., Tolmie, P. & Button, G. (2009) 'Ethnography considered harmful', in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. [Online]. 2009 ACM. pp. 879–888.
Crabtree, A., Rouncefield, M. & Tolmie, P. (2012) Doing design ethnography. London; New York: Springer.
Delamont, S. (2009) The only honest thing: Autoethnography, reflexivity and small crises in fieldwork. Ethnography and education. 4 (1), 51–63.
Dourish, P. (2006) 'Implications for design', in Proceedings of the SIGCHI conference on Human Factors in computing systems. [Online]. 2006 ACM. pp. 541–550.
Durrant, A. & Kirk, D. (2018) On Ethical Responsiveness: Being Answerable to Others as an HCI Researcher. Interacting with Computers. 30 (2), 99–115.
Feis-Bryce, A. (2015) Why the Third Sector Must Be Political. HuffPost UK
Fuller, D. (1999) Part of the action, or ‘going native’? Learning to cope with the ‘politics of integration’. Area. 31 (3), 221–227.
Garfinkel, H. (1967) Studies in ethnomethodology. Englewood Cliffs, N.J.: Prentice-Hall.
Goldschmidt, G., Hochman, H. & Dafni, I. (2010) The design studio ‘crit’: TeacherStudent communication. Ai Edam. 24 (3), 285–302.
Goodwin, D., Pope, C., Mort, M. & Smith, A. (2003) Ethics and ethnography: An experiential account. Qualitative Health Research. 13 (4), 567–577.
Hales, D. (2013) Design fictions an introduction and provisional taxonomy. Digital Creativity. 24 (1), 1–10.
Hansmann, H.B. (1980) The role of nonprofit enterprise. The Yale law journal. 89 (5), 835–901.
Harper, R. (2009) Inside the IMF. Routledge.
Heath, C., Jirotka, M., Luff, P. & Hindmarsh, J. (1994) Unpacking collaboration: The interactional organisation of trading in a city dealing room. Computer Supported Cooperative Work (CSCW). 3 (2), 147–165.
Heath, C., Knoblauch, H. & Luff, P. (2000) Technology and social interaction: The emergence of ‘workplace studies’. The British journal of sociology. 51 (2), 299–320.
Joseph, J. & Donnelly, M.K. (2012) Reflections on ethnography, ethics and inebriation. Leisure/Loisir. 36 (3-4), 357–372.
Jungk, R. & Müllert, N.R. (1996) Future workshops: How to create desirable futures. London: Institute for Social Inventions.
Kanuha, V.K. (2000) ‘Being’ Native versus ‘Going Native’: Conducting Social Work Research as an Insider. Social Work. 45 (5), 439–447.
Kuutti, K. & Bannon, L.J. (2014) 'The turn to practice in HCI: Towards a research agenda', in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. [Online]. 2014 pp. 3543–3552.
Liebling, A. & Stanko, B. (2001) Allegiance and ambivalence: Some dilemmas in researching disorder and violence. The British Journal of Criminology. 41 (3), 421–430.
Newcastle University (2021) University Ethics Form Version 3.
Nilsson, M. (2005) 'Workplace studies revisited', in 28th Information systems Research seminar In Scandinavia (IRIS 28). [Online]. 2005 Citeseer.
Plowman, L., Rogers, Y. & Ramage, M. (1995) 'What are workplace studies for?', in Proceedings of the Fourth European Conference on Computer-Supported Cooperative Work ECSCW’95. [Online]. 1995 Springer. pp. 309–324.
Randall, D., Harper, R. & Rouncefield, M. (2007) Fieldwork for design: Theory and practice. Springer Science & Business Media.
Rogers, Y. (1994) 'Exploring obstacles: Integrating CSCW in evolving organisations', in Proceedings of the 1994 ACM conference on Computer supported cooperative work. [Online]. 1994 pp. 67–77.
Sanderson, D. (1992) 'The CSCW Implementation Process: An Interpretative Model and Case Study of the Implementation of a Videoconference System', in Proceedings of the 1992 ACM Conference on Computer-supported Cooperative Work. CSCW ’92. [Online]. 1992 New York, NY, USA: ACM. pp. 370–377.
Schmidt, K. (2000) The critical role of workplace studies in CSCW. Workplace studies: Recovering work practice and informing system design. 141–149.
Schuler, D. & Namioka, A. (1993) Participatory design: Principles and practices. CRC Press.
Schwartz, M.S. & Schwartz, C.G. (1955) Problems in participant observation. American journal of sociology. 60 (4), 343–353.
Strohmayer, A., Marshall, M., Verma, N., Bopp, C., McNaney, R., Voida, A., Kirk, D.S. & Bidwell, N.J. (2018) 'Untold Stories: Working with Third Sector Organisations', in Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. CHI EA ’18. [Online]. April 2018 Montreal QC, Canada: Association for Computing Machinery. pp. 1–8.
Suchman, L. (1995) Making work visible. Communications of the ACM. 38 (9), 56–64.
Suchman, L.A. (1987) Plans and situated actions: The problem of human-machine communication. Cambridge university press.
Sultana, F. (2007) Reflexivity, positionality and participatory ethics: Negotiating fieldwork dilemmas in international research. ACME: An international journal for critical geographies. 6 (3), 374–385.
UK Government (2021) Basic DBS checks: Guidance. GOV.UK
Vines, J., Clarke, R., Wright, P., McCarthy, J. & Olivier, P. (2013) 'Configuring Participation: On How We Involve People in Design', in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI ’13. [Online]. 2013 New York, NY, USA: ACM. pp. 429–438.
Virtual College (2021) Statutory & Mandatory Training: Safeguarding Children.
Wright, P.C. & Monk, A.F. (1991a) A cost-effective evaluation method for use by designers. International Journal of Man-Machine Studies. 35 (6), 891–912.
Wright, P.C. & Monk, A.F. (1991b) The use of think-aloud evaluation methods in design. ACM SIGCHI Bulletin. 23 (1), 55–57.
Don’t mistake us (CPB) for the CPGB-ML, or the CPB-ML though! Splitters.↩
I later became, and have remained, a trustee myself.↩
See Section 4.2.1 for details.↩
Open Lab filed an incident report with the company, who blocked the driver responsible from seeing requests from my number as well as jobs on the Open Lab and Newcastle University accounts. After this incident I began cycling to the majority of field sites to minimise the use of taxis, and also ensured that I sat in the back of the car when I was forced to use one.↩
An ex-supervisor once outright accused me of going native, which could have been taken as a friendly warning about reflexivity if it wasn’t for his tone. When I mentioned this to Patchwork one of the members there pointed out that was quite a loaded statement which retained colonial attitudes about “lower cultures”. I tend to agree.↩
During an event, Patchwork staff once introduced me to some other charities in the area as “Matt, he’s from Open Lab but don’t worry he’s not like most of them. (sic)”↩