This chapter presents an empirical account of deploying and evaluating the novel accounting systems produced in the previous chapter, and the design requirements that arise as a result.
Chapter 4 outlined the ‘Accountability Work’ that is performed in a charity and its implications for design, and Chapter 5 presented the design process that made use of that material to build systems. This chapter picks up immediately following its predecessor by discussing how the systems seen in Chapter 5 were deployed and evaluated.
I first introduce the ways in which the systems were deployed and evaluated, accounting for problems in the deployments and how these were overcome to ensure that the systems could be evaluated properly. I also give a brief over of the organisations involved in the study to contextualise the findings; introducing some new participants who are not charity workers themselves but who operate in this space as stakeholders and partners. Their presence supports the evaluation of the tools from additional perspectives.
I then present the key findings of the evaluation, grouped by three main areas of focus: the barriers to system use in everyday work practice; the key work practice of ‘tagging’ items to account for them; and how external stakeholders use materials provided by charities to construct narratives for accountability. I then discuss the key take-aways from the evaluation by elaborating on how systems need to support the dialectic nature of ‘Accountable Objects’ as well as lessons for representing these in data structures, and how interfaces may be designed to support interactions with such data.
This study and evaluation covers a staged series of evaluations of the tools designed in Chapter 5 across mid 2017 to late 2018. This comprises of three sets of deployments with participant organisations Patchwork and GOPA as well as a separate short series of interviews with other stakeholders. This is visualised in Figure 6.1.
The organisations that participated in the deployments were Patchwork, Edbert’s House, and Gateshead Older People’s Assembly (GOPA). The latter two of these were introduced briefly in Chapter 05 as participants who also contributed to the design process alongside Patchwork. They were involved in evaluation as a natural continuation of their participation and to evaluate whether the systems which had primarily been design with Patchwork in mind could be deployed effectively in organisations with other working practices.
All three of these organisations participated in the early stages of deployment in June 2017 (Labelled unshepherded deployments in Figure 6.1), however after this initial set of evaluations Edbert’s House needed to drop out of the study due to constraints on worker time. The administrative staff member who was the most directly involved in the study was moving to another city and Edbert’s House felt that they couldn’t spare the additional labour to train up another member of staff to take part in the study atop their existing duties. This stage of evaluation was mostly “hands off” and involved regular check-ins with the organisations and informal interviews captured through field notes.
The early deployments resulted in some changes to the Qualitative Accounting (QA) standard, Accounting Scrapbook, and Rosemary Accounts applications which were covered in the previous chapter. These changes were then followed with a second series of deployments with Patchwork and GOPA across a longer period of time from October 2017 to April 2018. The methods used in this stage of evaluation were more involved in order to ensure that the systems were being used and overcome challenges to performing the research that I encountered in the previous stage. These are discussed in Chapter 03 as well as in the following sections in detail.
I then turned to seeking other stakeholders in charity transparency and accountability to show them the systems and and further evaluate them, in relation to their needs as organisations and stakeholders within the charity accounting process. Four organisations or participants were involved in this evaluation: Patchwork’s former accountants (which they’d explicitly left during the events in Chapter 04); their new, independent, accountant who signed off their accounts yearly; a local Community Foundation (small local funders); and a representative from the Big Lottery Fund (now the National Lottery Community Fund). These took the form of semi-structured interviews and show-and-tell sessions with participants that were recorded and transcribed. Sadly the interview with the Big Lottery Fund was lost when the data was corrupted before it could be backed up and thus I do not draw upon it in the corpus of data presented in this chapter.
Finally, since I felt that the evaluation could benefit from further engagement I constructed a short final “challenge” to Patchwork and GOPA to stimulate system use. This lasted about a week and I interviewed each organisation about their experiences before holding “exit interviews” at the conclusion of the research with both Patchwork and GOPA to reflect on its performance overall and elicit final reflections.
I now turn to describing in greater detail the people involved at each stage of the evaluation, and the characteristics of each of these evaluation stages. I cover the rationale for each as well as the methods and data that resulted from them.
This evaluation encapsulated deployments with several organisations and contact with people within them in addition to my main collaboration at Patchwork. I briefly introduced Edbert’s House and Gateshead Older People Assembly (GOPA) in Chapter 5, however as I will be reporting on quotes and observations of individuals it’s important to be able to identify key participants in these organisations. Unlike Patchwork Edbert’s House and GOPA didn’t give permission for me to use their real names in reporting this research; so I have assigned them pseudonyms which are reported in Table 6.1.
|Pseudonym||Organisation||Role / Position|
Further to this, I will also account for individuals from the interviews with stakeholders. As noted the nature of the organisations involved in this stage were involved in the more legal forms of accountability and transparency such as the Tyne & Wear Community Foundation, the National Lottery Community Fund (then the ‘Big Lottery Fund’), Patchwork’s former accountants “Ellison’s” (operated under the Newcastle Centre for Voluntary Services, now “Connected Voice”), and their new accounts examiner1 who is independent. I have noted these participants in Table 6.2.
|Pseudonym||Organisation||Organisation Type||Relationship to Patchwork|
|Alan||Ellison’s||Accountancy firm specialising in charities||Ellison’s was Patchwork’s former accountants|
|Barbara||N/A||Independent||Patchwork’s current accounts examiner|
|Charlie||Tyne & Wear Community Foundation||Local Funding Organisation||Patchwork have received funding from this organisation before|
|Darren||Big Lottery Fund||Large National Funder||Patchwork have received funding from this organisation before|
The initial deployments of the systems occurred in June 2017 following the conclusion of the design period described in Chapter 5. These deployments spanned the three organisations (Patchwork, Edbert’s House, and GOPA) that had participated to varying degrees in the design process. The goal of this deployment was to see how each organisation would (or could) make use of the tools that we had built; with each organisation having different modes of working, different reporting needs, and different working patterns — this was a good test to determine whether the systems were indeed flexible enough to be appropriated in each settings.
After giving each organisation an initial introduction to the systems (including a tutorial and a Q&A session) I hoped to take a “hands off” approach to the deployment to avoid influencing how staff may use or not use the tools. I arranged with each organisation a rhythm for checking in on the progress of the deployments to reflect with them and offered to adjust systems in response to their needs if required. These began fairly ambitiously with weekly check-ins at Patchwork integrated into my regular volunteering and field visits, while at Edbert’s House and GOPA I arranged a fortnightly staggered check-in. This meant that on any given week I would check in with Patchwork and either Edbert’s House or GOPA. As the check-ins progressed it became clear that the staff were not engaging with the systems much and in some cases not at all. This did provide material on my new partner organisations and reasons that the staff were unable to sit down and use the tools, however I was initially surprised at Patchwork’s lack of engagement with “Accounting Scrapbook”. Throughout my regular field visits I saw that this was largely due to Patchwork’s busy summer schedule and workload of the staff.
As we approached September I realised that three months of deployment had yielded little engagement but fair insight into the challenges of deploying these systems in on-the-ground environments. I decided to change tack and performed a quick reflective interview with each organisation to conclude this stage of evaluation and using this direct feedback implemented changes to the systems (namely to the
media functionality noted in Chapter 5). Following these changes and the initial deployments I tried to design the next stage of deployments to account for the non-use of systems I’d seen in the initial round. These expanded deployments were more structured in nature and involved more intervention, and had the goal of directly demonstrating the usefulness of the systems through directed use. At this point Edbert’s House needed to drop out of the study citing capacity issues.
One of the key insights from the prior stage of evaluation was that some organisations didn’t have an instinctive idea on when or how they were supposed to make use of the systems. This was mostly prevalent in GOPA where staff were not involved in design stages, however was also present among Patchwork staff (Sonia and Lynne). To address this I structured my deployment around the performance of Co-operative evaluation methods (P. C. Wright & Monk, 1991a) and outlined a series of tasks (Figure 6.2) to guide staff through use of the systems and elicit reflections. The tasks were designed to be completed independently by staff, on a weekly basis to account for scheduling issues. I would then discuss each task with each organisation at the end of the week. After a month of these tasks, however, it became clear that the organisations were still not using the systems on their own prerogative. To work around this I established a rhythm where I would visit each organisation and we would undertake these tasks as a more traditional think-aloud co-operative evaluation (P. C. Wright & Monk, 1991a, 1991b). At Patchwork I’d visit Lynne explicitly in addition to my regular field visits, while at GOPA I would visit on a monthly basis to work through several tasks at a time. Stretching the evaluation over this time period also meant it was subject to the interruptions of daily life and the evaluation period extended into April of 2018. During these deployments I collected field notes while the think-aloud exercises were audio recorded and later transcribed. Unfortunately the audio data of several of my sessions with Lynne were corrupted beyond retrieval before transcription and these were lost, however several field notes survived.
In the summer of 2018 I also set out to get a broader perspective on the systems that had been deployed. My efforts thus far had given me a good sense of how the systems could support the work practices of staff in organisations such as Patchwork and GOPA; but the other half of Transparency and Accountability requires someone to observe the subject (Oliver, 2004). Therefore I wanted to explore if the features and interfaces of data within Rosemary Accounts went some way in supporting this. To accomplish this I undertook a short show-and-tell session with four different potential stakeholders in a local charity’s ecosystem (described in Table 6.2): two accountants who would want to interrogate financial data to “sign off” accounts; and two representatives from charity funders (one local and one national). These participants were chosen because they represented the existing formal Transparency and Accountability pathways that Patchwork and GOPA would need to engage with. Chapter 4 shows how Patchwork saw themselves as directly accountable to their immediate community and throughout my fieldwork I witnessed many examples of this; for this reason I felt that it was more pressing to understand the needs of stakeholders and actors that held more formal or procedural stakes in charities2 as this was a missing piece of the puzzle. I wanted to keep the research as contextually-grounded as possible and got enthusiastic permission from Patchwork to chase up some of their funders and their accountants both past and present.3 The result was a spread of participant perspectives with diverse organisational work practices that had experience and direct stakes in Patchwork to ground their evaluation of the systems.
With these stakeholders I performed a total four separate semi-structured interviews; one per participant. We sadly couldn’t organise a group session due to a matter of logistics. During the interviews I began by asking questions to understand how each participant conceived of their work and what their role in the organisation was, as well as what they thought a charity’s responsibilities were to them as stakeholders. The interviews were conducted in a range of settings; mostly the offices of those organisations however Barbara visited Open Lab and I interviewed Darren in a café before he caught a train. In each case I interviewed a single person but during the interview with Charlie at the Community Foundation fetched his colleagues to contribute to the session. In all cases I audio recorded the interview and took notes, however the interview with Darren (Big Lottery Fund) was lost soon after it was performed due to an accident. This meant I was unable to draw upon this interview in the corpus of data in this chapter as my notes were not sufficient when alienated from the recording.
Finally, having been engaged more fruitfully with GOPA and Patchwork throughout 2018 I sought to trial a reduced form of the original deployment I had envisioned from 2017. This was in order to see whether their engagements with the systems had resulted in them seeing their value or if barriers to use were otherwise removed. After establishing the feasibility of this with the teams we commenced in July 2018. These deployments were presented as a task to capture a “Week in the life” of each organisation and were relatively successful; seeing engagement from staff at each organisations. I then followed each up with an exit interview which were both pleasant and informative. These deployments and exit interviews gave me a final chance to speak to staff about their experiences using the tools and their reflections of the research as a whole. It also provided copies of their data from the systems in the form of screenshots and database copies which were useful for me to explore how they’d interacted with the tool independently.
These accounts focus on key findings from the evaluations and deployments. I first consider the challenges and contradictions in adopting the system. Secondly I report participants experiences of tags; a key element of the design. Finally I focus explicitly on how the the involvement of participants in Table 6.2 highlighted needs to balance monitoring and reporting as well as providing material for them to construct narratives to others in a chain of Transparency and Accountability.
In each of these sections I present my own ethnographic observations of the participants as well as direct quotes taken from interviews or sessions.
It should be clear from the previous sections in this chapter that the deployment of the systems did not go particularly smoothly and required several attempts to see strong engagement. Although I had a role to play in this lack of engagement, there were also conditions in the settings that created this that are worth highlighting. This is because they should be addressed to inform future design of either tools similar to these or within setting such as these.
The purpose of conducting the work practice study in Chapter 4 was to then use these accounts of how the members of Patchwork organised their work to accomplish Accountability and Transparency within the setting. Despite the fact that I had been embedded within Patchwork for over a year, and had used these accounts to support the system designs (as reported inChapter 5), I found that there was still a tension in some areas of work practice where the system was intended to be used.
Rather mundanely, ease of use was a key factor in the low uptake of the system. This was mostly reported by workers at Patchwork who, despite being more involved in the system design, still reported that this was a barrier to system use. The ease of collecting data via the Accounting Scrapbook interface proved to be a barrier for many of the workers at Patchwork as the system struggled to communicate its internal state and support staff in recognising what content they had already contributed:
Andi: “Some things were easier, like putting up photos, but because it didn’t always show you what photo it was […] Then you weren’t quite sure, you know? Also, there were some things where you created things and then you weren’t sure what you’d created, do you know what I mean? So, things like putting an accounting spend, like say you’re going underneath it, ‘Oh, I really should put that into a such and such folder’, which you didn’t have. So, then you’d have to go back and create that.”
Andi here describes bugs in the Accounting Scrapbook application around photos not displaying properly which meant that she didn’t know which photographs she had. This had implications for her work-in-the-moment as she didn’t know which activities had been accounted for, in what ways, and she could not do the important work of curating the dataset to later provide an account.
Andi also critiques the workflow that was imposed by Accounting Scrapbook with regards to the collections (“Oh, I really should put that into a such and such folder, which you didn’t have. So, then you’d have to go back and create that”). This was also separately picked up on by Mick:
Mick: “I have forgotten which folders I did not have. Then I would be like, ‘Oh, right, I need to put this in the bike folder. Oh, there is not a bike folder,’ and I cannot add a folder at that point”
The application requires that a collection is created in order to then place items into it. This was an important step in the design as a collection is required to be “tagged” with metadata, to support generating rich contexts around an item. However, this presented a contradiction with how Patchwork account for their work internally as Andi and Mick each describe wanting to create new collections on-the-fly as new dimensions of their work become apparent to them in-the-moment. The application provided a barrier to this part of work practice around accounting for these dimensions and thus actively slowed the work practices that the system was originally designed to support.
The conceptualisation of their work also meant that Patchwork and GOPA experienced other usability issues concerning the language in some areas of the systems. This was also noted in Chapter 5, but is discussed in more depth here:
Andi: “The ‘events’ one was a bit confusing, I don’t think I understood what you were seeking in there. So, I suppose, it’s funny, if you look at a group activity… So, maybe you could put ‘activity’, rather than ‘event’, that would have had a different meaning to us. ‘Event’, to us is, like, I don’t know… Like, a fun day, something big, something that you’re planning towards, but then, everything we do has got planning towards. I suppose, because Patchwork is a really busy project …”
This forefronts something that was not captured during the design process; that Patchwork’s organisation of their work considers larger public-facing events such as a “fun day”4 differently to activities that are ongoing as a staple of their work. This language may come from their use of Facebook which has provision for public events in the platform. Importantly, Patchwork acknowledge that both “events” and “activities” take work to plan which means that they should be accounted for. This was also reported at GOPA:
Heather: “It’s like once a year […] The things like Tai Chi are more activities than events, I always think of events as a one off.”
Heather’s statement affirms Andi’s conceptualisation of the dichotomy of “Events” and “Activities”. The Tai Chi example mentioned by Heather is a regular activity performed at GOPA’s main offices, comparable in scope to Patchwork’s group activities with young people. This simple misalignment of language as used by the systems’ interface in contradiction with the mundane, and account-able, way that settings members conceptualise their work was evident elsewhere in the system as well:
I managed to catch up with Lynne and Mick today re Rosemary, which I think Lynne was glad of as she’s been sitting on some questions apparently. We discussed the costing screen and how she didn’t realise it was related to ‘funding source’ despite the design of the screen forefronting the grants and the spends. I’d checked my interpretation of costing work w/ Mick earlier when designing the feature and he seemed OK with it at-a-glance, but clearly Lynne has a different way of doing it. There were also some misc things around auto-completion for tags and not realising she should tag financial stuff as ‘[she] knew others were tagging photos’; and she was stressed out at having to maybe use numbered budget codes.
The original interface for doing ‘costing’ was designed after Mick had demonstrated this during my initial fieldwork (Chapter 4). While it was based off of Mick’s demonstration this was poorly translated across to Lynne’s later engagement with the spreadsheet. Not being able to access Lynne had resulted in the design of this feature without a fully grounded understanding of how she made sense of her work. Further to this she highlighted usability issues with tags not auto-completing; meaning she had to retain a working knowledge of the organisations tags in her head when using the system. Importantly while Rosemary provided the flexibility to use numbered budget codes or not; Lynne was worried that Rosemary was imposing a normative practice of budget codes and was hesitant to use the system. There was also confusion as to whether she should be tagging the financial entries such as income/expenditure. Tagging things in this way was not part of her regular work practice and while other members of staff were engaged with social media, Lynne was not. This created a conflict in her expectations of what was required of her by the system and forfeited her agency to appropriate and use the system in a way she wanted and organise her work practice around it.
This presents an internal contradiction in the system; the original purpose of the tags was to allow Patchwork (and other organisations) to appropriate the tagging mechanism to flexibly organise and curate their data but when that flexibility is achieved through an interface that is abstracted from existing work practice it actually hinders engagement. Use of new systems generally requires introductions and training but this flexibility was, in-part, an attempt to undermine this need by providing a set of interfaces which could be appropriated by a worker with their own work practice. It remains an open question as to what would occur if the system was less flexible and mandated the tagging process in Rosemary; imposing new work practice but possibly leading to less dissonance as to how the system was designed to be used.
Other issues which had an effect on practice were around the collection and sharing of data which would be used for reporting. The staff at Patchwork did this as part of everyday work practice and Account Scrapbook was designed to support this by providing interfaces to facilitate sharing collections of items; in this case with the Rosemary Accounts system. This was most apparent when staff considered the collection of photos which, as noted throughout this thesis, is an foundational example of this practice. Sonia discussed with me the drawbacks of an earlier version of Accounting Scrapbook which only supported attaching a single photo:
Sonia: “If that is going to be one-by-one then, no, I will not do it […] But, if you could quite a few together… I remember when WhatsApp only used to allow you to do 10, that was really frustrating. Because sometimes the signal would be running low and then that crashes it. But if it is a good few together then I do not see why not.”
Sonia emphasised the importance of being able to transmit a large number of photos to Andi here, something she achieves by sending them via a messaging application, and that Accounting Scrapbook’s provision for only capturing a single photo at a time was a distinct barrier to her. This, alongside similar feedback from Andi, lead me to re-designing this section of the interface. I detail in Chapter 5 how adjustments were made to allow for multiple images to address this limitation and the Accounting Scrapbook app was re-deployed. However Mick reported that there still existed a tension in capturing the multi-faceted nature of Patchwork’s work using the interface:
Mick: “So, for example, I ended up repeating myself, as in, that was the annoying thing… again, again and again. For example, you could be having a spend, having an activity, taking a photograph, and they are all the same thing.”
Here Mick was explaining how he didn’t like having to add separate items to describe what he conceived of as facets of a particular tangible moment or thing to be captured. Accounting Scrapbook provided interfaces for collecting different types of information as discrete items (e.g. spends, activities, photographs) which could be linked together through tags; however, the underlying data structure would have allowed each of these to be added as dimensions onto a single item but this would have also made it more difficult to re-use and recombine specific pieces of data such as an individual quote or image. This presents a contradiction between two competing models of dimensionality with regards to modelling an accountable object. One the one hand there is a clear desire from the workers to capturing a richer, multi-faceted, example in-the-moment; fleshing this out in a single ‘item’ or context. On the other there is the previous design requirement to capture discrete items which can be recombined or assembled to provide context based on later reporting needs; which would lend itself to a greater flexibility in how staff organise and present their accountability through these more discrete and reusable examples.
I observed that one of the key barriers to adoption of the system was lack of provision to support their work practices around coordination and co-operative tasks. Dean was the first to express this in summary in an interview:
Dean: “Surely that’s got to be an easier way for everybody to use one tool.”.
Dean was referring to the fact that there was a separation between the two systems that he, Andi, and Sonia were using (Accounting Scrapbook) as opposed to the other system, Rosemary Accounts, that was being utilised by Lynne and Mick for the curation stage:
Dean: “The week after, it just so happened that I bought a couple of things and got receipts, so I got to play with a couple of the tools and then it made more sense. I logged into Lynne’s computer a couple of times, just to check that stuff was going through. It is detailed in quite a good report and stuff. I can certainly see a use for it.”
While Dean reports on the potential usefulness of the systems in the future, he explicitly says that he logged into Lynne’s computer to check whether the items were going through. This shows that he needed affirmation that his contributions were being added to the team’s central repository of data and was abstracted from this by not having explicit access to the tool in order to engage with these objects in a larger context. Andi also reported similar feelings of abstraction alongside concerns that the act of sharing to Rosemary was far-enough removed from her existing practice so as to feel like replication:
Andi: […] at the minute, it does feel like replication. I go on a website and put photographs up, but that’s not into the ether, the purpose of that is to share it with the groups, families, young people, community. So, there’s a wider purpose to that than just my own benefit of just having photos. The same with, like, quotes and things, it would be a useful place to store quotes, because that’s one thing that we always like having, I’m the one who remembers them, but that’s because I write them down somewhere. I’m sure other people hear stuff and whether that’s a better way of collecting quotes, that would be good.
Andi and Dean report alienation from the end result of the collection stage where they’ve contributed items towards repository of data but are removed from seeing what happens with their contributions; similar to how factory workers on an assembly line are alienated from the end-product. Dean needed to log into Rosemary Accounts, an interface originally designed for Lynne, while Andi describes her contributions as going “into the ether”. Importantly Andi also contrasts this with her existing practice of sharing photos for a specific purposes of sharing directly with people she feels Patchwork are accountable to. This presents a contradiction to how Patchwork reported photographs being used earlier (ie in Chapter 4) where they’re explicitly used as multi-purpose, reusable, objects to provide accountability.
The production of the annual report, a key event in Patchwork, relies heavily on both: discrete contributions towards a shared set of reusable quotes and photographs; and a collaborative process to curate these into the report itself. Despite being at Patchwork for some time I’d sadly not been around when a report was physically produced, so I asked Dean during an interview how the curation of work was done:
Dean: That’s a team effort, again. We’ll have an away-day, where we write a report, choose photos, then we send it up to the printer."
Here Dean gives a glimpse into how the work of creating the report is organised; it’s conceptualised as a discrete activity in the Patchwork calendar with its own practices (having an “away-day”) and photos and quotes are chosen collaboratively as a team. This presented a contradiction to how Dean, Sonia, and Andi had described their alienation from the process of mundane collection that was supported by Accounting Scrapbook, however. The everyday work practices around collection and curation is made clearer when Andi says “I’m the one who remembers [the quotes]”; indicating that this responsibility is devolved to her by the group.
This devolution of responsibility for curation of particular things (e.g. spends, photos) thus forms an important part of the work practices surrounding accountability at Patchwork that the systems didn’t adequately support. While Accounting Scrapbook allowed the sharing of collections including photographs to Rosemary Accounts this did not match how Patchwork wanted to organise this work:
Sonia: “But then what is Lynne going to do with the photos? […] If I have [sent photos to Lynne] then I am still going to have to send photos to Andi because of her… Do you know what I mean?”
Dean: " […] in terms of what, the photos themselves, Andi does the photos on an evening, late evening […] Right, so she, sort of, collates everything; she’s got the database, which is easier. However, I think that’s going to be coming my way soon; I’m getting a laptop, and evening workloads are coming. So, to be honest we probably should share the burden of putting photos and stuff on, etc."
Both Sonia and Dean describe the key, devolved, role that Andi has in curation of particular items used for presenting accounts and how, by abstracting this, the systems were not facilitating this work practice. Rosemary Accounts was designed and presented as being more supportive of Mick and Lynne’s reporting needs; but didn’t account for the more peer-to-peer management performed through devolution such as we see with Andi’s curation of photos (this is also reported earlier when Sonia notes sending photos to Andi via a messaging app). Sonia picks up on this when she challenges what the use of sending photos “to Lynne” via the system is when it is a job of work to send these photos to Andi for processing.
Importantly, Dean recognises that the work of curating the photos is work that could, or perhaps should, be shared at Patchwork. He does this both through use of the term “burden” but also provides foresight into the fact he will be expected to participate in this work before long: "“I think that’s going to be coming my way soon; I’m getting a laptop, and evening workloads are coming.. Andi’s earlier example also hints at the transferrability of this work when discussing quotes: ”I’m sure other people hear stuff and whether that’s a better way of collecting quotes, that would be good.". This shows that while the interfaces provided in this iteration of the systems didn’t support the work practices adequately; the desire to coordinate and collaborate in curation and collection is still work that it is desirable to support in the setting. I also saw that this was not exclusive to Patchwork’s work practice but also shared by the staff at GOPA, as this conversation between myself and two staff demonstrates:
Heather: I think the pictures. If we all had that app on our phone, every time we took a picture, say if we nipped in, and took a picture of the tea dance, then it would all be in one central place with the dates on. Then you just go, ‘I want the pictures between April and March,’ and they would all be in the same place and Laura’s not going through…
Chris: And you would be choosing from a pre-existing list of hashtags rather than unlimited.
This exchange shows that while Rosemary Accounts was structurally a repository of information and photographs, the abstraction of this from the daily work of the team alienated them from it and they couldn’t coordinate in curating the photos from which to derive their reporting or other forms of Transparency and Accountability. Chris’ statement that choosing from a “pre-existing list of hashtags rather than unlimited” gives another dimension to this coordination; that the language and categories used to describe their work is also shared and organised in the setting. That Accounting Scrapbook has no knowledge of tags that were being used across the team thus provided an additional barrier.
This need for coordination was also expressed when discussing and observing the use of tags throughout the system, which I now turn to describing in full.
One of the main characteristics of the design of the system was the support for ‘social media style’ tagging of individual items. The rationale behind this is described in detail in Chapter 5, but the intention was fundamentally to support a flexible way of generating indirect links between items which could then be used to support the configuration and contextual interactions outlined as requirements in Chapter 4. It follows, then, that the act of ‘tagging’ and the management of tags formed a substantive part of the evaluation.
I found at Patchwork there was an initial ambiguity reported over whose role it was to be tagging particular things as part of their engagement with the system. For example, Lynne was using Rosemary Accounts and uploading information to it unshepherded and, when I asked about her experiences, she questioned what relevance the tags had for her and what was expected of her:
Lynne: “Yes. When it says ‘tags’, is that something further down the line, when you have been talking about tagging it to photos or tagging it to…?”
Lynne explicitly asks if there’s a stage ‘further down the line’; essentially demonstrating that she does not know whether she should be part of the tagging process or not, as well as querying whether the tags were relevant for the financial data she was inputting to the system. The act of ‘tagging’ items was an entirely new concept for Lynne, who didn’t engage with Patchwork’s social media in a way similar to Andi or Dean. She also knew from our previous discussions that the systems were intending to support collaboration and asked about whether the tagging was “further down the line” and thus something that another member of the team would be concerning themselves with. Since she was also removed from the performance Patchwork’s direct delivery work, she also did not think about or account for spending in the same rich way that the other workers did. The system, or at least the interface in Rosemary Accounts, was therefore not set up to support her in understanding the purpose of tags or what was expected of her in inputting the financial information.
Despite the rest of the Patchwork staff being familiar with social media, this difficulty in understanding the role of tags was also present with other members of the team who were using Accounting Scrapbook. Andi discussed with me how she initially struggled with tagging using the app:
Andi: “I also didn’t know about tags, so at first, the tags, they were sentences and I didn’t realise you’re supposed to put them all as one thing. I, sort of, worked that out, because I was thinking, ‘Oh, this can’t be right.’ Then I thought, Oh, that’ll be what it is.’ So, they probably make more sense later on.”
Here there is a direct contradiction between: how the interface of Accounting Scrapbook both required Andi to add tags to an item to communicate its context, or the dimensions of the item’s status as an ‘accountable object’; and how the interface also failed to communicate the purpose or role of this. Andi produces a work-around where she adds an item and fills the mandatory tag input with sentences in order to add the item, until she realises how tags worked as individual words that added context to an item. This adds weight to Lynne’s highlighting of this problem that the purpose of the tags are not made clear through the interfaces of the system, but also shows how they were less appropriate-able than initially thought. Andi says that she thought “they probably make more sense later on” which would indicate that she is not appropriating the tags to convey her thoughts on how an item in the system could be used to account for various elements of her work. This is also conveyed by her initial use of the tags field to write complete sentences; effectively using it to add an additional long-form description of the item rather than thinking about it in terms of its accountable dimensions.
The accountable dimensions of an item or ‘accountable object’ was also a matter of discussion when participants were reflecting on their use of the system. Opening up questions of what constituted an appropriate tag, how granular a tag should be, and how many tags should even be allowed:
Working with Heather (GOPA) we were running through a task on Rosemary where she had to upload some images and tag them with things. The image was of a member of the GOPA community holding a lettuce that they’d grown in the centre’s garden area. I prompted Heather to think about tags and she tagged the entry with three or four tags including ‘lettuce’. Later when I was stewarding a reflection on the tags used; Laura, announced “Who put, ‘Lettuce’?”
The above vignette illustrates how Laura could not account for Heather’s tagging of an item with ‘lettuce’ despite being a member of the same setting, and a worker at the same organisation. This shows that, around the new work practice of tagging, there was still an open question as to how an item should be tagged and thus what its accountable dimensions were. Laura’s exclamation indicates that, to her, there is an organised and orderly way of deciding what constitutes an ‘accountable object’ to report. This must, ironically, be made account-able as work practice in order to determine what the elements of a discrete item constitute its nature as something worth recording or reporting. Clearly “lettuce” had no immediate bearing on GOPA’s funding and reporting requirements.
Laura went on to give examples of tags that she described as “Black and White” or unambiguous. Examples of these were “Art Group”, “Social Event”, and “Greggs Foundation”. She also then qualified her original contention with the tag “lettuce”:
Laura: “I can’t see a reason for the ‘Vegetables’ tag. Because we will never go, ‘Hey, blog every picture we have got of vegetables.’ I am a fan of less is more… In terms of tags, because – this is just my personal – it makes me really anxious. I like systems to be really tidy … I want there to be 15 tags. I don’t want there to be one picture with, ‘Vegetables’ tagged on it and nothing else. If there is a tag, I want it to be used frequently. Because there is too much room for error there. If I tagged that, ‘Vegetables’, but didn’t tag it, ‘Garden’. If I tagged it, ‘Lettuce’, but didn’t tag it, ‘Tesco’, because they paid for the thing, then we would never find that picture.”
Laura tells us that the language used to attribute the characteristics, position, and context of a discrete item is tied directly to how GOPA account for their work to others. There is a negotiation of ‘what is worth reporting’ as Laura cannot understand the motivation for deciding upon ‘vegetables’ as a dimension of their work as that is not something that anyone else is interested in. Laura also relays how this feeds into her work practice of going through previously collected records and searching for them; wanting the system to be tidy and worrying that “we would never find that picture” when it came to reporting on it. Underpinning this practice is the need for the tags to be of a shared language; Laura comments that she doesn’t want to encounter a picture with the tag “Vegetables” “… and nothing else”. This demonstrates her understanding of the collaborative potential of this type of practice and how it may go wrong if people are not on the same page:
Laura: “I think we should agree as a group what our pool of tags is.”
This shows that an unsupported part of the account-able work for doing the tagging and curation was the collaborative development of either shared codelist (similar to budget codes), or of an agreement as to what the dimensions of their work are. This requirement to actually account for the tags, or dimensions of an ‘accountable object’, effectively gives a tag the dual-role of existing as something that can be used to describe something else; but also something that should be accounted for. The work of inputting the data, including tags, is distributed and there is coordination work involved; the dimensions of an ‘accountable object’ must be themselves account-able to everyone engaging with them. Laura’s idea for achieving this is to agree a shared vocabulary and to limit the potential set of tags to ones that are “used frequently”, meaningful and account-able to the team, and thus supportive of the work involved in curating items and reporting on their work.
Laura demonstrates how making the work practice of tagging account-able by restricting choice actually frees her to use tags flexibly which would be paradoxical if it did not remove the ‘room for error’ for her when accounting for her work. Laura went on to say how she would organise tags in the system, and around an activity:
Laura: “No. I want, ‘Tea dance’, ‘Community garden’, ‘Social groups’, ‘Volunteers’, ‘Board’. What else do we need? ‘Staying steady’. I would like a tag for each class or activity that we offer […] A tag for each funder. A tag for the three main points, and then a tag for each of the five ways to well-being. […] If Greggs came along and said, ‘We want to know about the community garden.’: ‘Greggs’, ‘Community garden’, ‘17-’18’. That’s it. No, ‘Lettuce’. No, ‘Vegetables’”
She lists several dimensions including the name of the activity itself and some key characteristics; the larger project (‘Staying steady’) from which the activity derives which would tying together a project as a context with the discrete activities that make it up. Laura then states that the funder needs to be a dimension of the item and that a tag for each funder would be the way to achieve this. The example she gives of the funder (Greggs) asking for reporting on the community garden demonstrates how this would work in practice; listing the key tags and dimensions that she would need to retrieve the materials for reporting and account for that work. Interestingly Laura also outlines how she would account for what she described as the “three main points” of an activity. In the context of the conversation Laura is referring to the aims of the project or activity and the ‘outcomes’; things that are notoriously hard to account for directly.
Laura also mentioned the ‘five ways to well-being’ which is GOPA’s framework or theory of change and it forms a central, driving, part of their work. This shows that there is an established account-able vocabulary that could be ‘ported’ across to the system and the work practice of tagging. This shows the dialectic, important, and mutually-defining relationship that the activities and the tags or dimensions have; an activity is defined by accounted for through its dimensions (here funders, outcomes, projects etc) which are in turn defined and made meaningful to GOPA by the activities they allow them to account for. This means that the dimensions which are used to account for activities, spends, and other ‘accountable objects’ form something discrete ‘accountable objects’ in-and-of-themselves. This is to say that they in turn must be account-able, accounted for, and understood by members of the setting in order to be used effectively for insight into their work and activity.
The organised, collaborative, and account-able way of establishing these dimensions of their work as ‘accountable objects’ ahead of time also needs to accommodate the practice of recognising and recording dimensions of the work as-it-occurs to staff during the tagging process. The latter of this potentially contradictory pair was noted earlier by Mick when discussing how the systems didn’t support this for him, but Dean elaborates here:
Dean: “But I like the tagging. But it just gets the workers thinking about what they’re actually- you know, what occurred in that sessions, and what you’re recording some of the soft stuff as well, so the soft- you know, you’re having a one-to-one conversation about sexual health, or confidential stuff, or… You know, you’re just tagging stuff, or linking them to quotes, which you can link to particularly the annual report later on in the year. I think that would be- that is how I envisage usefulness of it.”
Dean describes how some of more intangible elements of Patchwork’s work (“the soft stuff”) comes about through the reflective process that is prompted by tagging and trying to discern what the dimensions of his work are. This poses a challenge to Laura’s outlining of a more rigid and account-able system. Dean demonstrates how he can conceive of tagging as a worthwhile piece of work practice itself (“you’re just tagging stuff”) that’s used for multiple purposes. Firstly, to tie together otherwise disparate pieces of Patchwork’s activity, and secondly to provide language for the deeper value and outcomes of their work that could potentially be made account-able to others.
The practice of recording emergent or newly-apparent dimensions of work on ‘accountable objects’ was also something I discussed with GOPA during the investigation. Laura discussed with me how this is not contradictory to her stated need for a well-defined framework and synthesises a potential way forward that is rooted in work practice:
Laura: “Yes. As long as we have got room to add new things when they come in, but as long as it is understood that adding new things has to fall in line with the same protocol or whatever that we have already got in place, I can’t see [why not]”
Laura here explicitly outlines the need to establish a protocol for developing new dimensions or tags as ‘accountable objects’ within the setting. The existing vocabulary is used as a basis to decide upon the validity or membership of a new tag as something that can be accounted for and in turn be used to account for their work. This poses an interesting question when it comes to deciding the vocabulary that is useful to funders; earlier in this section Laura establishes that she would appropriate tags to describe the funder of an item and that this funder would be interested in a particular aspect of their work (the Community Garden). This means that there must be a way that this vocabulary could be established as a matter of practice between the funder and the organisation. This means that the account-able nature of these dimensions and of their work within the organisations also becomes account-able to those to whom they have reporting obligations. I got insight into how this occurred during my exit interview with GOPA where I asked how the language used by Laura to describe her work was established:
Laura: “Yes. That is the language that we speak for the remainder of that funding period. We know, from day one of getting that money what it is that we are going to have to report on in a year, two years, three years. We can front-load that when set up the tags for that project, for that funder.”
Chris: “There is a significant lump of your funding application which is about how you are going to measure it, what you are going to measure, how you will demonstrate this, how do you know that this has worked? Far more than was previously, it was just a case of, ‘What do you want to do?’ ‘Oh canny.’”
Laura confirms that the language and terminology used for the reporting is established through interactions with the funders and that these dimensions are going for form ‘accountable objects’ that are applied to their work going forward. Laura then explicitly says how she would interact with the systems in this situation; by front-loading tags and thereby ensuring that these dimensions of their work are established for use within the organisation. Chris also adds to this that large part of the funding application process is establishing “how you are going to measure [your activity]” and account for the outcomes to them; meaning this collaborative process involves some configuration of the ‘accountable objects’ that must then be reflected in the dimensions that GOPA use to account for their work.
With the funders and other stakeholders potentially having such an impact on the dimensions of an ‘accountable object’ that are used to make an organisation Transparent and Accountable; I will now be turning to focus on the needs of these stakeholders to understand what configuration and interactions the systems need to support.
One of the original purposes of the systems was to explore the requirements of interfaces to allow charities and organisations to present accounts of their work and their spending to others. As explained throughout this thesis; charities have obligations to stakeholders such as funders as well as presenting documentation on their accounts to the UK Charity Commission which are structurally enforced. Therefore I began to explore their needs and perceptions of the system during the evaluation in order to understand whether it supported these interactions.
This part of the evaluation revealed that central to the needs of funders was the ability to understand the work and financial practice of a charity in narrative terms. This was particularly true of the Tyne and Wear Community Foundation, as Charlie revealed how they sat as a link in a chain of accountability; with the funders themselves being accountable to others such as donors or larger funders. Part of their work to account for the funds that they were responsible for was to produce an ‘Impact Report’, which relies on the funder being able to use materials provided by grantees to feed into this larger account:
Charlie “Now the impact report has got two purposes really, from our point of view. Its purpose is to ensure that the donors know what’s happening with their money. So they get stories, they get some of the data, you’ve helped so many people, you’ve addressed these issues but they also get the words of the people that actually got the money. They get to see some of them and that kind of thing.”
This shows these accounts in the ‘impact report’ are narrative in nature and focus on the stories, a key feature is the careful framing of specific issues that are important to the donors which is accomplished with “data” being used as part as part of the narrative rather than standing on its own without context; contributing its relevance as a dimension of, and evidence for, the ‘accountable object’ which the donor is actually interested in. Further to this, the importance and relevance of quotes as a means of establishing this narrative is reinforced when Charlie relates how the donors will “get the words of the people that actually got the money”, demonstrating that charity workers such as Dean are indeed tuned in to the ability of this type of data to tell their story.
A charity’s ability to provide useful accounts of work to the funder was shown to have a direct effect on how the funder interacts with them in the future; and how likely they are to receive further funding from them. This ability to present narratives to the funder was shown to, in the funder’s eyes, belie their professional competency at accomplishing their work:
Charlie: “We identify good groups who feedback, invest in them early on, build them up. Then you’ve got organisations in each area which are more sustainable than you might otherwise have.”
Charlie makes it very clear that charities and other groups who are able to produce good reports and good narratives (which can then be reported upstream) get the cash. This results in a positive feedback loop whereby the organisations that are successful at telling a story will be the ones they “invest in” and built up. This is framed as being sustainable, indicating that the funders believe that the presentation of a good narrative may be tantamount to professional conduct generally and an overall indicator of quality as it relates to their work. Essentially; if the narrative of work is account-able to the funder and useful to them as an ‘accountable object’ in their own practice then this results in the charity being seen as trustworthy and reliable.
It became clear throughout the conversation that making the work account-able between a funder and a grantee was itself organised work as they negotiated language around objectives for a piece of funded work. This is supported through a systemic requirement to agree on these objectives before work can proceed:
Charlie: “We, at the moment, and it can always change again, we at the moment specify one, two or three objectives. So effectively while things work and it’s, like, have you met the objectives? Have you done 12 workshops in a 6 month period with 15 kids? That’s almost your tags.”
Charlie details how the funder and the grantee organise objectives as part of the grant which is how the organisation then reports on their performance. This shows again the dialectic relationship between a dimension of work and an activity; where the former is an attribute of the latter ‘accountable object’ which indicates its relevance to particular parties (in this case the funder) but can itself become an ‘accountable object’ when an overhead view must be taken. Charlie also hints at the type of “data” that is important to the funder; the number of workshops and attendees at them implies a form of monitoring which was not part of the systems designed for this research, and presents a contradiction to Mick’s testimony in Chapter 4 that nobody looks at monitoring information. Focusing on the systems I was presenting, Charlie explicitly and instantly saw the role of tags as things which could used to represent this (“That’s almost your tags”). This would mean that he shared Laura’s view that they could be used to model the objectives of a grant and provide a common vocabulary that is account-able to both parties. He went on to clarify how this may work for him:
Charlie “So I’m going to have a tag for this objective, a tag for that objective.’ Then you could say, okay, from your end would you be wanting to click on this objective and see, ‘Oh, here’s your 12 workshops.’”
Charlie further ties the status of objectives and tags together as ‘accountable objects’ by demonstrating how they’re used to explore the work of the organisations from the funder’s perspective. By selecting the dimension of work they’re interested in they can explore the individual items that make it up, but ultimately it is the larger view that is important and is forefronted; the details are readily available but the curation work, having been performed downstream by the charity, frames these in context for the funder to make use of.
The systems that support this work were also discussed; demonstrating how the charity must interact with a funder’s system to package and send the information to them. This allows the funder to collate all of the information and data on charity work to report to donors on the funds that they manage:
Charlie: “At the end of the year they get a reminder […] using the same system they used to apply, they feedback to us, did they achieve their three objectives? … We also asked them for information about their impact which can be anything from- it’s generally a written report photographs but it could be a video, it could be anything that they want to give us really. Then we use that data, we gather it up every year into an impact report.”
This reveals some mundane, but important, aspects of how this work is being accomplished now. Firstly, the reporting process is currently being supported with a digital system, and that the format of the report is flexible. Charlie relays how discrete items are useful contributions towards the narrative which describes how a charity accomplishes their objective; and as such draw their meaning from it. The funder will then perform some curative work of their own to organise these discrete ‘accountable objects’ into their own impact report which is used to give an indication on the impact of a particular funding stream.
An interesting element of this practice I gleaned insight on was how the ‘charity accounts’ (ie the financial records) were not seen as separate to the narrative aspect of a charity’s conduct but in fact contributed directly to it as an ‘accountable object’ in and of themselves. From the perspective of the accountants, the review and ‘signing off’ of charity accounts was actually not a pressure on the charity to become accountable in another way, but to contribute material towards their narrative to the funder:
Alan: “The charity commission don’t have time to check anything; the main purpose of charity accounts is really for funders […] One of the key things they’re looking for is the balance which is the main income and expenditure or profit and loss of a charity, […] You know, that the accounts were done correctly and the correct notes are there and stuff. Then obviously if [they are] and it’s a nicely-presented document, it’s telling the story”
Alan, an accountant at a firm, makes evident the role of the signed off accounts; they’re not used by the regulatory body (Charity Commission) to make decisions about financial management or inspection; that responsibility is deferred and entrusted to the examiner / auditor. The role of charity accounts as a discrete and tangible boundary object is to encapsulate a charity’s financial management into an ‘accountable object’ which may then be attributed as part of an charity’s work. This is then woven into the narrative that the organisation is trying to tell (or sell!) to funders and ultimately does not sit as a separate form of accountability. This is shown clearly when Alan states that the document is “telling the story” to those who will be looking at it in context with the charity’s other work.
Because the systems were designed to support the work of becoming Transparent and Accountable to the accounts examiners and provide materials for constructing this ‘accountable object’ I wanted to understand a little of the work of interacting with these materials. This was to unpick how the ‘accountable object’, which encapsulates the financial management of a charity was produced through its individual financial records. The fulcrum of this was the act of financial reconciliation. This is the matching work between various artefacts (bank statements, receipts, etc) which provides evidence that a charity’s expenditure was appropriate and reported correctly. This was spoken of by both Alan and Barbara as a key interaction that they needed to be supported by a system in order to accomplish their investigation or inspection work and thus ‘sign off’ on accounts:
Alan: “First of all, we can actually re-perform it, which obviously we don’t really want to do that. The second way we can do it, if it was on Sage, we can see that it’s been reconciled. If it’s on Excel we can do a thing called a bank reconciliation calculation. Where we take the opening bank balance, we take the closing bank balance, we see what has been through the bank and what has come out of the bank, and the two things should then balance off.”
Alan describes several methods that they can use to determine whether an item of expenditure or income has been appropriately reconciled. One of these is to perform reconciliation work themselves (Figure 4.4) but that is undesirable to him, and that if it’s presented in Sage he can see that an expenditure line is reconciled (recall that Mick’s utterance in Chapter 4 demonstrated some knowledge of this practice). He then details how a calculation may be performed on spreadsheet-based systems, where an automatic reconciliation has not already been performed by software – this involves some level of manual calculation but also leverages the existing reconciliations that are in place.
From this we see that the reconciliation of records is understood work where the goal is to effectively check whether the individual transactions in a charity’s financial records are genuine, and that the totals in the accounts are a reasonable match. Rosemary did not support this type of interaction as it was deemed out of scope for the system but, given the importance of this work to producing ‘signed off’ accounts this indicates that its absence would present a contradiction to achieving an effective narrative around financial management.
Alan describes how he can reconcile a charity’s accounts that are in a spreadsheet format using a calculation. This was interesting to me as Patchwork infamously contended with Alan’s organisation in Chapter 4 around the formatting of their spreadsheet where Lynne marks transactions as reconciled. How these reconciliations are recorded in the data is important as material to support this work, as Barbara reflected on her current engagements with Patchwork’s spreadsheets and modified it to suit her own practice:
Barbara: “[…] Whoever designed this spreadsheet is pretty clever but I don’t think Michael or Lynne have got the nous to actually- I mean I would be able to add whatever. I’ve used spreadsheets for years but I don’t think they- they do what they were trained to do with it but they haven’t got the skills to take it to the next level. When I get it, I basically modify it quite a bit at my end, put columns in and do whatever, sort it by different things.”
Barbara relates how the spreadsheet can be “set up” or designed by an individual to support the work of others, but that the ability to modify and adapt it remains a specialised skill in this context; and one that Patchwork are lacking in-house support for. She then gives a brief overview of her work in “pre-processing” the data to make it ready to engage with. In this way we see how Barbara used the inherent flexibility within the spreadsheet as a resource that may be manipulated to adapt and better suit her work practice to check over the accounts. This provides insight into how the data structures and the interfaces around them may be oriented to suit particular work practices while retaining their semantic meaning, but more importantly demonstrates how there is a “burden” where interacting with the data is concerned — and one that may be shifted between the different parties. From Alan’s earlier statement he is clearly most comfortable receiving data in a format that is account-able to him already via systems such as Sage; implying that the burden is on the producer of the data to present it in a particular format. On the other hand Barbara is comfortable assuming the burden by manipulating the system to suit her needs, and she can reconfigure the presentation of the data herself into a format that she is capable of working with and which ultimately shows she can reshape the information to make it account-able to her.
This shows how the format and presentation of the data forms an ‘accountable object’ in this case, in much the same way other dimensions of the data do. The data needs to be understandable and account-able to the parties who must engage with it, before the financial data’s position as an ‘accountable object’ in its own terms is accessible and apparent to them so that it may ultimately contribute towards the charity’s narrative as a trustworthy charity worthy of funding.
This section reflects on the findings from the deployment and distils them into lessons. In doing so, I first focus on the underlying mechanisms by which ‘Accountable Objects’ in a system operate in a dialectic fashion and how this must be supported at a system level. I then present how this may be accomplished by modelling the ‘Commitments’ and ‘Actions’ within data structures, and finally how the existing work practices around collaboration may be catered for by designing new interfaces and making use of existing ones in new ways.
Throughout my findings I have have referred to ‘Accountable Objects’ as discrete and palpable things which are accounted for in some way by the workers and stakeholders. Examples I give of these Accountable Objects are: activities performed by charities as part of their work; the data that represents that activity; the dimensions of that activity that give it meaning; the tags that represent that dimension; the objectives agreed between a funder and grantee; and other items such as a set of ‘signed off’ accounts. These Accountable Objects are the result of an organised, account-able (in the ethnomethodological sense of the word (Crabtree et al., 2012)), practice of collaboration between workers within a charity and also between a charity and grantee to establish the haecceities (Crabtree et al., 2012) of Accountable Objects; that is it begs the question of what makes them Accountable Objects in the first place?
I have shown how in this system tags are used to delineate the dimensions of something to be accounted for (e.g. an activity, or a spend), and that these allow Accountable Objects to be account-able to others through these dimensions. As has hopefully been demonstrated through this thesis, a single activity could embody a multitude of different dimensions which are meaningful to different audiences at different times and in this sense they represent a ‘boundary object’ to interact around (Leigh Star & Griesemer, 1989). Boundary objects are both plastic in the sense that they may be interpreted differently across different communities or in different contexts, but integral in that they are immutable and retain an identity across contexts so that they can be individually recognised. The ‘coherence’ and account-ability of a boundary object is, however, dependent on the performance of the ‘invisible work’ that it encapsulates (Leigh Star, 2010; Crabtree & Mortier, 2015). This in turn means that the objects do not simply carry meaning but also carry the organised interactional work that makes them account-able (Crabtree & Mortier, 2015). I demonstrate some of this work in action when I present how various members of the settings at Patchwork and GOPA engage with their work. Laura has a very firm stance on what constitutes a dimension of an Accountable Object that needs to be accounted for; definitely not ‘lettuce’. She also professes a desire to develop a shared, account-able codelist of terms and Charlie, as a funder, engages in work to establish concrete terminology and ‘objectives’ for work to be performed. In this sense the dimensions of an activity each exist as boundary objects in and of themselves which embody this work. That they must be accounted for on their own terms also gives to them the status of Accountable Object and demonstrates that one of the haecceities of an Accountable Object is that they exist in a dialectic relationship with other Accountable Objects.
The dialectic relationship is made up of a contradiction (ie clashing, meeting of opposites) between the prominence of one Accountable Object in relation to each other. This notion of contradiction (Mao, 1937) is central to understanding the relationship between discrete activities of work and their dimensions as Accountable Objects as shown above, as it explains how Accountable Objects are defined in relation to other Accountable Objects. This means that, dialectically, Accountable Objects such as an activity and its dimensions are mutually defining and conditional on each other for their existence as Accountable Objects (Mao, 1937). This is perhaps demonstrated best with an example from the physical world; there can be no definition of ‘night’ without its opposite (contradiction) of ‘day’, and also there can be no concept of ‘day’ without the presence of ‘night’ to contradict it. In this way they are mutually defining and dependent on each other for their existence. In a similar way we see that Accountable Objects are dependent on other Accountable Objects: it is impossible to know which activities are worth accounting for without knowledge of its dimensions, but those dimensions are not present without a definition of the work as an example of them in practice.
Further evidence of the dialectic nature of the Accountable Objects is seen in how the dimension of an activity rise to prominence as the Accountable Object which must be accounted for; that the two opposites (activities and dimensions) undergo the transformation of opposites and the change of position (Mao, 1937). The discrete activity as an Accountable Object lies in contradiction to its dimensions; the activity is the dominant or foremost Accountable Object but through accounting for it in an structured and orderly way the dimensions of it rise to prominence as Accountable Objects ahead of the activities, and the former recedes. This occurs when Laura and Charlie demonstrate that they need to establish the ‘shared vocabulary’ around tags and objectives. In practical terms the discrete activities then transform into being dimensions of the Accountable Object that is the “tag” or “objective”, but by the same token; keep their status as Accountable Objects themselves. This allows the continuation of the dialectic process where focus may shift back to the discrete item, which is in turn again given meaning by its dimensions.
The notion of ‘accountable artefacts’ is one that has been discussed within HCI before (Benford et al., 2016). Benford et al detail an object which has a presence in both physical and digital space, where the digital record grows over time and is built upon mappings from various physical properties to parts of the digital record (Benford et al., 2016) This differs somewhat from my definition of an ‘Accountable Object’ as it lacks the dialectic relationship to other objects and the two halves of the artefact (digital record and physical object) are arguably understandable on their own terms, but is nonetheless useful to demonstrate how things which exist in the real world and digital space may be interacted with as a single boundary object. Benford et al detail how the concerns of ‘accountable artefacts’ are that of provenance (origins) of the artefacts and the ability to associate them with rich stories — something unarguably related to the reporting of charity work and spending. Erete et al demonstrate that charities can make use of open data to establish narratives about their work with the support of digital systems. (Erete et al., 2016), and Elsden et al open the question of whether contextualisation should be occurring through data or the conversation around it (Elsden et al., 2016). However, the dialectic relationship between Accountable Objects could be built into systems by supporting the mutually-defining relationship between entries and their dimensions. This would have the benefit of not providing a contradiction with this nature and leveraging it for sense-making.
Building the dialectic relationships of Accountable Objects into systems and interfaces would also benefit the process of sense-making and understanding around charity data by transforming a mass of linked Accountable Objects into an account-able view of an organisation as a whole. This itself leverages another dialectic mechanism — the transformation of quantity into quality (Stalin, 1940). Just as water transforms into steam with the addition of one hundred individual, discrete, degrees; each additional, mutually defining, Accountable Object which both inherits meaning and gives meaning to its peers transforms a quantified ‘mass’ of records into a qualitative, broader, and more nuanced understanding of a charity’s work. In this way the contradiction posed by Elsden et al around whether data should contextualise or whether contextualisation should occur through conversation is synthesised — both occur as the contextualisation is accomplished through using Accountable Objects to give meaning to each other; while also giving rise to a single understanding of the larger narrative through the process of engaging with them.
What remains is how this dialectic nature will be modelled structurally to be made use of appropriately in digital systems, which can be achieved by capturing the main contradiction of commitments and actions.
One of the key parts of the dialectic of Accountable Objects we see in play is the relationship between discrete actions towards an established and agreed-upon aim, outcome, or important dimension of the activity. The Qualitative Accounting Data Standard which underpinned the systems and the Accountable Objects within them took as its base metaphor only discrete ‘items’, although these could be linked via ‘tags’. Since ‘tags’ represented account-able dimensions of work which were themselves Accountable Objects; this shows that they should be given more support within the underlying structures.
Importantly we see how these dimensions of activity as Accountable objects are negotiated between funders and grantees as ‘objectives’. Charlie explicitly draws a link between the systems tags and the objectives agreed upon between the funder and the charity. In addition to this GOPA have an existing theory of change which they’ve established as something that needs to be accounted for, and is account-able to them. This sits outside (or at least goes beyond) the scope of an ‘objective’ agreed upon between a funder and grantee. It could be said to encapsulate their commitment(s) to their community and beneficiaries as a discrete element of their work. Both of these go to show that modelling only discrete actions through the Qualitative Accounting data structures did not provide for capturing the commitments a charity makes and their actions towards them.
There is some evidence of parallel approaches being taken already within HCI and the Third Sector. Beltran et al discuss their platform for ‘conditional donations’ known as Codo (Beltran et al., 2015). Through the Codo platform it was possible to synthesise a ‘grammar’ of conditions that defined when funds were to be released or returned. A similar, yet distinct, approach is explored by Elsden et al with ‘Programmable donations’ (Elsden et al., 2019). Here Elsden at al engage with ‘escrows’, third-party services or entities which hold funds until particular conditions are met and the beneficiary is reimbursed. Elsden et al unpick the ‘conditions and triggers’ that individuals use to determine their contributions and highlight the importance of escrows as being immutable and able to “preserve present intentions into the future” (Elsden et al., 2019).
While the systems I designed and deployed here are not concerned with donations and meeting conditions, both Erete et al and Elsden et al contribute important insight into how particular forms of ‘pledges’ may drive charities to particular actions and that it is possible to represent commitments in data, in a way that is useful and account-able. This would then enable others to interact with them and understand them later. The direct implications for Qualitative Accounting data standard as presented in Chapter 5 are simply that it must be iterated upon to explicitly forefront the commitments made by an organisation, and that the discrete items of work (photos, spends, activities, etc) may then be linked and presented as actions towards those commitments. For example: a commitment by Patchwork to provide employable skills for young people could be captured, and then a variety of activities, media, and quotes could be linked to this as actions towards this.
To keep in with the dialectic nature of these ‘Commitments’ and ‘Actions’ as Accountable Objects there should also be embedded in the data models a way to represent their mutually defining nature. Further to this, ‘Commitments’ as discrete Accountable Objects may come from many different places; as a funder negotiates commitments in the form of ‘objectives’ with a grantee, but also GOPA have encapsulated their commitment to beneficiaries as a framework and important context from Chapter 4 is that accountability to the immediate community is perceived as a commitment by members of Patchwork. However re-framing the loose groupings of ‘tags’ present in Qualitative Accounting’s models into explicit ‘Commitments’ and ‘Actions’ helps to shift the burden of interpretation back from those making sense of the data and makes it easier to explore. This shifting of a burden is discussed in the practice of designing data standards (Open Data Services Co-operative, 2017) as sitting between ‘data owners’, ‘intermediaries’, and ‘users’ of data. Where data is unstandardised the burden falls on the ‘users’ to bulk of the interpretation work; and this is evidenced when Barbara engages in manually reconfiguring Patchwork’s spreadsheet to suit her need. A shift in Qualitative Accounting away from loosely joined ‘items’ towards a model of linked-but-flexible ‘Commitments’ and ‘Actions’ would create a bit more work for the charity to do in curating and organising their data in this way; but would make the data more meaningful and account-able to others and support their engagement with it.
In practical, data-structure, terms this would require the ability to identify discrete Accountable Objects and draw relations between them which belies the need for identifiers. Identifiers are crucial in information infrastructure as it allows the relationships between discrete items to be drawn (Lynch, 1998; Eriksson & \AAgerfalk, 2010). Further to this, drawing identifiers from shared codelists of identifiers (e.g. UK charity numbers) would allow the data to be combined and seen in context alongside related data sets such as 360Giving (Open Data Services, 2020; 360Giving, 2020a).
Finally, both Dean and Mick each understood the multilayered value of their work and the reflective process involved in accounting for it; where dimensions of their work became apparent only in the accounting process. This demonstrates that in any system modelling ‘Commitment’ and ‘Action’ at its core must also provide the ability to account for dimensions of work which are not yet established as negotiated commitments but may become such in the future. I turn now to describing these collaborative and reflective interactions that interfaces must support in order to make it possible for workers to collect and curate Accountable Objects more effectively.
One of goals of the system was to support workers in charities collecting and curating data on their work and spending to contribute towards a larger narrative; however as shown in the evaluations of the systems this was not accomplished sufficiently by the interfaces. If systems are to model commitments and actions in order to support the dialectics of Accountable Objects, then they must also support the collaborative and organised work practices that surround them.
One of the clearest barriers to collaboration in the systems was the positioning of Rosemary Accounts as an ‘administrative tool’ and the provision of interfaces oriented towards certain staff concerned with formal reporting such as Lynne, Mick, and Heather. The issues highlighted by the staff show clearly that this needed to be reframed as a data store to which everyone contributed and everyone had access. This is exemplified when Dean explicitly reports that he needed to know where his data was going, and how he wanted everyone to use one tool. Previous digital platforms such as App Movement by Garbett et al (Garbett et al., 2016) may provide lessons in understanding how systems may be designed with collaboration in mind. While it is not concerned with the curation of data, App Movement demonstrates that communities may be supported in collaborating over producing a shared digital artefact; in the case of Patchwork and GOPA this may be a shared dataset about their work where the communities are self-creating as being comprised of each setting’s membership. In the scope of data systems, Puussaar et al showcase Data:In Place (Puussaar et al., 2018) which supported interactions between citizens accessing existing data but also contributing to it in a way that that was apparent and meaningful to them.
This also brings the importance of the ‘negotiability’ of data into focus, as an interaction between people and data (Crabtree & Mortier, 2015). Mortier et al articulate ‘negotiability’ of data as a challenge for Human-Data Interaction (HDI), and this is further illustrated by Bowyer et al in the context of ‘Family Civic Data’ (Bowyer et al., 2018). In this context, it is not enough to provide interfaces for interpreting data but to allow an interaction where data may be discussed and situated as a matter of conversation. While the information contained by the systems discussed here does not necessarily pertain to personal information in the traditional sense, it does collectively represent data about the staff at the charity and as such the lesson holds that they must be able to see it and negotiate with it to ensure that it is telling the correct narrative. Rosemary Accounts and Accounting Scrapbook, with their one-way interactions tailored for use in separate work practices failed to present at all ‘the data’ which had been collectively sourced from the staff and contributed towards the communal pile, and it was not available as a resource for them to use. Without Andi or Dean’s ability to engage with the data as they would’ve normally done (photos, quotes etc) they were prevented from appropriating the technology (Dix, 2007), and the lack of visibility over the collective dataset also represented a failure to adequately support a situated Horizontal Transparency (Heald, 2006) between members of staff, and also between the collective staff and the dataset.
This simple realigning of interfaces around collaboration, and shifting Rosemary Accounts from being oriented towards ‘administration’ towards being a communal data store may also address some of the collaborative issues around ‘tagging’. If the systems and interfaces supported a two-way communication between a datastore and a re-designed Accounting Scrapbook this would allow staff visibility not only over the collective dataset but also on the Accountable Objects that were the tags and dimensions being used to describe it. This would allow them to coordinate over this and support the ordered way that these dimensions of work are made account-able to members of staff. However, there may still be the need to provide more specialised interfaces to the system which are ‘configured’ for specific engagements (Marshall et al., 2018) (also Chapter 4). Lynne’s interactions with Rosemary Accounts was fraught due to lack of account-ability around tags as she wasn’t sure what her role in ‘tagging’ items was. They did not fit into her work practice and she was focused on porting the financial information from her spreadsheet. Outside of the charity, other actors performing the work of ‘signing off’ on accounts require particular interfaces aligned to their work practice as well. This is shown when Barbara and Alan both wish the data to be presented to them in a particular format and using their own tools; although it is notable Barbara is comfortable manipulating the data to suit her ends while Alan prefers it to be ‘pre-packaged’. In the context of designing systems, this type of interaction may be supported by drawing upon lessons from Unplatformed Design (Lambton-Howard et al., 2019, 2020) where Lambton-Howard et al show that existing platforms and services may provide design material for developing new interactions. It was my aim in the design of the systems in Chapter 5 to explicitly challenge existing monopolistic platforms and provide a set of inter-operable tools, however it is an undeniable fact that many workplaces and work practices make use of proprietary systems to accomplish tasks. The synthesis of these two opposing forces should be to focus on further developing ‘transparency infrastructure’ to support the transmission and manipulation of data using interfaces and systems that are already in use.
This does not necessarily sit in contradiction with other interface requirements I’ve outlined in this section. In this study I designed systems to support existing work practices around sharing of data around a workplace, but a key lesson from the deployment is that people are already performing some work with existing tools; such as spreadsheet systems. If there were a way for Lynne to engage with Rosemary Accounts , or a more datastore-oriented successor, then there is no reason that Lynne’s existing toolkit could not act as her interface to the system. As it stands the system could ingest spreadsheets, and it would be reasonable to develop a two-way system that allows Lynne to manipulate records using her tools. In a similar way I highlighted how the sharing of photos via Accounting Scrapbook was experienced by staff as a replication of their work; since they were already sharing these things via chat platforms to post on social media. While it can be postulated that this practice may recede if there was a collaborative way to do this via a shared datastore; it’s also possible that it could work the other way and simply pull information to a central place. That way staff could engage in existing practices without much intervention at all and using their existing tools as interfaces to the system.
The use of such existing tooling to contribute to a set of data also reflects existing open data practice, where publishers of grants data and procurement data may do so via spreadsheet formats, for it to be converted to other formats using infrastructure (Open Contracting Partnership, 2020; 360Giving, 2020b). Although since I am discussing active manipulation of the data via a variety of systems and interfaces with different actors both within and surrounding an organisation, this echoes the concerns of the Dataware model discussed by McAuley et al (McAuley et al., 2011), where the shared resource is held as an organisational-equivalent of the Dataware ‘personal data store’; including systemic provisions for devolved access and manipulation. This will need to be accounted for if future design work seeks to support the collaborative processes of collecting, curating, and working with data on charity work and spending.
This chapter has provided an account of deploying and evaluating the systems that I previously designed in Chapter 5, using the materials for design that were uncovered in Chapter 4. I introduced the participants and manner of evaluation before discussing the key findings from the deployment and how these provide further material for design. These were: to support the dialectics of ‘Accountable Objects’ in systems to leverage the way that discrete items in a system and their dimensions are mutually-defining; to model ‘Commitments’ and ‘Actions’ explicitly in data to support drawing these links and allow for the relationships between established objectives as ‘Accountable Objects’ and activities to be made explicit in the data; and to provide interfaces that afford workers and staff to collaborate on datasets together by making them aware of the data and supporting their existing digital tools as interfaces.
Chapter 5 also touched upon the performance of design work in these spaces and how it may be adjusted to accommodate the challenges of small charities with limited time and attention for the design of systems. As can be seen in section 6.2.2 the deployment and evaluation of these systems also presented similar challenges. These will be picked up and discussed in context within Chapter 7, as I now turn my attention to highlighting the contributions of this thesis as a whole.
360Giving (2020a) 360Giving Data Standard documentation.
360Giving (2020b) Spreadsheet format - 360Giving Data Standard documentation.
Beltran, J.F., Siddique, A., Abouzied, A. & Chen, J. (2015) 'Codo: Fundraising with conditional donations', in Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology. [Online]. 2015 ACM. pp. 213–222.
Benford, S., Hazzard, A., Chamberlain, A., Glover, K., Greenhalgh, C., Xu, L., Hoare, M. & Darzentas, D. (2016) 'Accountable artefacts: The case of the Carolan guitar', in Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. [Online]. 2016 pp. 1163–1175.
Bowyer, A., Montague, K., Wheater, S., McGovern, R., Lingam, R. & Balaam, M. (2018) 'Understanding the family perspective on the storage, sharing and handling of family civic data', in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. [Online]. 2018 pp. 1–13.
Crabtree, A. & Mortier, R. (2015) 'Human data interaction: Historical lessons from social studies and cscw', in ECSCW 2015: Proceedings of the 14th European Conference on Computer Supported Cooperative Work, 19-23 September 2015, Oslo, Norway. [Online]. 2015 Springer. p. 3.
Crabtree, A., Rouncefield, M. & Tolmie, P. (2012) Doing design ethnography. London; New York: Springer.
Dix, A. (2007) 'Designing for appropriation', in Proceedings of HCI 2007 The 21st British HCI Group Annual Conference University of Lancaster, UK 21. [Online]. 2007 pp. 1–4.
Elsden, C., Mellor, S., Olivier, P., Wheldon, P., Kirk, D. & Comber, R. (2016) 'ResViz: Politics and Design Issues in Visualizing Academic Metrics', in Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. CHI ’16. [Online]. 2016 New York, NY, USA: ACM. pp. 5015–5027.
Elsden, C., Trotter, L., Harding, M., Davies, N., Speed, C. & Vines, J. (2019) 'Programmable Donations: Exploring Escrow-based Conditional Giving', in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. [Online]. 2019 pp. 1–13.
Erete, S., Ryou, E., Smith, G., Fassett, K.M. & Duda, S. (2016) 'Storytelling with Data: Examining the Use of Data by Non-Profit Organizations', in Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing. CSCW ’16. [Online]. 2016 New York, NY, USA: ACM. pp. 1273–1283.
Eriksson, O. & \AAgerfalk, P.J. (2010) Rethinking the meaning of identifiers in information infrastructures. Journal of the Association for Information Systems. 11 (8), 1.
Garbett, A., Comber, R., Jenkins, E. & Olivier, P. (2016) 'App Movement: A Platform for Community Commissioning of Mobile Applications', in Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. CHI ’16. [Online]. May 2016 San Jose, California, USA: Association for Computing Machinery. pp. 26–37.
Heald, D. (2006) 'Varieties of transparency', in Christopher Hood & David Heald (eds.) Transparency: The Key to Better Governance? [Online]. Oxford, UK: Oxford University Press. pp. 25–43.
Lambton-Howard, D., Anderson, R., Montague, K., Garbett, A., Hazeldine, S., Alvarez, C., Sweeney, J.A., Olivier, P., Kharrufa, A. & Nappey, T. (2019) 'WhatFutures: Designing Large-Scale Engagements on WhatsApp', in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. CHI ’19. [Online]. May 2019 Glasgow, Scotland Uk: Association for Computing Machinery. pp. 1–14.
Lambton-Howard, D., Olivier, P., Vlachokyriakos, V., Celina, H. & Kharrufa, A. (2020) 'Unplatformed Design: A Model for Appropriating Social Media Technologies for Coordinated Participation', in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. [Online]. 2020 pp. 1–13.
Leigh Star, S. (2010) This is not a boundary object: Reflections on the origin of a concept. Science, Technology, & Human Values. 35 (5), 601–617.
Leigh Star, S. & Griesemer, J.R. (1989) Institutional ecology,Translations’ and boundary objects: Amateurs and professionals in Berkeley’s Museum of Vertebrate Zoology, 1907-39. Social studies of science. 19 (3), 387–420.
Lynch, C. (1998) Identifiers and their role in networked information applications. Bulletin of the American Society for Information Science and Technology. 24 (2), 17–20.
Mao, Z. (1937) On contradiction. Peking: Foreign Languages Press.
Marshall, M., Vines, J., Wright, P., Kirk, D.S., Lowe, T. & Wilson, R. (2018) 'Accountability Work: Examining the Values, Technologies and Work Practices That Facilitate Transparency in Charities', in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. CHI ’18. [Online]. 2018 New York, NY, USA: ACM. pp. 275:1–275:12.
McAuley, D., Mortier, R. & Goulding, J. (2011) 'The dataware manifesto', in Communication Systems and Networks (COMSNETS), 2011 Third International Conference on. [Online]. 2011 IEEE. pp. 1–6.
Oliver, R.W. (2004) What is transparency? New York: McGraw-Hill.
Open Contracting Partnership (2020) Serialization 1.1.5 documentation.
Open Data Services (2020) A beginners guide to identifiers: Linking beneficial ownership and open contracting data. Medium
Open Data Services Co-operative (2017) Standards Lab Handbook 0.1.
Puussaar, A., Johnson, I.G., Montague, K., James, P. & Wright, P. (2018) Making Open Data Work for Civic Advocacy. Proceedings of the ACM on Human-Computer Interaction. 2 (CSCW), 143:1–143:20.
Stalin, J. (1940) Dialectical and historical materialism. New York: International Publishers.
Wright, P.C. & Monk, A.F. (1991a) A cost-effective evaluation method for use by designers. International Journal of Man-Machine Studies. 35 (6), 891–912.
Wright, P.C. & Monk, A.F. (1991b) The use of think-aloud evaluation methods in design. ACM SIGCHI Bulletin. 23 (1), 55–57.
The term ‘examiner’ and ‘auditor’ are each defined in government documentation; examiners are often accountants but are not limited to this. I’m not sure what language Barbara, my informant, uses to describe herself but her relationship in Patchwork is that of examiner who ‘signs off’ on the accounts once they’ve been handed to her and she has examined them.↩
This is not to lessen the importance of these community or less institutionalised forms of Transparency and Accountability. As any ethnomethodological investigation could reveal; the social world of transparency between a service user and an organisation like GOPA or Patchwork is organised. However, these interactions do not constitute legal or contractual bindings and this research sought to lessen the administrative burden of Transparency and Accountability that the latter requires.↩
Although Mick laughed at the idea I would get anything useful from their former accountants.↩
See Chapter 4 for an example!↩