Lotus Development Corporation
One Rogers St
Cambridge MA 02142
whittaker@crd.lotus.com
+1 (617) 693 5003
Science, Technology, Society Program
Massachusetts Institute of Technology
Cambridge MA 02139
schwarz@mit.edu
+1 (617) 497 5827
Despite these compelling advantages, we found that in the company that we studied, a number of groups had abandoned on-line tools in favour of what seem to be outmoded technologies. Instead of on-line tools, we observed the use of large, publicly located, wallboards, with individual tasks and milestones written on paper slips which were then pinned to the boards (see Fig 1). These groups had access to, and expertise with, on-line project scheduling tools, email and workflow applications, but nevertheless preferred these "outmoded techniques" for managing software development.
Our empirical case study set out to address this paradox. We begin by examining the planning, co-ordination and communication processes involved in this type of complex multiperson project. We then contrast the effects of using electronic versus material co-ordination tools by comparing the software development process in two different groups. The first group relied on electronic scheduling tools and the other on the wallboard tool just described.
One group was developing a presentation package. For their development process they used a mixture of project bulletin boards, electronic mail, a scheduling tool (MS-ProjectTM) and face-to-face meetings. Electronic bulletin boards and face-to-face meetings were used to compile design specifications for product features. These specifications were broken into a list of development tasks. These tasks, along with developer estimates of how long they would take to implement, were then combined into the scheduling tool and a schedule generated. Ideally, each week, individual developers reported their progress to their project manager via email. The manager entered the data into the on-line schedule. This data was then compiled and distributed via email, or hardcopy, to both project members and external groups. The output each person received consisted of the list of tasks for the entire project, with each task allocated to an individual, with start and completion dates attached, along with progress on each task.
FIGURE 1. Image of part of the wallboard
The second group was writing software components to allow different software programs to interact together. This group had initially used an on-line scheduling tool, but had abandoned this, in favour of pen-and-paper procedures. During initial planning, project members entered both task descriptions on a large wallboard (Fig 1). The board depicted the names of all the people in the project vertically and a time-line horizontally. Against each person's name was a series of labelled pieces of card each representing different tasks: the length of the card indicated the expected duration of the task.
There were different coloured pieces of card to represent different task types, and the labels described the task. Each individual handwrote the labels for their own tasks. There were also various symbols to represent deadlines and review points. Unallocated tasks and an envelope containing notes about unforeseen tasks or potential problems were located in neutral areas of the board. People also pinned up photographs showing prior versions of the board, as well as cartoons, postcards and joke tasks.
The board was large: it was about 5ft high and 25ft long. It was located in a prominent public location in the hallway, so that team members walked past it 8-10 times a day, on their way to the printer, lunch and the bathrooms. It was also visible to external groups passing by the project area. Ideally, each week, group members reported their progress face-to-face, either with their project manager or in small project groups. They stood in front of the board, discussing the progress of old tasks already on the board for the last week, and planning the next week. They did this by moving around and cutting up the different pieces of card, to represent new tasks they had done or intended to undertake. In addition, some replanning meetings among subgroups of the project took place in front of the board.
The groups were similar in size, and both had participatory and open styles of management. One difference lay in their history: most of the presentation package group had worked together for over two years, whereas the components group had only been in existence for 6 months. Both groups predominantly communicated externally with other development groups, management and marketing in face-to-face meetings. The wallboard group sometimes met in front of the board for these meetings. There was no corporate requirement to produce an on-line schedule, although the presentation group distributed their electronic schedule to relevant external groups.
There are few established methodologies for case studies and the analysis of semi-structured questionnaires. We first analysed our interviews for descriptions of the software development process itself, its individual phases and the problems that arose in executing those phases. We collected every user comment about each phase, to see how the phase was affected by the medium. Thus for each phase, for both the wallboard and the electronic schedule, we collected the perceived benefits and drawbacks of the two scheduling methods. We present representative quotes from participants about these benefits and drawbacks, as well as participants' explanations about why they occurred. Where there was substantial disagreement or inconsistency between participants' opinions, then we present quotes representing alternative points of view.
"If this were a space mission and you underestimated by a factor of 2, you'd be off. But this is not. This is software design. Being off by a factor of 10 is bad, but not out of the realm of possibilities."
Our informants identified four major issues with scheduling their projects:
Coordination: There are multiple dependencies within and between projects, necessitating careful sequencing of tasks, and frequent communication about progress. This is to avoid people duplicating or redoing tasks, or being idle waiting for prerequisite tasks to be completed. Constructing a shared schedule is a technique for group co-ordination. It informs people inside and outside the group about what tasks should be completed by specific dates, allowing people to co-ordinate their actions accordingly. It therefore serves as a plan, about what the group and its members have committed to do. For a schedule to be an effective co-ordination tool, however, people must be able to understand it, and they must be committed to it. This requires careful work in generating, maintaining and modifying the schedule.
Initial planning: Planning realistic schedules for novel software is inherently difficult. The true extent of coding tasks may only emerge once coding has begun. Features may also need to be altered part way through development, in the light of marketing or UI feedback. Furthermore, given the competitiveness of the software market, there is political pressure for short (and sometimes unrealistic) deadlines. These factors mean that there is often a discrepancy between what was planned in the schedule for a given date and the actual state of the developed code, when development has begun. Repeated experiences with trying to execute unrealistic schedules can lead people to ignore the schedule, further reducing its utility as a co-ordination tool.
Updating: Given the uncertainty of the development process, discrepancies between schedule and reality are inevitable. The best that can be done is therefore to ensure that the schedule is kept up-to-date. This way at least people will be clear about what has been done and hence what remains to be completed in the original schedule. Without regular updates it may be unclear exactly where the project actually stands.
Replanning: In the course of development, new features are sometimes added to the initial design, often in response to competing products. Combined with the emerging discrepancies between schedule and reality, this means that replanning is a constant feature of the development process. It is therefore crucial as part of this process, to support learning: access to previous project data should enable people to be more accurate about the replanning process, by casting their new plans in the light of prior experience.
In what follows, we compare the utility of material versus electronic schedules for group co-ordination, based on their capacity for supporting initial planning, updating and replanning.
"Somehow, it seems more real when it's on the board, I have to say. Perhaps, when I see it on the screen, I think here's another attempt to automate this stuff. The board somehow seems more real".
In contrast, weekly paper printouts of schedules generated from software tools lacked reality:
"Maybe because you get a new printout like every week and it's a little bit different. I don't know, it's like a copy. It's not the master database, it's not like you're looking at the real thing".
And because the software schedule is not considered real, it fails in its function as a tool for communicating information and co-ordinating actions:
"managers really sort of use the electronic software tools to derive schedules for the groups that they manage, and that, whether you intend it or not, that information doesn't get shared as well as it could ... The information conveyance seems to be somewhat inconsistent. Even though you distribute it to everybody... people may file it. Or people may not share. It just doesn't seem to sort of get out there in any sort of consistent manner... It's dead on arrival. "
Why then, was the board perceived and used differently from electronic tools? We will argue that three physical characteristics of the board afford a different style of co-ordination usage. First the board is large and visual. This promotes a straightforward representation of the global aspects of the complex scheduling problem. Secondly, the board is public, which means it can serve as a site for communication and collaborative planning, as well as engendering commitment to the schedule. Finally, the board is material: the fact that people physically manipulated and moved task cards leads to a more reflective style of scheduling activity. We now discuss the impact of these factors in the context of the scheduling problems described earlier: namely the generation, updating and replanning of the schedule, as well as its use for communication and co-ordination.
"in the mechanical process of cutting a task bar, maybe you're thinking of, 'Jeez, did I cut it right to really represent the duration?' And that may cause you to think about, 'Did I really represent this duration realistically? Did I talk to all my people about it?' And then in the act of posting. . . So all of that takes a certain amount of time, and in your mind, you could be thinking ... about some questions that relate to this. So, it tends to draw you in -- at least me -- a little bit more than doing it electronically, where I can just sort of read from my notes what it is, type it in, recalculate, and consider myself done with it. This [board] forces me ... It gives me the opportunity to think about the schedule a little bit more, because the process of building it requires more time."
The electronic form appears to make the process less concrete, and less reflective. It seems to promote a more cursory style of planning:
"And they'll say, Look this over and change some things. So you're not even looking at the date because it's too hard to try to really think about, so you're just looking at the numbers. Cross it out, you know, make 5, make that a 7, make this 3 a 2, and then you hand it back. That's how the process goes."
Furthermore, the concrete character of the wallboard representation gives the planned tasks more reality and permanence:
"having a pile of cards that have to be placed, it becomes clear what is possible and what not: you can't cheat. The cards don't go away when you wait a couple of minutes."
The public nature of the wallboard is another crucial characteristic that distinguishes it from electronic tools. This has at least two effects. First, transactions performed with it, are potentially done in a social situation. Second, the displayed information is permanently publicly visible. Together, these encourage collaborative planning which promotes awareness of the plans of others: this is vital given the number of dependencies between team members.
"the board ... seems pretty clearly to be a matter of different people's work, and that board couldn't have come out of one person sitting in his office and throwing something together that looks pretty fancy in an hour or two and that has things arranged in a certain way."
"[with the board] if you're leaving stuff out, it's possible to see it. Somebody will say, we forgot."
Public activities at the board also seems to have social consequences. Awareness of the public nature, as well as social interaction, seem to encourage reflection and commitment.
"there was something about the board and the arrangement of the tasks that said the developer ... was on the carpet discussing with the manager what these various things were and making certain contracts. I'm less impressed with that fact when I see you've got another Windows application doing it."
There are major limitations of the board for planning, however. Software can better represent the complex dependencies among tasks, that need to be included in initial planning. The current instantiation of the wallboard does not provide a way of showing the relations between tasks.
"This got changed a lot last week to be more of a reflection of how reality had progressed instead of what it was supposed to have been".
Several conditions can support or impede the updating process. As with planning activities, the material, social, and public qualities of the wallboard mean that people treat the updating process differently from software updating.
"just by changing the number on a spreadsheet with a couple of key strokes, as opposed to walking out the door, going down the hall, out in the public, where I move my date out... The public stuff seems to have heavier weight to it, in terms of the import."
The public and visible nature of the board also motivates updates because people want to be seen to be working:
"people .... got really nervous that they didn't have things after their name. They got jumpy.... So they're eager to get up there"
With software the process is organized in a way that seems to promote less responsibility and interest in updating:
"whereas the developers in my group couldn't care less about the dates. Really they see it as my role to track that and to wave a flag if the dates are going haywire or whatever."
The material nature of the board makes it harder, however, to do detailed updating. Every change means producing a paper strip of the right length, naming it, putting it on the board. There is a minimal size of the paper that limits tasks to one-day-length, so updating brief tasks becomes a laborious process. This may reduce update accuracy and frequency.
As against this, the public nature of the board promotes accuracy in other ways: Often updating happened between the developer and the manager, or in small groups. They would stand in front of the board, look at the last week's tasks and talk about what was actually worked on, changing the board accordingly. These conversations not only revealed where the individual developer really was, but also why delays had occurred and which unplanned additional tasks may have come up. The board therefore reflected the past and present state of the project after this type of collaborative updating. Furthermore, the updating process on the board was seen as the responsibility of the developers, since they had access to the schedule themselves. This resulted in project organisation being devolved from the manager:
"I like the fact that the developers are getting involved. They're starting to take some ownership of .... the way their tasks are... they've taken responsibility for all the tasks they're working on together, and they're beginning to reallocate them via each other as necessary."
"we were rating tasks that were on the board, everything out past the deadline wasn't in, and we were able to pull things in and rearrange them... Everyone had a main task list in hand. We put them on the board, pushed everything out, got everything lined up, tried to get estimates from people on when they thought they could make it, realigned task estimates on things, added new tasks, added tasks for things like the stuff we discussed, filling in .... adding integration [time] .... saying, no, I've got too much on my stack, can you do it? Oh, Chris is doing this and that's related to this, wouldn't it be better if he took that and I gave this to Dave who's got something else that's kind of like that or related to that? There is a lot going on in half an hour. A lot of last-minute load levelling assignments, task definition, changing the size of the tasks, all happening in a very face to face [manner]."
Contrast the above process, with a description of replanning activity among two managers of a group that uses electronic management tools. The discussion takes place in the absence of the development teams. If the former situation can be called collaborative, this one can be titled private:
"we move tasks around between developers and I decide which tasks we move and how we move them ..I try to concentrate on moving tasks that are at the end of the schedule and tasks that, you know, different people could do equally well ... if this person has too many things and I can move this to this person, but not to this person, and then I can move something else from this person to this person and that sort of thing happens."
The board was also more flexible. People tacked up new unplanned tasks that occurred to them in the course of their current work, but which were unforeseen in the original plan. These were placed in neutral areas of the board. They could then be invoked in updating, which encouraged the board being used as a continuous replanning tool.
Given the uncertainty of the development process, learning from project history is essential when replanning during development. This process can be supported by a tool that helps recalibrate the original plan in the light of the actual work that was done and actual time needed. The board supports registering what actually happened. It can therefore give a systematic sense of which tasks were originally underestimated, which kind of overheads forgotten, and where in the schedule that delays occurred. The plan can be adjusted and refined once some of these factors are better understood.
"We're scheduled three days for it [a certain task]. We don't know if the three days are accurate, but we know it's a task that's going to be out there at the time. So we put an estimate out there that's on the board ... that gives us a record so that we're learning from this as we go."
However, despite attempts to do this manually, using the board, a major advantage of software scheduling tools lies in their ability to keep track of the set of changes a schedule goes through over time. Different versions of the plan can be saved without much effort, reflecting different states of the project, while a material schedule loses older versions after each change. "I know of no other way to capture the state of the board at a given frame of time other than doing it photographically. So, while it seems hokey, it may be the only practical thing to do."
"I'd say that that's a big weakness of [MS-Project] ... the whole communication .... of a visual higher picture is totally missing"
"it's harder to flip through pieces of paper when you've got a large number of people, than it is to go out here [to the board] and see what everybody's up to and how we're converging on them [our goals]."
It also provides easy public access to the group's goals as well as individual's tasks and commitments, serving a crucial co-ordination function:
"it was good to see all of us up there from the beginning, and an idea of what we were going to work on and when. Because you came in and you knew what you were going to work on."
The board was used as a co-ordination tool between a local developer W, and another developer, X, who is collaborating from a remote site:
"[W] is sitting here -- he's phoning [X], who's also up on the board -- saying, well, this is where we are now, you've got this coming up and I'll get that -- why don't I take this and you take that? And then he [W] will get up and move them around on the board. And -- it's working real well as a -- a mechanism for communicating information about the project."
This same developer worked part time. He frequently used the board when he came back to work after a four day absence, to re-orient himself. He would look at last week's work, and his work for the coming week. He said it also helped him to get initial information about the work and progress of his collaborators before he talked to them in person.
Having access to this kind of information is a prerequisite for developers assisting each other. According to the requirements of the situation, developers may also help each other spontaneously.
"[the developers] are beginning to reallocate [the tasks] via each other as necessary. If somebody finds that they have more work coming up all of a sudden, something happened, if they had to go up and do something else, then they're working well to reallocate it"
Electronic schedules in contrast seem to be less a method for group co-ordination than private progress checking:
"so once a week, they'll send us an Email going, the new updated development schedule is there. I look at that probably every couple of weeks.[...] I just check every couple weeks to make sure I -- you know, I didn't forget something big"
The privacy of the electronic process has consequences for individual's awareness about the progress of others. Developers using electronic schedules reported little knowledge of how the project in general was progressing, nor about the details of others' work. This may arise from the immense amount of detail in electronic schedules, making it difficult to obtain a general overview. One schedule had 1200 tasks allocated between 19 project members, organised on a person-by-person basis, extending to 25 pages when printed out. Developers typically updated on-line by accessing their own tasks, which were often spread over several screens. The complexity of accessing their own tasks and the sheer amount of detail meant that they were unlikely to serendipitously notice the progress of others while updating.
The benefits of the board for internal communication partially extend to external communication processes. The visibility and public nature of the board, seem to promote certain types of external communications in a way that is harder to achieve with software:
"Bob from Product Z is ... scheduled to integrate our code. I came in two mornings ago and found him standing there in front of the board looking at it to see where things were, when he's going to be able to start doing his integration."
"I've seen a lot of schedules here which are on PCs and then get sort of -- never really make it to management; they never really see the project. They get reports -- But they don't see what that schedule [is] -- how it looks."
The central location of the board also offers a place for opportunistic communication. Although this doesn't happen frequently, activities around the board can draw the attention of other people and invite opportunistic conversations. Updating situations encourage people to watch and join in conversations.
The biggest limitation of a material schedule for external co-ordination however lies in distribution. The board, as one manager pointed out, "is not very portable". This is not a problem for collocated team members, but for team members at remote locations, the difficulty of distributing the board becomes a major disadvantage.
(about a group member in another city): "He can't see this here. The best he can do is imagine what it might look like. I could maybe mail him a Polaroid [of the board], or I could scan one and send it in bit-map. [...] So he's getting his information coming very static on a one week granularity. That's not good.
Furthermore the information on the wallboard is difficult to integrate with other electronic information:
"I tried to get some [deadline] dates out of the [board] group, .... and their response to me was, come down and look at the wall..... the down side of it is the communications outside of the group, and because we're all so dependent on each other these days"
Other negative aspects to the board's public nature, are that sensitive or confidential information may be publicly displayed.
Other research work has identified the importance of synchronous interpersonal communications in long term collaboration [6,13]. The public forum of the board provided an ideal physical location for arranged and opportunistic meetings. The board was not only a place to meet, however. An important secondary function of the board was in providing tools and support for such meetings. Our results extend other work on synchronous collaboration processes and tools. In synchronous meetings, the wallboard served as an artifact in promoting group problem solving, it helped record progress in complex tasks and provided material, visual conversational props to aid reasoning and discussion. Research on shared workspaces shows they offer some of the same benefits for distributed collaboration [9,10,12,14]. Work on support for collocated design meetings is also consistent with our findings here: it shows the importance of large recording surfaces, and some of the advantages of pen and paper over computer techniques in complex planning tasks [5,7].
Our current data, mainly based on interviews, indicate a higher degree of satisfaction, and greater use of the wallboard schedule (1). This group completed code development to deadline, whereas the electronic group reported slips against the schedule. The two groups were developing different code however, so we should be careful about the generality of the conclusions we draw. Given the limitations of this case study, we need to supplement these observations with more systematic long term evaluations of the impact of the technologies.
(Footnote 1) The perceived success of the board is supported by the fact that in the course of our four month study, of the six groups we contacted who were using MS-Project when the study started, a further two had abandoned software in favour of the board.
We also need to look at different types of groups using the scheduling technology. This will help distinguish the effects of inherent media characteristics from the ways in which groups chose to use the technologies. In our current study these are hard to separate because we only looked at two groups. Thus for example, the big/visual characteristic of the board seems to directly promote overview information. In contrast, the fact that people chose to hold replanning meetings in front of the board is not the immediate result of media properties, but is part of the group operating principles. The public character of the board may facilitate such meetings, and it may have increased the likelihood that update meetings took place at the board, but people could clearly have decided to update and replan elsewhere. More research investigating groups with different culture and habits using these technologies is needed to disentangle media effects from group operating principles. Furthermore we have to understand the types of groups in which this type of technology is most useful. Is the tool as effective for groups where there are fewer dependencies between the activities of the members, and where tight co-ordination is less important? We also suggest a number of different potential benefits of the wallboard, and future experimental work should evaluate which of these benefits are most significant for the types of groups and activities we studied.
What are the implications of this work for the design of both scheduling tools and groupware in general? We have identified several limitations of electronic media when compared with existing paper based processes. Electronic tools may: (a) fail to promote face-to-face communications which have been shown to be critical for group collaboration [6,13]; (b) decrease awareness of the actions of other group members [1,3], which may reduce helping behaviour and collaboration; (c) suffer from lack of visibility and permanence which may compromise their function for reminding and co-ordination [11,12]; (d) fail to engender belief and commitment [4]. However, despite their benefits, material tools also fall short on several dimensions such as distribution, versioning and complex dependency tracking.
How then can we combine the benefits of electronic and material systems? One possibility is to use a pen-based large display such as [2] in a public location, running a modified scheduling application, that is updatable by all group members. This would satisfy the requirements of size and visibility, and allow public interactions. The pen-based interface would allow handwritten inputs, with tasks being placed and moved using direct manipulation techniques, to simulate material interaction. The benefits of software would be that the information could be remotely distributed. Coworkers at remote locations could also have Liveboards with fast audio (and possibly video) connections, so that collaborative planning and updating meetings could be held "in front" of the board at short notice. Furthermore the computational capabilities of the application could be used to depict dependencies, save previous versions of the "board" to facilitate learning, and integrate the data with other corporate information.
In future, we should design electronic group tools with the benefits of the material wallboard. Our electronic tools need to be public, to promote commitment and conversation; material in affording engagement and reflective use of the tools; and they need to simulate the dimensions of size and visibility in supporting ready access to complex information.
