|
Can
You Handle It? Individual Management of
Incoming Information in the
Workplace Sarah
Waterson, James Reffell December 12, 2001
Appendix B: Some Interesting Comments Executive SummaryThe purpose of this study is to identify whether or not individuals in the workplace feel overloaded with information. We look at the amount and quality of information people take in, their sources of information, and some strategies they use to handle that information. In doing so, we hope to identify whether information overload on an individual level should be a concern in the workplace, and understand how businesses can aid individuals in managing their information. We used an online survey clearinghouse to gather 165 responses to a 27 question survey that ran from October 16th, 2001 thru October 20th, 2001. With this survey, we attempted to ascertain:
We found that the majority of individuals in the workplace feel only somewhat overloaded with information. People also feel they receive some or most of the information they need to do their jobs. Most are satisfied with their company’s involvement in managing information, as well as with their handling of incoming information sources. Feelings of information overload correlate negatively with feelings of receiving needed job information, meaning the less people feel they are receiving the information they need for their job, the more overloaded they feel. A small correlation exists between working outside of the home and feelings of information overload. Frequent use of e-mail, voicemail and/or the telephone, correlate positively with feelings of information overload. People that rely heavily on computer-based information sources, especially knowledge workers, experience more frequent feelings of information overload, as do those people that frequently check their email, voicemail, or the Internet. Additionally, women report feeling overloaded more frequently than men. In general, the total number of email or voicemail messages does not correlate with feelings of overload. Rather, certain types of messages and behaviors for handling information seem to be more related. The findings in this study collectively support the idea that quality and appropriateness of information received, and not just quantity of information received, are linked to feelings of overload. These findings do not give us a clear picture of what characterizes individuals who feel overloaded with information at the workplace. They do, however, begin to show us that the quality of information received, the context in which it is received, and general work habits may prove more indicative of overload than sheer quantity of information received. In addition, they also reveal how difficult it is to characterize and quantify information overload. IntroductionAccording to a 2000 Reuters study [3], 49% of managers feel that they are quite often or very frequently unable to handle the volume of information they receive. They also report that the amount of information required to execute their jobs effectively is large and growing, and they feel that much of the information they do receive is not important or unsolicited. In addition, 41% of managers agree that their working environment is extremely stressful on a day-to-day business. How much of stress and overload is related to the information one receives at work? Does being inundated with large amounts of data – both useful and not useful information – lead to a sense of overload? There is a growing concern in the literature [4,5] and in the workplace about this sense of overload, whether or not it exists, where it comes from, and how to deal with it. However, in order to answer these questions, more work needs been done to identify the types and amount of information people deal with, how they receive and gather information, and what people do to manage it. MethodPilot StudyUsing online survey software from www.surveymonkey.com, we distributed two 43 question surveys consisting of a mix of quantitative, categorical and short answer open-ended responses. Six of the 43 questions were open-ended questions about the survey itself, asking for feedback on the wording, confusing questions, etc. We had 10 trial participants, each taking approximately 20 minutes to fill it out. Based on the feedback, we found that instant messaging (IM) was used surprisingly often by our sample population, so we added questions about IM use similar to the Internet and Intranet questions. Additionally, we re-worded a number of questions for more clarity. For the actual survey, we switched from surveymonkey.com to an online survey clearinghouse, Zoomerang, which could gather an appropriate sample population and distribute our survey. However, switching to Zoomerang required a number of changes from our original design, as their software could not accommodate the format for some questions that we initially desired and limited us to a 30 questions maxim. The combination of these two factors required considerable modifications to our original survey. The final survey consisted of 27 questions, requiring responses to quantitative, categorical and short answer open-ended questions. Again, it took approximately 20 minutes to complete. Survey DesignIn designing our survey, we were attempting to both ascertain how much information overload professionals in the workplace feel, and to identify those characteristics associated with the most overloaded individuals. Toward this end, we asked questions in the following areas:
Participants & DeploymentThe survey that ran for 5 days from October 16th to October 20th, 2001. In this time period, we received 165 valid responses from Zoomerang. Zoomerang’s sample was randomly selected from individuals working at businesses that had some form of Internet connection.
ResultsUnless otherwise indicated, all significance results are reported at the two-tailed 0.05 level. All means are reported with standard deviation. DemographicsOur sample population of 165 was nearly evenly split across gender, with 79 female and 84 males respondents. 3 respondents did not indicate a gender. The age of the respondents was fairly normally distributed across the categories. See Figure 1. Education levels of our respondents indicate most had at least a Bachelor’s or professional degree. See Figure 2. Our respondents cover a broad range of job categories (See Figure 3). We also asked participants to indicate their job title, which we used to categorize them as knowledge workers or not. The sample was split fairly evenly, with 72 classified as knowledge workers, 77 not, and 16 we were unable to classify. Figure 1: Age.
Figure 2: Education Level.
Figure 3: Job Category.
Source Use, Importance, & Interruptions In order to begin understanding how the different information sources people used at their job related to overload, we looked at (1) the frequency of the source use, (2) the level of importance of each source, and (3) the frequency in which a source interrupted somebody during their work day. The first question of the survey asked the respondent to indicate the information source they typically use the most. Figure 4 shows that e-mail and the phone are used the most out of all 13 sources we offered as selections in our study. Respondents could select more than one source, though one third only indicated one source. Figure 4: Of the following information sources, which do you use the most for your job?
Each respondent was asked to rank the importance of each 13 information source for getting their daily work done. The five point Likert scale options were not important (1), somewhat important, important, very important, and critical (5). An n/a option was available as well. For analysis, we grouped the n/a results with the not important results. Figure 5 shows the mean responses for each source. Figure 5: Information Source Importance.
We were also interested in how frequently people were distracted from their work by the information sources. Figure 6 shows how frequently people felt the information sources interrupted their work. The five point Likert scale used here was never (1), rarely, sometimes, often, and always (5). Figure 7: Source Interruption frequency.
In order to create a more general measure of use, importance and interruption frequency, we created indices for each of these three questions. All three indices correlate significantly with one another:
In general, these correlations show that if you use an information source frequently, you find it important and frequently distracting. If you find an information source important, you frequently use it and find it distracting. Lastly, if you find an information source distracting, you probably use it a lot and it is important to you. While the indices give a general feel for the how these three aspects relate to one another, taking a closer look at the correlations of individual sources amongst the three categories reveals more interesting relationships. Correlating each source against itself for use, importance, and interruption followed the same trend as described for the indices – all correlate very strongly and positively with one another. What this means is that for no source listed, was it frequently used and not distracting. Nor do people frequently use information sources that they don’t find useful. Many of the sources also have a positive, significant, correlation with other sources in expected fashions, e.g. if you use the Internet a lot and find it important, you use e-mail a lot and find it important as well. The interesting correlations here, however, are the negative ones. Table 1 shows the significant negative correlations of source importance between sources – people who find paper important, do not find e-mail or IM important, and vice versa. Table 1: Interesting Significant Correlations of Source Importance
*Correlations
significant at the 0.05 level (2-tailed). **Correlations significant at the 0.01 level (2-tailed). Use of e-mail or IM does not correlate to use of paper significantly. However, use of e-mail correlates negatively with finding paper important, as well as the converse of that: use of paper correlates negatively with finding e-mail important. See Table 2. The use of paper does not correlate to finding IM or e-mail distracting, nor does the use of e-mail or IM correlate to finding paper distracting. The use of paper also strongly correlates negatively to voice-mail, meaning people that find voice-mail important do not use paper. Table 3: Interesting Significant Correlations of Source Use to Source Importance
*Correlations
significant at the 0.05 level (2-tailed). **Correlations significant at the 0.01 level (2-tailed). In addition, Table 4 shows significant negative correlations between the interruption frequency and importance for voice-mail and paper. Table 4: Interesting Significant Correlations of Source Interruption Frequency to Source Importance
*Correlations
significant at the 0.05 level (2-tailed). **Correlations significant at the 0.01 level (2-tailed). Identifying GroupsA factor analysis was performed on the responses for importance, resulting in two factors that explain 43.4% of the variance in source importance. The first factor, accounting for 24.5% of the variance, describes computer users – people who find e-mail, instant messaging, the Internet, and their intranet important. These people also tended to find voice-mail important and paper un-important. The second factor, accounting for 18.9% of the variance, describes more mobile workers – people that find pagers, cellular phones, calendars, phones, and personal interaction more important. We used these two factors to distinguish in further analysis the more computer-based workers from the more mobile workers. From the transformed factor scores (ranging from 0-5), we selected to identify those individuals scoring a four or higher (very important to critical for job) to fit in each of these categories. 61 of the individuals can be described as these heavily computer-based workers, and 53 can be described as heavily mobile workers. 32 individuals fall into both categories, meaning they find nearly all of the information sources very important for their jobs. Demographically, these groups show similar distributions in age, gender, education. and as knowledge workers. 33 individuals do not fall into any category, and interestingly enough, they are significantly more knowledge workers than not, c2(1, 32) = 4.50, p<0.05. Likewise, a factor analysis was performed on the responses for interruption, resulting in four factors that explain 65.0% of the variance in interruption frequency. Table 4 shows these four factors, the percentage of the variance they explain, and their distribution. Table 4: Interruption Factors
As with the importance factors, we use the interruption factors to distinguish identify groups. Transforming the factor scores to a scale of 0-5, we identified those with a score of four or higher to fit into each of the four categories as highly interrupted by these sources. Not very many people fit into these categories, except for those frequently interrupted by e-mail, voice-mail, and the phone (13 individuals). This can be seen by looking at the distributions of the factors in Table 4. Factor 2 is the only displaying a normal distribution, while the other three factors are skewed heavily to the lower values. Comparing the values in figure 7, we can see that this makes sense. Most sources, with the exception of people, the telephone, and perhaps e-mail, are noticeably interruptive. In fact, factor 2 includes the three most interruptive factors after people. What these factors also seem to explain are the differences between sources that one must more actively seek information from – factors 1 and 4 – and those that one must actively communicate with – factors 2 and 3. For both factor analyses, we excluded the responses on databases. We found that this category was not clearly understood by our respondents, and confounded the data. Information Overload and Information ReceptionMeasuresWe asked our respondents to use a 5-point Likert scale (never (1), rarely, sometimes, often, and always(5)) to rate how often they felt overloaded with information. Using the same 5 point scale, we also had them rate the frequency of the quality of information they received. These quality related measures were (1) how frequently they felt they were receiving all the information they were required to take in for their job, (2) how frequently they felt they were receiving all the information they would ideally like to take in for their job, and (3) how frequently they felt they were provided by others with all the information they needed to do their job. The mean responses for these four questions are displayed in Table 5. Table 5: Overload & Information Quality Means
The mean for overload is slightly lower than 3, indicating that, in general feelings of information overload exist, but do not dominate in the workplace. The means for getting the required amount of information, the ideal amount of information, and being provided with the necessary information are slightly higher than 3. This indicates that, generally, people are comfortable with the amounts and quality of the information they receive. For each measure of quality, we found a negative correlation between the frequency of information reception and the frequency of information overload. See Table 6. People who most often receive the information they need or would ideally have to do their job feel overloaded the least often, and that those who most often are provided the information they need to do their job feel overloaded the least often. This data indicates a potentially surprising relationship between the quality and quantity of incoming information and feelings of information overload. If overload were strictly related to quantity, then one would expect a positive correlation between incoming information and overload. Additionally, because the required and ideal measures correlate more strongly with information overload than the provided measure, this data suggests that overload is related to the information that individuals seek out as part of their job as well as the information that is given to them. Table 6: Correlation between frequency of feelings of overload and information reception measures
**
Correlation is significant at the 0.01 level (2-tailed). * Correlation is significant at the 0.05 level (2-tailed). As could be expected, we also found that the measures of information reception correlated very highly to each other. See Table 7. Table 7: Correlation between information reception measures
** Correlation is significant at the 0.01 level (2-tailed). Because of these strong correlations between quality of information and information overload, we report interesting results related to these three measures in the rest of our analysis as well the interesting findings relating to measures of overload. Demographic FactorsWe next looked at how feelings of information overload were distributed by various demographic measures, including gender, age, job category, level of education attained, and whether or not they were knowledge workers. Overall, we found no significant differences except in the case of gender, where we found that women reported feeling overloaded by information at a slightly higher rate than men, t(160)=2.109, p<0.05. Additionally, more so than women, men felt that they received all of the information they required to do their job, t(160)=-1.967, p=0.05. See Table 8 for the mean values. Table 8: Means by Gender
Within
Groups
We also looked at overload and information reception with respect to the groups created by the factor analyses for source importance and interruption frequency. For all respondents, the more important computer-based information sources, the more feelings of information overload exist, r(112)=0,250, p<0.01. Knowledge workers who fall into the heavily computer-based category feel particularly more overloaded (N=22, mean=2.91, std.dev.=0.75) than their peers that do not find computer-based information sources important (N=30, mean=2.3, std.dev.=1.09), t(50)=-2.389; p<0.05. For the people that are frequently interrupted by e-mail, voice-mail and phone(N=13, mean=3.31, std.dev.=0.98), they tend to feel overloaded with information as well r(158)=0.269, p<0.01, and significantly more so than the people that are not interrupted by these sources (N=147, mean=2.63, std.dev.=0.99), t(115) = -2.529; p<0.05. Information SourcesIn order to understand how different
information sources related to feelings of overload in general, we first
looked at the overall usage, importance, and interruption indices. The
usage, r(161)=0.174, p<0.05, and interruption,
r(159)=0.186, p<0.05, indices correlated with information
overload, while the importance index, r(161)=0.120, n.s., did not.
This indicates that feelings of information overload were sensitive to
overall usage and frequency of interruption by the various information
sources, but not to the importance ascribed to them. We next investigated the relationship between the individual information sources and feelings of overload, to see if particular sources were associated with feelings of information overload. We found that for all respondents, only three specific information sources—e-mail, voice-mail, and phone—correlated to information overload in terms of use, importance, or frequency of interruption. See Table 9. Again, for this analysis we excluded databases because the definition was not clear to the participants. For these three sources, all measures correlated to overload, except for use of voice-mail. Table 9: Correlation between frequency of feelings of
overload and use, importance, and interruption measures for specific
information sources.
** Correlation is significant at the 0.01 level (2-tailed). * Correlation is significant at the 0.05 level (2-tailed). This data mirrors what we found in the
group level analysis – that the phone, voice-mail, and e-mail are the main
information sources contributing to feelings of information overload. In
the following sections, we take a closer look at e-mail and voice-mail.
Unfortunately, our survey did not cover the use of the phone.
In general, we found no relationship between the total number of e-mails received in a day and information overload. However, we asked the respondents to include information on the number of various types of e-mail they received: pertinent to their job, requiring an action/ response, internal to the company, company broadcasts, left unread, contained duplicate information, and spam. For all respondents, we found a positive correlation between the number of internal e-mails received and frequency of feelings of overload, r(153)=0.195, p<0.05. This could indicate that intra-company communication by e-mail is more of a factor than inter-company communication information overload. In addition, for the people who indicated e-mail was very important or critical for getting their job done, the number of e-mails left unread correlates with feelings of not getting the ideal information for their job, r(98)=-0.212, p<0.05. E-mail organizational schemes also seem to contribute to feelings of overload. Particularly the use of filters correlates strongly to people feeling they are not getting the required information for their job, r(161)=-0.212, p<0.01, as well as to feeling they are not getting the ideal information for their job, r(161)=-0.154, p<0.05. We found no other significant results for various organizational schemes we listed. It seems, however, that the filtering strategy affects people’s perceptions on the information they receive. One possible explanation for this finding is that imperfect e-mail filtering algorithms are discarding or misfiling e-mails in such a way that users are not able to locate and read all the ones that contain needed information. The frequency of which people check their e-mail also correlates to feelings of information overload. For those who check their e-mail multiple times in an hour or constantly, they tend to feel more overloaded than those who check it hourly or less, t(161)=-2.417, p<0.05. Voice-mailSimilar to e-mail, we had respondents record not only their total voice-mail messages, but also information on the number of types of e-mail: pertinent to their job, requiring an action/ response, internal to the company, deleted without listening to, saved for later, and duplicates. For all respondents, redundant voice-mails correlated with feelings of not being provided with enough information r(149)=-0.202, p<0.05. For those that indicated voice-mail is very important or critical to their work, the number of pertinent voice-mails, r(71)=-0.255, p<0.05, and the number of internal voice-mails, r(67)=-0.281, p<0.05, correlate with a sense of not being provided with enough information. One possible explanation for this relationship is that these internal and relevant voice-mails are not reaching these people at the right times, or the information they contain is not complete or in a format that they can directly use. The frequency of checking
voice-mail correlates with feelings of overload, with those checking
multiple times in an hour or constantly tending to feel more overloaded,
r(159)=0.159, p<0.05. Internet, Intranet and Instant MessagingWe also looked at the frequency with which people reported using the Internet, their intranet, and instant messaging. In general, use of the Internet correlates with feelings of information overload, r(159)=0.171, p<0.05. This trend is stronger for those people who do not use the Internet frequently, r(126)=0.264, p<0.01. For people who find their intranet important to their job, frequently using it correlates with a sense that they do get the information required for their job, r(51)=0.321, p<0.05, as well as feeling as if they are provided with the information to perform their job, r(50)=0.307, p<0.05. For instant messaging, no interesting correlations or relationships were found. Other ItemsWork outside of
work
In terms of work outside the traditional office and office hours, we found that of information overload positively correlated with both the amount of time people spent on work outside the office or normal workday, r(161)=0.158, p<0.05, and the number of days a week they spent telecommuting, r(161)=0.164, p<0.05. When we looked at telecommuters (those who reported telecommuting one or more days a week) and non-telecommuters separately, however, both correlations disappeared for both groups. This suggested the possibility that most of the increase in information overload was associated with the difference between telecommuters and non-telecommuters, but further testing indicated no significant difference in the mean information overloads of the two groups for any of the work outside of work factors. Making
Changes
We asked participants what their company could do to help them better manage their information. The resulting categories and totals can be seen in Figure 7. 43% of the respondents left this field blank, were not sure what their company could do better, or felt that their company did not need to do anything. This indicates that, on the whole, companies and/or individuals are doing a good job managing their information. Figure 7: If your company could do one thing to help you manage your information, what would it be?
Figure 8 shows how each improvement category felt about their information overload and information quality. Notice that the few people that wanted more information from their company, not only feel less overloaded than the rest and feel as though they get the information they require to perform their job, but they also feel they do not get the ideal information or are provided with information. Those that have the highest overload scores are those that would ask for better filtering of information, more training, or the removal of an information source. No tests were done for the significance of these differences. They are indicated here as relative measures of interest. Figure 8: Information Overload and Information Reception for Improvement Categories
Lastly, we asked participants which, if any, information source they most wished to remove. The sources and totals are shown in Figure 9. Figure 10 shows how each of these groups of people rate themselves on information overload and information quality. In particular, notice that the people that would not remove any source feel the least overloaded. Again, no tests were done for the significance of these differences. They are indicated here as relative measures of interest. Figure 9: If you could remove one information source from your workday, what would it be?
Figure 10:
Source to remove and Information Overload and Reception
measures.
DiscussionSource Use, Importance, InterruptionsOur preliminary findings relating source use, importance, and interruptivity show that the three factors are very closely related to each other, not only in terms of each specific information sources, but across sources as well. Individuals who use, find important, and are interrupted by one source are likely to use, find important, and be interrupted by related sources. The exception seems to be with paper: the negative correlations between these factors in terms of paper versus e-mail, voice-mail, and IM could indicate either that paper is being replaced with those sources in some situations or that high-paper use individuals simply work differently than low-paper use individuals. Source importance can be factored into two descriptive categories, those people that find computer-based information sources important, and those that find more mobile sources of information important. These factors are not mutually exclusive. In fact, a large number of individuals find most of the information sources important. A large number of individuals do not fall into either of these categories. Source interruption can be factored into four categories: inter/intranet interruptions, e-mail/voicemail/phone interruptions, pager/cellular phone interruptions, and fax/paper interruptions. Of these factors, only interruptions from e-mail/voicemail/and phone have a noteworthy population that is frequently distracted. The other three factors contain predominantly people not interrupted by the sources. Overload
We found that the majority of individuals in the workplace feel only somewhat overloaded with information. People also feel they receive some or most of the information they need to do their jobs. Most are satisfied with their company’s involvement in managing information, as well as with their handling of incoming information sources. Feelings of information overload correlate negatively with feelings of receiving needed job information, meaning the less people feel they are receiving the information they need for their job, the more overloaded they feel. A small correlation exists between working outside of the home and feelings of information overload. Frequent use of e-mail, voicemail and/or the telephone, correlate positively with feelings of information overload. People that rely heavily on computer-based information sources, especially knowledge workers, experience more frequent feelings of information overload, as do those people that frequently check their email, voicemail, or the Internet. Additionally, women report feeling overloaded more frequently than men. In general, the total number of email or voicemail messages does not correlate with feelings of overload. Rather, certain types of messages and behaviors for handling information seem to be more related. The findings in this study collectively support the idea that quality and appropriateness of information received, and not just quantity of information received, are linked to feelings of overload. The lack of some expected correlations was particularly interesting. In general, the number of e-mails received and voice-mails received, total or in any category, did not relate significantly to overload, with the notable exception of the internal messages. Probing more deeply into the relationship between internal information exchange and overload could provide some interesting results. Because the frequency with which individuals checked their e-mail, voicemail and used the Internet relate to overload, it may be that an individuals work habits may be more of an issue than the actual quantity of information received. On the other hand, at least for e-mail use, the various methods used to organize information flow did not seem to be a factor, except for filtering. These issues need to be tease apart in future studies. Running our survey online limited our study to people that use the Internet and e-mail, which means we do not represent the greater working population. Additionally, we intentionally limited our information sources to a list of thirteen. Other sources, such as television and radio were not included, both of which are used frequently in certain professions. A number of other possible important factors were not possible to include in this survey, such as job satisfaction, stress factors and levels within and outside of work, and hours worked are just a few. These findings do not give us a clear picture of what characterizes individuals who feel overloaded with information at the workplace. They do, however, begin to show us that the quality of information received, the context in which it is received, and general work habits may prove more indicative of overload than sheer quantity of information received. In addition, they also reveal how difficult it is to characterize and quantify information overload. The survey and data is available online at: http://www.sims.berkeley.edu/academics/courses/is271/f01/projects/Individual_Overload/ About the Authors Sarah Waterson is a graduate student in Computer Science at the University of California, Berkeley. Her research interests are human-computer interaction, information visualization, sculpture, and painting. James Reffell is a Master’s student in the School of Information Management and Systems at the University of California, Berkeley. His research interests are human-computer interaction, information visualization and usability.
This survey was
sponsored and funded by BackWeb Technologies (http://www.backweb.com/).
References and Resources [1] Ahmed, Z. “The Demise of Digital Dysfunction.” The Next Big Thing, 2001. (http://tnbt.com/jsp/TNIP.jssp?x=1&y=0&p=TNAsset.jsp&a=79692) [2] Bezroukov, Nickolai. “Information/Work Overload Annotated Webliography.” 2001. (http://www.softpanorama.org/Social/overload.shtml) [3] “Dying for Information? An Investigation into the Effects of Information Overload Worldwide,” Dow Jones Reuters Business Interactive Limited, 2000. (http://about.reuters.com/rbb/research/overloadframe.htm) [4] “More Online, Doing More.” Pew Internet & American Life Project, 18 February 2001. [5] Owen, Jim. “Coping with Information Overload: Too much of a good thing can hurt your job performance.” CareerBuilder, 2001. (http://www.careerbuilder.com/wl_work_9905_overload.html) [6] Zoomerang (http://www.zoomerang.com) [7] Survey Monkey (http://www.surveymonkey.com) Appendix A: Survey(27 Questions)
Appendix B: Some Interesting Comments If you could remove one information source from your
working day, what would it be? “My supervisor. Why? Obviously, you've never dealt with him.” “bosses.” “assistant supervisor.” “too many chiefs.” If your company could do one thing to help you
manage your information, what would it be? “Clone me...” (2) “Provide matches to burn all the unwanted magazines! Otherwise, nothing.” “Give me more of it.” “leave me alone. Because non techs ask a lot of dumb questions” “honesty. It makes things quicker, easier, more beneficial to work and wellness of workers” “I don't know. I'm the President. Tough question.” “put a woman in charge :)” “Put in my hand directly rather than let it die a slow death going through the proper channels” “Get off my back because I know my job better than the bosses do.” “strap on an extra brain to my head”
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
<![endif]>