Abstract

As the cellular phone market continues to grow rapidly, customer service is becoming a key factor in attracting new customers and retaining existing customers. Using a two- part design, the Cellular Service Provider Customer Satisfaction Survey (CSPCSS) explores customer service issues in the cellular service industry by investigating consumers’ interactions and satisfaction ratings with their customer service providers. The web-based survey is used to determine consumer perceptions of customer service, and factors contributing to these perceptions. The survey also aims to better understand how consumer background, cell phone usage and experience with customer service affect overall consumer satisfaction when accessing customer service. A small, related experiment also provided additional information on actual consumer experience when interacting with customer service. For a sample population of mostly university students and young professionals, customer service is considered somewhat satisfactory and is not an important consideration when selecting cellular service provider. Customer service satisfaction is sensitive to timeliness and ease of access, and many customers new to a provider need to call customer services several times to resolve issues like as billing. Customer service satisfaction is a likely indicator of whether consumers would recommend a provider to others.

I. Introduction

As rates for cellular service fall and the number of new subscribers continues to increase, the demand for wireless technology is shifting from a luxury service to a need-to-have service that relies more on customer service than ever before. A recent study by JD Power and Associates reports that the market penetration for cellular phone service in the top 25 markets in the U.S. is now above 50%, with the cost of acquiring a new customer ranging between $350 - $475. It is more important than ever for cellular service providers to retain existing customers rather than to merely add new ones. As competition becomes fiercer and the market becomes more saturated, retaining existing customers and overall customer loyalty are becoming key competitive factors for any industry player, with the focus being on superior customer service once the initial sale has been made.

Additionally, JD Power reports that overall monthly cell phone service usage has grown, increasing 32% in just one year (2000 to 2001). Along with more usage comes more cellular problems. 59% of users reported having problems that required customer service interaction. Customer satisfaction is relying more on the overall customer service experience, an area that has been traditionally ranked low when compared to other company initiatives.

Study Goals

In an effort to explore customer service issues and rate consumer satisfaction, we wanted to answer some of the following questions:

§ Who are the cellular consumers and what is their cellular phone usage background

§ What features drive customer selection of service providers and whether customer service is critical to the selection

§ In a growth service-based industry, what are post-purchase customer service satisfaction levels among consumers

§ How do satisfaction levels for different service providers compare to one another

§ What is the model of actual customer behavior

· What is consumers’ actual experience with customer service like

· How would they act upon their perceived notions of customer service

We also wanted to find out

§ Are there contradictions between customer behavior and perception?

§ What indicators, if any, can predict customer satisfaction within a growth service industry?

§ What is the underlying assumption for each indicator?

We designed a two-part exploratory study to attempt to answer these questions and gain some insights for a potentially larger market-based study on customer service satisfaction with cellular phone service providers.

II. Methodology

Study Design

The study design consisted of a broad online survey and a small interaction experiment with customer service representatives. (See Appendix A and B for the survey instrument and experiment kit respectively.)

Online Survey

To better understand users’ perceptions of consumer satisfaction with cell phone providers’ customer service, we designed a quantitative survey of service, issues, and customer satisfaction. The survey aimed to reveal consumer attitudes towards cellular service providers and specifically focused on post-signup customer service experience. The topics include:

· Frequency of customer service interaction

· Overall satisfaction with customer service

· Reasons for contacting customer service

· Customer service interaction preference

· Satisfaction with specific areas of customer service

· Recommendation to friends and family

Potential confounding variables included participants’ varied experience with cell phone usage and service provider. To better understand our participants as cell phone consumers, we measured cell phone demographics. This includes:

· Current provider

· Length of usage

· Cost of plan

· Monthly minutes spent on the phone

· Reasons for choosing provider.

We also measured traditional demographics such as gender, age, education background and income level.

Interaction Experiment

To gauge actual interaction with customer service representatives on a standardized level, a small experiment modeled after usability task testing was designed. The experiment aimed to discover:

· Events leading up to interaction with live customer service representatives

· Overall customer service reaction to a standardized battery of questions

· Determining how well representatives can react to questions that involve their suggestions, rather than just facts

· Checking consistency of specific answers by representatives of the same company

Participants were to complete the tasks of contacting their respective customer service and obtaining responses to the standard set of questions. While trying to get in touch with a customer service representative, participants were asked to document number of events they encountered, and where possible, the sequence of these events. Possible events include:

· Selecting from menu options

· Entering account information

· Being put on hold

· Being transferred to various departments/lines

· Reaching a customer service representative

· Being put on hold/transferred by the representative to another representative

To gauge standardized responses, respondents were given a “script” to follow which explored different areas of service including:

· Charges for current customer service call

· Explanation of and specific charges for “roaming”

· International calling charges, specifically exploring the cost of calling South Africa

· Reviewing previous monthly service usage and providing a recommendation of plan if current plan is not effective

The total time to complete the experiment in one call session was recorded. After completing a call, participants were also asked to fill out a post-experiment questionnaire to measure their satisfaction with customer service based on the experience.

Pilot Study of Instruments

The survey instrument and experiment kit were designed separately in several iterations. Team members drafted the initial survey and developed the experiment using personal experience, other search-related survey instruments and past research results as sources. The survey was initially piloted with 2 participants, on paper with a team member at hand to observe and answer questions. Subsequent iterations tested both the online survey and experiment with at least one team member present. 3 participants assisted at this stage and allowed us to fine-tune the survey instrument and experiment procedures and forms, as well as test the web-based survey for technical problems. Pilot participants were not participants in the full deployment, to avoid the possibility of introducing bias in the results.

Incentives for participants

In order to attract responses, all participants who completed the survey and experiment received a $5 gift certificate. The remaining certificates were awarded based on a random drawing of online survey participants who provided email addresses for contact. In recruiting emails, the certificates and method of award was also advertised.

No other compensation or disclosure agreements were associated with participation in the survey and the experiment, and participants were assured of anonymity.

Implementation

The survey was designed to take approximately 10 minutes for the respondents to complete. It primarily consisted of multi-choice or check box answers instead of open-ended or fill-in answers. Initial pilots were performed on paper, while the final iteration and ultimate deployment used ColdFusion forms connected to an Access database as our web-based method.

An experiment kit was prepared for each participant. At least one team member was present to facilitate the session and observe and take notes on the process. The experiment kit for each participant consisted of:

· Informed consent form

· Instructions to the participants

· List of questions to ask as well as space for response

· One page for recording the details related to the call, including a chart listing events and space for marking frequency, sequence and extra notes on event occurrence

· A short 2 page post-experiment questionnaire

The Participation

The online survey of current cellular service targets consumers who had contacted customer service at least once in the past 12 months. Respondents were invited to participate by a short email introducing the survey and our intentions, with a link to the web-based survey. They were directed to follow the link and fill in their responses electronically. Upon deployment, the survey instrument was posted for 10 days before being closed to participants.

The known total sample size is approximately 200. 78 participants responded successfully to the online survey (40% response rate). The web-based survey was "hit" approximately 100 times, and completed successfully 78 times. For the purposes of this analysis, we assume each participant completed the survey only once. The final number of survey responses used for analysis is 63, after ruling out those who have had no experience contacting customer service.

A small subset of online survey participants also completed the short experiment. 11 participants completed the experiment successfully. Having filled out the online survey before completing the interaction experiment can affect participants’ post-experiment responses. One early intention of the design was to see if there are significant changes to satisfaction ratings before and after the interaction experiment.

III. Survey Results

1. Basic Demographic

The study does not reflect and was not intended to reflect the total Cellular Service Provider (CSP) customer base. Rather, it focuses on specific customer groups associated with the School of Information Management and Systems at University of California Berkeley. Our sample consisted primarily of technologically savvy students and friends, a group that can also be considered a key target market for CSPs.

Some basic demographic distributions representing the characteristics of our sample can be seen in the charts below.


Figure 1: Distribution of age for respondents	Figure 2: Distribution of age for respondents


Figure3: Distribution of education level for respondents	Fig 4:Distribution of household income

The graphs above demonstrate that our population consists primarily of college students and young professionals ranging from 18 to 34 in age (78%). They have high education levels and varying levels of household income. For this sample, the reported household income is not the best indicator of the sample’s actual financial standing. It is also not the best measure for determining whether financial resources have clear affects on attitudes towards CSPs’ customer service. Many young professional tend to live and work in areas with high cost of living despite high salaries while many students also receive additional financial support from school and family and may not have considered these when reporting.

Our participants represent only a limited sample of the types of cellular phone service customers. The results from this study do not necessarily represent the feelings or behaviors of all cellular customers.

2. Cell Phone Demographic Findings

To better understand our participants as cell phone consumers, we measured the participants’ cell phone demographics.

Cellular Service Providers

The following chart displays the distribution of service providers used by survey participants. The CSPs displayed here represent the cellular providers that provide services in California and do not include service resellers like MCI WorldCom.

Figure 5: Distribution of Cellular Service Providers

As can be seen in the above chart, Sprint and Cingular were the most highly represented service providers in this study. It is also important to note that this distribution is not necessarily representative of the actual market share of each of the providers studied. The varying number of participants representing each service provider also has implications for the analysis that we performed. This will be discussed further in the relevant sections.

Monthly Bill, Monthly Usage and Purposes of Use

The data from this study indicates that cellular phone usage of students and young professionals has become common but is not becoming overly used. Most of the participants in this study indicated that they are mid-range users. 48% reported a mid-range monthly bill (between $30 to $75) and 44% reported mid-range monthly usage (between 300 to 750 minutes). Findings also indicated that participants in this study are willing to spend a relatively larger amount of money on cellular service than with other phone services regardless of their financial standing. (See Appendix C)

The charts below show the average month usage and monthly bills for study participants.

Figure 6: Distribution of Monthly Bill

Figure 7: Distribution of Monthly Usage

This study asked participants to indicate their primary cell phone use. The chart below shows the percentage of participants who use their cellular phone primarily for personal reasons, work, and equally for work and personal.

Figure 8: The Purpose of Cellular Service Use

The graph shows that 63% of respondents reported using their cellular phone mostly for home / personal usage while only 4.8% of respondents use it only for work. This results is most likely due to our sample population consisting of a large number of students who do not have jobs that would require cellular phone usage. The purpose of cell phone use and monthly usage and billing patterns point to the adoption of cellular phone as a basic personal communications tool that many use substantially on a regular basis.

Usage Patterns of Cingular/PacBell Users

One point of interest is that a high percentage of Cingular/PacBell users have similar monthly usage and average monthly bill trends. 35% of users indicated low monthly phone usage (under 200 minutes per month) and 87% of users have mid-range monthly bills (between $30 and $80). The average monthly usage and average monthly bills among users of other CSPs do not indicate such noticeable trends. These numbers indicate that Cingular may be targeting users who need cellular phones for small usage. Appendix C includes tables indicating the averages by CSP.

Cellular Provider Switching Behaviors

Survey data supports the observed market phenomenon of high customer turn over rates among cellular providers. By comparing the amount of time that participants have had any cellular service with the amount of time that they have been using their current cellular provider, we found that a startling 61% (17 out of 28) of respondents have switched their CSPs in the past year alone. Figure 7 shows the number of users who have converted to each provider as well as new cellular users who have been using each service provider for less than one year. The top portion of each bar represents cellular users that have switched providers and have started using the respective service provider and the bottom portion of each bar represent new cellular users for each provider.


Figure 9: The winner in the switching game (Convert Rate of Cellular Service Providers)

From this chart, we can conclude that Cingular/PacBell is the most successful at obtaining users who have previously used other service providers. We can also conclude that AT&T Wireless is the most successful provider of gaining first time cellular users.

Reasons for Choosing Service Provider

The results of this study have shown that customer service reputation does not play a large role in a consumer’s choice of service providers. Participants were asked to select their top three choices (from a list of nine options) for choosing their service provider. The following chart indicates that the overwhelming top three reasons consumers selected a provider were the coverage area associated with the service, the cost of the service plan, and the minutes included in the service plan. As you can see, customer service reputation does not play much of a role, if any, when consumers make decisions about service provider.

Figure10: Top Reasons for selecting service provider

Figure 12 was obtained by assigning weighted scores to reasons selected by participants based on the order of selection. Points for each reason were summed to produce the overall importance of each with respect to others.

Cellular industry players often design their marketing campaigns based on the assumption that a cell phone’s features and other associated equipment are among the top reasons consumers select a service provider. Our data contradicts this assumption. The Equip/Phone Features also received low scores as the reason for provider selection. We can guess that our sample population may not have chosen equipment or phone features as a top choice due a change in the market. It could also indicate that equipment and phone features are not as important to our demographic group as they are to the market as a whole. This finding is an interesting deviation from what previous studies have shown and may be a potential focal point for future studies.

The top reasons for selecting service providers and the observed high switching rate among providers could also be part of the bigger trend indicating that cellular consumers are only interested in the short term, immediate benefits and costs when selecting a provider instead of long term relationship with a provider. Given that the cellular market is still fairly young and that many changes are expected in the overall telecommunications market, this trend is not surprising.

3. Cellular Service Experience and Attitude

This section aimed to reveal consumer attitudes towards cellular service providers and specifically focused on post-signup customer service experience.

Since the survey was conducted to better understand customer service satisfaction levels, we screened respondents for recent interaction with their primary cell service provider’s customer service department. The screening criteria (Question 7) asked respondents if they had contacted their primary cell service provider’s customer service department for any issues. Respondents who answered positively, continued on with the survey. The following are results from these participants.

Actual customer service experience

57% of the respondents have called their customer service more than 3 times in the past year. This shows customer service is in non-trivial in the post-signup customer relationship with the CSP. Almost half (49%) of those who called customer service needed to resolve billing/invoice issues. Another 41% called because of service, coverage, or connection issues.


Figure 11: Customer service experience in the past year	Figure 12: Reasons of Contacting Customer Service

Overall Satisfaction

Results indicate that users in general are somewhat satisfied with their customer service. On a 1-5 Likert scale, participants were asked to rate their satisfaction with their CSP’s customer service in two ways. (The scale ranged from 1: Very Dissatisfied to 5: Very Satisfied.) They were first asked to rate their overall satisfaction with customer service (Question 9). Then various aspects of customer service, such as the timeliness and the knowledge level of operators, were presented for rating (Question 13).

Figure 9 displays the results of Question 13 and Question 9, users’ satisfaction levels associated with various aspects of customer service and users’ overall satisfaction levels with customer service. The bar on the right hand side of the graph represents question 9.

Fig 13: Comprehensive report for ranking of each aspects in customer service in Question 13 and Question 9.

This graph indicates that in general most users feel somewhat satisfied with various aspects of customer service. However, a large percentage of customers are very dissatisfied with the timeliness of their customer service and a large percentage are very satisfied with the polite and professional manner exhibited by customer service agents. Open-ended questions also find that timeliness of customer service is a key factor to customer service satisfaction. Additional finding is presented in the section addressing open-ended questions.

In addition to using the 1-5 Likert scale used for detailed analysis, we also checked the satisfaction measures using the common industry Percentage Scale[1]. Percentages of customers satisfied with various aspects of customer service are shown in Figure 11.

Figure 14: Percentage of Satisfied Customers for each aspect of customer service (Percentage Scale)

Again a high percentage (over 60%) of participants are satisfied with customer service representatives’ polite and professional manner while less than 40% are satisfied with both timeliness and the knowledge of representatives.

Analysis indicates that each of the six aspects of customer service is highly correlated with each other. We concluded that participants do not differentiate each of the aspects too much when considering satisfaction with customer service. After calculating the average for each participant over the six satisfaction variables from Question 13 and the overall rating from Question 9, we confirmed that the new average is highly correlated to each of the variables at a 0.01 significance level. The new average based on overall rating and the aspects’ ratings is used as a reliable final overall satisfaction measure for further analysis. The average of all respondents’ overall customer satisfaction is 3.63, a rating of somewhat satisfied. The associated correlation table can be seen in Appendix B.

Satisfaction by Monthly Bill

With the new satisfaction value computed by averaging the overall satisfaction level and those towards various aspects in customer service, we are able to conduct some further study to understand the contributing factors towards the satisfaction level. By correlating different variables, including gender, age, educational level, income level, monthly bill, monthly usage, and purpose of use, we found monthly bill had noticeable negative correlation with satisfaction level. The Pearson correlation is –0.281 reported at the significance 0.05 level.

Figure 15: Customers satisfaction level for users among different monthly bill group (Reported by average satisfaction value)

Figure 15 shows the tendency of the decrease on satisfaction level along with higher monthly bills. By looking closely into the most common monthly bill groups, we found the trend that customer satisfaction level goes down with the increase of the bill in the low and medium usage groups. In higher monthly bill group, the satisfaction varies based on their usage. In the group of which the users hold $75-99 monthly bill with 500-749 minute monthly usage, we found a contradictory high satisfaction tendency. The observation reveals that the different attitudes among varying customer groups. Given the same customer service quality, customers will expect more if they pay more. Within the groups of low and medium monthly bill/usage, the phenomenon is especially true with no exception. Thus, we can conclude that monthly bill is an important contributing factor to the satisfaction level especially for the low to medium monthly bill/usage groups.

The findings are important for CSP targeting young professional market since the low and medium monthly bill group consisted the biggest pie (30% and 41% respectively) in the young professional market.

Satisfaction by Providers

Figure 13 shows the distribution of each cellular service provider by each satisfaction level. For each CSP, participants who are satisfied with the customer service constitute highest percentage of the five rating levels. Given the varied number of participants for each CSP, comparison by CSP at each level is not reliable in this case.

Figure 16: Satisfaction Level by CSP

Comparing Satisfaction Rating of Each Provider to All Other Providers

T-test analysis produced more reliable comparisons of the means and standard deviations for each individual service provider against all other providers.

Figure 17: Means and Standard Deviations for Individual Companies and the Rest of the Providers

This graph gives us a better indication that customers four each of the four major service providers are similarly satisfied with the customer service offered by their respective providers. Though this graph does not indicate any drastic differences between the means of each provider when compared with the rest of the providers, it does offer some insight about differences between service providers. Cingular customers have a higher satisfaction level on average than the rest of the service providers while SprintPCS customers express an overall lower satisfaction than other customers. The satisfaction level of AT&T customers is almost similar to the average satisfaction among competitors’ customers, and Verizon customers also tend to be more satisfied with their customer service than others. However, only results for Cingular and SprintPCS are statistically significant.

The standard deviations are all very similar (approximately 0.90), however, the standard deviation for SprintPCS is much higher than all other providers (1.19). This may indicate that SprintPCS users have significantly varying feelings about customer satisfaction, however, the larger standard deviation may be related to the fact that the number of participants using SprintPCS service was much larger than our number of participants using all other services.

4. Open Ended Questions:

In open-ended questions, participants were asked to give what they liked the most and the least about their provider’s customer service.

Most Liked Aspects of Customer Service

Of the 35 who discussed aspects they liked the most about a service provider, 29% cited short wait on the phone or ease of access to customer service to be what they liked the most. Others found their customer service representatives to be responsive, polite, capable of resolving issues and genuinely trying to assist with the issues at hand. One participant responded: “They answered my call and questions quickly and were very pleasant throughout the conversation. They did not rush me off the phone or have to pass me on to another representative.”

Figure 18: Percentages of Positive Open Ended Responses

Least Liked Aspects of Customer Service

34 participants discussed aspects they liked the least about their customer service. 38% cited long wait or difficulty in getting through to customer service as the aspect they are most unhappy with. 40% had problems with billing and invoicing. Issues include being confused about the exact terms of their plan, receiving their bills late and having problems resolving billing discrepancies.

Figure 19: Percentages of Negative Open Ended Responses

IV. Experiment Results

Experiment results reflect positive experiences as a result of interaction with their cell phone service provider’s customer service. 11 users completed the experiment successfully. Four participants use AT&T as their service provider, four use Cingular/PacBell, two use SprintPCS, and one uses Verizon.

Given the small sample, it is difficult to generalize findings based on individual service providers. All participants encountered 4-5 events before reaching a customer service representative. This is consistent across all companies and within the companies. The time of day when the calls were made did not have any significant effect on the duration of a call. For all providers, the average call lasted 11 minutes. Only three participants, all with different providers, experienced long call duration from 15 to 20 minutes. Cingular had the lowest average call duration of 9.5 minutes while Sprint had the highest of 14 minutes. This is not conclusive again due to small sample size.

Figure 20: Total Call Duration by Provider.

Customer Responses to Standardized Questions

After having reached a customer representative, most participants were able to get satisfactory answers to the standardized questions. We examined the responses by providers for consistency in response. AT&T and Cingular were both represented by 4 participants. For both companies some inconsistencies appeared. The other providers did not have adequate representation for comparison. (See Appendix F for response data.)

Post-Experiment Satisfaction Ratings

One early intention of the design was to see if there are significant changes to satisfaction ratings before and after the interaction experiment. Participants gave the customer service overall positive rating after the experiment. Overall satisfaction with the call is 4.17 with standard deviation of 0.57. The positive rating from the experiment is even higher than findings from the survey. Having participants complete the online survey before the experiment could have affected ratings. Team members observed participants giving lower rating on the online survey. After a relatively successful call session, those same participants would rate the customer service much higher. In future studies, it is recommended that experiment participants be a separate sample from survey participants. Discovering whether participant attitudes have changed as a result of the interaction is not as significant as understanding the general overall perceived attitudes towards CSPs’ customer service. The standardized questions also need to be revisited for level of difficulty.

Figure 21: Overall Call Satisfaction by Company.

V. Discussion

These sections include more details on the statistical methods used and further interpretations from the data analysis. The survey and experiment reveal some design issues that may have affected the findings in the report. Difficulties and problems involved in this study will also be discussed. Final conclusions highlight the findings of our study.

Comparing Satisfaction Rating of Each Provider to All Other Providers

To make more reliable comparisons of customer satisfaction among service providers, we performed several t-test comparisons. Each t-test compared customer satisfaction means between an individual provider and the set of the rest of the providers. These statistics help to determine how the customer satisfaction levels for individual service providers compare to those of other providers. Below is a table displaying the t-tests values that were obtained for each of the major service providers.

Group	Observations (N)	Mean	Std. Dev	Std. Error Mean
AT&T	12	3.75	0.87	0.25
Not AT&T	51	3.73	0.98	0.14
Cingular	14	3.86	0.86	0.23
Not Cingular	49	3.69	0.98	0.14
Sprint	20	3.5	1.19	0.27
Not Sprint	43	3.84	0.81	0.12
Verizon	11	3.82	0.87	0.26
Not Verizon	52	3.71	0.98	0.14

Table 1: Means and Std. Deviations for groups

The t-test values and respective significance are shown in the table below.

Group	t	significant?
AT&T/NonAT&T	0.086	no
Cingular/NonCingular	0.604	yes
SprintPCS/NonSprintPCS	-1.147	yes
Verizon/NonVerizon	0.36	no

Table 2: t-test values for groups

Because t-tests can be used to measure groups against one another without requiring a large number of observations, the t-tests used here are much more appropriate methods by which to compare service providers with all other companies in the industry.

Customer Perception and Behavior & Their Satisfaction

Most of the respondents were somewhat satisfied with their customer service and do not consider customer service important when selecting a service provider. Though the switching rate from one service provider to another is very high, it is not conclusive whether customer service played a major role since consumers considered coverage area, cost of plan and minutes included the top reasons for selecting a provider.

There are no significant demographic variations in the attitude and behavior. Regardless of the potential of sampling or non-sampling error, we can at least conclude the patterns observed in our study are common among young professional consumer group.

We observed some contradictions from the satisfaction level questions (Question 9 and Question 13) and recommendation questions (Question 15). By the overall satisfaction level analysis presented in the Results Section, it is apparent that satisfaction levels are relatively positive. In comparing the results of questions 9 (along with question 13) and question 15, we can see that 59% of participants are at least somewhat satisfied with their cellular provider’s customer service, however, only 48% of participants would recommend their cellular provider based on the customer service.

One contributing factor to this finding maybe the phenomenon of participants’ tendencies to report satisfaction when they in fact have a neutral opinion or no opinion about a topic. One way to determine the effect of this phenomenon on our data is by comparing users’ satisfaction levels to the likelihood to recommend their service provider based on customer service alone (see next section).

Good Predictors To Customer Service Satisfaction Level

a. Likelihood to recommend the service

To determine if the relationship between the new satisfaction value and the likelihood of a user to recommend his or her service (see question 15), we correlated the two variables. From the scatter plot one clearly sees the positive relationship between the higher overall satisfaction value and the stronger likelihood of one to recommend a service provider. The correlation demonstrates that a higher customer service satisfaction level leads to a stronger likelihood to recommend the service provider to friends.

Since the correlation showed significance at a 0.05 significance level, we conclude that the satisfaction value is significantly related to whether a person would or would not recommend their service, and hence can be used as a good indicator of this variable. (See Appendix E)

b. Other possible methods to predict Customer Satisfaction

We attempted to predict customer satisfaction using multiple regression. The attempt to obtain an equation by which customer satisfaction could be obtained from other variable values was unsuccessful for two reasons. Not enough data points existed for performing multiple regression. Also problematic were the high correlations among the variables that the multiple regression was using and the dependent variable. To successfully predict customer satisfaction, future studies should include more variables in the experiment in order to test what other factors could contribute to customer service satisfaction. Many more participants are also required.

Limitations due to Design issues

Several statistical methods were used in order to draw conclusions from survey data. Many of the statistical analyses performed, however, showed insignificant results or could not be reliably performed due to data limitations.

a. Sampling problem

Our survey participants represent only a small proportion of the types of cellular phone service customers. This limits how much our results can be used to represent the feelings or behaviors of all cellular customers.

a1. The margin of error (theoretical sampling error rate)

Before starting study, we estimate a sample size of 100 respondents can give us a margin error with 10% at 95% confidence level. After gathering data, we computed margin of error again as 12.13%. (This survey was designed so that we can distinguish respondents with actual customer service experience from those without. As a result we did not take the change of the number of responses as the survey progressed into account. Instead, we only use the portion of the respondents who have actually used customer service before. ) All the question except for those with a large portion of respondents answering “not sure” are reported with a margin of error at 12.13%.

a2. Number of Data Points

A chi-square test could be used on the type of data obtained from this study in order to test differences between groups such as differences among customer satisfaction from different providers. However, the number of data points obtained for this study is not adequate to calculate reliable chi-square values to make firm conclusions. Limited data points was also problematic for obtaining reliable multiple regression results. Consequently, future studies should focus on obtaining several more data points in order to perform such data analyses.

a2. Other empirical sampling problems

Given limited time and resources and since the study is a pilot study for a larger market study, sampling was limited to a population of known individuals. Respondents consisted primarily of colleagues, friends, and family. This introduces the possibility of respondent bias and thus limits the generalizability of the study. The online survey method also limits participants to those who have access to the Internet.

Due to initial technical problems, information linking online survey participants to those who also completed the experiment was lost for enough users that comparing satisfaction ratings before and after is not feasible. Since this kind of comparison would provide more insight to how consumer attitude can be changed than insight to general attitudes and perceptions, in a full blown test the survey participants and experiment participants should be two separate groups. Aside from qualitative evaluation of customer service, other quantitative information gathered from the experiment is still valid.

b. Non-sampling Errors

By looking at a considerable percentage of respondents who answer “other” or “not sure,” we can assess non-sampling uncertainties and ignore the answers to some ill-designed questions.

Besides these non-responses, we have found other difficulties when trying to analyze our data. The high correlation between the various aspects of customer service has caused the result to be hard to interpret for individual aspects. For example, brand name and customer service reputation is highly correlated. The same concern applies to various aspects for satisfaction evaluation, though to a smaller degree. More careful selection of a set of various aspects for customer service selection could provide more meaningful results. This also placed additional limits on the multiple regression analysis.

Recommendations for Future Studies

It is recommended that a future study similar in nature and scope obtain at least 400 participants from the larger general population to allow meaningful analysis and interpretation. Researchers may also consider using paper-based survey to include those who other wise could not respond. Some questions need to be reconsidered, including the different aspects of customer service. Presentation and order of questions used in the survey study could also be improved.

Participants for the experiment should be a separate group from those who respond to the survey. Researchers may also want to reconsider some of the questions asked of customer service representatives and increase the level of difficulty with two questions. Having participants log sequence of events before reaching a live customer service representative does not provide meaningful data and is very confusing for participants. This should be eliminated from future experiments.

Conclusion

Despite anecdotal evidence to the contrary, most consumers from a sample population of mostly university students and young professionals are somewhat satisfied with their cellular service providers’ customer service. The average satisfaction level is 3.68 reported in this survey. Customer service quality does not play a perceptible role when consumers are selecting service providers. However, after signing up for with a provider, 57% of participants contacted their customer service more than 3 times in the past year. Resolving billing and invoice issues are main reasons for calling. Timeliness of and ease of access to customer service are critical to satisfaction level with customer service. High level of satisfaction does correlate with likelihood of recommending a provider to others. Monthly bill is also an important contributing factor to the satisfaction level especially for the low to medium monthly bill/usage groups, which consisted the biggest pie (30% and 41% respectively) in the young professional market.

There is no major difference in the cellular phone usage and CSP choice based on gender, age, education level, or income level reported by our sample population. The purpose of cell phone use and monthly usage and billing patterns point to the adoption of cellular phone as a basic personal communications tool that many use substantially on a regular basis. However, 61% of the participants have switched their providers in the past year alone. Contrary to previous studies, our sample population does not consider equipment or phone features as a top reason for choosing a service provider. Instead, coverage area, cost of plan and monthly minutes included with the plan are the top three considerations for provider selection.

As a result of the nature and extent of the sampling method, these results can only be generalized to a limited population.

VI. Reference

Search Engine Survey

http://www.sims.berkeley.edu/~sinha/teaching/Infosys271_2000/SearchFeatures/index.html

Reciprocal Compensation Survey

http://www.sims.berkeley.edu/~sinha/papers/ReciprocalCompensationSurvey.html

“Tricks of the trade (How to think about your research while you’re doing it).” Howard S. Becker. The University of Chicago Press, 1998.

“The craft of research.” Wayne C. Booth, Gregory G. Colomb, Joseph M. Williams. The University of Chicago Press.

“What is a margin of error?” American Statistical Association 1998.

VII. Appendix

Appendix A: Questionnaire for Online Survey

Appendix B: Experiment Kits

Appendix C: Monthly Bill And Usage Report

Appendix D: Correlation Table for Customer Satisfaction, Overall And Aspects

Appendix E: The relationship between satisfaction level and likelihood to recommend

Appendix F: Correlation Tables (attached)

Appendix G Experimental Data

[1] The Percentage Scale sums all of the very satisfied and half of the somewhat satisfied to find the percentage of customers that are ‘satisfied’. Only half (50%) of the somewhat satisfied customers are considered because their satisfaction level is not as strong as the very satisfied customers (i.e. that are also ‘somewhat dissatisfied’). While we did not use this statistic in further calculations, it is a commonly used measurement across industries and is an alternate way of representing data that our industry readers may like to see.