Presenters:
Linda Stinson, BLS
Kerry Levin, WESTAT
Angie Becher, WESTAT
Barbara Forsyth, WESTAT

Clyde Tucker discussed results from tests of different methodologies for collecting data about time use. These are survey methodologies, no economics. Also, the presenters will discuss diary procedures for collecting time-use data.

Linda Stinson is a psychologist, trained to look at behavior. Time-use data can be used for many research purposes. WESTAT used the time-use approach to measure the frequency and duration of non-market work. Below is a summary of some problems and solutions.

Their focus is on incorporating the value of non-market work into our SNA. Non-market work includes activities such as housework, home maintenance, care of children and elders, food production, and volunteer work.

The methodological options for collecting time-use data include direct observation, random time sampling (beepers), questionnaires and time diaries. They chose the latter. They began with cognitive interviews and a pilot test.

Measurement difficulties:

1. Detailed recollection — how to get people to give the desired amount of detail
2. Lack of "time stamps"
3. Retrieving memories of routine or automatic activities during the day

Some cognitive interview findings:

1. Unusual events or patterns which were different from the daily routine were more memorable.
2. Daily routines served as timetables and provided mental markers to help respondents remember when or in what order events occurred.
3. Appointments and time commitments helped people reconstruct time use.
4. Some people have difficulty reporting exact starting and stopping times of activities. They reported durations instead.
5. Interviewer probes seemed to help participants remember events.

Some commonly forgotten activities:

1. Housekeeping tasks such as making the beds, cleaning up after meals
2. Purchasing gasoline
3. Talking on the telephone

The field test’s experimental conditions:

1. Both the Standard protocol and Enhanced protocol had 500 cases in each (n=500)
2. In the Standard version, they asked, "Where were you?" and "Who else was with you?"
3. Half-way through the study, they realized they did not know if a person was getting paid for the activity (e.g., baby-sitting a grandchild), so they added a question on that topic.
4. In the Enhanced version, they asked, "Were you doing anything else during that time?" and "If working, did you stop what you were doing to do something for yourself or someone else that was not related to your work?"
5. Child-care probe in the Enhanced version: "Are there any activities with children for which you were not paid that I should add to what you have already told me?" Asked participants to think about it.

The call schedule for the designated day was:
Round 1 — four calls
Round two — four calls.
Round three — average 5.3 calls

To measure data quality, they verified answers by comparing results of two related probes.

Measures of feasibility:

1. Length of interviews
2. Number of calls
3. Cost per case, i.e. survey processing time

Her colleagues will discuss the field test’s results.

Kerry Levin talked about the administrative details and costs, and assessment of feasibility.

They examined differences in two versions of the telephone interview. The main difference was that the Enhanced version focused on more probes to capture non-market activities. It was a multi-dimensional feasibility study. Practicality-wise, it was more qualitative. It used a "designated day" design as the mode of implementation and did interviewer training. It was productive, in that it did capture interviewer effort and time, an assessment of the number of codeable activities, and it was assessed quantitatively.

Assessing Productivity: It took about three hours to complete a case (longer than usual). To understand why it take so long, they looked at:

1. Interviewer survey processing time (a paper/pencil instrument)
   Interview rushed
   Shorthand used
   Had to go back and recapture information and clean-up
2. Interviewer down time
   A serious problem
   Spent time waiting to call a case. They could not do anything after making four calls per case.
3. Retraining time for unusual interviews
   They monitored every interview, and retrained whenever they found something problematic.

The actual "real" time on the telephone was longer by two minutes with the Enhanced version, but the total time was the same. Interview completion time decreased with each round. Interviewers became more comfortable with the time. The additional time on the phone with the Enhanced version was very important. It resulted in more reporting of non-market activities. It took an average of 7.24 calls to complete the Standard version and 6.95 calls to complete the Enhanced version. Both versions were similarly productive, in that each minute of coding time resulted in slightly more than one codeable activity.

Assessing Practicality:

1. With the "designated day" design, the proportion of interviews completed by round decreased dramatically between Rounds 1 and 2, with a slight increase between Rounds 2 and 3.
2. The "designated day" design did reduce productivity with a limit on the number of calls and too much down time. They recommend using a "scheduler" to reduce the down time and boredom, and using bigger sample sizes. (Remember, this was a feasibility study.)

In assessing feasibility, there are two issues. One is the complexity of the interviewing task. It requires considering other modes of administration. They recommend using CATI, although it could have some drawbacks, such as it being harder to backtrack. Second is the issue of improving interviewer training. They recommend:

Longer training
Practice with "real" cases
Practice with probe techniques
Practice coding data
Special training for "refusal converters"
Longer field periods
Separate "reluctance" training.

The trade-off between the survey methods depends on whether the additional information is worth the extra time (the 2 minutes). They believe that capturing non-market time from the Enhanced instrument was worth it. The cost for obtaining additional information is minimal.

Angie Becher: Identifying Non-market Work: A Look at Coding Issues

Accurate and reliable coding are critical. Some problems and solutions with how they coded:

1. They used established BLS activity codes. (See her paper, "Identifying Non-market Work: A Look at Coding Issues," for details.)
2. The focus was non-market work, so those types of activities were coded in more detail than leisure activities.
3. Three levels of coding were available to deal with various levels of detail provided by respondents.
4. The imbedded child-care code captures "passive" child care which occurs simultaneously with another activity.

Overall, the coding system was good, but there were some difficulties. Some problems encountered:

1. Often, adequate detail was not provided to allow coding at the most specific level, e.g., eating can take place at home, at work, or in a social setting. Standard probes were used to get additional information. Standard probes:

   Where were you? (e.g., home, work, social occasion)
   Who were you with? (e.g., household adult, household child, non-household member)
   Were you paid for the activity? (to help identify paid vs. unpaid work)
   What was the purpose of the activity? (e.g., work, household administration, leisure)

2. Disentangling simultaneous activities: The duration of the two activities may not be the same, but they were coded as though they were. (Coders can use Multiple Activity error code to correct.)
3. Reporting travel time: Respondents often included travel time with another activity (e.g., went to the store, went to work). Another complication was when a respondent did multiple errands in one trip.

A well-trained coding staff and constant availability of supervisors for answering questions and doing consistency checks is important. A long-term continuous staff is desirable for these reasons.

Barbara Forsyth: Assessing Data Quality: How well did this approach work?

Key features of the Standard questionnaire:

Report main activities
Location and participant probes
Survey participation cooperation rates were 57.3%.

Key features of the Enhanced questionnaire:

Report consecutive activities and times
Additional activity probes (We expected probes in the Enhanced questionnaire to elicit the reporting of more activities)
Survey participation cooperation rates were 54.2%.

Below are some reporting details:

Mean Number of Reported Time Intervals for the Standard version was 22.99, and for the Enhanced 17.39. This difference is not in the expected direction.

Mean Number of Codeable Activities per day for the Standard version was 21.8, and for the Enhanced 24.7. This is a significant difference in the expected direction.

Mean Number of Activities by the two Versions and four Categories (see her paper, "Assessing Data Quality: How well did our approach work?" for details)

Market/Standard version had more activities
Non-market/Enhanced version had more activities
Volunteer — about the same for each
Social/Personal — Enhanced version had more activities

Mean Activity Duration (in minutes) by each Version and Category:

Market — no difference between versions
Non-market — Enhanced version greater with nearly three hours more per day
Volunteer — no difference
Social/Personal — Enhanced version was greater with over 6 hours more per day.

The average number of data hours collected per day with the Enhanced questionnaire was 40, and with the Standard questionnaire it was 31. The Standard questionnaire was constrained to 24 hours, but they used multiple activity codes and assigned time to both codes because they wanted to track errors. The Enhanced questionnaire allowed simultaneously occurring activities.

Mean Number and Duration of Activities by each Version and Type:

On average, respondents reported less initial activities than when asked, "What did you do next?"

On the Enhanced questionnaire, simultaneous activities contributed four hours to the day. Simultaneous activities were not due to time allocation strategy. This information was usually lost with the Standard procedures.

Non-Market Work: Mean Activity Durations (See paper for details.)
Allowing simultaneous activities added an additional hour per day.

Percentage of Time Reported by Version and Category showed that time distributions under the Standard and Enhanced questionnaires differed. The portion working was 17% of total time in the Standard version and 12% of total time in the Enhanced. Non-market activities took 16% of total time in the Standard and 20% in the Enhanced. Volunteer and Social/Personal activities did not differ. They got a larger amount of non-market work reported with the Enhanced questionnaires.

Data Quality Conclusions:

The Enhanced questionnaire increases reports and accounts for more time.
It is particularly effective for reports of non-market work.
Simultaneous activities are an important factor.
The challenge is regarding the methods for valuing simultaneous activities.

They believe the Enhanced questionnaire is better at collecting information about non-market work. They recommend using probes to capture non-market work.

Nancy Mathiowetz: The measurement of non-market work is difficult. These are some sources of error in the process and agendas related to them.

Errors can be due to:

1. The questionnaire
2. Respondents (response refusal, omission of some answers, etc.)
3. Interviewers
4. Post-data processing
5. Others: coverage, non-response, sample design.

Cognitive interviews generated some potential ways of improving quality and completeness of the information received. One concern is that people report their normal time use and not for the specific day in question. Cues help respondents to recall events.

The Enhanced questionnaire did not include some of the techniques which the cognitive interviews would suggest. The Enhanced questionnaire also excludes some features of the first questionnaire, such as cues from the environmental context.

She recommends using more cues, and allowing people to report durations in ways they are more comfortable with, rather than limiting them to a specific way of reporting duration.

With regard to the interviewers, probing skills are difficult to master. With an open-ended interview like this, results can be greatly affected by the interviewer.

Regarding data processing, if information is critical for coding, then questions should be included in probes, e.g., coders needed to know why someone had been sewing. If we want detailed coding, we should build that detail into the questionnaire. Otherwise, we should not expect that level of detail from coders.

Other comments and recommendations:

1. Mode of data collection: By doing phone interviews, a coverage error is generated since 5-7% of the population has no phone. Interviewing via telephone also limits communication.
2. Recommend use of computer-prompted interviews.
3. Data quality — do we want to allow people to have more than 24 hours accounted for in a day?

David Paton: Cognitive work was helpful in aiding respondents in remembering the details of their day.

At Statistics Canada in a recent test of a hybrid survey, they used a combination of CATI and paper/pencil which needed less calls. They got more activities than WESTAT did, and limited the accounting to 24 hours.

Some Coding Issues:

1. We can no longer use paper/pencil. He recommends using CATI and doing the coding in the field. He does not know if this procedure will cause problems. For example, if the interviewer chooses the wrong screen, it will cause some difficulties because it is hard to backtrack.
2. Interviewers gave positive feedback about this new method, with a few negative comments about some of the ordering of the codes, such as problems with meals and conversations.
3. It would be possible to have another interviewer in the field taking responses down on paper and then compare the results.
4. We were not successful in past attempts to use secondary activity questions to get at child-care activities. We have added a child-care diary portion (like WESTAT’s) and have collected more child-care information . We find that asking about time spent looking after children, as opposed to being with children (like WESTAT), gets at more child care.
5. We found it very difficult to get information from both spouses. There was a drop-off in recall with a weekend’s activities unless the diary was seven-day. We concluded that it was best to try and only get information from one spouse.

Group Discussion and Comments

Luisella Goldschmidt-Clermont, France: With respect to coding of activities as paid work, unpaid work, or leisure, are these mutually exclusive? Do we want to ask if the respondent considers the activity leisure even if it is paid work? Leisure is a very subjective concept, and perhaps one of the worst words/concepts used in time-use analyses.

In Europe, we would find socio-economic differences with a telephone interview. There may be cultural differences in the ability to answer questions on the telephone. Some people are more comfortable on the phone than others.

Becher: We used a hierarchical scheme.

Stinson: We have not gotten at all the differences of leisure/paid work.

Forsyth: We did not ask whether it was paid or unpaid leisure, only if it was paid or unpaid.

Sarah Fenstermaker, UC at Santa Barbara: Question directed to Paton: Did you do any operationalizing about the word "important"? What did you mean by that? How do you define which is the most important activity?

Paton: We did not do anything special to interpret what "important" meant.

Fenstermaker: Asked how important to the household is her work.

Thomas Juster, University of Michigan: We would prefer to have 24 hours in a day. There is no such thing as a simultaneous activity. We do need to ask about the principle activity and secondary activities, but it is not OK to assign the whole time period to all activities. We should somehow split the time among the activities, and have interviewers sort out the time allocation during the interview. He is not sure how to measure time-diary quality. Recording more activities does not necessarily indicate better quality data.

Stinson: Passive activities can truly be done simultaneously with other activities, e.g.,. child care, listening for the phone. I think it is an important finding for us that there was layering of activities.

Forsyth: There is a difference between respondents’ and economists’ concept of time.

Jan van Tongeren, United Nations: She is unclear what is a paid-for and a not-paid-for activity. Which of the non-market activities should be added to GDP, and which are already embedded in the market? Which non-market activities are carried out as inputs into market activities? These should not be included as outputs.

Session ended at scheduled time.

back to top