An Evaluation of Retail Outlets as Part of a Community Prevention Trial to Reduce Sales of Harmful Legal Products to Youth

Evaluating the Retail Component Using Retailer Surveys and Youth Purchases

Table 1 presents the operational definition of each of the targeted intermediate variables, the data source, and the schedule and frequency of data collection activity for both the telephone survey of retailers and the youth purchase attempts.

The telephone survey of retailers utilized a survey instrument focused on measuring a number of constructs including: (a) perceived ability of youth to purchase legal products that can be abused; (b) perceived ability of youth to shoplift legal products that can be abused; (c) knowledge about legal products that youth can use to get high; (d) levels of concern about legal products that can be inhaled or otherwise abused by youth to get high; (e) feelings of responsibility about preventing youth from obtaining legal products that can be abused; (f) policies and procedures related to the availability of legal products that can be abused by youth; and (g) actions taken by retailers to restrict availability to youth. The telephone survey of retailers was fielded from late October to early December, 2005. Respondents for the survey were owners and managers of retail establishments that sold legal products that youth could inhale or ingest to get high. Across the four communities a total of 35 businesses were included in the sampling frame and interviews were completed with 29, yielding a response rate (AAPOR Response Rate 1) of 83%. Interviews were completed with a wide variety of different types of retail establishments, including craft stores, office supply stores, pharmacies, gas stations, grocery stores, and “big box” supercenters. Although the retail establishments included in the sample had 25 employees on average, the number of employees in these establishments ranged from a minimum of three to a maximum of 125.

There are no previously existing precedents for conducting youth purchase attempts of legal potentially harmful products from retail outlets. In the HELP prevention project, youth purchases were used to determine the rate at which early teen youth buyers (between ages 12 and 15) were able to purchase inhalants and other harmful legal products in retail establishments in the four communities participating in the project. Youth purchasers were male youth volunteers who were between the ages of 12 and 15 in each of the experimental communities. Only places that sold one or more of the targeted products were included in the purchase attempts—this included a total of 45 businesses across the target communities. The types of businesses targeted included supermarkets, small “Mom and Pop” groceries, convenience stores (e.g., 7-Eleven), pharmacies, hardware stores, hobby shops, gas stations, and paint stores. Youth purchasers attempted to purchase five types of products in the following quantities:

1. Gasoline—½ gallon of gasoline

2. Spray paint—12 oz. can of Krylon or Rustoleum (any color)

3. Aerosol whip cream—7 oz. can of Redi-Whip

4. Hobby glue (or other glue such as Super Glue that has the chemical tolulene in it)—(2) 1 oz. tubes

5. Cough syrup—(2) 4 oz. bottles of Robitussin DM

The quantities specified were such that, if the retailer was concerned about potential abuse of the product, the amount should stimulate the attention of the clerk. For example, a small amount of gasoline would be unusual since the amount (and the age of the purchaser) is unlikely to be practically useful for legal application but is sufficient to be abused.

Youth purchase attempts began in July 2005, and were completed by September 30, 2005. A total of 95 purchase attempts were completed across the four participating communities. Local community data coordinators trained three to four youth purchasers in each community to conduct the purchase attempts after providing prevention materials to both the youth and their parents. The community data coordinators also accompanied the youth purchasers in each community during each set of purchase attempts. The coordinators provided transportation if needed, and directed each youth purchaser to the store to be visited and told the youth purchaser the specific product (brand, size, and quantity) to purchase on that visit. Purchase attempts were made a variety of day and evening times and also were conducted on a variety of weekday and weekend days. Each youth purchaser was provided with sufficient money only to purchase the product specified. The coordinator recorded the results of the purchase attempt. The Appendix contains a sample Product Purchase Outcome form, along with the coding rules that the community data collectors used to categorize the outcomes of each purchase attempt.

Based on preliminary interviews conducted in each community and on published research on inhalants, three purchase attempts were completed with each retail establishment that sold a targeted product in each community. Some retail establishments had multiple businesses within them and were considered two businesses for the purposes of the purchase attempts (i.e., a grocery store that also has a gas station attached or in the parking lot). These establishments were listed twice, as separate businesses, and three purchase attempts were completed with each of the businesses.

As can be expected, implementing the youth purchase attempt involved overcoming some difficult practical and logistical challenges. The specific challenges faced by the project in implementing youth purchase attempts as a data collection strategy included:

1. Conducting an inventory of all businesses in each community that potentially carried legal but abusable products. Although gas stations, office supply stores, grocery stores, pharmacies, and craft stores could be presumed to carry one or more of the targeted products, implementing the purchase attempts with rigor required the project team to be certain that the business did indeed carry one or more the targeted products. As a result, community data collectors hired as part of the project visited each business in all four communities and determined which of the targeted products was carried by each business, along with the brands and sizes that were carried by the business. One key learning from this inventory was that a few of the retail establishments in each community that the research team expected would carry one or more of the targeted products did not, in fact, stock any of the products. Those businesses were subsequently excluded from the sample.

2. Targeting products to be purchased and determining purchase amounts. As described above, conducting the youth purchase attempts in a rigorous fashion required the project team first to understand the legal but harmful products carried by each targeted retail establishment and then to specify the type of product, the brand, size, and quantity to be purchased at that retailer. As described above, the project team also specified the quantity to be purchased—setting those amounts in such a way that the purchase would be more likely to appear suspicious to retail staff members. In a few cases, unusual situations arose as purchase attempts were being conducted (for example, a pharmacy was out of Robitussin and delays in receiving new orders of it prevented purchase attempts from being conducted). Even when businesses regularly stocked the targeted products, rural communities in Alaska, which rely on the sometimes unreliable, federally subsidized Bypass Mail for stocking goods, were sometimes out of stock of certain items for weeks. Community data collectors were trained to contact the research team to review these situations. The research team handled these unusual instances on a case-by-case basis—in some instances, youth purchasers were permitted to purchase another type of product or another brand or size of the originally specified product, while in other instances the community data collectors were instructed not to make additional purchase attempts at the business.

3. Human subjects review and approval. Protecting the safety and rights of the youth purchasers was of paramount importance to the research team and a key step prior to the implementation of the youth purchase attempts was obtaining review of the component by Pacific Institute for Research and Evaluation (PIRE)��s Institutional Review Board (IRB) and approval for activities to begin. The IRB’s concerns were focused on two areas: first, a concern that purchasing the targeted products would teach the youth what products could be abused and in doing so, make them more likely to abuse the targeted products. A second and related concern was the benefit of the research to the youth purchasers.

   To ensure that the youth purchasers were not harmed during the purchase attempts, community data collectors informed the youth during their training that they would be purchasing products such as gasoline, cough syrup, and aerosol whip cream that youth sometimes have used to get high. The community data collector also distributed prevention materials provided by the research team and conducted an interactive prevention education session that focused on the dangers of inhaling and misusing legal products to get high. No information was provided to the youth on how the targeted products might be abused to get high.

   Written parental consent was obtained prior to a youth participating in the purchase attempts. Youth also were required to sign a Youth Product Use Contract that confirmed that they had been told about the risks and dangers of misusing household products. As part of this contract, the youth purchasers agreed that they would not use any of the products purchased for the research study and that they would return all products to the community data collector after each purchase attempt had been completed.

   The IRB also required the youth purchase attempts to have a direct benefit to the youth. As a result, the research team adopted a strategy in which each youth purchaser was provided with gift certificates to local businesses (such as McDonald’s, Blockbuster, or similar local businesses) to thank him for assisting with the purchase attempts. The exact amount of the gift certificates depended on the number of purchase attempts the youth participated in but had a minimum value of ten dollars.

4. Training community data collectors. Implementing the youth purchase attempts with fidelity and rigor required that the community data collector be fully trained on how to inventory local businesses, identify youth to participate, administer the youth purchase attempt protocol, and keep records of both the outcomes of the purchase attempts and the money used to make the purchases. The community data collectors were trained by members of the research team during a 3-day intensive in-person training seminar conducted in June 2005, in Anchorage (a central point to meet for both the research team and the community data collectors). In addition, the community data collectors were provided with a detailed reference manual created for the project that included background information on the project, written instructions for carrying out each of their activities, and samples of all data collection instruments.

5. Identifying youth volunteers to participate. To ensure that the youth purchase attempts were realistic, the research team needed to develop criteria to guide the community data collectors in selecting the youth volunteers. To be eligible to participate in the youth purchase attempts, youth had to be males between 12 and 15 years of age; had to have written parental consent to participate; had to complete and sign a Youth Product Use Contract; and had to not be directly related to store owners, managers, or employees. Ideally, the research team would have liked to require that the youth purchasers not be known at the retail establishment being targeted; however, the very small size of the participating communities made this criterion unworkable.

6. Training youth volunteers to make purchase attempts. Implementing the youth purchase attempts with fidelity and rigor required also that the youth volunteers be trained to carry out the purchase attempts. The youth purchasers in each community were trained by the community data collector in in-person sessions that lasted two hours on average. Topics covered in the training included: how to dress; products being targeted for purchase; how to make the purchase attempt; how to respond to questions from retail staff; how to handle situations in which friends and/or relatives might be present in the retail establishment; and what to do after the purchase attempt is complete and the youth exits the retail establishment. The training session for the youth purchasers also included a discussion facilitated by the community data collector on the danger of inhalant use (see above).

7. Need for scripted answers for youth to respond to questions from retail staff. After the first few purchase attempts were completed, the youth volunteers asked for guidance on how to respond to questions from retail staff about the purchase attempt. The research staff developed a list of reasons for making the purchase (such as, “I am making a fort” as a reason for purchasing spray paint or “we are having a party” as a reason for purchasing aerosol whip cream). These reasons were designed to be plausible in Alaskan culture and were developed in consultation with community data collectors in each community. Common scripted answers follow below in Table 2.

Youth were trained to use their own words but to stick to the “story” in the scripted answers. Community data collectors were encouraged to conduct a practice session with the youth purchasers in which they practiced the scripted responses. In a few instances, the youth volunteers or their parents were uneasy using the reasons listed above because they were not truthful. The research team asked those youth volunteers not to use the scripted response that they were uncomfortable with or to reply with something generic. In one largely Alaska Native community, the data coordinator, youth, and parents felt strongly that being asked to be untruthful was a significant problem. The data coordinator requested that the volunteers be able to reply, “My friend asked me to buy this” if asked why they were purchasing these products. A universal and simple response, the data coordinator reported great success with this response that was based upon the friendship she developed with the volunteers.

Retailer Survey

Results from the telephone survey of retailers are shown below. Two key outcome constructs measured how easy or hard it would be for youth to purchase and shoplift thirteen different legal products that youth can abuse to get high. Table 3 presents mean scores for retail respondents’ ratings of how easy or hard it was for youth to purchase the five products specifically targeted by the retail intervention. For each product, respondents were asked to rate whether it would be very easy, somewhat easy, somewhat difficult, or very difficult for youth to purchase that product.

Mean scores for the full sample of cases approached 3.0 (a rating of “somewhat easy”) with the exception of respondents’ ratings of the ease of purchasing aerosol sprays, which had a mean of 3.25.

Table 4 presents mean scores for retail respondents’ ratings of how easy or hard it was for youth to shoplift the five products specifically targeted by the retail intervention.

As can be expected, the results in Table 4 show that retailers generally perceived that it was more difficult for youth to obtain legal but harmful products through shoplifting than through purchasing the products. Gasoline was rated on average as very difficult for youth to shoplift, while spray paint, aerosols, and cough syrup were rated on average as being between somewhat difficult and somewhat easy to shoplift. One notable difference between the results in Tables 3 and 4 is that retailers on average rated it to be easier for youth to shoplift hobby glue or super glue than to purchase it. The results in Tables 4 and 5 also demonstrate that most retailers in the sample felt that although it was possible for youth to purchase or shoplift legal but harmful products, most respondents perceived that it would not be very easy for youth to obtain these products.

One important secondary outcome measured in the retailer survey instrument was retailer knowledge of legal products that youth can use to get high. The construct was measured with a series of 10 “knowledge test” questions that asked retailers whether specific legal products could produce a high if they were abused by youth. Table 5 presents the overall findings about retailers’ level of knowledge about legal products that youth can use to get high.

Table 5 shows that the retail staff in the four participating communities had fairly high levels of knowledge about legal products that youth could abuse to get high.

The retailer survey also measured retailers’ levels of concern about legal products that can be inhaled or otherwise abused by youth to get high. Respondents in the survey were asked how concerned they were about the abuse of legal products by youth between 11 and 15 years of age. Table 6 presents the overall results from the four participating communities summarizing retailers’ level of concern about legal products that youth can abuse to get high.

Table 6 shows that the vast majority of retailer respondents in each community reported that they were very concerned about legal products that youth could abuse to get high. It should be noted, however, that these responses may be impacted by social desirability biases that would motivate respondents to overstate their actual levels of concern.

The evaluation team hypothesized that levels of concern about legal but harmful products might be higher among retail managers and owners than among other staff. Although project budget constraints precluded including retail staff members in the sample for the telephone survey of retailers, questions were added to the instrument asking retail owners and managers to rate the level of concern by other employees at the establishment about legal products that youth could abuse to get high. Table 7 presents the overall results from the four participating communities summarizing the perceived level of concern of other retail staff about legal products that youth can abuse to get high.

The results in Table 7 show that respondents indicated that the vast majority of other staff in retail establishments was at least somewhat concerned about legal products that could be abused by youth to get high. One difference in the distributions of the results in Tables 6 and 7 bears an additional note: survey respondents were more likely to rate their own level of concern as “very concerned,” and were more likely to rate the level of concern of other employees as “somewhat concerned.” This was the pattern expected by the evaluation team.

Respondents also were asked to rate whether they felt that retail establishments were responsible for preventing youth from obtaining harmful legal products. Table 8 presents the results from this question.

Table 9 presents descriptive results from a similar question that asked respondents whether retail establishments were responsible for asking questions when youth purchasers attempt to purchase legal products that can be abused.

The results in Table 9 show that despite high levels of reported concern by respondents and other retailers, a significant proportion of retail respondents did not feel that retail establishments were responsible for preventing youth from obtaining legal products that could be abused. However, Table 10 demonstrates that almost 82% of respondents felt that retail establishments were responsible for asking questions when youth attempted to purchase legal but harmful products.

The retailer survey instrument included questions measuring whether retailers had policies and procedures in place related to the availability of legal products that can be abused by youth. Of the 29 businesses included in the baseline retail survey data, only 9 (31.0%) reported that they had a policy in place related to sales of legal but harmful products to youth. Although 8 of the 9 (88.9%) businesses with policies in place reported that (a) they shared the policy formally with employees during training sessions or orientations and (b) that they monitored compliance with the policy, only one of the nine businesses reported that the policy had been put into writing and posted publicly in the business. Because of the small proportion of businesses in the sample reporting that they had a policy on the sale of legal but harmful products to youth, community-specific data is not reported here.

Finally, the retailer survey instrument included questions measuring whether the retailer had taken any actions (such as asking questions, using product placement, or refusing sales) to restrict the availability of harmful but legal products to youth. Table 10 summarizes the strategies used by retailers at baseline to limit sales of inhalants and other legal but harmful products to youth.

The most striking aspect of the findings presented in Table 10 is that retailers reported using each action or strategy, and a substantial minority of retailers reported using product placement, asking questions, and refusing sales as mechanisms to limit access by youth to legal products that can be abused to get high.

Youth Purchase Attempts

A total of 95 purchase attempts were made across the four communities that participated in the project. Results from the youth purchase attempts are summarized in Table 11.

In 87 of the purchase attempts (92%), the youth purchaser was able to purchase the targeted product in the specified quantity with no questions from the retailer. In 8 cases (8.0%), the youth purchaser was allowed to purchase the product, but only after answering questions from a retail staff member about the intended use for the product. Questions came from a variety of different types of businesses—including three grocery stores, two craft stores, an auto parts store, a gas station, and a pharmacy. All of the eight questions involved clerks asking the youth purchasers about the intended use of the products. In no case was the purchase of the inhalant or other harmful product refused, and in all eight cases the product was sold to the youth purchaser as soon as they answered the clerk’s questions.

Seven of the eight businesses that asked questions of the youth purchasers also completed the retailer survey interview; of note, there were no systematic differences between these seven businesses and the other 29 businesses included in the sample for the retailer survey. However, three of the seven businesses reported having policies that were designed to prevent youth from obtaining harmful legal products, and in one of the businesses the policy was publicly posted in view of both customers and staff. All three of these businesses reported that they monitored staff compliance with the policy.