The Work Speaks—Part 2: Sample-Minded Research

In April, Conservation Biology published a comment authored by Christopher A. Lepczyk, Nico Dauphiné, David M. Bird, Sheila Conant, Robert J. Cooper, David C. Duffy, Pamela Jo Hatley, Peter P. Marra, Elizabeth Stone, and Stanley A. Temple. In it, the authors “applaud the recent essay by Longcore et al. (2009) in raising the awareness about trap-neuter-return (TNR) to the conservation community,” [1] and puzzle at the lack of TNR opposition among the larger scientific community:

“…it may be that conservation biologists and wildlife ecologists believe the issue of feral cats has already been studied enough and that the work speaks for itself, suggesting that no further research is needed.”

In fact, “the work”—taken as a whole—is neither as rigorous nor as conclusive as Lepczyk et al. suggest. And far too much of it is plagued by exaggeration, misrepresentations, errors, and obvious bias. In my previous post, I presented examples of researchers “reinterpreting” the work of others to better fit their own arguments. For the next few posts, I’ll focus on some of the major flaws in the feral cat/TNR research itself—beginning with the reliance, by some, on small sample sizes.

Size Does Matter
There are all kinds of reasons for small sample sizes, perhaps the most common being limited resources (e.g., time, funding, etc.). And they are often a fact of life in real-world research, where investigators have less control over conditions than they might in a laboratory environment. Studies employing small sample sizes are not without value; indeed, they often serve as useful pilot studies for future, more comprehensive, work. They do become problematic, though, when broad conclusions are drawn from their results. Below are three (among many!) examples of such studies.

Impressive Estimates
In “Free-Ranging Domestic Cat Predation on Native Vertebrates in Rural and Urban Virginia,” [2] published in 1992, the authors estimated that the state’s 1,048,704 cats were killing between 3,146,112 and 26,217,600 songbirds each year. “This number,” they note, “is certainly inaccurate to some degree, although the estimates are impressive.” [2] Impressive? I suppose. Maybe incredible is more fitting—since the study from which they were derived included exactly five cats, four “urban” and one “rural.”

Mitchell and Beck acknowledged “the limitations of extrapolation to large areas from relatively small data sets such as ours,” suggesting that their work was intended to provoke future “careful and detailed studies that can reveal truer estimates of the impact of this introduced species.” Hawkins [3] and Dauphiné and Cooper [4], however, seem to take them at their word, regardless of any disclaimers.

Many Cats, Multiple Seasons
In a recent study on Catalina Island, the researchers “examined the home-range behavior and movements of sterilized and intact radiocollared feral cats living in the interior” [5] of the island. Although Guttilla and Stapp concede that “sample sizes, especially for males, were relatively low” despite having “tracked many cats across multiple seasons,” they nevertheless come to some rather dramatic conclusions. Among them: “sterilization likely would not reduce the impact of feral cats on native prey.” [5]

So what do the authors mean by many and multiple? Actually, there were just 27 cats in the study (of an estimated 614–732 on the island). “Four cats were tracked during all four seasons, 9 cats were tracked for three consecutive seasons, 4 cats were tracked for 2 consecutive seasons, and the remaining cats were tracked for 1 season.” [5] And these numbers were effectively cut in half, because the researchers were comparing sterilized and non-sterilized cats. At best, this is a pilot study—though it’s already morphed into something more substantial in the mainstream media.

Myth vs. Math
In their 2004 study, “Ecological Impact of Inside/Outside House Cats Around a Suburban Nature Preserve,” Kays and DeWan observed hunting cats, concluding that their kill rate (13%) is “3.3 times greater than the rate estimated from prey brought home.” [6] Not surprisingly, this figure has been used as an instant multiplier (much in the same way William George’s work has been misused) for researchers interested in “correcting” (inflating?) prey numbers. [4, 7-11]

But this ratio, 3.3, hinges on the hunting behaviors of just 24 cats—12 that returned prey home, and another 12 (11 pets and 1 feral) that were observed hunting for a total of 181 hours (anywhere from 4.8–46.5 hours per cat). It’s interesting to note that the cat observed the most (46.5 hours) was only a year old—the youngest of the 12 observed, and likely the most active hunter. This factor alone could have had a significant influence on the outcome of the study.

Also, as several studies have shown [7,8,12,13], the distribution of prey catches tends to be highly skewed (many cats catch few/no prey, while a few catch a lot). In other words, the distribution is not the familiar bell curve at all—making it inappropriate to use a simple average for calculating estimations (a topic I’ll address in detail later). What’s more, with only 12 cats being monitored, how can we be sure their behaviors accurately represent any real distribution at all?

But the key to their calculation is the average time spent outdoors. This, too, tends to be a highly skewed distribution [14, 15], although—curiously—Kays and DeWan’s data suggest otherwise. By way of example, a 2003 survey conducted by Clancy, Moore, and Bertone [15] revealed that nearly half of the cats with outdoor access were outside for two or fewer hours a day. And 29% were outdoors for less than an hour each day. A survey conducted by the American Bird Conservancy revealed similar behavior, reporting that “35% keep their cats indoors all of the time” and “31% keep them indoors mostly with some outside access.” [14]

Kays and DeWan’s average of 8.35 hours/day, then, seems rather out of line with other studies. This, in addition to a number of unknowns (e.g., influence of time of day/night on hunting success, actual time spent hunting by each cat, etc.) raises serious questions about their conclusions.

By way of comparison, using an average of 2.5 hours/day (which is not out of line with the surveys described above) would yield a ratio of 1:1. In other words, no difference between predation rates predicted by actual hunting observation and those predicted by way of prey returned home. Which is not to say that I agree with Kays and DeWan’s underlying methods—we don’t know the possible effects of seasonal variation, for example, or differences in habitat. I’m only pointing out how sensitive this one factor—with its enormous consequences—is to the amount of time cats actually spend outdoors (and, just to introduce one more complication: I’d be very surprised if the amount of outdoor time cats spend hunting is normally distributed; it, too, is probably skewed).

Ironically, while the authors express disappointment that “biologists have rarely sampled both cat and prey populations in such a way that direct effects on prey populations can be shown,” [6] they seem to have had no misgivings about how their work—suffering from its own sampling issues—might be used to misrepresent those same effects.

*     *     *

Next, I’ll discuss the difference between compensatory and additive predation, and how that affects predictions of feral cat impacts on wildlife.

References
1. Lepczyk, C.A., et al., “What Conservation Biologists Can Do to Counter Trap-Neuter-Return: Response to Longcore et al.” Conservation Biology. 2010. 24(2): p. 627-629.

2. Mitchell, J.C. and Beck, R.A., “Free-Ranging Domestic Cat Predation on Native Vertebrates in Rural and Urban Virginia.” Virginia Journal of Science. 1992. 43(1B): p. 197–207.

3. Hawkins, C.C., Impact of a subsidized exotic predator on native biota: Effect of house cats (Felis catus) on California birds and rodents. 1998. PhD Dissertation, Texas A&M University.

4. Dauphiné, N. and Cooper, R.J., Impacts of Free-ranging Domestic Cats (Felis catus) on birds in the United States: A review of recent research with conservation and management recommendations, in Fourth International Partners in Flight Conference: Tundra to Tropics. 2010. p. 205–219

5. Guttilla, D.A. and Stapp, P., “Effects of sterilization on movements of feral cats at a wildland-urban interface.” Journal of Mammalogy. 2010. 91(2): p. 482-489.

6. Kays, R.W. and DeWan, A.A., “Ecological impact of inside/outside house cats around a suburban nature preserve.” Animal Conservation. 2004. 7(3): p. 273-283.

7. Baker, P.J., et al., “Impact of predation by domestic cats Felis catus in an urban area.” Mammal Review. 2005. 35(3/4): p. 302-312.

8. Baker, P.J., et al., “Cats about town: is predation by free-ranging pet cats Felis catus likely to affect urban bird populations? Ibis. 2008. 150: p. 86-99.

9. van Heezik, Y., et al., “Do domestic cats impose an unsustainable harvest on urban bird populations? Biological Conservation. 143(1): p. 121-130.

10. Nelson, S.H., Evans, A.D., and Bradbury, R.B., “The efficacy of collar-mounted devices in reducing the rate of predation of wildlife by domestic cats.” Applied Animal Behaviour Science. 2005. 94(3-4): p. 273-285.

11. MacLean, M.M., et al., “The usefulness of sensitivity analysis for predicting the effects of cat predation on the population dynamics of their avian prey.” Ibis. 2008. 150(Suppl. 1): p. 100-113.

12. Churcher, P.B. and Lawton, J.H., “Predation by domestic cats in an English village.” Journal of Zoology. 1987. 212(3): p. 439-455.

13. Woods, M., McDonald, R.A., and Harris, S., “Predation of wildlife by domestic cats Felis catus in Great Britain.” Mammal Review. 2003. 33(2): p. 174-188.

14.  ABC, Human Attitudes and Behavior Regarding Cats. 1997, American Bird Conservancy: Washington, DC. http://www.abcbirds.org/abcprograms/policy/cats/materials/attitude.pdf

15. Clancy, E.A., Moore, A.S., and Bertone, E.R., “Evaluation of cat and owner characteristics and their relationships to outdoor access of owned cats.” Journal of the American Veterinary Medical Association. 2003. 222(11): p. 1541-1545.