As someone who went from a research heavy semester in an Applied Criminology master’s program to becoming an apprentice midwife, I feel that I have a duty to this crazy birth world to put some research methods information out there.

We talk a lot about evidence based care, but it is silly to assume that all evidence is created equal.  And although I think it’s important to understand quantitative research, I find lots of things wrong with the idea that quantitative or number based research is what we should base our decisions on.  So I guess my desires for this post are three-fold-

1) I would like it if birth workers could stop putting out poorly interpreted information they are masquerading as “cold, hard numbers”.
2) It would be great if more birth workers had a basic ability to spot bad research and studies with incorrect conclusions.
3) What would be SUPER awesome is if more birth workers and the world in general had a deeper understanding of the limits and flaws of quantitative research so that we could all move forward with our own complementary ways of knowing that come from the heart/soul/intuition.

The first part of this post is more philosophical, but is important to understanding what quantitative research is, where it came from, what the limits are, and why we should be skeptical.  The second part is more concrete, and is about what good research should include.  In the third part I look at a particularly bad blog post as an example and pick a few of the more obvious flaws in their research design.  I also show a better example of research.

Part 1 – What is Positivism?  What are the Limits of Quantitative Research and the Scientific Method?

Positivism is the school of thought that believes research based in numbers (quantitative research) and the scientific method are the best ways for us as humans to understand our world, including our social world. Positivism’s primary beliefs are that the world is testable and knowable, unchanging and potentially predictable, and that a reality and a truth exist which we can discover through using (quantitative) scientific methodologies.  Most importantly, positivist thinking believes that once we understand something more clearly we can then control it (for example, if we do enough studies, we will find the way to control Rh sensitization).  Hard positivism thus lends itself to being deterministic, or denying free will.  Most professional social science researchers today have moved away from hard positivism as well as soft positivism.

As a school of thought, positivism arose in the time of industrialization where there were many unfilled industrial jobs and huge shifts in what life was like.  Social life was fraught with overcrowding, urbanization and hard labor conditions.  When labor became mechanized and alienating in the new factories, scientists also began mechanizing and stripping the scientific process of context so that social life could be as well controlled as the means of production.  Social scientists began to feel pressure to start quantifying their research in order to become legitimized. When trying to influence policy and society, numbers were supreme.  It should be mentioned that this time period was coming hot off the heels of thinkers like Descartes who set the stage for these dysfunctional philosophies.  He said, roughly, “I think, therefore I am”, which led to a huge shift in the way we understand ourselves as humans, as our minds and bodies being separate, and the mind being valued above the body.  In the same way, scientists began privileging cold, disconnected numbers over contextualized understanding.  This changed the social sciences from largely theoretical and philosophical fields into extensions of legitimate academia where they were deeply integrated into the structure of power and privilege. 

So positivism is what made quantitative research really popular, but number based research is not only a part of positivist tradition. While quantitative scientific is touted in Western culture as a method of reducing the errors associated with human inquiry and coming closer to ‘the truth’ than possible with other methodologies, many critiques have been presented to challenge this notion.  Human subjectivity (our beliefs, culture, assumptions, emotional ties, etc) cannot be removed from science in any meaningful way.  From the selection of the research topic to the way data analysis is interpreted, subjectivity is an undeniable factor and a priori assumptions (things you assumed before starting the research) can never be fully accounted for. 

The scientific method only gives us an idea or piece of reality, not a full picture.  Specifically with the social sciences, quantitative methodologies are believed to capture some semblance of truth in the form of hard and factual numbers.  This ignores the fact that there are truths that cannot be quantified or even pinned down by human language.  Positivism led to the supreme arrogance that allowed those who already had power and privilege to further exploit those powers and privileges in ways that were harmful to immigrants, people of color, and women.  Since primarily rich, white men had (and still have) access and legitimacy within positivist social science movements, science was used to allow the sterilization of non-white women, to lock up and experiment on people with mental illness, and to justify slavery and segregation.  Scientific methodology can only overcome these critiques by recognizing this sinister history and admitting that scientific inquiry can only reach so far.  To use quantitative research effectively, we have to understand its limits and simultaneously hone alternative ways of knowing what is right for our families.

Part 2- So How CAN We Use Quantitative Research as Birth Workers?

DISCLAIMER: The following information is going to make it sound as though I think people should be intensely interested in scientific research.  I am primarily providing this information for people who insist on using quantitative research as the basis of their arguments and decision making in the hopes that they will at least do a better job of it.  I also think this is important so that other birth workers and mamas can recognize bad research.  I personally believe that we should be building our own subjective ways of knowing rather than relying on patriarchal and philosophically bankrupt science.

I think that quantitative research should only be performed by and used by people who know enough to be skeptical of its limits.  The number one goal in science is to be objective, so if you can’t be knowledgeable and objective when reviewing studies, you are already NOT BEING SCIENTIFIC!!!  If you can’t look at the abstract of a study and determine if they used sound methodology, how will you ever be able to be objective about which studies to show clients or incorporate into your “evidence-based practice”? There are definitely studies being done that use good/great, conscientious methodology out there.  There are also some not so good studies.  Then there are some really really terrible examples of people butchering statistical analysis.

This is all pretty abstract, and is really just meant to get your mind working a little bit.  If you are really invested in quantitative research based studies, I suggest that you take a class on research methods, or at the very least that you read some books on your own.  Here is what a class should teach you:
– choosing research topics
– setting parameters on those topics
– picking and defining populations
– determining and defining your independent and dependent variables
– deciding your levels of measurement
– how to construct your measures and test their validity and accuracy
– types of sampling
– ways of gathering information
– ethical considerations
– hopefully how to do some basic analysis   
– and you should probably take a statistics class too**

At the very basic level, true research should have all of these components well thought out and documented for the reader to look at:

1) Statement of the Problem : What is it that you’re interested in?  Why is it important?
2) Hypothesis: Your hypothesis has to state a clearly defined relationship between clearly defined and measurable independent and dependent variables.
3) Methodology: Talk about what methods you are going to use to test your hypothesis.  Secondary data? Surveys? Ethnographic/interview research? 
4) Identification of Independent and Dependent Variables – This includes both conceptual and operational definitions, meaning, you have to
5) Data collection – What is being collected? From where?
6) Sample – Who or what will be studied? How will it/they be selected?

And when analyzing data, we need to understand some basic things about the difference between causation and correlation.  Here is the example I was taught with-  

 When ice cream sales go up, the number of sexual assaults also goes up.  Why would this be? There is an association here between ice cream and the number of sexual assaults, but it is not because ice cream causes sexual assault.  There has to be an alternative explanation even though there is a statistically significant relationship between ice cream sales and sexual assault. Well, the alternative (and correct) hypothesis is that sexual assault goes up when the weather is warmer.  This just shows that there are many things that are related, but causation is very, very hard to prove.  We have to always be exploring alternative hypotheses, even after we have done a great job with our research methodology.   

This is just an overview and sample of the sorts of things to look for and think about when looking at research. 

Part 3- Let’s practice! 

For the sake of giving examples, lets take a look the inspiration for this blog post.  Granted, this is not a peer-reviewed, published journal article, but this is the sort of ridiculousness that can happen when people who don’t understand research start trying to “crunch the numbers” for themselves. Heather, a contributor over at the 10 Centimeters blog, is on a crusade to eradicate direct entry midwives, CPMs and homebirths that are not “low-risk” (according to which standard I wonder?).  The link to her post is here- http://www.10centimeters.com/no-matter-how-you-run-the-numbers-the-result-remains-the-same/

In this post about how she believes more babies die at home than at the hospital, she writes,
    
    “These are the cold, hard numbers. Excel did all the math for me, so I’m pretty sure they’re   
     correct! These clearly preventable deaths are the reason that I am so passionate about this.
     Please, the next time someone tells you that homebirth is “as safe or safer” than hospital birth,
     do me a favor and tell them they haven’t done their research.” 

Well, I would say that she hasn’t done her research.  If it isn’t scary enough for you to read that excel did the math for her and that she believes these are “cold, hard numbers”, read on. 

This author frequently uses the CDC Wonder data set to create her charts that she then edits in some rudimentary paint program.   So the most obvious problem in her post is that she has not told the reader about the quality of the methodology used by her secondary data source, how they measured their variables, or where the CDC even gets its data.  If you are using secondary data, you MUST MUST MUST MUST MUST investigate their method of data collection.  Does the CDC go off of birth certificates? Or hospitals’ and midwives’ quarterly reports?  Or do they sample populations?  Does a birth get counted as a “CNM attended birth” if it ends in a c-section, or if an OB takes over during the labor and delivery?  Does it count as a midwife attended birth if they transport?  I tried to find this information myself so I could better interpret her charts.  The best I could find is this – “The Natality data are available from a county-level national data collection derived from the U.S. records of birth”.  They get them from birth certificates.  Well I’m no expert in birth certificates, but if Heather wants to use CDC data then she needs to find out how county’s collect this data that goes on the birth certificates.  Does each county use the same form?  What is the validity and accuracy of this method of data collection?  If you don’t have data that was collected in a uniform and quality way, you can’t make valid interpretations.   More on using birth certificates and vital statistics later on.

The author starts out the post saying, “Homebirth, with or without a non-CNM midwife, is more dangerous than giving birth in the hospital”.  This is the closest thing I could find to a hypothesis, but it follows none of the rules of a research hypothesis.  You can’t hypothesize about variables that aren’t measurable – they have to be concrete.  “Safety” and “danger” are not measurable, but “feelings of safety” are.  “Dangerousness” is not measureable but “neonatal death rates” are.  Her hypothesis might be something like “Planned, low-risk, out-of-hospital births have higher neonatal death rates than low-risk hospital births”.  And if this is truly her hypothesis, she should probably also take a writing course because her argument is all over the place.  But I digress, she needs to define her variables clearly and show how she is measuring them.  If she is using CDC data then she needs to provide us with their operational definitions (or how EXACTLY they categorized things like “CNM births, midwife births, in and out of hospital births), which would answer some of the early questions like “Does a neonatal death count under midwife statistics if the baby was transported and born in the hospital?”

Now, she did understand at some level the basic idea that if you want to do research, you need to define the population and set parameters so you can make inferences later on.  She did include the parameters she searched under, but again didn’t define them or how the data were collected by the CDC.  She says:

Because the argument I’ve often heard against using the CDC database numbers is, ‘It includes all those babies born in the car on the way to the hospital,’ I did the complications of labor and delivery search twice, once with precipitate delivery removed as a cause of death. I also only included data from 32 weeks of gestation and beyond, because I imagine that even the most hardcore homebirthers would head for the hospital if they went into labor earlier than that.

Hopefully the mistake here is obvious, but if it isn’t, think about how she is trying to control the population of her sample.  She is trying to exclude unplanned out of hospital births.  In order to do this, she thinks it is sufficient to simply run two searches, one with precipitate deliveries and one without.  This assumes (incorrectly) that all the unplanned out of hospital births (that she is trying to exclude) were also precipitous deliveries.  Further, the population needs to be clearly defined and thought through, even when it seems obvious.  By using CDC data she can only really look at women who have their babies in the US and who file birth certificates.  Therefore, when she starts drawing conclusions, she would have to limit them to that same population.

These are just a few obvious issues that were overlooked.  I am certainly not the foremost expert on pregnancy and birth research, but I know enough to spot a bad study and bad information when I see it.  Of course, the scary part of all of this is that someone without this knowledge might buy into these poorly argued ideas simply because they have used “cold, hard numbers”. 

To truly and earnestly do a study on neonatal deaths in in-hospital versus out of hospital births, it would take much more than an afternoon of fiddling with the CDC Wonder data.  Please, when reading studies or non-peer reviewed information, remember that there is more to research than using an excel spreadsheet. There are concepts, ethics, methodology that go into creating good research.

Here are links to three true studies done on homebirth outcomes –
**Outcomes of Care for 16,924 Planned Home Births in the United States: The Midwives Alliance of North America Statistics Project, 2004 to 2009
http://onlinelibrary.wiley.com/doi/10.1111/jmwh.12172/abstract
– This study is new since I originally posted this article, and is the best available information on homebirth outcomes in the US to this date.

Outcomes of planned home births with certified professional midwives: large prospective study in North America
– http://www.bmj.com/content/330/7505/1416.short

Outcome of planned home and planned hospital births in low risk pregnancies: prospective study in midwifery practices in the Netherlands
– http://www.bmj.com/content/313/7068/1309.abstract

These link to the abstracts, but you can access the full pdfs via the links on the left.  But even from the abstracts, it is clear that they have followed the process of scientific inquiry.  They give us the needed information about what they studied and how they studied it.  Just in the short abstract they give us their objective (rather than a hypothesis), their study design, population, and main outcome measures (or their operational definitions of the dependent variables). 

A good example of a thoughtful conclusion is found in the Netherlands Study.  They write, “The outcome of planned home births is at least as good as that of planned hospital births in women at low risk receiving midwifery care in the Netherlands.”  Notice that this is a measured statement that clearly defines what they are studying and only makes statements about that specific population – planned home and hospital births of low risk women in the Netherlands. 

Above the conclusion they defined exactly how they were measuring the outcomes – “Perinatal outcome index based on “maximal result with minimal intervention” and incorporating 22 items on childbirth, 9 on the condition of the newborn, and 5 on the mother after the birth.”  So when you read the conclusion and see that planned home births have “at least as good” of outcomes, you know exactly how they measured that and what it means (and maybe more importantly, what it doesn’t mean). 

Another crucial aspect to research is that the researchers discuss the strengths and weaknesses of their own study, showing that they did the best they could, but acknowledging that there are limitations to their work.  The researchers for the North American study talk about how they would have preferred to have performed a randomized controlled study and then they go on to explain why that wouldn’t be feasible at this time since homebirth has not become institutionalized within the larger health care system.  They also go on to say:

     [T]he main study limitation was the inability to develop a workable design from which to collect a national prospective low risk group of hospital births to compare morbidity and mortality directly.  Forms for vital statistics do not reliably collect the information on medical risk factors required to create a retrospective hospital birth group of precisely comparable low risk (pg 5).

This is especially interesting after reading and thinking about the terrible blog post that was based on CDC data, since she was effectively saying she HAD created one of these retrospective groups.  The researchers in the North America study also cite that there is one study that has successfully used birth certificate data, and that was Schlenzka’s PhD thesis study done in California using data from 1989-1990.  These birth certificates could be used because they did include the necessary information to accurately create a “retrospective cohort” of low-risk planned home and hospital births.  And the kicker is that the conclusion of this thesis was that home births actually “had non-significantly lower perinatal mortality in the home birth group. The results were consistent regardless of liberal or more restrictive criteria to define low risk, and whether or not the analysis involved simple standardisation of rates or extensive adjustment for all potential risk variables collected.”  Non-significantly means that the perinatal mortality rates were lower for homebirth, but that it didn’t pass the statistical test to see if it was “significant”, or in other words, if we could confidently say it wasn’t based on chance.  Of course, this is something that most blog authors, like the one in our example, don’t even attempt to address, even though statistical analysis also has to be precise and thought out before drawing any conclusions, as seen with the Schlenzka example.

I think its pretty clear that if we are going to use research in the practice of midwifery, that we need to have a baseline understanding of how it works, and we need to have a healthy respect for the art of conducting research.  It should be pretty obvious that true scientific research (with all of its flaws and shortcomings) take more than plugging numbers into an excel spreadsheet.  Before buying into any new idea, it’s important to check the quality of our sources.  Read with a keen eye, and in the end, know that numbers can be manipulated to prove, or seem to prove, just about anything if framed the right way.  And perhaps the most important thing to remember is that when we are present within our lives, our heart is a fine tuned instrument that man made methodology cannot compete with.