You have all probably heard somebody say, in the middle of trying to persuade somebody else to come to their way of thinking, the words “do your own research”(DYOR). Taken at face value, this is very good advice. Let the person research it for themselves, and come to a conclusion based on the facts. One researcher studied this idea of doing your own research, and came up with some surprising results.
Sedona Chinn, in her paper in the Harvard Kennedy School’s Misinformation Review, started by asking three questions
- How are perceptions of “doing your own research” associated with trust in science institutions?
- How are perceptions of “doing your own research” associated with holding accurate beliefs about COVID-19?
- How are perceptions of “doing your own research” associated with concerns about COVID-19?
To sum up a lengthy study in a few words, she found that when somebody tells you to do your own research, most of the time they don’t want you to study the science behind it, but they want you to use your intuition (feelings) formed from alternative ideas. From her paper:
Our results revealed that positive DYOR perceptions were associated with more COVID-19 misperceptions and less trust in scientific institutions, even when controlling for previous beliefs and levels of trust. This may be because DYOR reflects epistemic beliefs that one’s gut feelings and “alternative facts” from independent research are as or more legitimate means of forming beliefs as scientific evidence and that truth is relative to each person’s unique experience . As people more strongly adopt DYOR epistemic beliefs, they may have greater justification for engaging in motivated reasoning and cherry-picking expert voices to support appealing beliefs, even when those beliefs oppose a consensus of institutional experts.
What is the implications of this? Chinn listed three in her paper which I’ve summarized here
First, people often overestimate their abilities to seek and interpret information and tend to search for information that aligns with preexisting values, beliefs, and identities. Perceptual biases can lead to inaccurate conclusions, particularly when individuals lack topic knowledge, training in scientific methods or rely on gut feelings.
Second, calls to DYOR may promote skepticism of science institutions and mainstream information sources by highlighting how they might mislead the public. These calls may reflect perceptions that science institutions or mainstream news media are corrupt or have a hidden agenda hostile to one’s worldviews and goals.
Finally, doing your own research may not be about seeking information but instead be expression of anti-establishment political identity. In cases of anti-vaccine sentiment, DYOR is often invoked in conjunction with resentment towards doctors and scientists who dismiss personal experience and intuition.
As I read the paper, I could see direct parallels in the Church when hard questions are brought up. The main idea of DYOR is that mainstream information sources might mislead you, so you need to DYOR, and then trust your gut (spirit). We are told by Church leaders to study for ourselves to find answer. The message here, sometimes implied and other times overt, is to use only approved Church material to study, and that “mainstream information sources” are less trustworthy. We are told to use the spirit to to guild us in our reasoning, just like the paper above referenced relying on “one’s gut feelings”
What has been your experience with DYOR, either as the one giving the advice, or the one receiving it?
(If you are interested in this subject, but don’t want to read the paper, you can listen to a podcast where the author is interviewed on the subject. That is how I found it. It is my new favorite podcast, “You are not so Smart”.
Wheat & Tares 
Great and timely post, considering what’s happening in both the church and in the country right now. As someone who regularly conducts research, and who teaches college students how to do it, I find I agree with Chinn’s conclusions. One of the things we try to teach at universities isn’t just “doing research”, but is also about information literacy. Looking stuff up is pretty easy, especially with google, the internet, etc.; but doing research about various sources of information is just as important. What the New England Journal of Medicine says about, say, measles or COVID vaccinations, is infinitely more reliable than what Joe Rogan (or Aaron Rodgers, for that matter) says about the same subject. One of the unintended consequences of the way the internet democratized information is that all of the other checks on research (peer review, etc.) are out the window; one can just as easily (perhaps more easily because of paywalls) access Uncle Bob’s Covid Conspiracy Website as one can access the NEJM.
There is also a (perceived) class/political issue here as well. “I did my own research” is, I think, also sometimes code for “I don’t need some liberal elite/egghead telling me about how disease pathology works”. Granted, both the American Left and the American university system haven’t done themselves any favors when it comes to selling the importance of education to Americans generally, but the current manifestations of the conservative movement in this country have, for a number of years, been very clearly and deliberately fomenting suspicion of higher ed in general, at least in part to curtail people’s ability to think critically and to properly vet various sources of information, including that which comes from the politicians themselves.
You can see a similar suspicion very clearly in the church’s language and attitude towards other sources of information to such an extent that often the extremely detailed and assiduously vetted research of historians of Mormonism is generally dismissed with a condescending wave of the hand by church leaders because that work is not an “official source”. The church often associates internet usage with the “dangers of pornography”, but I think it’s even more concerned that church members will find accurate, verifiable information that contradicts the church’s party line. Once church leadership adopted the Boyd Packer line about “some things that are true aren’t very useful”, it stopped being concerned at all about truth with a capital “t” and became much more concerned with the curated “truth” that it’s still committed to selling. This, more than anything else, has helped drive me to the brink of no longer claiming membership in the church. I can’t sustain an institution or leaders who consistently and demonstrably lie in order to maintain an elaborate fiction that they assume comforts their followers.
Interesting.
I can recall one study during the pandemic that–although ultimately disagreeing with them–found that anti-maskers were actually more rigorous in their scientific research and justification for their position than maskers were. Were they right? Maybe not, but I think DYOR can, in reality, involve doing just that, rather than relying on one’s gut. I don’t think it’s always about justification for your intuition. I’ve seen many of my positions change over time from DYOR.
Plus, there really is no “everything czar” when it comes to science and a lot of other things. I did grow a little tired of “trust the experts” implored repeatedly during the pandemic. The family and friends who most often repeated this seemed to treat certain scientific fields as if they existed in a vacuum. They don’t. Scientific fields interact with each other constantly and often in more ways than we might immediately realize. So yeah, I trusted the experts. I researched a variety of them as I formulated decisions that would be the most beneficial for my family and society at large. Admittedly, after taking all this information in, praying to the only “everything czar” didn’t hurt either.
As far as the Church goes, I regularly read plenty of uncorrelated and anti-material, in a addition to Church approved resources. I weigh and analyze all of them and am still currently an active member. What’s surprised me the last few months as a Xennial approaching middle-age (or there by some standards), is that–in addition to analysis–I’m only now starting to fully appreciate the power of pondering. I realize the cynic might throw around words like “rationalization” or “comfort” or any number of other words in there, but I don’t think that accurately describes the experience and resulting growth. I don’t know that it matters a ton whether you believe God is involved with the process or not.
In terms of doing my own research, as a scientist for 30 years, I find that it can be frustrating, expensive, very time-consuming, full of dead-rends, yet still perpetually interesting. So many of my guesses turn out to be wrong, or at least not substantially supported by experimental results. The worst is when an apparently interesting result the first time doing a pilot study fails to replicate on the second or third time the study is done for reproducibility. Did we accidentally do something we can’t account for the first time? or the second time? If it is a study with cultured cells, its it due to passage number? an undetected contaminant in our reagents? an error by my research staff? In animal studies, is it some unknown change in the environment of the vivarium?
What makes me happiest is when things replicate again and again as we press deeper into mechanism, testing inhibitors of the potential pathway, etc, and when other labs do similar studies in a slightly different context and find similar effects.
But if course all of this costs money and now funding by NIH has become highly uncertain and subject to the whims of a would be king.
Its certainly cheaper and easier to do RFK Jr style “research”. Make up a theory, find some odd anecdote that if you squint hard enough and pretend there are no other factors at play, supports the efficacy of the treatment. Life has so much stochasticity, you can always find an n=1 to support your “truth”.
I suppose that is the biggest challenge with religious experience being a foundation for interpreting the world. It may really be that God inspired me to look somewhere for my lost keys or to call so and so to be my counselor in the EQ presidency. Or maybe it was just random. Maybe the still small voice is witnessing that what a speaker is truth or maybe it’s just a feel good story like Star Wars.
The longer I live, the more my experience seems to grind down the religious truths of my youth. The one thing that survives is this: By this will others know you follow me, if you love one another.
I love research. Pre-internet, I used to just sit down with an encyclopedia and read it.
My go-to for research is to find a big fat book with lots of footnotes and endnotes and a balanced tone. The author can have an opinion, but if they’re ridiculing one side or the other, I don’t finish the book.
Back before social media really took off, I was visiting teacher to a an anti-vaxxer. I didn’t know that until I mentioned that my baby had a fever after his latest rounds of vaccines. That set her off and she gave me a book to read. The book was a joke – all hysteria and wild claims with charts and graphs that were full of logical fallacies. So I went and found a book about the history of vaccines. The book was 450 pages of information about vaccines, with plenty of footnotes and endnotes. It also described the times when vaccines had caused problems when they were contaminated; it was very balanced in its investigation. I discovered VAERS through that book, the Vaccine Adverse Effect Reporting System. Did you know the government tracks adverse reactions to vaccines? Probably not anymore, or at least it isn’t advertised. I submitted my son’s fever through that website. And I kept vaccinating my kids. We all got the Covid vaccine too.
Unfortunately, the book’s title was “Vaccine” and I can’t remember the author. So I can’t recommend it to anyone because I can’t find the actual book I read anymore. I checked it out of a library almost twenty years ago. Pro tip: if you’re going to write a super useful and informative book, give it a title that isn’t the same word as a really broad topic.
The current attack on research and evidence based medicine has made me leery of anything published after 2020.
If DYOR means watching some youtube videos, then that’s pretty much a disaster, yeah.
“I suggest that research is not the answer”.
Dallin H Oaks, April 2019 devotional
He’s your next prophet, seer, and revelator folks
“I suggest that research is not the answer”. Dallin H Oaks, 2019 devotional
Bishop Bill, I love that library picture! Growing up, my family spent most Family Home Evenings at the public library. We would check out with a whole orange box full of books. That led me to enjoying time in libraries just browsing random books and subjects. There were times as a student at BYU, studying in the library, I would take a break of studying by wandering the aisles looking at books and finding new subjects, which would eventually lead to deeper research in new topics. @janey, I also used to enjoy reading the encyclopedia.
Unfortunately, I no longer venture the library aisles. The internet/YouTube has changed all of our research approaches. I believe in doing research in looking a multiple points of view especially if it is a controversial topic (sports/religion/politics/finances). If it is researched based, I look at who provided the funding.
I really appreciate it when someone else creates an awareness for me about researching a topic. Many times that can lead to a superficial glance or to a deep dive, depending on the theme. However, I know few people enjoy wandering library aisles or doing deep intellectual dives. This is where institutions can take advantage of the masses. “Trust me I have the knowledge”, or people think “He/She is has done the research, and I can rely on them” On a personal level, I wish I would not have trusted the promoters of COVID vaccine (I ended up with long term cardiac side effects). I believe in vaccines, but there was not enough research when COVID was released.
On the LDS slant, too often members are asked/required to trust the leaders. However now research and experience is showing that many of their recommendations/mandates are more harmful than beneficial. I believe in being fully informed and empowered with knowledge, prior to giving consent. However, that does not work with business sales, politicians, or religious institutions that have a secondary gain.
When I was raised as a child in the 60’s, my parents were John Birchers. There were lots of books in our house and we were encouraged to read. As I went to college at Ohio State and then went on a mission, I came in contact with a lot of different kinds of books including anit-mormon literature. I noticed that the anti-mormon stuff sounded a lot like the John Birch stuff loaded with trigger words like “obviously, you must conclude, and the answer is.” The church histories I was reading didn’t have that kind of language and just presented things. After I got back from my mission and returned to college, I found that there were a lot of books that just presented things and didn’t elicit conclusions. Granted many of those books had a point to prove or an idea to support but they just wrote and presented information and let you draw your own conclusions. After college I continued to read a lot of different books and realized that people disagreed all the time and presented different point of views but I was the one that needed to draw my own conclusions. I was amazed to find there were books outside the church that supported things I learned in church but there were also books that pointed out different ways at looking at things without even mentioning the church. It was then that I started to make up my own mind about things and realized that truth was something that needs to be constructed from multiple sources and is not necessarily found in one convenient location. It was easier to see that with political left/right interpretations but eventually I could see it with things in the church. I also found that it was really hard to explain your journey to someone else. All you really could do was to try and open a door here and there to a new way at looking at something and let them travel through it themselves when they were ready.
It’s hard to be patient in today’s world. It seems we all have to instantly agree to whatever absurdity is presented and if we don’t we’re seen as “the enemy.” I’ve found that whenever anyone presents something as an us vs. them kind of truth, it’s not truth they are pushing. So I just keep on doing my own research, drawing my own conclusions, and try to make sense of the world the best I can.
Faith, my father was a University Librarian, and I have found memories of going to the closed library on a Sunday afternoon, and playing hide and seek in the stacks with my brothers. The photo in the post reminds me so much of the school’s library.
The DYOR people are not doing research. That would require reading the original documents and familiarity with the concepts and practices in the field they are curious about, or at least serious lay explanations. What DYOR doi is find opinions about the research which agree with the position they have already assumed. If they find opinions which disagree with their already formed opinions, they usually attribute it to malevolent forces or conspiracy. “[They] don’t want you to know” is a common shibboleth which assuages their ignorance by demonstrating that the DYOR are in the very, very special group the “knows.”
Janey, growing up we had the full World Book Encyclopedia. On Sunday afternoon I would pick a volume at random (E was the biggest) and go through every page, readying anything that caught my eye. We also had the “Time-Life” books on science that I would read.
I also grew up in a WB Encyclopedia home. It was always exciting to receive the updated versions and discover what had changed. Good times.
Contrary to vajra2’s claim that “DYOR people are not doing research”, it is possible to make reasonable conclusions based on using an intellectually disciplined approach, recognizing underlying assumptions and evaluating reliable information. It is called critical thinking.
For example, years ago a fellow ward member introduced me to a scholarly treatise on the sources of the Book of Abraham. I was surprised to learn that, without exception, non-LDS Egyptian scholars agreed that the purported BoA source papyri are funeral scrolls and bear no link to the so-called ‘translation’. That discovery led me to thoroughly read and research the works of FAIR contributors such as Gee, Muhlestein, Peterson, et al.
Such DYOR confirmed to me the built-in biases of Mormon authors when it comes to the study of issues that may be contrary to established doctrine. Their apologetic approach typically relies on unproven (and often outlandish) arguments that appeal to shallow, undisciplined thinkers. The so-called BoA catalyst theory and missing scroll claims have zero factual basis and are merely smokescreens.
Unfortunately, all too often apologists rely on the same intellectually lazy approach when it comes to evaluating issues such as polygamy, race, BoM historicity, etc. I find it possible to evaluate reliable sources and reach an understanding of what is reasonably factual. Following the advice of Mormon leaders, it is possible to find answers based on study and prayer – just not always what they would consider the ‘right’ answers.
The irony of the Do Your Own Research crowd is that they usually don’t do enough research and tend to embrace extraordinary ideas that lack extraordinary evidence to back them up. They are usually easily misled by a doctored video or things taken out of context. They usually commit burden of proof fallacies and position themselves as right until proven wrong instead being right because of strong evidence.
Knowing any subject requires a good amount of study and thought. It isn’t just watching a video or two. The mainstream has mostly become the mainstream because it consists of people who do the best research that has been able to withstand lots of questioning. Occasionally, however, ideas emanate from non-mainstream areas that are so well-evidenced and so well-argued that they come to be mainstream. This usually doesn’t happen overnight, however.
Love the post. I concur.
I think that prior to “doing your own research” you should be required to take a course on biases, with an emphasis on confirmation bias. I also think that after doing your own research, you should be able to be able to express the opposing side’s argument coherently. If you have done your own research and have strong arguments that support your side, but you “just can’t understand why the other side thinks what they do”, then I don’t think you’ve done real research. You’ve just engaged in confirmation bias (which feels so much better and is so much more satisfying than research), but it generally doesn’t help you get closer to the truth.
Eli:
“What’s surprised me the last few months as a Xennial approaching middle-age (or there by some standards), is that–in addition to analysis–I’m only now starting to fully appreciate the power of pondering.”
As a late baby boomer who is now entering geezerhood I can attest to the power of pondering. After we’ve sifted through all of the material we can find on a particular subject–or at least enough material to wear us out–it’s a good practice to turn the boil down to a simmer and quietly meditate on the subject for a while. It gives our mind the space it needs to do the work that we can’t force it to do–putting pieces together and making connections at levels that are deeper than our waking thoughts. And on top of that we may need a night or two of sleep–after vigorous thought and research on a given subject, that is–to allow our mind to work at the deepest levels.
That said, I find it very interesting that many of the great visions seen by prophets–both ancient and modern–seem to follow a period of focused pondering–or “marveling” as it is sometimes called in the scriptures–on their part. And this pattern pertains to all of God’s children. Said Jesus in 3 Nephi 17 to the Nephites after his first visit to them:
“3 Therefore, go ye unto your homes, and ponder upon the things which I have said, and ask of the Father, in my name, that ye may understand, and prepare your minds for the morrow, and I come unto you again.”
Pondering may not always lead to a grand vision. But it will almost certainly lead to understanding–especially if we follow that pattern that Nephi speaks of in 1 Nephi 11:
“1 For it came to pass after I had desired to know the things that my father had seen, and believing that the Lord was able to make them known unto me, as I sat pondering in mine heart I was caught away in the Spirit of the Lord…”
For those who may be interested–here’s the context for the Dallin H. Oaks quote cited above:
“I hope all Latter-day Saint marriages are loving and free from serious conflict over important values and priorities, but we know that some are not. A faithful wife submitted this question: “My spouse has gone inactive due to doubts regarding Church history and doctrinal issues. As someone who believes in the gospel and wants to remain active, how should I go about researching and responding to these issues?”
“I suggest that research is not the answer. References to the Church’s many helps to answer familiar questions, such as the Gospel Topics Essays at ChurchofJesusChrist.org, may help one who is sincerely seeking, but the best answer to any question that threatens faith is to work to increase faith in the Lord Jesus Christ. Conversion to the Lord precedes conversion to the Church. And conversion to the Lord comes through prayer and study and service, furthered by loving patience on the part of the spouse and other concerned family members.”
The full text to President Oaks talk is found here:
https://www.churchofjesuschrist.org/study/ensign/2020/06/young-adults/keeping-the-faith-on-the-front-line?lang=eng
DYOR discussions must first be prefaced with a discussion on the Dunning-Kruger effect in which people wrongly overestimate their knowledge or ability in a specific area. This tends to occur because a lack of self-awareness prevents them from accurately assessing their own skills.
Jack, thank you for providing full context on the Oaks quote.
In context, I think I may actually agree with Oaks. The questions that lead people to question their literalistic faith in the church generally don’t have good answers that would support maintaining that kind of literalistic belief, and trying to Bible bash or church-history bash with your loved one is kind of a dead end and not healthy to maintaining relationships. There is a path of embracing a nuanced and less literal faith that can help people find a way to stay in the church, but the institution still seems to show little interest in actively promoting that path. But, if that is someone’s goal, then I would consider that a topic worthy of research.
Back when RadioLab was still putting out episodes worth listening to, one of them was about this topic. Something they reported on was in the conservative alternate facts universe, what they will do is come up with some random phrase. Then they will create blog posts about said random phrase and its nefariousness. Then they will get on a more popular show/podcast, talk about random phrase, and say “Do your own research”. So the listeners will enter random phrase into search engines and only find articles talking about nefarious doings of the “others”. By the time a fact-checker catches on and publishes something informative, it’s too late. Plus those types despise fact checking sites, so it doesn’t matter much.
Given that it’s St. Patrick’s Day they might come up with the Peach Leprechaun Insurrection. Something that’s never happened. But it’s nefarious, and evil, and un-american. Make some posts about it obscure sites, make sure the search engines have indexed it, then talk about it. Those “doing their own research” for the P-L-I will primarily get hits from those obscure websites and feel rewarded for being in the know.
I prefer a different framing on the matter of misinformation. Rather than focus on facts and sources to decide what and who is correct, I think it is more important to consider how one might be fooled. Nicolas Taleb coined the term the “Ludic Fallacy” to explain how smart people are fooled. The reason they are fooled is because they make assumptions about the world that are not so.
To illustrate the Ludic Fallacy, Taleb imagines two persons. One is a “Dr. John” who is a PhD in Statistics with many years of academic and industry experience. The other is “Fat Tony” who is a street smart real estate agent. Taleb explains to both that he has a coin that has a fair and equal chance of being heads or tails. Taleb then flips the coin 99 times and it is heads every time. Taleb then asks both Dr. John and Fat Tony what are the odds the next coin flip will be heads.
Dr. John answers that because he trusts the rules as explained by Taleb – that the coin being flipped is a fair coin – then the odds are 50-50 the coin will flip heads. Fat Tony answers that having observed 99 coin flips all be heads, the claim Taleb made about the coin being fair is wrong. Fat Tony asserts the next coin flip will also be heads.
Taleb credits Fat Tony for not being fooled. Dr. John, explains Taleb, put his trust on the rules of the game being valid – that when Taleb said the coin was fair it must be fair. Fat Tony correctly observed that the coin was not fair and from this Fat Tony concluded that the rules of the game as presented by Taleb were not being followed.
A particular challenge in not being fooled is that it is when we are most confident in our assumptions that we are least likely to challenge them. This blindness is captured in the classic 1970s song, “What a Fool Believes”. The chorus explains: “What a fool believes, he sees, and no wise man has the power to reason away.”
I hear many ex-Mormons tell of having arrived at their positions by Doing Their Own Research. Honestly that was how it was for me. In that vein, the apologists like to dismiss ex-Mormons as mindless overreactors who are like conspiracy theorists overreacting to some piece of misinformation they found online about how vaccines are bad or something akin. What is really going on, however, is that in a few pockets of Mormonism, there is a rather large intellectual presence that defends traditional Mormon beliefs. Because this group is quite prolific and outspoken, and widely accepted in the Mormon community, within these pockets of Mormonism, this group appears mainstream and the traditional ideas of Mormonism appear validated by experts and academics. Those who challenge these ideas appear to be going against the grain of intellectuals within these small pockets. Herein lies the problem. If we expand the picture to include all academics and intellectuals throughout the world, Mormon beliefs are not validated or mainstream. They are extreme. The only intellectuals defending Mormonism were deeply socialized in the Mormon culture often from very early stages of life. Mormon truth claims have no validation or acceptance outside very small pockets. Additionally, over decades, the church has indirectly funneled millions and millions of dollars to back faith-based research and researchers to build the extremely prolific world of apologetics. It would thousands and thousands of hours to read everything that has been published. And yet, almost none of this work has gained any acceptance outside Mormonism. The whole design of this research isn’t to expand or contribute to general knowledge. It is to create a smokescreen that will discourage already believing Mormons from being able to build a narrative to justify exiting the church. And it has been effective at that. But not at convincing outside intellectuals of the veracity of Mormonism.
So in the context of tightly-knit communities that have a large webbed narrative that the pump into its members from early ages and a network of protectors of the traditional narrative, it is incumbent upon individuals to do their own research to figure out a way out. But in the context of the larger mainstream scientific thinking in the free world where science is the least tainted by political influences, we can try to do our own research, but we must recognize that doing our own research in a productive and meaningful way is incredibly difficult (in both social and hard sciences) and we must rely extensively upon experts in getting to a place where we can produce independent thought on these subjects. By entertaining fringe theories that challenge the very fundamentals of mainstream thought, we risk going down conspiracy theory rabbit holes from which we may never exit. I’ve lost several family members to conspiracy theory. They will never return to reality.