Page 1 of 2 12 LastLast
Results 1 to 25 of 26
  1. #1
    151
    151 is offline
    RoadBikeReview Member
    Reputation: 151's Avatar
    Join Date
    Apr 2009
    Posts
    1,590

    life vs the scientific method

    I am not trying to argue against science or scientist or carbon bikes, just have a question that science'ish people can probably help with.

    Is it true that anecdotal evidence gets a bad rap in scientific thinking?

    The thought came to me earlier that maybe it should be considered in all our scientific experiments.

    The reason being, that there is no way for all variables to be controlled during an experiment. Even in the most simple there are going to be variables, however small, that we do not know about.

    But over years or centuries of anecdotal evidence all variables are included in the "study". Because by only considering (basically) the outcome, we include every possible variable.
    You. You are the frame pump noob.
    ----Creakyknees

    then I did it and crawled into bed with my gf and that aspect alone changed EVERYTHING. It felt soooo good
    ----Ghost234

  2. #2
    jaded bitter joy crusher
    Reputation: Fredke's Avatar
    Join Date
    Dec 2004
    Posts
    19,768
    Quote Originally Posted by 151 View Post
    But over years or centuries of anecdotal evidence all variables are included in the "study". Because by only considering (basically) the outcome, we include every possible variable.
    The problem is that with anecdotal evidence, you tend to remember some things and forget lots of other, equally relevant things. This gives you a bad, unrepresentative, and often biased sample that is likely to mislead you.

    There have been lots of studies that show that people tend to draw the wrong conclusions when they reason from anecdotal evidence. Google "availability heuristic" and "representativeness heuristic" if you want to learn more.
    Fredke commented in your thread. You won't believe what happens next!

  3. #3
    waterproof*
    Reputation: Creakyknees's Avatar
    Join Date
    Sep 2003
    Posts
    40,950
    can someone define for me the difference between "experience" and "anecdotal evidence"?
    * posted by Creakybot 2013 all rights reserved.
    * not actually waterproof.

  4. #4
    eminence grease
    Reputation: terry b's Avatar
    Join Date
    Jan 2004
    Posts
    18,561
    Science often stems straight from anecdotal evidence which is better known as "observation."

    The apple fell on Newton's head. He observed that sitting under a tree eating lunch, and relayed the story anecdotally. He took that, formulated a theory and proved it.
    You'd be better off with a netbook, they do everything better.

    My travel blog: http://tbaroundtheworld.blogspot.com

  5. #5
    eminence grease
    Reputation: terry b's Avatar
    Join Date
    Jan 2004
    Posts
    18,561
    Quote Originally Posted by Creakyknees View Post
    can someone define for me the difference between "experience" and "anecdotal evidence"?
    The most basic definition is that "experience" is one word while "anecdotal evidence" is two. I could get into the syllables and letter content but I just woke up from a nap and need a drink of something cold.
    You'd be better off with a netbook, they do everything better.

    My travel blog: http://tbaroundtheworld.blogspot.com

  6. #6
    waterproof*
    Reputation: Creakyknees's Avatar
    Join Date
    Sep 2003
    Posts
    40,950
    Quote Originally Posted by terry b View Post
    The most basic definition is that "experience" is one word while "anecdotal evidence" is two. I could get into the syllables and letter content but I just woke up from a nap and need a drink of something cold.
    if I had any rep to spread, you'd get a big juicy lump for that.
    * posted by Creakybot 2013 all rights reserved.
    * not actually waterproof.

  7. #7
    eminence grease
    Reputation: terry b's Avatar
    Join Date
    Jan 2004
    Posts
    18,561
    Quote Originally Posted by Creakyknees View Post
    if I had any rep to spread, you'd get a big juicy lump for that.
    I keep blowing my humor wad in dry holes.
    You'd be better off with a netbook, they do everything better.

    My travel blog: http://tbaroundtheworld.blogspot.com

  8. #8
    banned
    Reputation:
    Join Date
    May 2011
    Posts
    46
    Keeping all other variables constant while changing only one, is the accepted method of scientific investigation. Of course, you must vary the variable an adequate number of times in order to raise the confidence level of both your data and the results.

    After careful analysis of your data and the results, you are perfectly welcome to abort all forms of logic and default to the anecdotal.
    Last edited by SlimDandy; 05-28-2011 at 01:23 PM.

  9. #9
    RoadBikeReview Member
    Reputation: stevesbike's Avatar
    Join Date
    Jun 2002
    Posts
    5,112
    Quote Originally Posted by 151 View Post
    I am not trying to argue against science or scientist or carbon bikes, just have a question that science'ish people can probably help with.

    Is it true that anecdotal evidence gets a bad rap in scientific thinking?

    The thought came to me earlier that maybe it should be considered in all our scientific experiments.

    The reason being, that there is no way for all variables to be controlled during an experiment. Even in the most simple there are going to be variables, however small, that we do not know about.

    But over years or centuries of anecdotal evidence all variables are included in the "study". Because by only considering (basically) the outcome, we include every possible variable.
    that last sentence makes no sense. Anecdotal evidence just means evidence that is not systematically related to a hypothesis. Typically it involves some limited number of observations that do not have enough power to support some hypothesis (e.g., peanut butter sandwiches are great pre-race food because I felt really good this am after eating one). Anecdotal evidence lacks statistical power and people are also susceptible to various biases about it, such as representativeness, availability, base rate fallacies, etc.

    You don't have to control all variables in an experiment; you can have more than one independent variable in an experiment, you can have variables you can't measure, you do so by making assumptions about how they may interact (typically a linerarity assumption).

  10. #10
    Rep *****.
    Reputation: firstrax's Avatar
    Join Date
    Nov 2001
    Posts
    9,852
    Quote Originally Posted by Creakyknees View Post
    if I had any rep to spread, you'd get a big juicy lump for that.
    I covered it for you.

  11. #11
    jaded bitter joy crusher
    Reputation: Fredke's Avatar
    Join Date
    Dec 2004
    Posts
    19,768
    Quote Originally Posted by terry b View Post
    Science often stems straight from anecdotal evidence which is better known as "observation."

    The apple fell on Newton's head. He observed that sitting under a tree eating lunch, and relayed the story anecdotally. He took that, formulated a theory and proved it.
    Two things:

    First, Newton's anecdote was that he was sitting in his study, looking out the window and saw the apple fall, not that it hit him on the head.

    Second, the apple inspiration had nothing to do with evidence. It was inspiration. The evidence was thousands of systematic observations by Tycho.

    Experience is good for giving you ideas, but then you need a way to test the ideas and see whether there's anything to them.
    Fredke commented in your thread. You won't believe what happens next!

  12. #12
    RoadBikeReview Member
    Reputation:
    Join Date
    Dec 2004
    Posts
    1,076
    The plural of anecdote is not data. The scientific method is based on the collection and analysis of data, which is a systematic process. As was said before, anecdotal information can provide inspiration that leads to the collection of data and hence the use of the scientific method.

    I think anecdotal experience and qualitative data often get muddled. Good data, at least in social science, is not inherently going to be qualitative. Some things can't be readily or directly measured with quantitative scales. In those situations qualitative methods can be superior. They can also produce results that look similar to anecdotes, but the method of collection differs and this is critical.

  13. #13
    151
    151 is offline
    RoadBikeReview Member
    Reputation: 151's Avatar
    Join Date
    Apr 2009
    Posts
    1,590
    I think I just can't express my though well.

    if we are most concerned with the result, and not the cause that is when I think anecdotal evidence may be better than the scientific method.

    No matter how hard we try it is impossible to control all variables in a test, because there are going to be factors that we are unaware of.

    Where using experience based results, the variables are not relevant, only the result.

    Or, I'm just a moreon.
    You. You are the frame pump noob.
    ----Creakyknees

    then I did it and crawled into bed with my gf and that aspect alone changed EVERYTHING. It felt soooo good
    ----Ghost234

  14. #14
    RoadBikeReview Member
    Reputation:
    Join Date
    Dec 2004
    Posts
    1,076
    Quote Originally Posted by 151 View Post
    I think I just can't express my though well.

    if we are most concerned with the result, and not the cause that is when I think anecdotal evidence may be better than the scientific method.

    No matter how hard we try it is impossible to control all variables in a test, because there are going to be factors that we are unaware of.

    Where using experience based results, the variables are not relevant, only the result.

    Or, I'm just a moreon.
    To come at this a different way, are you saying that we don't need to gather information scientifically in order to make decisions? I think most people would readily agree that the scientific method isn't the only way of making decisions. We use other methods all the time, and often with acceptable results. Is that where you were headed? In terms of outcomes based action this is a departure from the scientific method, it's a different form of "knowing" so to speak.

    In terms of controlling variables in tests we can only do our best. We express our confidence in tests with statistical test of significance and the life. There is always potential for error and even if systematic collection this is accounted for. The possibility of testing errors, failed controls, etc are part of measuring the confidence we have in the results of an analysis.

  15. #15
    eminence grease
    Reputation: terry b's Avatar
    Join Date
    Jan 2004
    Posts
    18,561
    Quote Originally Posted by Fredke View Post
    Two things:

    First, Newton's anecdote was that he was sitting in his study, looking out the window and saw the apple fall, not that it hit him on the head.

    Second, the apple inspiration had nothing to do with evidence. It was inspiration. The evidence was thousands of systematic observations by Tycho.

    Experience is good for giving you ideas, but then you need a way to test the ideas and see whether there's anything to them.
    So much for me thinking I know anything.
    You'd be better off with a netbook, they do everything better.

    My travel blog: http://tbaroundtheworld.blogspot.com

  16. #16
    151
    151 is offline
    RoadBikeReview Member
    Reputation: 151's Avatar
    Join Date
    Apr 2009
    Posts
    1,590
    Quote Originally Posted by Scott B View Post
    To come at this a different way, are you saying that we don't need to gather information scientifically in order to make decisions? I think most people would readily agree that the scientific method isn't the only way of making decisions. We use other methods all the time, and often with acceptable results. Is that where you were headed? In terms of outcomes based action this is a departure from the scientific method, it's a different form of "knowing" so to speak.

    In terms of controlling variables in tests we can only do our best. We express our confidence in tests with statistical test of significance and the life. There is always potential for error and even if systematic collection this is accounted for. The possibility of testing errors, failed controls, etc are part of measuring the confidence we have in the results of an analysis.
    I guess I was just thinking that in some cases, long term healthy eating for example, anecdotal evidence gathered over a long period of time may be superior to a scientifically tested and proven diet.

    Because being healthy is more important than knowing why you are healthy, and because. Their are aspects of something as complex as the human body that we still do not fully understand, so we cannot control all the variables, so we do not truly know what caused a result.
    You. You are the frame pump noob.
    ----Creakyknees

    then I did it and crawled into bed with my gf and that aspect alone changed EVERYTHING. It felt soooo good
    ----Ghost234

  17. #17
    banned
    Reputation:
    Join Date
    May 2011
    Posts
    46
    Quote Originally Posted by 151 View Post
    I think I just can't express my though well.

    if we are most concerned with the result, and not the cause that is when I think anecdotal evidence may be better than the scientific method.

    No matter how hard we try it is impossible to control all variables in a test, because there are going to be factors that we are unaware of.

    Where using experience based results, the variables are not relevant, only the result.

    Or, I'm just a moreon.
    Don't worry 151,

    We're all morons in one way or another...One day somebody's going to push the button and prove it!

  18. #18
    Frog Whisperer
    Reputation: Touch0Gray's Avatar
    Join Date
    May 2003
    Posts
    36,811
    ok, all you real scientists PLEASE correct me if i am wrong. The way i understand the scientific method is that one formulates a hypothesis then sets about in an attempt to disprove it. And like Terry i would consider anecdotal evidence to be observation if it was documented.
    Of course I'm sure...that doesn't mean I'm right......

  19. #19
    haole from the mainland
    Reputation: jorgy's Avatar
    Join Date
    Oct 2005
    Posts
    5,966
    I'm going to suggest you read some of David Gorski's stuff over at science based medicine. Specifically, the use of Bayesian inference, which I think can be applied to what your concept of 'anecdotal information.' http://www.sciencebasedmedicine.org/?s=bayesian

    All that anecdotal information is prior information in Bayesian parlance, and shouldn't be ignored when we interpret the results of a single study.

    As far as variation and variables, that's why randomization is so important when conducting sampling or designing an experiment. Randomization ensures that all that unknown stuff (variety) is distributed without bias.

  20. #20
    151
    151 is offline
    RoadBikeReview Member
    Reputation: 151's Avatar
    Join Date
    Apr 2009
    Posts
    1,590
    Quote Originally Posted by jorgy View Post
    I'm going to suggest you read some of David Gorski's stuff over at science based medicine. Specifically, the use of Bayesian inference, which I think can be applied to what your concept of 'anecdotal information.' http://www.sciencebasedmedicine.org/?s=bayesian

    All that anecdotal information is prior information in Bayesian parlance, and shouldn't be ignored when we interpret the results of a single study.

    As far as variation and variables, that's why randomization is so important when conducting sampling or designing an experiment. Randomization ensures that all that unknown stuff (variety) is distributed without bias.
    That's really cool.

    My next thread will be about how just because one of us doesn't know something, doesn't mean smarter people haven't already thought of the same idea and sorted it out.

    How awesome would it be if everyone knew everything?
    You. You are the frame pump noob.
    ----Creakyknees

    then I did it and crawled into bed with my gf and that aspect alone changed EVERYTHING. It felt soooo good
    ----Ghost234

  21. #21
    We have met the enemy...
    Reputation: paredown's Avatar
    Join Date
    Oct 2006
    Posts
    6,911
    Quote Originally Posted by jorgy View Post
    All that anecdotal information is prior information in Bayesian parlance, and shouldn't be ignored when we interpret the results of a single study.
    The fallacy of the "scientific" method is that it pretends that scientists stand in some value-free box and formulate hypotheses which they can then test, which if it were literally true would make the formulation of hypotheses the hardest (if not the impossible) part of science.

    In fact, it is often the easiest part precisely because they have so much prior information.

    The great legal thinkers understood this, arguing that it was "prejudice" (literally--before judgement)--the mental furniture that we have--that makes solving complex puzzles possible, and that the process is to look for information to confirm or deny what often begins as an anecdotal piece of evidence, observation or the need to decipher the causal chain, as in a complex court case. This is probably truer to the actual process of doing science.

    For a philosopher's critique of the myths that scientists tell themselves about their own practices, I recommend a good reading of Feyerabend,--to quote from the Wiki article,
    Feyerabend was critical of any guideline that aimed to judge the quality of scientific theories by comparing them to known facts. He thought that previous theory might influence natural interpretations of observed phenomena. Scientists necessarily make implicit assumptions when comparing scientific theories to facts that they observe. Such assumptions need to be changed in order to make the new theory compatible with observations.
    Nothing "value free" about the process...
    "When the spirits are low, when the day appears dark, when work becomes monotonous, when hope hardly seems worth having, just mount a bicycle and go out for a spin down the road, without thought on anything but the ride you are taking."

    (Sir) Arthur Conan Doyle. Scientific American, January 18, 1896

  22. #22
    haole from the mainland
    Reputation: jorgy's Avatar
    Join Date
    Oct 2005
    Posts
    5,966
    Quote Originally Posted by 151 View Post
    How awesome would it be if everyone knew everything?
    According to the computers are freaking amazing thread, we should all be able to carry around everything that's known on our cellphones in just a few years.

  23. #23
    jaded bitter joy crusher
    Reputation: Fredke's Avatar
    Join Date
    Dec 2004
    Posts
    19,768
    Quote Originally Posted by paredown View Post
    The fallacy of the "scientific" method is that it pretends that scientists stand in some value-free box and formulate hypotheses which they can then test, which if it were literally true would make the formulation of hypotheses the hardest (if not the impossible) part of science.
    Coming up with interesting hypotheses is, in fact the hardest part. There's good discussion of this in Jacob Bronowski's "Science and Human Values."

    Think of who we think of a the greatest scientists: Newton, who came up with the hypothesis that the same force of gravity that pulled the apple from the tree also made the moon orbit the earth and the earth orbit the sun.

    Einstein, who came up with the hypothesis that the laws of nature are identical in all inertial reference frames. Einstein also came up with the hypothesis that light acts both as a wave and as a particle.

    Bohr who hypothesized that electrons in atoms also behaved as both waves and particles.

    Charles Darwin, who hypothesized that variation plus natural selection were what created the variety of species that we see.

    Stephen Hawking, who hypothesized, among other things, that black holes weren't actually black, but would emit radiation and eventually dissipate.

    Most of these guys came up with the hypothesis but didn't test it.

    No one I know says that science is value free. It's full of value judgments about what's adequate evidence and what's an interesting hypothesis.

    Feyerabend is great and I love his stuff, but do realize that Feyerabend says that voodoo is just as valid as modern medicine for treating illness because according to his philosophy there's no value-free way to judge which of the two is better. He also disagrees with assertions that astrology has been proved not to work. If you're comfortable with those conclusions, then go ahead and accept everything Feyerabend has to say, but for the rest of us these kinds of conclusions indicate that there are some problems with his point of view.
    Fredke commented in your thread. You won't believe what happens next!

  24. #24
    Bikespace member
    Reputation: moschika's Avatar
    Join Date
    Feb 2004
    Posts
    4,464
    doesn't it depend on what you are studying? if you're looking at atoms and all that kinda stuff, then "experience" may not give you the info you are looking for.

    but if you're looking at why people may do what they do or get an appreciation how people develop an aspect of one's self, then "experience" or "lived experience" will give you much richer information then counting widgets.

    and then there's the Grounded Theory approach where you study people first then develop a theory based on the observation.

    I think culturally we place a higher value on quantitative research then we qualitative research. quantitative research is good at showing a cause and effect/no effect relationship, while quantitative i think better explains human behavior and phenomena.
    Quote Originally Posted by Jesse D Smith
    You can always tell the guy that started his day with a proper shave versus the guy who started his day slappin' it to internet porn.

  25. #25
    We have met the enemy...
    Reputation: paredown's Avatar
    Join Date
    Oct 2006
    Posts
    6,911
    Quote Originally Posted by Fredke View Post
    Coming up with interesting hypotheses is, in fact the hardest part. There's good discussion of this in Jacob Bronowski's "Science and Human Values."
    Which is why most scientists only ever do "normal" science using hypotheses that are in fact constructed around an acceptance of basic theories, etc, and as Kuhn argues, only a few hit the level of interesting, or paradigm shifting as your examples show.

    I was making a slightly different point--emphasizing that the formulation of any testable hypothesis involves a whole set of assumptions about what questions to ask, what is interesting, how to test it, etc--and these are not generated in a vacuum but more often (as your Newton example suggests) by "inspiration" which is not too different from the OP's comment re anecdotal evidence. The significance of the "facts" is already suggested by the theory in your head--even with your Newton example, Brahe's observation became significant after Newton had made some sort of intuitive leap--until then no one knew what to make of Brahe, and certainly not to connect to a unified theory as Newton eventually did.

    (Once you are embarked on a project, I understand the need to randomize etc to prevent observer bias...)

    Quote Originally Posted by Fredke View Post
    If you're comfortable with those conclusions, then go ahead and accept everything Feyerabend has to say, but for the rest of us these kinds of conclusions indicate that there are some problems with his point of view.
    You're overstating here--I was only trying to correct the impression from your original poast that there was neutrality in "scientific" method--that fact and theory are independent things "out there" and the thinking about them not colored by anecdote, hearsay etc. I view Feyerabend as a bit of a tonic, but certainly not the whole story....

    As for voodoo--I may draw the line at that, but I have had great personal success with acupuncture, chiropracty etc--all things that "scientific" medicine has trouble accepting or explaining.

    So, like the OP--I think you can say "this works for me" and not be bothered if voices on high dismiss it as if it were voodoo, especially for complex systems like our own bodies.
    Last edited by paredown; 05-29-2011 at 06:36 PM.
    "When the spirits are low, when the day appears dark, when work becomes monotonous, when hope hardly seems worth having, just mount a bicycle and go out for a spin down the road, without thought on anything but the ride you are taking."

    (Sir) Arthur Conan Doyle. Scientific American, January 18, 1896

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

EUROBIKE

Hot Deals

Latest RoadBike Articles


Latest Videos

RoadbikeReview on Facebook