Yahoo Answers is shutting down on May 4th, 2021 (Eastern Time) and beginning April 20th, 2021 (Eastern Time) the Yahoo Answers website will be in read-only mode. There will be no changes to other Yahoo properties or services, or your Yahoo account. You can find more information about the Yahoo Answers shutdown and how to download your data on this help page.

Question for skeptics: Is this an appropriate use of the derivative operation?

The following snippet has been lifted from one of the papers that scientists complained about in those leaked emails (link: http://climatedebatedaily.com/southern_oscillation...

"To remove the noise, the absolute values were

replaced with derivative values based on variations."

You can use your pdf reader's search feature to view this in its full context, if you wish.

Now, my question is: Is this application of the derivative operation really a good way to reduce noise, and is it appropriate for use on data where you wish to determine the presence or absence of a long-term trend?

Once again, I'd to give the skeptics here the first crack at this.

Dana1981, pegminer, linlyons, et al., feel free to jump in after the skeptics have given it a go. ;)

Update:

Actually, no context is necessary, provided that you understand what a derivative (or difference operation in the case of discrete data) does.

Since the global-warming signal is a long-term trend in the data (the slowest time-varying component), a derivative/difference operation will essentially remove it. Remember that a derivative acts as a high-pass filter.

If you run temperature data through a derivative operator before correlating it against SOI data, you will have removed the global-warming signal before doing the correlation! You can't take data, remove the global-warming signal, and then conclude that there's no global-warming!

The authors of this paper committed the type of blunder that has caused many a hapless college freshman to flunk out of 1st-semester calculus.

At this point, nobody deserves a "best answer" vote. Hopefully, someone with a clue will supply an answer that I worthy of the designation.

Update 2:

correction: above sentence should read, "*is* worthy of..."

9 Answers

Relevance
  • JimZ
    Lv 7
    1 decade ago
    Favorite Answer

    Damn it jim, I am a jim and an engineer. It didn't work for me either but I doubt it matters. I would say that they have to clearly state what they are doing. I have no problems with models if they clearly list their limitations and assumptions. What happens is the results of those models and simulations often get those limitations filtered out and are given to the public and politicians as fact when in fact the assumptions are poorly understood and probably wrong. We are all waiting with baited breath to hear from the guardians of science, Dana et al.

  • 1 decade ago

    The link doesn't work (at least for me).

    There are multiple methods used to reduce noise and decern a signal especially from time series data. I'm not familiar with the research you linked to (nor can I access it), but dendrochronolgy studies typically go through two detrendings, the first being an exponential decay function to remove age related decline in tree growth and the second can be anything from a linear fit to a moving average to remove "stand dynamics" variation. However a dendrochronolgist in my lab is perfectly willing to admit that there is as much art to this process as there is science.

    Edit: I'm still waiting for one of the all-knowing to enlighten the rest of us.

  • 1 decade ago

    when comparing change and rates of change, derivatives is the only way to do it, in this case it's not a textbook mathematical derivative. This was simply a difference operation dSO=a*(SO(i)-SO(i+1))-c, performed on a data series. The author was simply trying to see the lag between the SO data set and certain UAH data sets.

    Just off hand, a script of 200 to 300 lines of code could have handled the data nicely. Whay does this have to do with sceptics and AGW?, the paper does not address AGW.

  • 1 decade ago

    Damn it Jim, I'm an engineer not a number cruncher.

    I'm not sure the point of this question but I'm going to guess that if I can't come up with the correct answer then I'll fail the test an need to hand in my skeptic membership card. I must say, you guys are really grasping at straws lately.

    Edit: The link doesn't work because there is punctuation at the end of it. Just remove that period and parantheis.

    Note for all, when you put in http links, make sure there is no punctuation around it. It must stand on it's own.

  • How do you think about the answers? You can sign in to vote the answer.
  • Anonymous
    1 decade ago

    "derivative values based on variations" can mean a lot of things none of which are necessarily good or bad. There is nothing wrong with mollifying curves to smooth graphs. This can be revealing. Using "derivatives" of surrounding values may or may not be more judicious.

    The question is whether the data have been kept pristine. Why THESE tree rings and not THOSE? How is outlying data handled? Where is data truncated...?

  • Anonymous
    1 decade ago

    Looking at the Yamal data, in which 1200 trees where available...you're saying that Briffa's removal of 1197 of the trees was just "removing the noise"?

  • Steve
    Lv 4
    1 decade ago

    Sometimes yes, but you must clearly state what you are doing and what your assumptions are. You will also need to IMHO, compare your results with the noise included, if the error is too large then you have falty data and need to run the test again.

  • BB
    Lv 7
    1 decade ago

    Is the link correct? I'm not able to open it.

  • 1 decade ago

    It is pathetic and telling when alarmists have to search for something that isn't criminal or unethical.

Still have questions? Get your answers by asking now.