Actions
Forum Thread Details
  • Replies 13 replies
  • Subscribers 608 subscribers
  • Views 1244 views
  • Users 0 members are here
Related

A perspective on Testing.

DAB

Testing is a natural scientific response to confirm or dispel an idea.

Testing falls into two primary functions, Verification and Validation.

 

Most people are familiar with Verification testing.  You use the tests to verify that the idea/device/system provides the expected outputs given the proper input stimulus.

 

In component testing, you want to verify that the device responds as designed or advertised.  I have had a lot of fun with the latter as some marketers just cannot help themselves when making outlandish claims for their new toys.

I worked as an Independent Verification and Validation engineer on some very large aerospace systems and I had a wonderful time deflating exotic ideas about the technology implementations being developed.

 

For the most part, Verification Testing is straight forward.  You establish the inputs and measure the outputs.  Standard black box testing technique.  The device either meets specification or it does not.

 

Validation Testing is similar but very different from Verification Testing.

 

Validation involves analyzing the resulting Verification Testing results to determine if the final product satisfies the intended uses of the product.

 

In essence, Validation assess the question of can the intended users, use the product to do the job they need it to do.

 

Depending upon your user base, this becomes a very difficult task.

Each user has a perceived notion of what the product needs to do for them.  Very subtle implementation decisions can result in a product that is technically perfect, but useless.  Sort of like contacting Microsoft Support!

 

In some cases, Validation testing takes you out of the comfortable area of specific input and output and into a more fuzzy world of user perception.  Not an easy task, thought there are some excellent analysis tools available to help resolve this level of testing.

 

Most of my current work involves Validating my current solution for a Unified Field Theory.

The issues involved are both technical and perception.  Many of the existing scientific theories are used with both verified test results and with rationalized validation through consensus.

 

This last issue is the biggest hurdle.  Especially since I bring into question the works of Nobel Laureates in Theoretical Physics for the past 120 years.

When I tried to verify their results, I discovered that most have never been really validated or indeed verified with hard testing.

 

That result surprised me.  I had been taught that science was built entirely by independently verified testing.

 

That is not the case with many of the current Theoretical Physics Theories.  The excuse has been based upon the level of complexity and untestable nature of the issues involved. 

 

Using standard analysis of public domain data, I have been able to show that most of these excuses are not true and that those theories are false.

 

So my advice on any of the current theories is to trust but verify.  There is nothing, including ideas, that cannot be tested.

If someone cannot provide a theoretical model that cannot be tested, then it should not be used until sufficient model definition is developed and adequate independent testing can occur.

 

DAB

  • in reply to DAB

    Remind me not to use your psychologist image

  • in reply to mcb1

    Hi Mark,

     

    I agree with being too stubborn.

    I probably should have said persistent.

     

    Still, by being open minded, you can assess new ideas and comments of your existing ideas and extrapolate potential issues.

     

    Just yesterday I had an interesting talk with my psychologist and he got me thinking about diffraction limited optics. (Don't ask how we go there!)

    So last night I came up with an explanation that shows why you need  aperture sizes of 1.22 of the wavelength.

    In itself, understanding why is useful, but not necessarily important.  However, once I understood what drove the limit, I may have stumbled onto a new way to focus all EM photons to unheard of levels of accuracy.  I wrote about four pages of notes, identified at least two potential physics papers and a number of potential patentable ideas.

     

    So you just never know where inspiration will come from.

     

    DAB

  • in reply to mcb1

    I agree with you Mark. A hypothesis is just a theory maybe expressed in a different form. We suppose something, maybe coming from an intuition then we try to deducts experimentally + previous knowledge + a lot of other possibilities that this same hypothesis is true usually in two steps: first starting from the hypothesis we formalise a theory then it's the experiment / testing loop to demonstrate true or false the theory.

     

    Enrico

  • in reply to mcb1

    Hi Mark,

     

    Don't take my example too literally. It is the process that is important.

     

    "BTW I'm a realist, not optimist or persimist.

    My glass is neither half full or half empty it's simply waiting for me to finish it " 

    OK,  but I would have guessed "Optimist" based on all the help you have given to this Forum and its denizens as well as the students that you help with electronics. There has to be an optimist in there someplace.

     

    John

  • in reply to jw0752

    John

    Sorry but that example is simply an attempt to prove an observation.

     

    For me the potential problems with this is.

    1. Are the blocks uniformly stacked each time

    2. Are the blocks the same size, weight and density.

     

    It could still prove correct, but given the variable at play, I don't think anyone could reasonable sustain a theory of 2 blocks.

    If you modified it to will displace one or more blocks ... then maybe it would be correct.

     

     

    Perhaps we have conducted the experiment 10 times and each time two square block pop out

    I'm sure there is some clever person that will demonstrate that probability has some bearing on the exact decision point.

    IMO 10 attempts is too small, as the margin for error has to be one, and is therefore 10%.

     

     

    BTW I'm a realist, not optimist or persimist.

    My glass is neither half full or half empty it's simply waiting for me to finish it   image

     

     

    Mark

  • in reply to mcb1

    Hi Mark,

    I will use a simple fabricated hypothesis and experiment to demonstrate. Suppose that we have a situation where we notice that when we drop a ball into a box two square blocks pop out. As scientists we decide to make a hypothesis that whenever a ball is dropped into the box 2 square blocks will pop out. We begin to experiment dropping the ball into the box from different angles and at different velocities. Suppose that on the second try three square blocks pop out instead of two. This is a false result and it means we have shown that our original hypothesis was wrong. We have easily disproved the hypothesis. It is false. On the other hand suppose that two square blocks continue to pop out time after time. Perhaps we have conducted the experiment 10 times and each time two square block pop out. With each experiment, as we vary the angle and speed of the drop, our confidence in the hypothesis grows. At what point however do you as a scientist want to say that in all the possibilities for choosing angle and speed there is not one that will result in 1 square or 3 squares popping out? We can't be certain that there isn't some circumstance whereby our hypothesis will fail in the next experiment. Ideas that continue to survive experimental tests like the Conservation of Energy for example have been tested so many times and in so many ways that there is high confidence in them but a truly scientific view is that even these pillars of our science are just ideas that can be removed with one experiment that shows that they fail under some new circumstance.

    John

  • in reply to jw0752

    Science is not able to prove that a hypothesis is true, only that it is false

    I'm curious about why.

    Surely you can prove that a theory is true, equally as well as proving it is false.

     

    Mark

  • in reply to DAB

    When we stop learning, we are all doomed.

    Correct.

     

    All I can do is plant the seeds, provide trail of bread crumbs, and let the curious follow the path to enlightenment.

    The books at least get my ideas documented world wide beyond the reach of anyone to suppress them.

    I mix with some really clever EE at a robotics club, where they cover just about everything except robots!.

    I sometimes find myself going WTF are they talking about, are they talking some foreign language because I only caught two or three words in those last few sentences.

     

    Physics and all the theory is something that hurts my head, so you'll find me sitting there.

    It's not that I subscribe to either camp, it's just that you need to understand something before you can even start to decide.

    I prefer to not be a sheep and simply follow what is considered 'mainstream', I prefer to have a basic understanding, even if it is simply to place it with all the blue pieces of the jigsaw.

     

    Plus I happen to be one very stubborn person.  What can I say, it is in my blood

    Stubborn is fine as well as it doesn't cloud objectivity.

    I've seen some stubborn people finally come a cropper in a big way because they failed to consider the alternatives.

     

    Mark

  • in reply to mcb1

    Hi Mark,

     

    When we stop learning, we are all doomed.

     

    I did not embark upon my journey for fame or fortune.  Like Steven Hawking, I want to know how the Universe Works before I die.

    I now have an advantage over Steven as I feel pretty confident that I now have a plausible answer and he is stuck with theories that will not work.

     

    Validation is the hardest test to pass.  Often, it takes years, decades and centuries, but in the end, the truth will come out.

     

    Many people will have to run the experiments and tests, assess the mathematical models and consider the evidence I cite as reasons why I think I am right.  I fully agree that I could be wrong, but then I apply my models to existing experiments and data to get more plausible answers than others have provided to date.

     

    All I can do is plant the seeds, provide trail of bread crumbs, and let the curious follow the path to enlightenment.

    The books at least get my ideas documented world wide beyond the reach of anyone to suppress them.

    I am encouraged after my first public talk that there are still educated people willing to listen and ask questions.

     

    Time will finally validate my ideas.

     

    I do not worry that I am trying to climb a mountain with a near vertical face.  New ideas always face that hurdle, but there is no mountain that cannot be climbed with planning, preparation and persistence.

     

    Plus I happen to be one very stubborn person.  What can I say, it is in my blood.

     

    As the Vikings used to say, I am standing at the crossroad with no desire to run.

     

    I am not afraid of the truth, I have presented my ideas, come read and discuss.   Open minds are welcome.  Closed minds have already lost.

     

    DAB

  • in reply to jw0752

    You make a great straight man John.

     

    You hit the nail right on the head.  No single test ever validated anything!

     

    Each test, no matter how elaborate can only provide you with a pass/fail on the specific question the test was to answer.  That is it.  No light bulb in the sky, no bolt of lightning, nothing.  Just one answer to a complex problem.

     

    As for "elevated" people, a good Phd friend of mine pointed out that as you get higher in the education and research structure, you actually know less and less about anything other than the specific issue you study or research.

    So anyone with letters after their name are really unqualified to speak on any issue other than their specific field of study.

     

    Plus you also identified the key issue.  Until a theory can be used to solve everyday problems, it is totally useless.

     

    That is why my specific investigations goes through the underlying issues, cites existing test and experimental data, but defines the "process" by which my theory would evolve over time to build what we can see and measure in the universe around us.

    I also introduce mathematical models that only require basic algebra and science to understand.

    The result is that my math, works, and you can apply all collected measurements and many observations to fully explain what happened and why.

     

    Try asking anyone with a Phd to do that and see what you get.

     

    Which brings us to the reason for my post and the identification of the Validation issue.

    When people can USE a theory, they validate it.

    If people cannot use a theory, then it provides no value.

     

    Believe me, by engineering use requirements are very high.  Hence my efforts over the last seven years to find answers about how the universe works so that people can solve everyday problems.  Not introduce alternate dimensions, uses, mass, energy, imaginary particles and fields, all of which are interesting but basically useless.

     

    DAB