Help Define the Ultimate Black Box Test
There have been some suggestions about us facilitating validation tests for various parties. Last week at ICCF, I ran into an excellent fellow who had already put a lot of thought into this. Using some notes he sent me, I started a document flesh out the process. My hope is that if we are asked to do a validation of device without being able to see inside the device everyone including the inventor, our team, and the general public will know what to expect and be able to trust the results. Furthermore, I hope this helps to set the bar for anyone doing a validation test or public demo.
Please help us refine this into a comprehensive, well articulated document. You can review it here and make suggestions, or request to get editing access to the original document.
Comments
* introduction with background information / define why do you want test your device
* describe test object and list all documents related to the test object
* what should be tested / what shouldn't been tested (what can be tested see en.wikipedia.org/.../IEC_9126)
* prioritise your the features you want to test. do a risk analysis for this. test the features with a higher prio more extensive.
* list requirement documents (do you want to evaluate against some standards? )
* draft your test strategy (helicopter view). for black box testing you need a good documentation basis. but decide the right test method for every feature independently.
* test environment (plan test environment like you have done it already in your draft, keep the environment parameter under control is a good thing. knowing all your input data and environment parameters makes your test reproducible. keep them constant and stable.)
* plan the test driver for test automation. plan what is done manually.
* calculate what you need - human resources and all test ware. (make or buy decision)
* describe when a test run is assumed as failed or passed
* describe how to proceed when a test run is interrupted - in this case it is important to proof that the test object is still in a defined state.
* plan the execution of the test run itself step by step
* plan all tasks you have to do to prepare the test environment, test run, etc.
* what are the deliverables out of the test activities (documents, etc.)
RSS feed for comments to this post