Is "personal trustworthiness" a big problem in safety-critical software?
Question:
be running a testing group.
You've got to ask youself if the test section has been effectively disbanded in some sort of cover-up.(After all you wouldn't want them to be too effective - they might find problems in systems you'd already foolishly released).
You'd shudder to think of any software, let alone "safety critical software" not going through a formal QA process including:
- Requirements analysis
- White box testing focusing on saftety critical items
- Black box testing and system testing done by an independent group using real-world scenarios.
As a developer, development manager, tester, and test manager you'd expect a clear process that yields predictable results
But how do you know if any genuine testing is taking place ?
Answer:
There is a methodology called bebugging, where you can test the effectiveness of the software testing process by putting bugs into the code and then seeing how many of your bugs are found. The bugs put into the code should be similar in type to the bugs that you expect are actually in the code. You can't put a bunch of simple bugs into the code and get good results at determining if testing is effective in finding hard-to-find bugs. You can estimate the effectiveness of the testing process by the percentage of the bebugging inserted bugs found by testing.
However, no development organization I know of does bebugging. They all review the test plans and then rely upon the honesty of the testers to follow the test plans. This includes mission critical software for NASA. I think the reason is that inserting bugs that mimic the expected bugs is very hard for people to do. Some bugs are very subtle, and coming up with subtle bugs is difficult.
There is a substantial amount of automatic testing software, like valgrind that will find some classes of bugs. Unfortunately it doesn't find many classes of bugs. The automatic testing of software if really difficult.
That is a good question, but also one that has always and will always plague software engineers. As you probably know the software development cycle is a long and tedious one, depending on the size of the program and the cycle you are using to create the program.
The requirements, documentation, implementation, and testing phases actually only take up 20% of the process time on average because 80% is being used for maintenance after the software is released.
This means that almost all software companies are not doing correct, or thorough testing, but are attempting to.
The fault of the testing could be the system(s) the software is being tested on (not the same as some of the workstation environments that this software will be deployed on) or in my opinion is probably the lack of testing from the testers.
I believe the best way to know if the program is being tested is to create a database on the server where the software is being tested. You can have the program insert into the database some sort of log events such as (pseudo):
"Hi, I am User blah-blah, I tested at this time, and I have just entered this module, function, subroutine, etc..."
It would definitely slow the program down, and if this is a giant program, putting queries in every function would be ridiculous. But careful placement can definitely prove better results. Bear in mind if a lot of testing is taking place this may eat up server bandwidth.
This is my off the cuff opinion, for I am not YET a professional software developer but still an undergrad.
I still hope this helps...
Good Luck
More Questions & Answers...