ADVANCES ON TESTING SAFETY-CRITICAL SOFTWARE - Goal-driven Approach, Prototype-tool and Comparative Evaluation

Guido Pennella, Christian Di Biagio, Gianfranco Pesce, Giovanni Cantone

2006

Abstract

The reference company for this paper – a multination organization, Italian branch, that works in the domain of safety-critical systems – evaluated the major tools, which the market provides for testing safety-critical software, as not sufficiently featured for her quality improvement goals. Consequently, in order to investigate the space of possible solutions, if any, the company’s Research Lab. started an academic cooperation, which leaded to share knowledge and eventually to establish a common research team. Once we had transformed those goals in detailed technical requirements, and evaluated that it was possible to realize them conveniently in a tool, we passed to analyze, construct, and eventually utilize in field the prototype “Software Test Framework”. This tool allows non-intrusive measurements on different hard-soft targets of a distributed system running under one or more Unix standard OS, e.g. LynxOS, AIX, Solaris, and Linux. The tool acquires and graphically displays the real-time flow of data, so enabling users to verify and validate software products, diagnose and resolve emerging performance problems quickly, and enact regression testing. This paper reports on the characteristics of Software Test Framework, its architecture, and results from a case study. Based on comparison of results with previous tools, we can say that Software Test Framework is leading to a new concept of tool for the domain of safety-critical software.

References

  1. Anderson T. and Knight J.C., A Framework for Software Fault Tolerance in Real-Time Systems, IEEE Trans. Soft. Eng., Vol. SE-9, no.3, pp.355- 364, 1983.
  2. Basili, V. R., Caldiera, G., and Rombach, H. D., The Goal Question Metric Approach, Encyclopedia of Software Engineering, Wiley&Sons Inc., 1994.
  3. Di Biagio C., Pennella G., and Cantone G., Comparing Tools for Testing Critical Software. The Case Study of “Software Framework 2.0”, TR 20060426.1, MBDA Italy, 2006.
  4. Di Biagio C., Pennella G., Lomartire A., and Cantone G., An Introduction to Characterization of Monitors for Testing Safety-Critical Software, Procs. of ICSOFT 06 (these Proceedings), Setubal, 2006.
  5. Cantone, G., and Donzelli P., Production and Maintenance of Goal-oriented Measurement Models, International Journal of Software Engineering & Knowledge Engineering, World Scientific Publishing Company, Vol. 10, No. 5, pp. 605-626, 2000.
  6. EPRI, Handbook for verification and validation of digital systems, Vol.1: Summary, EPRI TR103291, Vol.1, 1994.
  7. IEEE, IEEE/EIA 12207.0-1996 Industry Implementation of International Standard ISO/IEC 12207: 1995 (ISO/IEC 12207) Standard for Information Technology Software Life Cycle Processes, in IEEE/EIA 12207.0-1996, 1998, pp. i-75.
  8. Leveson. N. G., Software safety: Why, what, and how. Computing Surveys,18(2):125-163, June 1986.
  9. Isaksen U., Bowen J. P., and Nissanke N., System and Software Safety in Critical Systems, December 1996.
  10. Lilja D. J., Measuring Computer Performance, Ed. Cambridge University Press, 2000.
  11. QUEST SPOTLIGHT™ http://wm.quest.com/library/ docs/spotlightwindows/SpotlightWindows.pdf (last access, March 2006).
  12. Simser D. and R.E. Seviora, Supervision of Real-Time Systems Using Optimistic Path Prediction and Rollbacks, Procs. Int'l Symp. Software Reliability Eng. (ISSRE), pp. 340-349, Oct. 1996.
  13. SOLARIS PERFORMANCE METER™ 2.0.0
  14. http://docsun.cites.uiuc.edu/sun_docs/C/solaris_9/SUNWa be/CDEUG/p125.html (last access, March 2006).
  15. TOP™ - William LeFebvre's http://www.uwsg.iu.edu/UAU/system/top.html (last access, March 2006).
  16. Tsai J.J., Yang S.J., Monitoring and Debugging of Distributed Real-Time Systems, J.J. Tsai and S.J. Yang, eds., IEEE CS Press, 1995.
  17. Ubench 2.0™ , http://www.phystec.com/download/ubench.html (last access, March 2006).
  18. Weyuker E.J., On Testing Non-Testable Programs, The Computer J., vol. 25, no. 4, pp. 465-470, 1982.
  19. VxWorks, http://www.windriver.com (last access, April 2006).
Download


Paper Citation


in Harvard Style

Pennella G., Di Biagio C., Pesce G. and Cantone G. (2006). ADVANCES ON TESTING SAFETY-CRITICAL SOFTWARE - Goal-driven Approach, Prototype-tool and Comparative Evaluation . In Proceedings of the First International Conference on Software and Data Technologies - Volume 1: ICSOFT, ISBN 978-972-8865-69-6, pages 218-223. DOI: 10.5220/0001311002180223


in Bibtex Style

@conference{icsoft06,
author={Guido Pennella and Christian Di Biagio and Gianfranco Pesce and Giovanni Cantone},
title={ADVANCES ON TESTING SAFETY-CRITICAL SOFTWARE - Goal-driven Approach, Prototype-tool and Comparative Evaluation},
booktitle={Proceedings of the First International Conference on Software and Data Technologies - Volume 1: ICSOFT,},
year={2006},
pages={218-223},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0001311002180223},
isbn={978-972-8865-69-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the First International Conference on Software and Data Technologies - Volume 1: ICSOFT,
TI - ADVANCES ON TESTING SAFETY-CRITICAL SOFTWARE - Goal-driven Approach, Prototype-tool and Comparative Evaluation
SN - 978-972-8865-69-6
AU - Pennella G.
AU - Di Biagio C.
AU - Pesce G.
AU - Cantone G.
PY - 2006
SP - 218
EP - 223
DO - 10.5220/0001311002180223