software testing - 2

III. At what stage of the life cycle does testing begin?

                    Testing as an activity no longer waits till the first line of code is written. In today’s world of ever increasing complexity, testing has to begin right at the conceptual stages. Experts are evolving techniques to conduct model tests on the requirements model, domain model and design model.

                   Testing starts from the beginning - we can start testing from as early as Requirement specification stage of the project. Experience shows that close attention to the requirements in the beginning allows eliminating a lot of problems

When should testing start in a project?

                    The earlier you start the testing process, the better. The longer a defect stays in process, the more expensive it is to fix. It is said that on average, programmers create six to eight defects in every 100 lines of code (LOC) .They create 12 to 20 defects per 100 LOC if the code is not structured and is poorly documented. These defect rates improve to two to three defects per 100 LOC if the code is structured and documented. However, the rates only improve to one to 1.5 defects per 100 LOC for subsystem, programs, or modules after typical unit testing. Although the above metrics are alarming, we also know that 70 percent of the defects in most systems are caused during the analysis and design phases, and 30 percent are introduced during coding. Using one of the most widely referenced metrics in the industry, we can expect costs to fix requirements defects (found late in system testing or after delivery) to be twice orders of magnitude or more than when found during requirements analysis.

How automation fits into the overall process of testing?

                      Test automation provides the answer to an organization’s business problems by minimizing the effort required for testing systems, while ensuring uniformity in the testing process. Using automation, tests can be run faster, in a consistent manner and with fewer overheads. Automation is the replacement or supplementation of manual testing with a suite of test programs. Benefits include increased software quality, reduced time-to-market, reusable test procedures, and reduced testing costs.

What is base for test cases?

                    We can base the test cases on documents like System Requirement Specifications (SRS), High Level Design, Low Level Design, Use cases, Blue Prints etc. From SRS and High level design documents you can create System Test cases, from low level design docs you can create Integration Test cases and from use cases you can create Unit test cases.

What is an equivalence class?

                 In the software testing, if the input domain is usually too large for exhaustive testing, it is therefore partitioned into a finite number of sub-domains for the selection of test inputs. Each sub-domain is known as an equivalence class and serves as a source of at least one test input.

                Equivalence Partitioning: An approach where classes of inputs are categorized for product or function validation. This usually does not include combinations of input, but rather a single state value based by class. For example, with a given function there may be several classes of input that may be used for positive testing. If function expects an integer and receives an integer as input, this would be considered as positive test assertion. On the other hand, if a character or any other input class other than integer is provided, this would be considered a negative test assertion or condition

The roles of QA play in the software lifecycle.

                    Software QA Engineer works closely with the development team and provides technical leadership in testing to ensure the release of the highest quality products, including developing, maintaining, and implementing test strategies, plans, procedures, and cases. In addition, the Software QA Engineer drives the full software project lifecycle of each software project from gathering requirements to post implementation review and continually introduces improvements to streamline QA and development of processes and procedures

What is a "bug?"

                   A fault in a program, which causes the program to perform in an unintended or unanticipated manner. See: anomaly, defect, error, exception, and fault.

How do you feel about cyclomatic complexity?

                  Cyclomatic complexity is a measure for the complexity of code related to the number of ways there are to traverse a piece of code. This determines the minimum number of inputs you need to test all ways to execute the program

IV.Types of testing:

            Based on project stage     

                          i) Unit Testing,

                        ii) Integration Testing,

           Alpha testing

            Beta testing

Based on kind of testing

                         i) GUI testing,

                        ii) System Testing,

                      iii) Regression testing.

.

What is unit testing?

           Testing of a module for typographic, syntactic, and logical errors, for correct implementation of its design,  and for satisfaction of its requirements. Testing conducted to verify the implementation of the design for one software element; e.g., a unit or module; or a collection of software elements. Syn: component testing.

What is Regression testing?

                The complete retesting of a software system that has been modified to ensure that any bugs  have been fixed and that no other previously working functions have failed as a result of the reparations and that newly added features have not created problems with previous versions of the software. Also referred to as verification testing, regression testing is initiated after a programmer has attempted to fix a recognized problem or has added source code to a program that may have inadvertently introduced errors. It is a quality control measure to ensure that the newly modified code still complies with its specified requirements and that unmodified code has not been affected by the maintenance activity.

Difference between Integration & System testing:

                        Integration testing is testing of combined parts of an application to determine if they function together correctly. The 'parts' can be code modules, individual applications, client and server applications on a network, etc. This type of testing is especially relevant to client/server and distributed systems. This type of testing conducted after unit and feature testing. The intent is to expose faults in the interactions between software modules and functions. Either top-down or bottom-up approaches can be used. A bottom-up method is preferred, since it leads to earlier unit testing (step-level integration) this method is contrary to the big-bang approach where all source modules are combined and tested in one step.

                       System testing is the testing of a complete system prior to delivery. The purpose of system testing is to identify defects that will only surface when a complete system is assembled. That is, defects that cannot be attributed to individual components or the interaction between two components. System testing includes testing of performance, security, configuration sensitivity, startup and recovery from failure modes.

Validation and Verification.

                Verification – intended to show that software correctly implements a specific function; typically takes place at the end of each phase.

       Are we building the product right?

               Validation – intended to show that software as a whole satisfies the user requirements: typically uses black-box testing.

       Are we building the right product?

               Verification typically involves reviews and meetings to evaluate documents, plans, code, requirements, and specifications. This can be done with checklists, issues lists, walkthroughs, and inspection meetings. Validation typically involves actual testing and takes place after verifications are completed. The term 'IV & V' refers to Independent Verification and Validation.

Difference between Black & White box testing:

                 Black box testing. A testing method where the application under test is viewed as a black box and the internal behavior of the program is completely ignored. Testing occurs based upon the external specifications. Also known as behavioral testing, since only the external behaviors of the program are evaluated and analyzed.

                  White Box Testing (glass-box). Testing is done under a structural testing strategy and require complete access to the object's structure¡that is, the source code

V.Types of automated tools:

     Functional Testing Tools

a)         WinRunner versions 7, 7.5 from Mercury Interactive

b)         Quick Test Professional (QTP) 8

c)         Rational Robot from IBM Rational

d)         SilkTest from Seague

      Load Testing

A)        Load Runner from Mercury interactive

B)        Webload from Radview

C)        Silk Performer from Seague

      Unit testing tools

a)         JUnit

b)         NUnit

      Test Coverage Tools

a)         Test Coverage

b)         Rational Purify

      Test Management Tools

                Test Director

      Defect Tracking Tools

                Rational Clear Quest

                Buzilla

      Configuration Management Tools

                Rational Clear Case

                MS Visual Source safe

                               

VII. Why Software Testing?

To get adequate trust and confidence on the product.

-To meet the Organizational Goals like meeting requirements satisfied customers, improved market share, zero defects, etc

-Since the software can perform 100000 correct operations per second, it has the same ability to   perform 100000 wrong operations per second, if not tested properly

Some Software Failures:

1. Computer Glitch Causes £22m Tax Error; Mar 22, 2002: The Inland
   Revenue has said that computer problems were responsible for an estimated
   134,000 basic rate taxpayers being overcharged by a total of £22m. Individual  
   taxpayers paid an average of £148 over the odds because of data transfer
   problems between the national insurance computer system and the PAYE
   system.

2. Yahoo Glitch strikes again; Mar 20, 2002: Parts of Yahoo were shut down
   on Tuesday following software problems encountered in the integrating of
   Yahoo Groups and Yahoo Clubs.

3. Microsoft’s Anti-Unix Site Crashes; Apr 3, 2002: A marketing Web site,
    part of a multi-million dollar campaign by Microsoft and Unisys to get
    customers to switch from UNIX is turning into a major embarrassment.
    www.wehavethewayout.com which was powered by UNIX kit and that, when
    Microsoft switched it to Internet Information Server software, the site
    crashed completely.

4. Cisco Flaw Enables DoS Attack; Apr 2, 2002: Vulnerability in Cisco’s 
   Call Manager software can result in a memory leak in the computer telephony
   framework causing the server to crash, which could be used by a hacker to
   start a denial of service (DoS) attack.  The fault is most commonly seen when
   Call Manager systems are integrated with a directory such as Active Directory
   or Netscape.

VI.Standards

CMM and CMMI:

                Capability Maturity Models (CMMs) assist organizations in maturing their people, process, and technology assets to improve long-term business performance. The US based Software Engineering Institute has developed CMMs for software, people, and software acquisition, and assisted in the development of CMMs for Systems Engineering and Integrated Product Development. The latest development in this initiative is the CMM IntegrationSM (CMMISM) Product Suite.

                 The purpose of Capability Maturity Model Integration (CMMISM) is to provide guidance for improving your organization's processes and your ability to manage the development, acquisition, and maintenance of products and services. CMM IntegrationSM places proven practices into a structure that helps your organization assess its organizational maturity and process area capability, establish priorities for improvement, and guide the implementation of these improvements. Unlike CMM, which was based on software development, the CMM Integration is not restricted to software development. It encompasses the entire developmental organization processes.

What is Configuration management? Tools used?

               

                  Configuration management is the process of identifying and defining the deliverable product set in a system, controlling the release and change of these items throughout the system life cycle, recording and reporting the status of product items and change requests, and verifying the completeness and correctness of the product items. The common tools used are MS Visual Source Safe, Rational Clear Case, and CVS etc