Case Study 3:
Performance Testing

Our Client

One of the leading providers of SCM Services across Caribbean Islands

The Need

Every year dozens and dozens of major online retailer’s experience performance problems when it matters most back to school days, Black Friday, Cyber Monday, the holiday shopping season. And it is not like they have not been performance testing for peak traffic loads since the summer.

Most often the problem is that they are encumbered by an old-fashioned, difficult-to-use, manual labour-intensive load, and performance testing solution. As peak loads get more intense and eCommerce systems get more complex, performance testing gaps widen and risks multiply.

 

  • Their Business priority was to reduce “Time to Market”, so the Testing Cycle needed to be reduced

  • Testing had to be done across 13 Caribbean Counties, environment, and browsers

  • The Performance suite size was increasing with each release due to new functionality leading to increased test effort

Our Solution

IGT proposed Performance testing approach for faster test execution and reduced cycle time

  • IGT propose complete system study to identify every transaction TPS

  • LoadRunner Performance testing automation tool was used for automation.

  • A hybrid automation framework was designed using LoadRunner.

  • Execute the same script against multiple datasets and environment

  • Compatibility Test - Run the same script on IE, Chrome & Firefox

  • Only Test data sheets needs to be updated

  • Easier maintenance and Continuous Integration

  • Customized reports and remote execution

  • Concurrent execution of test script using LoadRunner which reduces the execution time

  • Every release Performance test has been bench marked.

  • In addition to this, performance of critical bugs related scenarios was also done

Phase 1 –Planning and development of Performance scripts

  • Load Runner installation

  • Knowledge transfer from QA team

  • Prepare Performance Test plan

  • Scripting Scenarios

  • Business Process Strategy preparation

  • VUser Specification document

 

Phase 2 – Execute, Bottleneck Identification

  • Project downtime notification / Black Friday test

  • Environment Preparation

  • Site Administration Freeze

  • Phased Backup of Project Database

  • Execution of the scripts and results analysis

  • Recommendation of fixes

 

Phase 3 – Re-execution of the Scenarios

  • Environment Preparation

  • Project downtime notification

  • Site Administration Freeze

  • Phased Backup of Project Database

  • Re-execution of the scripts and results comparison

  • Performance Summary Report preparation

  • Handover

Benefits

Performance issue resolved, identified major roadblocks

Find below benefits of technical solution proposed to the client

  • Traced the transactions taking long time to respond

  • Found missing indexes on Database

  • Code was optimized to reduce time taken to load page

  • SQL queries were tuned

  • Garbage Collection time reduced to free memory

  • Heap size increased

  • Client saw significant improvement in the transaction response time with the implementation of changes when compared the results from the first round of execution with the second round.

  • Increased the productivity of the team for manual testing by 40%

  • Good number of critical and major defects have been identified by the Performance test scripts

  • Scripts are executed in different browsers and various environments by just changing the required URL parameters in spread sheet.

  • HTML Report logs, spotlight and snapshots are generated for tracking defects and exceptions

  • Framework served at the enterprise level

Technology Platform

Technology Platform LoadRunner, Spotlight, Web HTTP/S, Selenium WebDriver, JAVA, Winium, Sikuli, Robot class/ TestNG/ Maven