Best Test Management Tool



  Is there such thing as single best test management tool?
                                                                        
              “Quality is free, but only to those who are willing to pay heavily for it.” – T. DeMarco


What is the best test management tool?
  The right answer is it depends. As with everything in life there is no black or white answer to it as it depends on number of factors.

First of all what is Test management???
Test Management encompasses anything and everything that we do as testers.  Our day-to-day activities include:
1. Creating and maintaining release/project cycle/component information
2. Creating and maintaining the test artifacts specific to each release/cycle that we have- requirements, test cases, etc.
3. Establishing traceability and coverage between the test assets
4. Test execution support – test suite creation, test execution status capture, etc.
5. Metric collection/report-graph generation for analysis
6. Bug tracking/defect management

What are the primary factors while choosing the test management tools?
Based on our long and broad experience working on multiple tools, we have come up with few prominent test management tools based on their rich features provided along with cost effectiveness as below:
Parameter
Test Rail
Zephyr
TestLink
Testopia
Pricing
TestRail -Server Approx. $230 per user.
TestRail -SaaS, $25 per user per month.
Community Edition-Free-10 users-1 project
Zephyr for Jira - $10 - $14,000(10-10K users)
Zephyr Enterprise - $80 per user per month.
Open Source - Free to Use
Open Source - Free to Use
Ease of Use & Configurability
TestRail has a clean and easy to use test management interface.
TestRail facilitates flexible custom fields, configurable user roles, unique UI Scripts and a powerful JavaScript-based option to customize TestRail.
TestRail's test plans and the built-in configuration feature helps verifying tests against all the platforms and operating systems that are critical to the success of the project.
TestRail's unique forecasting feature lets you track the progress based on your historical time data.
Very flexible test management system.
Zephyr offers push-based, real-time metrics.
Easy drag and drop interface for reusing test cases across various projects.
Provides custom dashboards.
ZQL based search capabilities.
Intuitive requirements management application.
Highly customizable because of user-definable Custom Fields.
Allocating the work either test case. Creation/execution/preparing any kind of documents is easy.
Centralized User- and Role-Management.
Testopia interface is clean, intuitive and easy to use.
Testopia Inherits Bugzilla's powerful searching capabilities
Testopia utilizes a tagging mechanism to help organize test objects instead of virtual folders.
Test cases can have dependencies.
Test cases and Test runs can be cloned for reuse.
Several methods for restricting access to test data.
Bug Tracking
Can be Integrated with Jira, FogBugz, Trac,
Redmine, Bugzilla, OnTime and more…
Built in defect tracking system, can also be integrated with Bugzilla or Jira.
No inbuilt Bug Tracker available
Can be integrated with Bugzilla, Mantis, Jira etc.
Bugzilla is the default bug tracking platform can be integrated with Jira.
Automation & Integration (API)
TestRail integrates with bug trackers, automated testing tools, requirements and single sign-on.
Provides web services based API for Java, Python, PERL and PHP.
Zephyr for JIRA can be Integrated with automation and CI tools using ZAPI (sold separately).
Can be integrated with popular test execution tools such as QTP, Silk Test, Rational Robot, Selenium, JUnit, Watir, Test Complete, Load Runner, Silk Perfomer etc.
Automation of the test management is possible through web service XML RPC.
Automation of the test management is possible through web service XML RPC.

With Testopia-Jenkins CI plug-in it is possible to manage tests in Testopia, schedule and control in Jenkins, and execute using popular test execution tools such as TestPartner, Selenium, TestNG, Perl modules, prove, PHPUnit etc.
Export  & Import Test Cases
Import/Export test cases as XML, providing CSV/Excel/TestLink to TestRail migration scripts.
Import/Export test cases as Excel files
Import/Export test cases as XML
Import test cases using XML from other testopia installations/Export test cases as CSV or XML
Reporting, Statistics & Graphs
Actionable reports and powerful metrics that assist in making informed decisions.
Provides real-time status, activity and progress reports.
Provides Live, comprehensive Metrics and Trends reports.
Very good Reporting functionality including Test Reports, Metrics, Charts, Test Execution and Monitoring. it generates reports in HTML/ CSV /Excel formats.
Build Coverage Report, Completion Reports, Top Bugs, Bugs Found In This Plan, Printable Percentage Report, Estimated vs Actual Time and Historic Status Breakdown
Support
Support, Professional Services and Training. Supports by Email and online.
SaaS and On-Premise Deployments.
Support, Professional Services and Training. Supports by Phone, Email and online.
SaaS and On-Premise Deployments.
Open source with documentation online.
Installation process is little tedious but very good documentation available online.
Email Integration
Besides hosting and displaying reports inside TestRail, it is possible to forward reports as HTML attachments via email or download reports to archive them.

Reports could be mailed directly from the tool
Uses In-built email feature from Bugzilla

Conclusion:
With the presence of open source tools, the space of test management tool has become very dynamic & flexible in terms of customization based on the project to project requirements so instead of going for given features by standard expensive proprietary tools, we can use much better,flexible & inexpensive options available in the form of open source tools based on individual project requirements.

Comments

Post a Comment

Popular posts from this blog

Testing Webservice with SoapUI Having Windows Authentication

Sometims SoapUI doesn't seem to work directly with NTLM authentication, but you can use a proxy such as Burp Suite to do the auth for you. Following are the steps you can perform: Burp Suite Configuration     Download Burp Suite from http://portswigger.net/burp/ and set it up.     On Burp's "Proxy : Intercept" tab, click the button to turn intercept off.     On Burp's "Proxy : Options" tab, make sure it's set to an unused port, the default is 8081     On Burp's "Options" tab, tick "do www authentication" and add a setting for the server you wish to hit. Also tick "prompt for credentials on authentication failure"     Switch to Burp's "Proxy : History" tab so you can see requests going through. SoapUI Settings     In SoapUI, choose File > Preferences, then select "Proxy Settings". Enter Host "localhost" and port "8081".     Use SoapUI as normal. It will s
Read more

How to Re-run Failed Scenarios in Cucumber

Usually we require to rerun the failed test cases again once they are fixed or for double verification.In cucumber we can do that easily without developing any customize code for it. Cucumber directly supports the feature to re- run the failed test cases as following: Run Cucumber with rerun formatter simply using :   cucumber -f rerun --out rerun.txt It will output locations of all failed scenarios to the file. Then you can rerun them by using cucumber @rerun.txt This can be further bundled as a rake task and can be run simply as : require 'cucumber' require 'cucumber/rake/task' Cucumber::Rake::Task.new(: jenkins_with_rerun ) do |t|   cucumber -f rerun --out rerun.txt   cucumber @rerun.txt end task :default => : jenkins_with_rerun  And then can be run as simply "rake" inside the project folder:   C:/>ProjectFolder>rake
Read more