E-commerce IT systems are usually quite complex. Business requirements tend to change frequently. New promotions and discount calculation formulas are apt to confuse developers. Unfortunately, publishing buggy software may lead to dire consequences.
Sales platform unavailability leads to measurable losses — if the website is offline, orders cannot be placed. If our price calculations are off, customers placing orders may take advantage of the bugs. We already bore witness to at least several serious mishaps of this kind, and a huge number of less significant ones. Software errors may lead to such consequences, as loss of customer confidence, unfavorable bias, and in severe cases — even legal actions.

Let’s have a look at tests, which are worthwhile for e-commerce software.

User Interface Usability Testing
These tests are conducted with the help of users, on the basis of interface models, prior to commencing deployment. They consist of assigning specific tasks to test subjects (e.g. “purchase a PC”), which are then performed on the basis of interface models. Test subjects provide answers as to how they would go about performing a particular task, and testers are responsible for noting, when users are frustrated, confused, or when they misunderstand the way the system works. Performing such research with the help of 3 to 5 subjects usually enables more than 80 percent of user interface errors to be identified. Such tests improve system functionality and ergonomics. They make it possible to find navigational gaps, function layout errors, and user message errors before deployment begins. Such mistakes would be quite expensive to eliminate at a later stage. In case of e-commerce, ergonomics-related errors are in a direct relationship with the conversion factor. The value of usability is commonly appreciated, and user interface testing is the key tool to improve user experience.

Security Testing
Along with developing a report on application vulnerabilities, tests are conducted using automated tools in a Black-box model. Skipfish, XSS-Me and SQL Inject-Me tools are used to test for most common errors: XSS, CSRF, and SQL Injections. Tests are also conducted at the level of analyzing module code added to Magento (White-box testing) in order to verify conformance to OWASP.org guidelines. Such tests are fundamental to e-commerce. If they’re omitted, the system is vulnerable to attacks, the consequences of which are difficult to anticipate. Transaction-processing systems and systems responsible for financial operations should be audited even at the source code level in order to be trusted. Such tests are often extremely expensive. Automated tools and heuristics are fortunately able to detect a large number of primary vulnerabilities, and therefore such tests should be considered a minimum in terms of security testing.

Error Handling Testing
This test scope involves checking, how the system reacts to such factors as: improper data, malformed URLs, no authorization, form validation issues, network issues (database failures), etc. Exceptions are certain to occur, it’s only a question when. One should know what messages are displayed to the user, if improper data is entered. If the database fails (lower level error), it will influence our application as well. This involves displaying information about outages, or switching to backup servers. Testing exceptional conditions may be compared to regular DHW or air vent overhauls performed in private apartments — they may be routine, but in case of failure they may save lives.

Basic Scenario and Path Testing
Upon completion of deployment, a list of scenarios is created based on application design. Such scenarios include all functional application paths, which are to prove that the application operates as specified in the agreement. Functional tests are the primary tool used to verify, whether our software works as intended. It’s the scenarios, which define the meaning of intended operation. Scenarios developed as part of acceptance testing are applicable to the remainder of the application maintenance cycle. If the budget is sufficient, automated application tests may also be developed (BDD — Behavior-Driven Development) using such tools as Selenium.

Regression Testing and Smoke Testing
Upon the introduction of new functionality (during maintenance) behavioral regression testing is performed based on the abovementioned test scenarios. Those tests should be conducted by an appointed developer representative or developers themselves, and they must cover at least functions, which underwent modification. It’s best practice to utilize those scenarios every time modifications are introduced to the production server. If system functionality is well regulated, they allow us to be certain that nothing important is missing, and that there will be no surprises awaiting us at the production server.

Recovery Testing
Recovery procedures are tested in a test environment (from scratch), in order to make sure, that they’re working properly. Backup archives are tested as well. It’s quite obvious, that backups are created. However, are they recoverable? Do they contain errors, or is using them overly difficult and time-consuming? The team responsible for application maintenance shouldn’t be answering such questions when a disaster occurs, because time is of the essence then. The development service contractor agreement should include maximum application unavailability periods, recovery time, and the interval which may be lost due to recovery (this implies the frequency of backups). By performing suitable tests a clear answer may be obtained, as to whether maintaining these parameters is possible.

Load Testing
An application may be tested for being able to serve an assumed number of concurrent users utilizing automated tools such as Apache Benchmark and jMeter. However, in addition to load testing, all application server parameters should be continuously monitored in order to diagnose potential performance issues. Load testing should be conducted in an environment which is as similar to the production environment as possible. These tests are designed to expose actual system capabilities. Server parameter monitoring is as vital as pre-launch testing. By monitoring CPU, memory, and hard drive load, it’s possible catch on to potential application issues early on. If such issues appear, we may be able to react, before the application is utterly crippled. Pre-launch load testing also proves, whether the agreement conditions are met, and whether software performance meets the assumptions.

Integration Testing
Basic user path testing includes communications with ERP and other realm systems (Web Services). These realm system communications should also undergo continuous monitoring. External system integration is that part of a deployment, which is most prone to errors, most time-consuming, and often the most difficult to handle. Except for technical challenges, which are usually not overly difficult to overcome, it involves communication issues, and a necessity for businesses responsible for integrated components to work together. In practice inter-system connections often experience issues — network issues for example. Therefore one-time validation is far from enough. Automated integration testing should be performed as part of maintenance. What should such a test involve? It’s usually enough to call Web Services methods, which don’t influence the system, and verify the results. There’s also an option to place test orders, and delete them afterwards, or verify inventory states. What’s most important is to discover the error before the user does. Integrations are usually clearly described in system analysis documentation. This documentation should be treated as a baseline for developing and conducting tests.

Acceptance Tests
Acceptance tests are conducted upon completion of system and integration tests. They are meant for the customer to confirm, whether use case scenarios and test scenarios are properly covered. In case of discrepancies from use case scenarios, acceptance tests may be performed in several iterations.

Testing — An Absolute Minimum
The above test definitions may sound like taken straight out of an encyclopedia. However, in case of large systems, most of the abovementioned verification procedures are indeed conducted. In case of small and medium deployments conducting the whole spectrum of tests described above may fail at being cost-efficient.
However, every single type of system should be covered with the following minimum of tests:

  • User Interface Usability Testing
  • Security Testing
  • Basic Scenario and Path Testing

It’s also difficult to handle development without periodic regression and smoke testing.