[NEW RELEASE] Mobile Performance Testing – Including Network and Client Emulation

Today, we introduced the first true cloud-based load testing Software as a Service for mobile apps, APIs and websites that can simulate traffic generated from a variety of mobile operating systems, popular browsers, and mobile networks – including 3G, GSM and LTE.

Current, only about half of companies with mobile sites or apps today test their mobile code, and a recent industry study reported that when a mobile app fails, 48 percent of users are less likely to use the app again, 34 percent will switch to a competitor, and 31 percent will tell others about their poor experience. [1]

Our new testing service for mobile apps, APIs and websites allows developers to emulate client behavior when downloading content to a phone, specify the number of concurrent downloads in total and per host, as well as the mix of different client applications or browsers, including Safari, Chrome, Firefox and Opera.

Developers can also use our new features to emulate mobile network characteristics including available bandwidth, network delay, packet loss, jitter and packet reordering.

So what’s REALLY changed?

What’s really new is that when we simulate a mobile client – whether it is a mobile user running a mobile web browser and accessing a standard web site, or it is a mobile user playing the Candy Crush app – we can generate the same kind of traffic for the servers to handle that real users would.

If the average mobile user has a network connection speed of, say, 384 kbit/s (old generation 3G) we will not let our simulated client load data faster than that from the servers.

In previous versions of  Load Impact, and in most other load testing tools, every simulated client/user in a load test will load things at maximum possible speed, at all times. This will of course result in a very skewed test result, that might tell you your site/app can handle max 1,000 concurrent users while in reality you could handle a lot more (or less).

Apart from simulating network connection speed, we also simulate network latency, which is equally important for performance as connection speed is  –  just like connection speed, latency also affects how “heavy” a client is for the servers to handle.

Our network/client emulation feature is currently available at the test level only, but you will soon be able to simulate mobile traffic at the user scenario level too. We’ll be sure to let you know when the update arrives.

Automated Acceptance Testing with Load Impact and TeamCity (New Plugin)

As you know, Continuous Integration (CI) is used by software engineers to merge multiple developers’ work several times a day. And load testing is how companies make sure that code performs well under normal or heavy use.

So, naturally, we thought it wise to develop a plugin for one of the most widely used CI servers out there – TeamCity by JetBrains. TeamCity is used by developers at a diverse set of industry leaders around the world – from Apple, Twitter and Intel, to Boeing, Volkswagen and Bank of America. It’s pretty awesome!

The new plugin gives TeamCity users access to multi-source load testing from up to 12 geographically distributed locations worldwide, advanced scripting, a Chrome Extension  to easily create scenarios simulating multiple typical users, and Load Impact’s Server Metrics Agent (SMA) for correlating the server side impact of testing – like CPU, memory, disk space and network usage.

Using our plugin for TeamCity makes it incredibly easy for companies to add regular, automated load tests to their nightly test suites, and as a result, get continuous feedback on how their evolving code base is performing. Any performance degradation, or improvement is detected immediately when the code that causes it is checked in, which means developers always know if their recent changes were good or bad for performance – they’re guided to writing code that performs well.

 

Here’s how Load Impact fits in the TeamCity CI workflow:

 

 

Once you have the plugin installed, follow this guide for installing and configuring the Load Impact plugin for TeamCity. 

Load Testing Prior to Holiday Season Rush Can Help Reduce Cart Abandonment Rate by up to 18%

The holiday shopping season is rapidly closing in and e-commerce sites and services all over the world are preparing for one of the busiest times of the year. With expected traffic spikes on November 29th – Black Friday and December 2nd – Cyber Monday.

The pressure to capture every last sale is even greater this year as it is the shortest holiday shopping season in over a decade. 

To understand the grandeur of what is at stake if you fail to meet customer performance demands, let’s recap some stats.

• According to comScore, approximately 57.3 million Americans visited at least one online retail site on Black Friday 2012 (an 18% increase from the year prior).
• On Cyber Monday 2012 alone, sales reached 1.98B and are expected to increase by 17% in 2013.

When it comes to shopping chart abandonment, the stakes get even higher…..

• Some US analysts expect to see a shopping cart abandonment rate peak of 88% in 2013 (~6% above the average for the rest of the year).
• The total cost of abandoned shopping carts for online retailers has been estimated at more than $18 billion per year.
• Even a 1% reduction in cart abandonment can lead to a notable increase in revenue, a $2.6 billion increase in relation to total 2013 U.S. online retail sales forecast.

At this point, you might be asking yourself, but what impact does website performance really have on all this anyway? The answer is, quite a lot actually.

According to Tammy Everts’ blog, one out of five of those carts are abandoned due to slow website performance. Simply put, 18% of shoppers will abandon their cart if pages are too slow. If 18% of that loss can be attributed to slow pages, then this correlates to more than $3 billion in lost sales (across US e-commerce sites) due to poor performance.

Now, while some e-commerce sites are making appropriate preparations for expected visitor load, others are just holding their breath and suffering from ‘the ostrich effect‘ – basically just avoiding to deal with an obviously risky business situation by pretending it does not exist.

Instead of burying their heads in the sand, they should just accept that the risk is very real and extremely probable and start performance testing before it’s too late.

It’s almost embarrassing if they don’t, since cloud-based load testing tools are so accessible and affordable. It was somewhat excusable when you had hardware to install and licenses to buy, but nowadays… seriously?!

In fact, our recent State of Web Readiness report found that while shoppers demand page load speeds in the milliseconds, most e-commerce sites have response times closer to 8 seconds. This could be due to the fact that those same  e-commerce site owners surveyed overestimated their website capacity by roughly 3.4 times.

A lot of companies are preparing to meet the upcoming traffic spike and increased activity by taking appropriate measures. Some of those measures are quite easy, we wrote about a few of them a while back in another blog post called “Different types of website performance testing – Part 3: Spike Testing“.

On the up side, you already have some general data about what to expect in terms of traffic spikes. Simply knowing how traffic will trickle in on those key dates will help you to configure more realistic test execution plans.

But make no mistake, if you don’t try out the durability of your site you can’t really be sure that the correlation of all active components of your services – 3^rd parties resources or content, feeds, content management platforms, databases and internal systems – will provide for an acceptable customer experience.

Basically what we’re saying is: don’t pull an ObamaCare, load test before its too late.

Listen to Load Impact CTO and CEO discuss performance testing  prior to holiday ramp-up on the Rackspace Google Hangout.