Saving Your Super Bowl Bacon: How One Advertiser Performance Tested Early and Avoided Disaster

During Super Bowl  XLVII, Coca Cola, Axe, Sodastream, Calvin Klein had their hands full. Not so much serving online visitors as running around looking for quick fixes for their crashed websites. As reported by Yottaa.com, no fewer than 13 of the companies that ran ads during that Super Bowl saw their websites crash just as they needed them the most.

If anything in this world is ever going to be predictable, a large spike in traffic when you show your ad to a Super Bowl audience must be one of those things.

Thats why one leading digital agency – ROKKAN –  worked with us to guarantee the stability of their client’s campaign website just days ahead of their Super Bowl XLVIII TV commercial.

Getting this level of performance assurance not only eased their client’s concerns, it also showed how committed ROKKAN was to the quality of execution.

As with any advertising campaign, its not only about the glitz and glamour of the ad, its about the ability to motivate action on the part of the audience. And if that motivation is to visit a website, you better be damn well sure that website can take the heat and deliver an exceptional experience. Anything less translates to lost customers, degraded reputation and fewer sign ups – as was this advertiser’s campaign objective.

Background:

The auto industry was the biggest sponsor of the Super Bowl for the third straight year this year. No fewer than seven brands advertised during Fox TV’s telecast of the National Football League’s championship game on Sunday, February 2, 2014.

On average, the carmakers spend as much as $6 million per 30-second spot, for the chance to get in front of an estimated 100+ million viewers. 

“When these types of investments are made, if your website doesn’t work, it becomes very ugly, very quickly for all parties involved in the campaign,” said Jim Blackwelder, chief technical officer for ROKKAN, a Publicis Groupe company that’s done award winning work for American Express, jetBlue, Chipotle, Ceasars Palace, and TAG Heuer. “Everybody just expects technology to work these days, and if it doesn’t, everyone notices.”

Test setup:

The Super Bowl Sunday challenge for ROKKAN – a 2014 Advertising Age A-List Agency for the third consecutive year – could not have been clearer: the target user load for its automaker client’s microsite promoting the features of an upcoming luxury sedan, was over 250,000 visitors per hour coming in from across the country and around the world.  But initial performance tests of the website using Load Impact failed far short of that goal.

“That first test was a total eye-opener,” said Blackwelder.  “Load Impact showed us we had a lot of work to do to get performance where it needed to be.”

At the advice of Load Impact, ROKKAN split performance testing to separately hit the website run by the hosting company Rackspace. The tests were done by going through and around the Rackspace content delivery network (CDN) to uncover crucial performance and configuration issues with both scenarios.

Daily meetings and testing ensued. 

Challenges:

“The team at Load Impact realized that without a full court press – a really integrated effort with all of the stakeholders including ROKKAN, their service providers, as well as Load Impact – the success of the project was far from guaranteed,” said Charles Stewart, Load Impact’s general manager for U.S. operations. 

Solution:

With less than one week from kickoff, ROKKAN made changes and optimizations to problems identified by Load Impact at the caching layer, application layer and hardware layer, that brought page load times to under 4 seconds with just over 10,000 simultaneous users.  

It was 20 times better than when they started, but still a long way from their goal. As an added complication, eleventh hour changes requested by the automaker adversely impacted performance dramatically.

Load Impact and the ROKKAN team examined the data and identified network bandwidth as the likely cause of the performance degradation, and produced reports to support the findings.

With data in hand, Blackwelder was able to go to Rackspace’s network team, which quickly identified and resolved the remaining bottlenecks, increased throughput to the site, and gave ROKKAN’s platform the capacity it needed to scale.  

Results:

The automaker’s website was ready for game day.

“Had we not done performance testing we never would have uncovered a range of issues with both our work and our partners,” said Blackwelder. “Without Load Impact our hitting the project goals and schedule would have been at risk. But it wasn’t at all.  It was a huge success.”

The automaker’s site performed perfectly under the Super Bowl traffic, far exceeding expected connections per second without breaking a sweat, and delivered the customer sign-up conversions that were the automaker’s key success metric.

“For many of ROKKAN’s projects we thought of performance testing as a nice-to-have. But for this project and any of its type, it’s an absolute need-to-have. We needed large scale capacity testing, real-time viewing, and robust reporting, with a platform that could respond to change quickly, and Load Impact delivered on all counts.” – Jim Blackwelder. 

———–

This is a perfect example of how NOT to waste $4 million (not including production costs). If you’ve got a big marketing campaign coming up, make sure your audience can actually fulfill the action you want them to take – performance test your campaign website/app early. 

Scenario Testing: Four Tips on How to Manage Effectively

This article was originally written for Software Testing Professionals.

————

Testing software has always been complex. The minute you add more than a handful of features to any system, theoretical complexity sky rockets.

All the buttons to click, links to follow, client browser versions, client bandwidth and what have you, will soon add up to a near infinite number of things you’d need to test. At the same time, actual users will only engage with a small fraction of those features.

But how does one bring some order to this complexity when, barring a few exceptions, 100% test coverage is basically impossible to achieve?

One approach is to turn to scenario testing. This is where you use a real or hypothetical story that describes how a user actually uses the application. It may sounds very similar to a test case, but a test case is typically single step, whereas scenario tests cover a number of interconnected steps.

A good scenario is one that is based on a credible story of a user performing a complex task. The scenario should be:

• Critical to all stakeholders (i.e sales, marketing, management, customer support)
• It should be obvious that the scenario must work as expected
• The scenario must be easy to evaluate

Scenario testing is primarily thought of as a tool to facilitate feature testing, but performance is sometimes part of that.

Since application performance is very often a non-functional requirement, satisfactory performance is often assumed and lack there of is considered a bug – even if it was never mentioned in the requirements.

Therefore, scenario testing can be used to uncover important performance bottlenecks as well as test features.

Consider this test case for an e-commerce site.

Test case #1: Add valid coupon worth X%
Steps:

1. Add one or more products to the cart
2. Go to the checkout page
3. Add coupon code ‘TEST123’ and click ‘Add coupon’

Expected result:

• Page refreshes. Message “coupon successfully applied” is visible
• Discount clearly indicated in the cart summary section
• Cart total is reduced by X%

Now, imagine that the test case is performed and confirmed to work during development. One of the testers makes a note saying that sometimes, the page refresh takes 4-5 seconds when you have more than 10 products in the cart, but it’s not considered a major issue since it’s affecting a very small number of users.

Now, consider an actual user, Lisa, as she uses the e-commerce site:

Lisa gets a 15% coupon code for an eCommerce site she’s used before and really likes. She decides to order a few things she needs and also asks her mother and sister if they need anything. While she’s shopping, she talks once with her mother and three times with her sister to double check she get’s correct amount, sizes and colors of all items.

After about 20 minutes, Lisa has 20 items worth $900 in her cart. She hits the checkout page where she enters the discount code. The page seems to be doing ‘something’ but after 5 seconds with no visual feedback, Lisa decides that it’s most likely expected behaviour and hits ‘Pay now‘ to proceed. She’s a little worried that she can’t see her discount on the screen, but assumes that it will be presented on the emailed receipt.

Five minutes after completed checkout, she receives the receipt and realizes that she didn’t get the discount. At this point, Lisa feels let down and decides to try to cancel the order. Maybe she will try again later, maybe not.

The story of Lisa’s real world shopping experience makes a great base for a test scenario. A credible story of a user performing a complex task. It highlights to relevant stakeholders – like sales, marketing, management, customer support – that it’s important functionality that really needs to work.

It is, of course, possible to write a few test cases that would capture the same performance issue, but by putting the steps into a realistic and credible context, the coupon code response time suddenly stands out as an important issue.

It suddenly becomes easier to spot and it becomes apparent that, even if it’s a small fraction of all http requests to the server, it will likely seriously affect a customer that wishes to make a rather large transaction. Which, I would like to point out, was the main reason the marketing/sales team wanted to distribute the coupon code in the first place.

Finally, since scenarios are much easier to understand for people outside R&D, it’s easier to involve everyone with an interest in the project. In most organizations, stakeholders such as sales and marketing, customer support and management will find scenarios much easier to grasp than a long (and sometimes boring) list of small test cases.

The challenge is, of course, to find correct and credible stories that both motivate the important stakeholders to participate and at the same time covers as much of the application as possible.

Performance testing can benefit from a scenario approach in many ways. One of the most obvious benefits is that creating scenarios helps to highlight the important application flows that must perform well – just as the coupon code scenario above shows.

Test configurations can then be more focused when we know what the most important areas are. And since scenarios are stories that are easier to understand, it’s also easier for non-technical people to be part of the prioritization work, making sure that first things come first.

Another great benefit that can come specifically from performance testing multiple complex scenarios at the same time is that it can unveil dependencies.

Let’s say that one problem area with an e-commerce web application is slow internal search. While that’s a problem on it’s own, it’s not unlikely that if affects overall database performance. That in turn can affect more important functionality that also uses the database – like registration or checkout.

When applying the concept of scenario testing to your performance testing efforts, here’s a few things keep in mind:

1. Consider using scenarios in your performance testing. Use tools such as Google Analytics to analyze what paths users take through your site to help you come up with credible and critical scenarios.
2. Prioritize possible scenarios by thinking how valuable each scenario is. A user browsing your products is good, a user that checks out and pays is better. Make sure you cover the most critical scenarios first by ordering them according to how valuable they are to you.
3. Consider using Continuous Integration tools such as Jenkins or TeamCity to automate performance scenario testing. An automated test that gives you pass/fail results based on response time is very easy to evaluate.
4. When the number of scenarios grow, group different ones together based on what part of the system they test. Or group them based on complexity, making sure that all low complexity tests pass before you run the high complexity ones.

———

This post was written by Robin Gustafsson. Robin is currently CTO at Load Impact. Prior to his role as CTO, he held positions as Solutions Architect, Consultant and lead developer for numerous other tech startups and tech firms, including Ericsson. He also owned and operated his own web development company, Popmint Media, from 2002-2005. Robin specializes is performance testing, software architecture, cloud and distributed computing, as well as Continuous Delivery software development.

Test Driven Development and CI using JavaScript [Part II]

This is the second half of a two part article on Test Driven Development and Continuous Integration (CI) using JavaScript. Part I covers different types of testing and approaches to Test Driven Development (TDD).

Behavior-Driven Development (BDD) is a specialized version of Test Driven Development (TDD) focused on behavioral specifications. Since TDD does not specify how the test cases should be done and what needs to be tested, BDD was created in response to these issues.

It’s easy to talk about Behavioral Driven Development (BDD), but it’s more difficult to actually put it into practice. BDD is a fairly new concept, so it’s not completely intuitive for some coders who have been working with Continuous Integration (CI) for a long time.

This article gives a real-world example application using the concept of a “Shapes” object. The “Shapes” object contains classes for each shape and the application is a small JavaScript application that uses BDD for testing.

BDD and Software Testing

This tutorial covers how to use BDD to test your JavaScript code. In the following example, some test cases are written along with the corresponding code. The code is then refactored to fix bug issues.

Project: Create an application that contains a set of shapes. For every shape, calculate its area.

Application structure

You can create different kinds of folder structures for your applications. For example, you can divide your code into public and private folders to correspond to your class types.

Create a structure for your project using the following:

+ public 
   + javascripts 
       + app               
       + lib 
           - require.js 
+ private 
   + javascripts 
       + lib 
           + jasmine-2.0.0 
.	... 
       + spec 
- SpecRunner.html 

Save your Jasmine libraries in the private folder. In the public folder, store RequireJS to use in your models.

Creating a test

Using TDD as methodology, you start creating small test cases. Test cases require good design, but you also need to consider your code. You can also rewrite some of your test cases as you improve your code.

Start creating tests for your application. For example, the following is a list of considerations you could make about your application:

• Do you want to organize my shapes into classes? Since a shape could represent one object, it’s a good idea to design a class for each shape.

• You will probably need a method to calculate the area of your shapes. 

The following is a test case that fits the above two code requirements:

describe("Shapes", function () {
    describe("Square", function () {
        var that = this;

        beforeEach(function (done) {
            require(['Shape/Square'], function (Square) {
                that.shape = new Square();
                done();
            });
        });

        it("with side 0 should have an area of 0", function () {
            expect(that.shape.getArea(0)).toBe(0);
        });

        it("with side 2 should have an area of 4", function () {
            expect(that.shape.getArea(2)).toBe(4);
        });

        it("with side 4 should have an area of 16", function () {
            expect(that.shape.getArea(4)).toBe(16);
        });

        it("with side 123.123 should have an area of 15159.27", function () {
            expect(that.shape.getArea(123.123)).toBe(Math.pow(123.123, 2));
        });
    });
});

/private/javascripts/spec/Shape/SquareSpec.js

The above test case fits the division specifications.

• Suites: The method describe is used with the story’s name. In this case, you want to describe the actions to apply to Shapes and Square. The second argument is a function, which will contain Spec or more Suites. 
• Spects: The keyword it is used with a “with XXX should” sentence. The way the sentence is written varies with the test case, but you should always write it as if you are writing a user story.  The second argument is a function where you start using your code for testing.
• Expect: This statement helps you test code with simple sentences for output.  These statements usually do comparisons between values. They start with expect and find different kinds of “Matches,” which are comparison functions. An example of a “Match” is the function toBe .

This code gives you tips for organizing the code later.  The story for the first test looks like this:

• Given a Shape which is a Square
• And with side 0
• Then the shape should have an area of 0

In order to create this story and obtain the right result, you create expectations: 

• expect the area of 0 toBe 0

As you can see, it is very easy to read the tests, which allows you to create stories.

Creating the model

Now that we did a small test, we may have some idea about how to organize our code. We will create a base class called Shape.js and a class that represents the shape Square.

define("Shape/Shape", [], function() {
    function Shape() {
    }

    Shape.prototype.getArea = function() {
        return 0;
    };

    return Shape;
});
/public/javascripts/app/Shape/Shape.js

define("Shape/Square", ["Shape/Shape"], function (Shape) {
    function Square() {
        Shape.call(this);
    }
    Square.prototype = new Shape();
    Square.prototype.constructor = Square;

    Square.prototype.getArea = function (side) {
        return side * side;
    };

    return Square;
});

/public/javascripts/app/Shape/Square.js

As you can see, our shape contains a method to calculate the area. In the case of the Square, we need to pass it an argument for the side.

Running tests

A SpecRunner file is a file that runs all your test cases. You can organize them into suites and sets of SpecRunners. For this example, you create one file that runs all test cases.

Open the file SpecRunner.html and modify it with the following content:

<!DOCTYPE HTML>

<html>

<head>
  <metahttp-equiv="Content-Type"content="text/html; charset=UTF-8">
  <title>Jasmine Spec Runner v2.0.0</title>

  <link rel="shortcut icon"type="image/png"
href="private/javascripts/lib/jasmine-2.0.0/jasmine_favicon.png">
  <link rel="stylesheet"type="text/css"
href="private/javascripts/lib/jasmine-2.0.0/jasmine.css">

  <script type="text/javascript"
src="public/javascripts/lib/require.js"></script>
  <script type="text/javascript">
  requirejs.config({
  baseUrl: 'public/javascripts/app',
  paths: {
     jasmine: 'private/javascripts/lib/jasmine-2.0.0/jasmine'
  },
  shim: {
     jasmine: {
        exports: 'jasmine'
     }
  }
});
</script>

    <script type="text/javascript" 
src="private/javascripts/lib/jasmine-2.0.0/jasmine.js"></script>
    <script type="text/javascript" 
src="private/javascripts/lib/jasmine-2.0.0/jasmine-html.js"></script>
    <script type="text/javascript" 
src="private/javascripts/lib/jasmine-2.0.0/boot.js"></script>

    <!-- include source files here... -->
    <script type="text/javascript" 
src="public/javascripts/app/Shape/Square.js"></script>

    <!-- include spec files here... -->
    <script type="text/javascript" 
src="private/javascripts/spec/Shape/SquareSpec.js"></script>

</head>

<body>
</body>

</html>

/SpecRunner.html

The content is divided in three main sections:

• Load and configure libraries for requireJS (public/javascripts/lib/require.js).
• Load the necessary libraries for Jasmine (private/javascripts/lib/jasmine/…).
• Load the application source files (public/javascripts/app/Shape/Square.js).
• Load the test source files (private/javascripts/spec/Shape/SquareSpec.js). 

If you execute (open) the file with your favorite browser, you see the following result:

All green labels show us that all tests have been passed correctly.

Refactoring code

In most cases, you’ll need to do a least a little refactoring after running your test cases. The following is an example of some real-world questions you could have about the test case results:

• You could set the size of “side” using the method setSide instead of passing it though getArea method.

Do the following changes to the test cases, which create a new method setSide: 

describe("Shapes", function () {
   describe("Square", function () {
      var that = this;

      beforeEach(function (done) { 
            require(['Shape/Square'], function (Square) { 
              that.shape = new Square(); 
              done(); 
            }); 
      });

      it("with side 0 should have an area of 0", function () { 
         that.shape.setSide(0); 
         expect(that.shape.getArea()).toBe(0); 
      });

      it("with side 2 should have an area of 4", function () { 
         that.shape.setSide(2); 
         expect(that.shape.getArea()).toBe(4); 
      });

      it("with side 4 should have an area of 16", function () { 
         that.shape.setSide(4); 
         expect(that.shape.getArea()).toBe(16);
      });

      it("with side 123.123 should have an area of 15159.27", function () { 
         that.shape.setSide(123.123); 
         expect(that.shape.getArea()).toBe(Math.pow(123.123, 2)); 
    }); 
  }); 
});

/private/javascripts/spec/Shape/SquareSpec.js

After you make the changes to the test cases, refresh your browser. In this example, the test cases fail.

Since the test cases failed, you know you’ve broken your application. The failed test cases are the advantage of BDD since these test cases let you know that there are errors in your application – before you write too much code . Because you’ve only written a small amount of code, you only have a small amount of code to refactor from the failed test cases. If you had written the entire application, you would have hours of refactoring ahead of you. Even if you have one class used in several modules, every module has its own test.

Now we can fix the model:

define("Shape/Square", ["Shape/Shape"], function (Shape) {    
    function Square() {
        Shape.call(this);

        this.side = 0;
    }
    Square.prototype = new Shape();
    Square.prototype.constructor = Square;

    Square.prototype.setSide = function (value) {
        this.side = value;
    };

    Square.prototype.getArea = function () {
        return this.side * this.side;
    };

    return Square;
});

/public/javascripts/app/Shape/Square.js

After you refactor your code, the tests will now run successfully.

In this tutorial, a basic use for TDD in JavaScript was used. However, you can use TDD in any language that supports test cases.

————

 

This post was written by Miguel Dominguez. Miguel is currently Senior Software Developer at digitallabs AB but also works as a freelance developer. His focus is on mobile application (android) development, web front-end development (JavaScript, CSS, HTML5) and back-end (mvc, .net, java). Follow Miguel’s blog.

Test Driven Development and CI using JavaScript [Part I]

In this tutorial, we will learn how to apply TDD (Test-Driven Development) using JavaScript code. This is the first part of a set of tutorials that includes TDD and CI (Continuous Integration) using JavaScript as the main language.

Some types of testing

There are several approaches for testing code and each come with their own set of challenges. Emily Bache, author of The Coding Dojo Handbook, writes about them in more detail on her blog – “Coding is like cooking“

1. Test Last: in this approach, you code a solution and subsequently create the test cases.

• Problem 1: It’s difficult to create test cases after the code is completed.
• Problem 2: If test cases find an issue, it’s difficult to refactor the completed code.

2. Test First: you design test cases and then write the code.

• Problem 1: You need a good design and formulating test cases increases the design stage, which takes too much time.
• Problem 2: Design issues are caught too late in the coding process, which makes refactoring the code more difficult due to specification changes in the design. This issue also leads to scope creep.

3. Test-Driven: You write test cases parallel to new coding modules. In other words, you add a task for unit tests as your developers are assigned different coding tasks during the project development stage.

 

TDD approach

TDD focuses on writing code at the same time as you write the tests. You write small modules of code, and then write your tests shortly after.

Patterns to apply to the code:

• Avoid direct calls over the network or to the database. Use interfaces or abstract classes instead.
• Implement a real class that implements the network or database call and a class which simulates the calls and returns quick values (Fakes and Mocks).
• Create a constructor that uses Fakes or Mocks as a parameter in its interface or abstract class.

Patterns to apply to unit tests: 

• Use the setup function to initialize the testing, which initializes common behavior for the rest of the unit test cases.
• Use the TearDown function to release resources after a unit test case has finalized.
• Use “assert()” to verify the correct behavior and results of the code during the unit test cases.
• Avoid dependency between unit test cases.
• Test small pieces of code.

 

Behavior-Driven Development

Behavior-Driven Development (BDD) is a specialized version of TDD focused on behavioral specifications. Since TDD does not specify how the test cases should be done and what needs to be tested, BDD was created in response to these issues.

Test cases are written based on user stories or scenarios. Stories are established during the design phase. Business analysts, managers and project/product managers gather the design specifications, and then users explain the logical functionality for each control. Specifications also include a design flow so test cases can validate proper flow.

This is an example of the language used to create a BDD test story:

Story: Returns go to stock

In order to keep track of stock

As a store owner

I want to add items back to stock when they’re returned

---

Scenario 1: Refunded items should be returned to stock

Given a customer previously bought a black sweater from me

And I currently have three black sweaters left in stock

When he returns the sweater for a refund

Then I should have four black sweaters in stock

---

Scenario 2:  Replaced items should be returned to stock

Given that a customer buys a blue garment

And I have two blue garments in stock

And three black garments in stock.

When he returns the garment for a replacement in black,

Then I should have three blue garments in stock

And two black garments in stock

 

Frameworks to Install

1. Jamine

Jasmine  is a set of standalone libraries that allow you to test JavaScript based on BDD. These libraries do not require DOM, which make them perfect to test on the client side and the server side. You can download it from http://github.com/pivotal/jasmine

It is divided into suites, specs and expectations. 

.Suites define the unit’s story. 

.Specs define the scenarios. 

.Expectations define desired behaviors and results. 

Jasmine has a set of helper libraries that lets you organize tests.  

2. RequreJS

RequireJS is a Javascript library that allows you to organize code into modules, which load dynamically on demand.

By dividing code into modules, you can speed up the load-time for application components and have better organization of your code.

You can download RequireJS from http://www.requirejs.org

Part II of this two part tutorial will discuss Behavioral Driven Testing and Software Testing – how to use BDD to test your JavaScipt code. Don’t miss out, subscribe to our blog below. 

————-

This post was written by Miguel Dominguez. Miguel is currently Senior Software Developer at digitallabs AB but also works as a freelance developer. His focus is on mobile application (android) development, web front-end development (javascript, css, html5) and back-end (mvc, .net, java). Follow Miguel’s blog.

[NEW RELEASE] Mobile Performance Testing – Including Network and Client Emulation

Today, we introduced the first true cloud-based load testing Software as a Service for mobile apps, APIs and websites that can simulate traffic generated from a variety of mobile operating systems, popular browsers, and mobile networks – including 3G, GSM and LTE.

Current, only about half of companies with mobile sites or apps today test their mobile code, and a recent industry study reported that when a mobile app fails, 48 percent of users are less likely to use the app again, 34 percent will switch to a competitor, and 31 percent will tell others about their poor experience. [1]

Our new testing service for mobile apps, APIs and websites allows developers to emulate client behavior when downloading content to a phone, specify the number of concurrent downloads in total and per host, as well as the mix of different client applications or browsers, including Safari, Chrome, Firefox and Opera.

Developers can also use our new features to emulate mobile network characteristics including available bandwidth, network delay, packet loss, jitter and packet reordering.

So what’s REALLY changed?

What’s really new is that when we simulate a mobile client – whether it is a mobile user running a mobile web browser and accessing a standard web site, or it is a mobile user playing the Candy Crush app – we can generate the same kind of traffic for the servers to handle that real users would.

If the average mobile user has a network connection speed of, say, 384 kbit/s (old generation 3G) we will not let our simulated client load data faster than that from the servers.

In previous versions of  Load Impact, and in most other load testing tools, every simulated client/user in a load test will load things at maximum possible speed, at all times. This will of course result in a very skewed test result, that might tell you your site/app can handle max 1,000 concurrent users while in reality you could handle a lot more (or less).

Apart from simulating network connection speed, we also simulate network latency, which is equally important for performance as connection speed is  –  just like connection speed, latency also affects how “heavy” a client is for the servers to handle.

Our network/client emulation feature is currently available at the test level only, but you will soon be able to simulate mobile traffic at the user scenario level too. We’ll be sure to let you know when the update arrives.

Automated Acceptance Testing with Load Impact and TeamCity (New Plugin)

As you know, Continuous Integration (CI) is used by software engineers to merge multiple developers’ work several times a day. And load testing is how companies make sure that code performs well under normal or heavy use.

So, naturally, we thought it wise to develop a plugin for one of the most widely used CI servers out there – TeamCity by JetBrains. TeamCity is used by developers at a diverse set of industry leaders around the world – from Apple, Twitter and Intel, to Boeing, Volkswagen and Bank of America. It’s pretty awesome!

The new plugin gives TeamCity users access to multi-source load testing from up to 12 geographically distributed locations worldwide, advanced scripting, a Chrome Extension  to easily create scenarios simulating multiple typical users, and Load Impact’s Server Metrics Agent (SMA) for correlating the server side impact of testing – like CPU, memory, disk space and network usage.

Using our plugin for TeamCity makes it incredibly easy for companies to add regular, automated load tests to their nightly test suites, and as a result, get continuous feedback on how their evolving code base is performing. Any performance degradation, or improvement is detected immediately when the code that causes it is checked in, which means developers always know if their recent changes were good or bad for performance – they’re guided to writing code that performs well.

 

Here’s how Load Impact fits in the TeamCity CI workflow:

 

 

Once you have the plugin installed, follow this guide for installing and configuring the Load Impact plugin for TeamCity.