Locust is a programmer-friendly load testing tool (certainly compared with jmeter!). Traditionally, once you needed to generate more load than a single host could easily support, you would set up a swarm. However, if you’re willing to live without the web UI, there is another option.

Once you have a containerised version of your scripts, you can go “serverless”, and run them as a task on AWS Fargate. You can use the wizard to set up a cluster &c, or define them with cloudformation:

AWSTemplateFormatVersion: 2010-09-09

Resources:

  Cluster:
    Type: 'AWS::ECS::Cluster'
    Properties:
      ClusterName: ${cluster_name}

  TaskExecutionRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - ecs-tasks.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Path: /
      Policies:
        - PolicyName: logs
          PolicyDocument:
            Statement:
              - Effect: Allow
                Action:
                  - 'logs:CreateLogGroup'
                  - 'logs:CreateLogStream'
                  - 'logs:PutLogEvents'
                Resource: '*'
        - PolicyName: ecr
          PolicyDocument:
            Statement:
              - Effect: Allow
                Action:
                  - 'ecr:BatchCheckLayerAvailability'
                  - 'ecr:GetDownloadUrlForLayer'
                  - 'ecr:BatchGetImage'
                Resource: !Join
                  - ''
                  - - 'arn:aws:ecr:'
                    - !Ref 'AWS::Region'
                    - ':'
                    - !Ref 'AWS::AccountId'
                    - ':repository/your-repo'
              - Effect: Allow
                Action:
                  - 'ecr:GetAuthorizationToken'
                Resource: '*'

  LogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !Sub /fargate/${AWS::StackName}
      RetentionInDays: 7

  TaskDefinition:
    Type: 'AWS::ECS::TaskDefinition'
    Properties:
      Family: ${name}
      Cpu: 256
      Memory: 512
      NetworkMode: awsvpc
      RequiresCompatibilities:
        - FARGATE
      ExecutionRoleArn: !Ref TaskExecutionRole
      ContainerDefinitions:
        - Name: ${name}
          Cpu: 256
          Memory: 512
          Image: ${account_id}.dkr.ecr.${region}.amazonaws.com/${image_name}:latest
          LogConfiguration:
            LogDriver: awslogs
            Options:
              awslogs-region: ${region}
              awslogs-group: !Ref LogGroup
              awslogs-stream-prefix: !Ref AWS::StackName

Once the stack is created, you can run a task on the cluster:

aws ecs run-task --launch-type FARGATE --cluster ${cluster-name} --task-definition ${task-name}:${latest-revision} --network-configuration "awsvpcConfiguration={subnets=[${public-subnet-id}],securityGroups=[${security-group}],assignPublicIp='ENABLED'}" --count 1 --overrides '{"containerOverrides":[{"name":${name},"environment":[{"name":"TARGET_URL","value":${target-url}},{"name":"LOCUST_OPTS","value":"--clients=100 --no-web --only-summary --run-time=1h"}]}]}'

(You can use a public subnet/sg from the default vpc). That will spawn 100 VUs, for an hour, against your chosen target. And you can just keep adding more. Any logs from locust will be available in the AWS console.

Locust is a load testing tool that allows you to script your actions in python (much more pleasant than fiddling with jMeter).

I was investigating an issue that only happened under load (turned out to be due to starvation of the db connection pool). The trouble was, after it had happened, the logs filled up with errors, which weren’t useful information. I wanted the load test to stop, as soon as the first error had occurred.

I couldn’t find anything in the documentation, but a quick nose in the source revealed the existence of a StopLocust exception:

from locust.exception import StopLocust

...

response = locust.client.post(uri, data=json.dumps(data), headers=headers)
if response.status_code != 200:
    raise StopLocust()

At any point, you can raise it, and that locust will stop. If you wanted to stop them all, you could set a flag and check it in your other tasks (not ideal, but the best I can offer).

Remember this is undocumented behaviour, and could change at any time, but it works in v0.7.2.