kashtan404 Jun 5 2019 at 14:00

Moto. Mock AWS

3 min

3.7K

Amazon Web Services*DevOps*Systems engineering*IT systems testing*

Testing is an important part of the development process. And sometimes developers need to run tests locally, before committing changes.

If your application uses Amazon Web Services, python library moto is the perfect thing for that.

Full list of implementation coverage is here.

I found the repo of Hugo Picado on Github — moto-server. Ready to start'n'use image. The only nuance is that after launch, no AWS resources will be created.

Well, it's pretty easy to fix it.

Whereas we must define env variable MOTO_SERVICE to specify type of resource at launch, all we have left is to define resource creation.

Update start.sh:

Replace

moto_server $MOTO_SERVICE -H $MOTO_HOST -p $MOTO_PORT

with

if [ -f /opt/init/bootstrap.py ]; then
  moto_server $MOTO_SERVICE -H $MOTO_HOST -p $MOTO_PORT & (sleep 5 && echo "Executing bootstrap script." && python /opt/init/bootstrap.py)
else
  moto_server $MOTO_SERVICE -H $MOTO_HOST -p $MOTO_PORT
fi
wait

The file should be like:

start.sh

#!/bin/sh

# validate required input
if [ -z "$MOTO_SERVICE" ]; then
  echo "Please define AWS service to run with Moto Server (e.g. s3, ec2, etc)"
  exit 1
fi

# setting defaults for optional input
if [ -z "$MOTO_HOST" ]; then
  MOTO_HOST="0.0.0.0"
fi

if [ -z "$MOTO_PORT" ]; then
  MOTO_PORT="5000"
fi

echo "Starting service $MOTO_SERVICE at $MOTO_HOST:$MOTO_PORT"

if [ -f /opt/init/bootstrap.py ]; then
  moto_server $MOTO_SERVICE -H $MOTO_HOST -p $MOTO_PORT & (sleep 5 && echo "Executing bootstrap script." && python /opt/init/bootstrap.py)
else
  moto_server $MOTO_SERVICE -H $MOTO_HOST -p $MOTO_PORT
fi
# Prevent container from exiting when bootstrap.py finishing
wait

Build new image and push it to your registry.

Futher, lets make a script of resource initialization, using the library to work with AWS — boto3. E.g.SWF domain:

bootstrap_swf.py

import boto3
from botocore.exceptions import ClientError
import os

os.environ["AWS_ACCESS_KEY_ID"] = "fake"
os.environ["AWS_SECRET_ACCESS_KEY"] = "fake"

client = boto3.client('swf', region_name='us-west-2', endpoint_url='http://localhost:5000')

try:
    client.register_domain(
        name='test-swf-mock-domain',
        description="Test SWF domain",
        workflowExecutionRetentionPeriodInDays="10"
    )
except ClientError as e:
    print "Domain already exists: ", e.response.get("Error", {}).get("Code")

response = client.list_domains(
    registrationStatus='REGISTERED',
    maximumPageSize=123,
    reverseOrder=True|False
)

print 'Ready'

Logically:

Mount our bootstrap script to /opt/init/bootstrap.py
If file mounted — it will be executed
If not — the only moto-server will be launched

So, we can mock the whole resource by just run only one container:

docker run --name swf -d \
    -e MOTO_SERVICE=swf \
    -e MOTO_HOST=0.0.0.0 \
    -e MOTO_PORT=5000 \
    -p 5001:5000 \
    -v /tmp/bootstrap_swf.py:/opt/init/bootstrap.py \
    -i awesome-repo.com/moto-server:latest

Let's try in the interactive mode:

It works!

Thus, we can create docker-compose.yml which can help save time for testing changes:

docker-compose.yml

version: '3'
services:
  s3:
    image: picadoh/motocker
    environment:
      - MOTO_SERVICE=s3
      - MOTO_HOST=10.0.1.2
    ports:
      - "5002:5000"
    networks:
      motonet:
        ipv4_address: 10.0.1.2
    volumes:
      - /tmp/bootstrap_s3.py:/opt/init/bootstrap.py
  swf:
    image: picadoh/motocker
    environment:
      - MOTO_SERVICE=swf
      - MOTO_HOST=10.0.1.3
    ports:
      - "5001:5000"
    networks:
      motonet:
        ipv4_address: 10.0.1.3
    volumes:
      - /tmp/bootstrap_swf.py:/opt/init/bootstrap.py
  ec2:
    image: picadoh/motocker
    environment:
      - MOTO_SERVICE=ec2
      - MOTO_HOST=10.0.1.4
    ports:
      - "5003:5000"
    networks:
      motonet:
        ipv4_address: 10.0.1.4
    volumes:
      - /tmp/bootstrap_ec2.py:/opt/init/bootstrap.py
networks:                             
  motonet:                          
    driver: bridge                
    ipam:                         
      config:                       
        - subnet: 10.0.0.0/16

By the way, we can use this approach not only on the developer's laptop. Preliminary tests with mocks after build will help us to get rid of possible problems on the dev* environments.

Links:

Motocker repo — github.com/picadoh/motocker
Moto repo — github.com/spulec/moto
Boto3 Docs — boto3.amazonaws.com/v1/documentation/api/latest/index.html

Tags:

Hubs: