Docker In AWS Lambda
AWS Lambda allows developers to deploy Serverless solutions, but can sometimes be cumbersome to package application dependencies. Docker support has recently been introduced for those...
Highlighting updated container support
AWS Lambda allows developers to deploy Serverless solutions, but application dependencies can sometimes be cumbersome to package. AWS has recently introduced support for those who employ Docker container-based development strategies in their workflow. This can not only reduce the overhead involved with traditional deployment packaging but also allow for previously unsupported runtime dependencies to be employed.
The article will highlight this updated container support - the example project below provides the times when the International Space Station will rise above and set below the horizon for the next 24 hours. To keep this example simple, we will hard-code orbital data for the ISS (as of Feb 4, 2021), and assume the observer is located at Cape Canaveral Space Force Station.
from datetime import datetime, timedelta
import pytz
from skyfield.api import EarthSatellite, load, wgs84
def handler(event, context):
ccsfs = wgs84.latlon(+28.4889, -80.5778)
tz = pytz.timezone("US/Eastern")
satellite = EarthSatellite(
"1 25544U 98067A 21034.93850296 .00000995 00000-0 26262-4 0 9997",
"2 25544 51.6455 281.7841 0002223 334.9147 115.4374 15.48938034267947",
)
position = satellite - ccsfs
now = datetime.now(pytz.utc)
t = load.timescale().from_datetimes([now, now + timedelta(hours=24)])
t, events = satellite.find_events(ccsfs, *t)
for ti, event in zip(t, events):
alt, az, distance = position.at(ti).altaz()
print(
ti.astimezone(tz).strftime("%Y-%m-%d %H:%M:%S %Z"),
("rises", "culminates", "sets")[event],
alt if alt.degrees > 1 else "",
)
if __name__ == "__main__":
handler(None, None)
The code calculates how many times the satellite rises above the horizon over the span of a single day. For each such event, our program will print the time it rises, the moment and altitude of greatest ascent, and the time it sets.
2021-02-07 17:37:12 EST rises
2021-02-07 17:41:56 EST culminates 13deg 57' 59.5"
2021-02-07 17:46:40 EST sets
2021-02-07 19:13:31 EST rises
2021-02-07 19:18:53 EST culminates 41deg 07' 40.6"
2021-02-07 19:24:13 EST sets
2021-02-08 08:40:04 EST rises
2021-02-08 08:44:09 EST culminates 08deg 43' 02.6"
2021-02-08 08:48:17 EST sets
2021-02-08 10:15:12 EST rises
2021-02-08 10:20:34 EST culminates 50deg 18' 24.1"
2021-02-08 10:25:59 EST sets
2021-02-08 11:54:26 EST rises
2021-02-08 11:58:01 EST culminates 05deg 23' 08.6"
2021-02-08 12:01:36 EST sets
Our local development will include a simple Dockerfile
in addition to a docker-compose.yml
file and requirements.txt
file.
FROM python:3.8
WORKDIR /usr/src/app
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["python", "./app.py"]
version: "3.9"
services:
web:
build: .
volumes:
- .:/usr/src/app
pytz
skyfield
Traditional Deployment Package
To deploy a Lambda function, we must create a deployment package consisting of all code dependencies, in addition to the code itself. This isn’t a difficult process but involves manual steps that are not otherwise required in our container-based environment. We can use our existing requirements.txt file to set up a virtual environment and `pip install` our dependencies as usual.
$ python3 -m venv venv
$ source venv/bin/activate
(venv)$ pip install -r requirements.txt
Our libraries are now located within the `site-packages` directory of our virtual environment. These need to be archived alongside our lambda function to create the deployment package.
$ cd venv/lib/python3.9/site-packages
$ zip -r ../../../../app.zip .
$ cd ../../../../
$ zip -g app.zip app.py
If we were to use this .zip
file as our Lambda function, we would get the following error in production: Importing the NumPy C-extensions failed
.
If you look closely, you’ll notice that our virtual environment was created using the locally installed python version: 3.9
. Our Dockerfile is set to the AWS supported runtime of python:3.8
, but is not used with python’s virtual environment. We could assume this to be the cause of our production error, but the error will persist even after resolving local and production environment discrepancies. Unfortunately, C-extension libraries (such as NumPy
) are not supported by the AWS Lambda python runtime.
Container Image Deployment
As the local environment already supports Docker containers, it would be much more convenient to have our deployment process use the same tools. AWS provides base images for all supported Lambda runtime environments. This allows us to locally test our lambda function, invoking it directly through the runtime interface emulator.
With our Dockerfile
updated as follows:
FROM public.ecr.aws/lambda/python:3.8
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["app.handler"]
We can now test with a local container:
$ docker build -t app:latest .
$ docker run -p 9000:8080 app:latest
$ curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}'
The runtime interface emulator invoked through the /2015-03-31/functions/function/invocations
endpoint, acts as a local proxy for Lambda’s Runtime API. The lambda handler specified in the updated Dockerfile
will receive event and context data, just as in production.
Here are the results from a local test invocation:
START RequestId: 00ca084f-159f-46ec-9d69-9c638573e081 Version: $LATEST
2021-02-07 17:37:12 EST rises
2021-02-07 17:41:56 EST culminates 13deg 57' 59.5"
2021-02-07 17:46:40 EST sets
2021-02-07 19:13:31 EST rises
2021-02-07 19:18:52 EST culminates 41deg 07' 40.8"
2021-02-07 19:24:13 EST sets
2021-02-08 08:40:04 EST rises
2021-02-08 08:44:10 EST culminates 08deg 43' 02.6"
2021-02-08 08:48:17 EST sets
2021-02-08 10:15:12 EST rises
2021-02-08 10:20:34 EST culminates 50deg 18' 24.2"
2021-02-08 10:25:59 EST sets
2021-02-08 11:54:26 EST rises
2021-02-08 11:58:01 EST culminates 05deg 23' 08.6"
2021-02-08 12:01:36 EST sets
END RequestId: 00ca084f-159f-46ec-9d69-9c638573e081
REPORT RequestId: 00ca084f-159f-46ec-9d69-9c638573e081 Init Duration: 0.27 ms Duration: 160.37 ms Billed Duration: 200 ms Memory Size: 3008 MB Max Memory Used: 3008 MB
For our lambda function to use the container, we must deploy the image to Amazon ECR using docker push
.
$ docker tag app:latest 123456789012.dkr.ecr.us-east-1.amazonaws.com/app:latest
$ docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/app:latest
Now, when you create a function in the Lambda console, you will have the option to enter your new container image URI as the base image.
Custom Base Image
In addition to AWS base images, you can also update an existing Linux image to support Lambda container deployment. The Lambda Runtime Interface Client must be added to the Dockerfile
.
RUN pip install awslambdaric
ENTRYPOINT ["/usr/local/bin/python", "-m", "awslambdaric"]
Optionally, you can install the runtime interface emulator to allow for local testing with docker-compose
.
$ mkdir -p ~/.aws-lambda-rie
$ curl -Lo ~/.aws-lambda-rie/aws-lambda-rie https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie
$ chmod +x ~/.aws-lambda-rie/aws-lambda-rie
version: "3.9"
services:
web:
build: .
entrypoint: /aws-lambda/aws-lambda-rie
command: /usr/local/bin/python -m awslambdaric app.handler
volumes:
- ~/.aws-lambda-rie:/aws-lambda
- .:/usr/src/app
ports:
- 9000:8080
The JBS Quick Launch Lab
Free Qualified Assessment
Quantify what it will take to implement your next big idea!
Our assessment session will deliver tangible timelines, costs, high-level requirements, and recommend architectures that will work best. Let JBS prove to you and your team why over 24 years of experience matters.