-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: commit with entirely functional code for bug repro
- Loading branch information
Showing
10 changed files
with
172 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
*.pyc | ||
/.vscode/* | ||
.vscode/* | ||
/.vscode | ||
aws_env.env | ||
*.tar.gz | ||
# package directories | ||
node_modules | ||
jspm_packages | ||
# Serverless directories | ||
.serverless | ||
.requirements.* |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
# Python support can be specified down to the minor or micro version | ||
# (e.g. 3.6 or 3.6.3). | ||
# OS Support also exists for jessie & stretch (slim and full). | ||
# See https://hub.docker.com/r/library/python/ for all supported Python | ||
# tags from Docker Hub. | ||
FROM python:3.6 | ||
WORKDIR /usr/app | ||
ADD . /usr/app | ||
|
||
# install necessary node things | ||
RUN apt-get update | ||
RUN apt-get -y install curl gnupg | ||
RUN curl -sL https://deb.nodesource.com/setup_11.x | bash - | ||
RUN apt-get -y install nodejs | ||
RUN npm install -g serverless | ||
RUN npm config set bin-links false | ||
RUN npm install | ||
# install necessary python things | ||
RUN pip install -r requirements.txt | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,46 @@ | ||
# bug-aws-lambda-infinite-timeout | ||
Repository to reproduce the aws lambda bug of non-stop timeout after a first timeout | ||
|
||
# Bug Trigger | ||
|
||
When the lambda package is too large, after the lambda times out once, it times out forever. It does not matter if you increase the timeout or memory in AWS. | ||
|
||
In this repo, the dependencies are large when zipped to upload to AWS in order to reproduce the bug. | ||
|
||
# How to run it | ||
|
||
copy the aws credentials file like below | ||
|
||
``` bash | ||
cp aws_env.env.rename aws_env.env | ||
``` | ||
|
||
fill the credentials file that will be used inside your container | ||
|
||
Starting the container | ||
|
||
``` bash | ||
docker-compose run --rm bug-lambda-service bash | ||
``` | ||
|
||
Deploying the application | ||
|
||
``` bash | ||
serverless deploy | ||
``` | ||
|
||
Run the script to continuously invoke the lambda function | ||
|
||
``` bash | ||
python main.py | ||
``` | ||
|
||
|
||
Then go to AWS console and reduce the timeout to **1 seconds**. The function will start to timeout. | ||
After it starts to timeout, increate the timeout in AWS' console to something greater than 3 seconds. Even putting it to 30 seconds. It will continue to timeout. | ||
|
||
The image below shows the execution of the lamda. | ||
Even though it is only a sleep of **2 seconds**, after it starts timing out, it never recovers. Even after **10 seconds** of waiting time. | ||
|
||
|
||
![image](execution-output.JPG) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
AWS_ACCESS_KEY_ID= | ||
AWS_SECRET_ACCESS_KEY= | ||
AWS_DEFAULT_REGION= |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
version: '2.1' | ||
|
||
services: | ||
bug-lambda-service: | ||
image: bug-lambda-service | ||
build: . | ||
volumes: | ||
- .:/usr/app/ | ||
env_file: | ||
- aws_env.env | ||
environment: | ||
# needs this for deploying using windows | ||
- VIRTUALENV_ALWAYS_COPY=1 | ||
command: python main.py | ||
|
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
try: | ||
import unzip_requirements | ||
except ImportError: | ||
pass | ||
import time | ||
import boto3 | ||
import json | ||
|
||
def handler(event, context): | ||
time.sleep(2) | ||
return 'finished' | ||
|
||
|
||
if __name__ == '__main__': | ||
lambda_client = boto3.client('lambda') | ||
while True: | ||
lambda_request = lambda_client.invoke( | ||
FunctionName='bug-lambda-timeout-dev-bug-lambda-event-function' | ||
) | ||
print('published: ', time.time()) | ||
print(lambda_request['Payload'].read()) | ||
time.sleep(0.5) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
{ | ||
"name": "bug-lambda-timeout", | ||
"version": "1.0.0", | ||
"description": "", | ||
"main": "handler.js", | ||
"dependencies": {}, | ||
"devDependencies": { | ||
"serverless-python-requirements": "^5.0.1" | ||
}, | ||
"scripts": { | ||
"test": "echo \"Error: no test specified\" && exit 1" | ||
}, | ||
"repository": { | ||
"type": "git", | ||
"url": "" | ||
}, | ||
"author": "Pedro Torres", | ||
"license": "", | ||
"homepage": "" | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
boto3==1.10.12 | ||
botocore==1.13.12 | ||
docutils==0.15.2 | ||
jmespath==0.9.4 | ||
lambda-packages==0.20.0 | ||
numpy==1.17.3 | ||
python-dateutil==2.8.0 | ||
s3transfer==0.2.1 | ||
six==1.13.0 | ||
urllib3==1.25.6 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
service: bug-lambda-timeout | ||
|
||
package: | ||
exclude: | ||
- aws_env.env | ||
|
||
custom: | ||
pythonRequirements: | ||
zip: true | ||
useDownloadCache: false | ||
useStaticCache: false | ||
|
||
plugins: | ||
- serverless-python-requirements | ||
|
||
provider: | ||
name: aws | ||
runtime: python3.6 | ||
memorySize: 2048 # optional, in MB, default is 1024 | ||
|
||
|
||
functions: | ||
bug-lambda-event-function: | ||
timeout: 3 | ||
handler: main.handler | ||
description: lambda to test aws timeout bug |