Select Page

Build Lambda SFTP client with S3 integration

In this article, I’ll explain how we can build a client for SFTP server using Lambda. It’s a secure environment, also lambda has a static IP address. A brief view of infrastructure:

  1. We have an S3 bucket, and each time when the file will be added to the bucket we should transfer it to the SFTP server.
  2. For security reasons Lambda should has a static IP address, this address will be whitelisted on the customer side
  3. We have a classic VPC – 1 web subnet with Internet access, 2 application subnet with NAT attached, 3 Database subnet.

You should clone this repository https://github.com/yunojuno/lambda-s3-sftp and follow instructions to build a package.zip file. Basically just build a zip archive using basic docker commands.

 

Create a IAM Role for Lambda function, your function should have specific permissions to be executed under NAT subnet and permissions to read and receive events from S3 bucket.

Assigne to Role S3ReadOnly permissions, and create a customer policy with next rules:


{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1586855062068",
            "Action": [
                "ec2:CreateNetworkInterface",
                "ec2:DescribeNetworkInterfaces",
                "ec2:DeleteNetworkInterface"
            ],
            "Effect": "Allow",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "logs:CreateLogGroup",
            "Resource": "arn:aws:logs:us-west-1:<YOUR_ACCOUNT_ID>:*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": [
                "arn:aws:logs:us-west-1:<YOUR_ACCOUNT_ID>:log-group:/aws/lambda/lambda-sftp:*" ] } ] } 

 

First of all, go to Lambda and click “Create Lambda function”. Chose a Python3.6.

create Lambda

Now you should add an S3 trigger to Lambda. Click “Add trigger” and chose S3, then prefix or put “/” for the whole bucket.

create Lambda Trigger

Then upload the package.zip file to Lambda.

upload Lambda file

After that you should add environment variables for the SFTP server:

  1. SSH_USERNAME
  2. SSH_PASSWORD
  3. SSH_FOLDER

Now you should add a network configuration and place Lambda under NAT subnet, in this case, lambda will use a NAT IP address.

VPC configuration