Lessons Learnt On Deploying GO Lambda Application on AWS - Part I

Posted on Views:

Recently, I started to build an application with Go, it is a quite simple application that does something very basic and then sends an notification to a telegram bot. It's quite obvious to me this kind of application is quite suitable to run as Lambda, and that's where I decided to deploy my application to once it is working well locally. It turned out that I had to solve several issues I encountered. Here I share how I solved those issues, so you don't have to scratch your head when encounter them.

Attempt 1: Deploy the application through the web interface.

For my first attempt on deploying the application, my goal is to make things as simple as possible.
Therefore, I chose to use the web interface. From the web interface, there's an option to upload zip file and that's where I began.

Problem: Compile GO in an static way

This problem happens quite often from what I see on the internet. The main issue with here is that, some of the libraries in GO uses a feature called CGO which means using C code in GO, and when this feature is in use, GO compiler will try to create dynamic binary.

To solve this problem, it is often as simple as compiled the code to a statically linked binary. Do note that, some of the code that's compiling use GCC was not working, this is because often times the GLIBC library is higher than the ones used in AWS lambda environment, at least that was the case for me (I am on an Linux laptop with Manjaro Linux).

I was able to find something called musl-gcc and then used it in my compilation 

1
2
build:
	CC="musl-gcc" go build --ldflags '-linkmode external -extldflags "-static"' ./main.go

This proves to be working fine, once I complied the binary, zip and upload it to lambda through the interface, everything seems to be working.

Attempt 2: Deploy the application through AWS SAM

Often, it is not efficient to manually upload the code using a zip file through every time, that's why I started to thinking about introducing SAM as a tool to simplified the process of deployment. This was when I encountered the second issue.

Problem: Asking SAM to compile the GO program in an static way.

As the line above says, SAM always compiled the code in a dynamic way which is why the binary fails to work again even locally using the command sam invoke local.

Now it's the time to tell SAM I don't want dynamically linked binaries. As a matter of fact, none of the article available online has a direct answer to my question, fortunately, I did find an AWS documentation on using custom runtime. Based on this article, a GO program that wants to utilize static linking can have the following template:

1
2
3
4
5
6
7
8
9
10
11
Resources:
  HelloGOFunction:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: HelloGO
      Handler: main
      Runtime: go1.x
      MemorySize: 512
      CodeUri: .
    Metadata:
      BuildMethod: makefile

And in the MakeFile, the following corresponding entries need to be added:

1
2
3
build-HelloGOFunction:
  CC="musl-gcc" go build --ldflags '-linkmode external -extldflags "-static"' ./main.go
  cp ./main $(ARTIFACTS_DIR)

Doing so will make sure compiled binary are statically linked and works on lambda when bundle and uploaded.