Learnlog

9:50am — Fenrir CloudFormation

I am taking today off from learning Bazel and instead I am looking at how to build a CloudFormation static site deployer for Fenrir.

Fenrir is a AWS SAM deployer that is basically sam deploy but in a Step Function. I want Fenrir to be able to deploy “full-stack” applications, including front end resources which can sit in S3 as a static assets. This would make it much easier for Coinbase engineering to work with serverless, as currently they have to deploy front end code separately from the serverless API’s that power them.

As I understand it, CloudFormation does not allow you to upload S3 objects. So I will have to build a “custom resource” to uploaded and extract files to a S3 bucket.

I have never built a custom CloudFormation resource before and I am anticipating a bunch of problems, mostly static typing in both the [goformation](https://github.com/awslabs/goformation) library, and the JSON schema we use to validate Fenrir input.

To start with I am going to go have a look at what others have done and try get a base knowledge of the moving parts.

10:30am — Reading and Scheming

Found a few useful blog posts

Also the Golang Lambda SDK already has all the types and a useful function wrapper to make custom resources. It does look like this library tries to hide all the complicated CF logic.

I am going to reduce my initial scope for today and not deal with file extraction, and just focus on getting a custom resource working. That will still probably be a full days work.

I have started making changes to step our Step Function framework which Fenrir uses, and to Fenrir. I am still unsure of what changes goformation will need so I will start from the top down and wait to start getting errors back. Typical trial and error development.

12:00am — Fighting with Types

Spent the last hour fighting types with Fenrir. I want the custom resource handler to be inside the Fenrir lambda. I like keeping related functionality together in Lambdas, rather than having lots of small Lambdas. I think that having lots of small lambdas is just trading off development costs for infra costs.

12:30pm — GoLang Random Dict Order Bug

Hit a super annoying bug involving and error that was not correctly being handled in my Fenrir test suite so I went down a rabbit hole.

The tests sometimes worked, which is infuriating. I eventually ran it down to an error being swallowed rather than exposed, and because of Go’s random order when iterating over a map this error would fail tests only occasionally.

Oh well an hour is now gone, great time for food.LUNCH#### 1:00pm — CloudFormation Custom Resources in Go Lambda

This is my new custom resource lambda handler, basically a copy/paste off the example in the README: func StaticSiteResources(awsc aws.Clients) cfn.CustomResourceLambdaFunction { return cfn.LambdaWrap(func(ctx context.Context, event cfn.Event) (physicalResourceID string, data map[string]interface{}, err error) { v, _ := event.ResourceProperties["Echo"].(string) data = map[string]interface{}{ "Echo": v, } return }) }

1:30pm — Lambda Wrapping

Had to refactor the LambdaWrap because it is impossible to stub the http client for tests… how I miss Ruby sometimes.

Now I am going to deploy this to my sandbox AWS account for testing.

To deploy I assume-role into my account then use the ./scripts/cf_bootstrap script to deploy. This builds Fenrir locally, uploads it to upload S3 and uses CloudFormation to update the resources.

Then I created a simple test event { "ResponseURL": "http://localhost", "ResourceProperties": {"Echo": "asd"} }

And directly invoked the function, which errored but logging was correct.

Now comes the hard part, getting it working with Fenrir and goformation.

3:00pm — goformation CustomResource Type

The reason why goformation is hard is because it pulls down the Cloudformation spec from AWS and then generates all static Go classes. Custom resources cannot be in the specification so there is no class for them to be unmarshaled into.

To work around this I have injected into the unmarshalling a check to see if the type starts with Custom:: and then to select a generalizable CustomResource class. This is pretty simple and might have problems later but for now it is working.

goformation also generates JSON schema, which I am ignoring. I will rely on the clients (Fenrir) to edit the schema they validate against. So Fenrir will need to build and inject S3File schema into the larger AWS SAM schema.

4:30pm — It Works, First Custom::Resource Deploy

image Custom CloudFormation resource working

It Worked!! Well it deploys without erroring. At the moment I have:

  1. goformation marshalling and unmarshalling my custom resources. I will work with their maintainers too get my PR merged https://github.com/awslabs/goformation/pull/213
  2. Step now allows for default handlers, so the lambda can be used outside of its Step Function https://github.com/coinbase/step/pull/43, e.g. for Custom CF resources.
  3. Fenrir now supports Custom::S3File type, though it doesn’t do anything yet https://github.com/coinbase/fenrir/pull/12

The schema for S3File is: var S3FileSchema = `{ "additionalProperties": false, "properties": { "Properties": { "additionalProperties": false, "properties": { "Bucket": { "type": "string" }, "Key": { "type": "string" }, "Uri": { "type": "string" } }, "required": ["Bucket", "Key", "Uri"], "type": "object" }, "Type": { "enum": [ "Custom::S3File" ], "type": "string" } }, "required": [ "Type", "Properties" ], "type": "object" }

As you can see, I actually don’t allow ServiceToken which is required by CloudFormation. Fenrir replaces the ServiceToken with its own Lambda ARN: if res.Properties["ServiceToken"] != nil { return resourceError(res, resourceName, "ServiceToken must be nil") } res.Properties["ServiceToken"] = lambdaArn

This means that the client doesn’t need to know where the Fenrir lambda is, so the template can be reused across accounts. It also makes S3File look more like other CloudFormation resources to developers.

One very ugly piece of code left is: `func JSONSchema() (string, error) {
…``` defs := newSchema[“definitions”].(map[string]interface{})
props := newSchema[“properties”].(map[string]interface{})
defs[“Custom::S3File”] = s3FileSchema```` res := props[“Resources”].(map[string]interface{})
pp := res[“patternProperties”].(map[string]interface{})
reg := pp["^[a-zA-Z0-9]+$"].(map[string]interface{})
ao := reg[“anyOf”].([]interface{})
reg[“anyOf”] = append(ao, map[string]interface{}{"$ref": “#/definitions/Custom::S3File”})
newSchemaStr, err := json.Marshal(newSchema)
if err != nil {
return “”, err
}
return string(newSchemaStr), nil
}``

This code injects the S3File schema into the AWS SAM schema Fenrir uses to validate input. This is some pretty ugly golang. Exploring a map[string]interface{}becomes a ton of wrapping code, which in Ruby would be much cleaner.#### End

This has been a productive day. I started without ever having created a custom CloudFormation resource, and now I have created a custom S3File resource AND laid the ground work for allowing more custom resources in Fenrir.

Also, I am enjoying writing these learn logs (as long as I can show my work). It is a useful motivator and a good rubber duck.