Serverless – Putting It All Together

After putting all necessary infrastructure pieces in place it is finally time to implement my serverless function using AWS Lambda. On the AWS management console there is an editor available to configure the function and with Cloud9 one even has an integrated IDE in the management console to write functions in a variety of languages.

However for my little project I wanted to go the traditional route with writing my code locally (I am a big fan of the Atom editor these days), being able to test it and then uploading it to AWS. My language of choice currently is Python so I am came across python-lambda, which provides excellent scaffolding around creating an AWS Lambda project locally and then takes care of uploading the archive (containing all required dependencies) to AWS.

At the heart of my serverless function is a Neo4J query to get events from venues which are in close proximity to the currently viewed event. While there are different ways to interact with Neo4J I choose the Cypher query language. In previous versions of this feature (which were not yet running as an AWS lambda function) I had the processing of the data split over several Cypher queries combined with Python magic in between. The goal was to change that into one query (and basically let Cypher do all the work). This is what I came up with:

// Lookup the venue specified by name as well as all other venues which are hosting events

MATCH (v1:Venue { name: "%s" }),(v2:Venue)-[r]->(e:Event)

// Filter all venues and only use the ones that have the geo labels set

WHERE EXISTS (v1.latitude) and EXISTS (v2.latitude)

// Remember the results and store how far apart these venues would be

WITH v1, v2, toInt(distance(point(v1),point(v2))) as distance, r as hosting, e as event

// Next filter the results and make sure the distance between the venues is below a certain max value and also only consider the venues which host events in the future

WHERE distance > 0 and distance < %s and hosting.startTime > timestamp() WITH v2 as venue, distance, hosting, event

// Sort the results ascending so that the closest venue is at the top. Also sort by start time of the event and take the first 10 results

ORDER BY distance ASC, hosting.startTime ASC LIMIT 10

// From the result list return the event name of each event alongside its id, distance of its venue, time and venue name

RETURN event.name as event_name, head(collect( [hosting.eventId, distance, hosting.startTime, venue.name])) as event_details

Running the above query in the Neo4J UI presents a result like shown in the following screenshot:

Same as with the Microservices I am leveraging Gitlab CI features to bring the function to work. The below shows the build and deploy phase of the AWS lambda function. During the build phase I am putting together a container which has the function code as well as python-lambda deployed and during the deploy phase the only thing left to do (besides fiddling around with some configuration) is to call lambda deploy to trigger the upload to AWS.

.build_script: &build_script

stage: build

script:

- export stack="${service_name}-${CI_COMMIT_REF_SLUG}-${CI_COMMIT_SHA:0:8}"

- docker build --build-arg version=${CI_COMMIT_SHA:0:8} --build-arg branch=${CI_COMMIT_REF_SLUG} --build-arg stack=$stack -t $registry/$service_name:latest .

- docker push $registry/$service_name:latest

.deploy_script: &deploy_script

stage: deploy

image: $registry/$service_name:latest

script:

- mv config.yaml config.yaml.orig

- sed "s/function_name:\ [a-z\_]*/&_${CI_COMMIT_SHA:0:8}/" config.yaml.orig > config.yaml.1

- sed "s/version:\ 'DEV'/version:\ '${CI_COMMIT_SHA:0:8}'/" config.yaml.1 > config.yaml

- lambda deploy

Obviously I also had to update my Angular UI to use this function and since all UI and API requests get orchestrated by Kong I had to update its configuration respectively. But now I am able to present events in close proximity to the currently viewed event on Harvee as it can be seen in the next screenshot (the event shown is organized by the Python user group in Düsseldorf, where I (try to) attend the monthly meetings and their Hackathons happening twice a year).