Accelerate transformation at supersonic speed with a well designed cloud program enabled by committed change agents
Werner Vogels launched the 2017 AWS Summits in Sydney with his typical non-conformist approach that transcends both keynotes and strategy. The Amazon CTO took center stage wearing a custom T-shirt emblazoned with “Werner Against the Machine” — the chest insignia of a modern superhero.
As the ultimate committed change agent disguised as a CTO, Werner is using his AWS superpowers to save businesses worldwide — using speed, analytics, flexibility, the skill to adapt, and the power to take flight from ‘hostile’ database vendors.
The Supersonic Speed of an Enterprise
TL;DR — speed matters
For most other enterprises, supersonic speed is the most elusive superpower for a heroic cloud journey — and the most necessary to harness.
During my 20-year career at Capital One, speed at scale was the modus operandi of the lean enterprise. Accelerated by an agile mindset and DevOps culture, the rapid rate of adoption enabled a cloud journey that transformed Capital One into what is now essentially a large FinTech startup.
achieving supersonic speed is the real superpower for companies like @CapitalOneDevEx #AWSSummit @Werner
Capital One’s accelerated technology transition is powered by the API-driven cloud computing services from AWS, enabled by real-time access to big data, and fueled by a commitment to the open source community. While technology plays a leading role, the hardest part of that transition and ultimate superpower is really a talent transformation.
The Gravitational Pull of Legacy
Gaining the supersonic speed to achieve escape velocity from on-premise data centers requires bold leadership with a long-term dedication to innovation. There is no magic pill for enterprises — although going all-in with AWS is a leap in the right direction.
The Cloud Center of Excellence is the tip of the spear for enterprises adopting the cloud
Cloud computing has posed a disruptive threat for years, but short-term incentive structures provide little reason for large corporations to invest in the disruptive technology. Many enterprises continue to be anchored by the weight of their own internal processes and platforms — and paralyzed by the FUD of vendors in survival mode contributing noise to their echo chambers.
“It’s not the big that eat the small … it’s the fast that eat the slow.“ — Laurence Haughton
The gravitational pull is preventing many enterprises from gaining momentum and advantage in the cloud — at least not at the speed required to avoid death by the fast and hungry. The only way out of the dilemma is to adopt a new approach — where continuous innovation is the new bottom line and speed is the name of the game.
Achieve Escape Velocity with a CCoE
For enterprises that are serious about cloud adoption and aspire to achieve supersonic speed, the executive leadership team must invest their time, resources and budget into the sponsorship of a Cloud Center of Excellence (CCoE).
starting your enterprise cloud adoption journey? please don't pass go and collect $200 without a cloud center of excellence #AWSSummit
The CCoE is an essential mechanism for large organizations planning to achieve the velocity required to escape the gravitational pull of their own death star — a private cloud or an on-premise data center.
Design the CCoE for Constant Evolution
TL;DR — transition matters
The Cloud Center of Excellence will continually evolve in order too keep pace with the rate of innovation associated with cloud adoption. Modern enterprises should be very purposeful with the organization design principles of their cloud program structure.
The leaders of cloud programs should observe the flow of their value within the organization, and design an adaptive structure that evolves alongside the internal needs of the enterprise. Simon Wardley infuses these concepts of flow and transition within his value chain mapping strategies — and his design principles apply as much to organizational design as they do towards product evolution.
All of this stuff - bimodal / two speed IT / dual operating - de-emphasises that important transition, the middle - https://t.co/hgpEvMurt5
Applying Simon’s design principles of Pioneers → Settlers → Town Planners (PST) toward an organization’s cloud adoption program offers an innovative approach for effectively navigating the evolving journey. Guided by astute situational awareness, organizations learn to continuously pivot through the cycle — from the exploration stage, through expansion, toward the enhancement of industrialized components and patterns.
Accelerate through the ‘trough of despair’ with an education program that achieves critical mass of cloud fluency
Stage 1: The Cloud Center of Exploration
The Pioneers Explore
Embarking on a cloud journey requires a tremendous amount of iterative experimentation with the underlying AWS utility compute services. During this stage of early adoption, the core team is focused on pioneering engineers with plenty of aptitude and attitude.
The two-pizza team leverages agile techniques to break things daily — and collect the data which will determine the patterns of future success.
The core team is stacked with battle-tested engineers with deep experience in understanding how critical functions currently operate, and know how to translate existing data center platforms into cloud services. These engineers have plenty of scars and war stories from previous tours of duty with security, network, and access control.
ProTip 1: An executive sponsor is absolutely essential during the early phases of the cloud adoption. The sponsor must be a strong advocate, provide plenty of air-cover, and actively engage with the core team for the purpose of removing impediments from the board.
Stage 2: Cloud Center of Expansion
The Settlers Expand
Once the pioneers solidify early successes into identified patterns, the focus turns toward scaling the prototypes into products and services that are consumable by the enterprise. By listening to a broad range of internal customers, the settlers refine the patterns and help the understanding grow.
When the cloud services start to scale across an enterprise, it’s natural to place a heavier emphasis on governance and controls. A key advantage of AWS is the ability to engineer your governance by leveraging the API-driven services to access real-time controls and compliance.
At this stage, it’s imperative to focus on scaling the early understanding of AWS to the enterprise — achieving critical mass of cloud fluency is the only way an organization can sustain a transition to the new operating model.
Invest the time and money into a multi-dimensional cloud education program. An engaged workforce armed with compelling context and content will dramatically accelerate your organization through the ‘trough of despair’ and ensure more attraction versus attrition.
During the enablement phase, the core team should no longer be considered the most cloud savvy department in the organization. By unleashing cloud superpowers upon thousands of developers throughout the enterprise, the core team should pivot and begin harvesting new and improved patterns from other divisions.
Elevating other departments beyond your core team’s existing capabilities is a key early indicator of achieving escape velocity.
ProTip: Instead of outsourcing your cloud training to Human Resources, tightly integrate cloud education as a core function on your program team. Leverage the AWS certifications as a benchmark for cloud fluency and set a minimum goal of 10% enterprise-wide.
Stage 3: Cloud Center of Enhancement
The Town Planners Enhance
As the cloud journey matures, it should lead toward the commoditization of services that result in more cost efficient, faster, and industrialized platforms. In highly regulated industries like financial services, the organization depends on these town planners to ensure customers and regulators can trust what’s built.
Innovation it’s not just limited to the early stages and pioneers — it’s also found in the operational stages of cloud adoption. For example, Capital One’s Terren Peterson is evolving the mindsets of their engineering teams by embracing the concepts of Site Reliability Engineering using innovative approaches to manage operations.
Their SRE teams leverages industrialized utility functions to manage compliance and controls with Cloud Custodian. The SRE’s also contribute new functions to the ever expanding platform — completing the cycle as higher order services continuously evolve.
Cloud Custodian is a function-based policy rules engine. The origin of the service demonstrates the PST design principles at work. Originally developed internally by their pioneers, it was scaled across the entire enterprise by settlers, and finally driven toward open source commoditization by the town planners.
ProTip: Involve your operational teams starting on day one of the cloud journey. Since operations is 24x7, consider a shift-and-lift for a subset of workloads. Leverage a cloud capable MSP for interim support to lighten the load during their talent transition.
Summary
Using these organizational design principles, a cloud program team can begin to continually cycle through the explore-expand-enhance stages. Over time, this approach will begin to harness Werner’s superpowers and accelerate cloud adoption at supersonic speeds.
Drew is an AWS Community Hero, Alexa Champion, and maker of dad jokes. Follow on Twitter @drewfirment. #WePowerTech
This is a walkthrough for deploying the Color App that was demonstrated at the AWS App Mesh launch. The new service helps you to run and monitor HTTP and TCP services at scale with a consistent way to route and monitor traffic.
The following diagram shows the programming model of this simple application. This is literally the programmer’s perspective of the application:
Figure 1. Programmer perspective of the Color App.
In this post, we’ll walk through creating specific abstract resources for AWS App Mesh that will be used to drive a physical mapping to compute resources to stitch our application together, providing us with fine-grained control over traffic routing and end-to-end visibility of application request traffic and performance.
The following diagram represents the abstract view in terms of App Mesh resources:
Figure 2. App Mesh abstract perspective of the Color App.
Finally, we deploy the services that will comprise our application to ECS along with proxy sidecars for each service task; these proxies will be governed by App Mesh to ensure our application traffic behaves according to our specifications.
Figure 3. Amazon ECS compute perspective of the Color App.
The key thing to note about this is that actual routing configuration is completely transparent to the application code. The code deployed to the gateway containers will send requests to the DNS name colorteller.demo.local, which we configure as a virtual service in App Mesh.
App Mesh will push updates to all the envoy sidecar containers to ensure traffic is sent directly to colorteller tasks running on EC2 instances according to the routing rules we specify through App Mesh configuration.
There are no physical routers at runtime since App Mesh route rules are transformed to Envoy configuration and pushed directly to the envoy sidecars within the dependent tasks.
This brief guide will walk you through deploying the Color App on ECS. The process has been automated using shell scripts and AWS CloudFormation templates to make deployment straightforward and repeatable.
Core networking and compute infrastructure doesn’t need to be recreated each time the Color App is redeployed. Since this can be time-consuming, resource provisioning is divided among a layered set of CloudFormation stack templates.
The App Mesh deployment is also partitioned into different stages as well, but this is for for performance reasons since App Mesh operations are very fast. The reason for the separation is simply so you can tear down the Color App without tearing down the demo mesh in case you also have other sample apps running in it for experimentation.
Infrastructure templates:
examples/infrastructure/vpc.yaml – creates the VPC and other core networking resources needed for the application independent of the specific compute environment (e.g., ECS) provisioned for cluster.
examples/infrastructure/ecs-cluster.yaml – creates the compute instances and other resources needed for the cluster.
examples/infrastructure/appmesh-mesh.yaml – creates an App Mesh mesh.
Application resource templates:
examples/apps/colorapp/ecs/ecs-colorapp.yaml – deploys application services and related resources for the Color App.
examples/apps/colorapp/ecs/servicemesh/appmesh-colorapp.yaml – creates mesh resources for the Color App.
Each template has a corresponding shell script with a .sh extension that you run to create the CloudFormation stack. These scripts rely on the following environment variables values that must be exported before running:
AWS_PROFILE – your AWS CLI profile (set to default or a named profile).
An Amazon Virtual Private Cloud (VPC) is a virtual network that provides isolation from other applications in other networks running on AWS. The following CloudFormation template will be used to create a VPC for our mesh sample applications:
examples/infrastructure/vpc.yaml
Set the following environment variables:
AWS_PROFILE – your AWS CLI profile (set to default or a named profile)
ENVIRONMENT_NAME – will be applied as a prefix to deployed CloudFormation stack names
Run the vpc.sh script to create a VPC for the application in the region you specify. It will be configured for two availability zones (AZs); each AZ will be configured with a public and a private subnet.
$ export AWS_PROFILE=default $ export AWS_DEFAULT_REGION=us-west-2 $ export ENVIRONMENT_NAME=DEMO $ ./examples/infrastructure/vpc.sh ... + aws --profile default --region us-west-2 cloudformation deploy --stack-name DEMO-vpc --capabilities CAPABILITY_IAM --template-file examples/infrastructure/vpc.yaml --parameter-overrides EnvironmentName=DEMO Waiting for changeset to be created.. Waiting for stack create/update to complete ... Successfully created/updated stack - DEMO-vpc $
Create an App Mesh
A service mesh a logical boundary for network traffic between services that reside in it. AWS App Mesh is a managed service mesh control plane. It provides application-level networking support, standardizing how you control and monitor your services across multiple types of compute infrastructure.
The following CloudFormation template will be used to create an App Mesh mesh for our application:
examples/infrastructure/appmesh-mesh.yaml
We will use the same environment variables from the previous step, plus one additional one (MESH_NAME), to deploy the stack.
MESH_NAME – name to use to identify the mesh you create (we’ll use appmesh-mesh)
Waiting for changeset to be created.. Waiting for stack create/update to complete ... Successfully created/updated stack - DEMO-appmesh-mesh $
At this point we have now created our networking resources (VPC and App Mesh), but we have not yet deployed:
compute resources to run our services on
mesh configuration for our services
actual services
Create compute resources
Our infrastructure requires compute resources to run our services on. The following CloudFormation template will be used to create these resources for our application:
examples/infrastructure/ecs-cluster.yaml
In addition to the previously defined environment variables, you will also need to export the following:
SERVICES_DOMAIN – the base namespace to use for service discovery (e.g., cluster.local). For this demo, we will use demo.local. This means that the gateway virtual service will send requests to the colorteller virtual service at colorteller.demo.local.
Waiting for changeset to be created.. Waiting for stack create/update to complete ... Successfully created/updated stack - DEMO-ecs-cluster $
Review
You have provisioned the infrastructure you need. You can confirm in the AWS Console that all of your CloudFormation stacks have been successfully deployed. You should see something like this:
Now that we’ve deployed our infrastructure resources for testing, let’s configure our mesh and finally deploy the Color App.
Configure App Mesh resources
We will now add our mesh resource definitions so that when we finally deploy our services, the mesh will be able to push computed configuration down to each Envoy proxy running as a sidecar for each ECS task. The following CloudFormation template will be used to create these resources for our application:
Waiting for changeset to be created.. Waiting for stack create/update to complete ... Successfully created/updated stack - DEMO-appmesh-colorapp $
Note: the App Mesh resources for the Color App are created before the app itself is deployed in the final step; this is so Envoy, which is deployed as a task sidecar, is able to communicate with the Envoy Management Service. If the mesh itself isn’t configured first, the sidecar will remain unhealthy and eventually the task will fail.
Deploy services to ECS
Deploy images to ECR for your account
Before you can deploy the services, you will need to deploy the images that ECS will use for gateway and colortellerto ECR image repositories for your account. You can build these images from source under the examples/apps/colorteller/src and push them using the provided deploy scripts after you create repositories for them on ECR, as shown below.
Deploy the gateway image:
# from the colorapp repo root... $ cd examples/apps/colorapp/src/gateway $ aws ecr create-repository --repository-name=gateway $ export COLOR_GATEWAY_IMAGE=$(aws ecr describe-repositories --repository-names=gateway --query 'repositories[0].repositoryUri' --output text) $ ./deploy.sh + '[' -z 226767807331.dkr.ecr.us-west-2.amazonaws.com/gateway ']' + docker build -t 226767807331.dkr.ecr.us-west-2.amazonaws.com/gateway . Sending build context to Docker daemon 1MB Step 1/11 : FROM golang:1.10 AS builder ... + docker push 226767807331.dkr.ecr.us-west-2.amazonaws.com/gateway The push refers to repository [226767807331.dkr.ecr.us-west-2.amazonaws.com/gateway] latest: digest: sha256:ce597511c0230af89b81763eb51c808303e9ef8e1fbe677af02109d1f73a868c size: 528 $
Deploy the colorteller image:
# from the colorapp repo root.... $ cd examples/apps/colorapp/src/colorteller $ aws ecr create-repository --repository-name=colorteller $ export COLOR_TELLER_IMAGE=$(aws ecr describe-repositories --repository-names=colorteller --query 'repositories[0].repositoryUri' --output text) $ ./deploy.sh + '[' -z 226767807331.dkr.ecr.us-west-2.amazonaws.com/colorteller:latest ']' + docker build -t 226767807331.dkr.ecr.us-west-2.amazonaws.com/colorteller:latest . Sending build context to Docker daemon 996.4kB Step 1/11 : FROM golang:1.10 AS builder ... + docker push 226767807331.dkr.ecr.us-west-2.amazonaws.com/colorteller:latest The push refers to repository [226767807331.dkr.ecr.us-west-2.amazonaws.com/colorteller] 69856c2b3fc6: Layer already exists latest: digest: sha256:ca16f12268907c32140586e2568e2032f04b95d70b373c00fcee7e776e2d29da size: 528 $
Deploy gateway and colorteller services
We will now deploy our services on ECS. The following CloudFormation template will be used to create these resources for our application:
examples/apps/colorapp/ecs/ecs-colorapp.yaml
In addition to the previously defined environment variables, you will also need to export the following:
ENVOY_IMAGE – see Envoy Image for latest recommended Docker image (currently: 111345817488.dkr.ecr.us-west-2.amazonaws.com/aws-appmesh-envoy:v1.9.0.0-prod)
COLOR_GATEWAY_IMAGE – Docker image for the Color App gateway microservice (see example below).
COLOR_TELLER_IMAGE – Docker image for the Color App colorteller microservice (see example below).
TIP: If you don’t see a newline after curl responses, you might want to use curl -w “n” or add -w “n” to $HOME/.curlrc.
5. Shape traffic
Currently, the the app equally distributes traffic among blue, red, and white color teller virtual nodes through the default virtual router configuration, so if you run the curl command a few times, you might see something similar to this:
In the following section, we’ll walk through how to modify traffic according to rules we set.
Apply traffic rules
Open up examples/apps/colorapp/servicemesh/appmesh-colorapp.yaml in an editor. In the definition for ColorTellerRoute, you will see the spec for an HttpRoute (around line 123):
... # repeated calls will increase the stats for black since it's the only color response now {"color":"black", "stats": {"black":0.21,"blue":0.28,"red":0.26,"white":0.25}}
The following query will clear the stats history:
$ curl $colorapp/color/clear cleared
# now requery $ curl $colorapp/color {"color":"black", "stats": {"black":1}}
Since there are no other colors for the histogram, that’s all you will see no matter how many times you repeat the query.
Simulate A/B tests with a 50/50 split between red and blue:
Any integer proportion will work for the weights (as long as the sum doesn’t exceed 100), so you could have used 1 or 5 or 50 for each to reflect the 1:1 ratio that distributes traffic equally between the two colortellers. App Mesh will use the ratio to compute the actual percentage of traffic to distribute along each route.
You can see this in the App Mesh console when you inspect the route:
Figure 5. Route weighted targets.
In a similar manner, you can perform canary tests or automate rolling updates based on healthchecks or other criteria using weighted targets to have fine-grained control over how you shape traffic for your application.
To prepare for the next section, go ahead and update the HttpRoute to send all traffic only to the blue colorteller.
In the next section we’ll experiment with updating the route using the App Mesh console and analyze results visually with AWS X-Ray.
Monitor with AWS X-Ray
AWS X-Ray helps us to monitor and analyze distributed microservice applications through request tracing, providing an end-to-end view of requests traveling through the application so we can identify the root cause of errors and performance issues. We’ll use X-Ray to provide a visual map of how App Mesh is distributing traffic and inspect traffic latency through our routes.
When you open the AWS X-Ray console the view might appear busier than you expected due to traffic from automated healthchecks. We’ll create a filter to focus on the traffic we’re sending to the application frontend (color gateway) when we request a color on the /color route.
The Color App has already been instrumented for X-Ray support and has created a Segment called “Default” to provide X-Ray with request context as it flows through the gateway service. Click on the “Default” button (shown in the figure below) to create a group to filter the visual map:
Figure 6. Creating a group for the X-Ray service map.
Choose “Create group”, name the group “color”, and enter an expression that filters on requests to the /color route going through the colorgateway-vn node:
(service("appmesh-mesh/colorgateway-vn")) AND http.url ENDSWITH "/color"
Figure 7. Adding a group filter expression.
After creating the group, make sure to select it from the dropdown to apply it as the active filter. You should see something similar to the following:
Figure 8. Analyzing the X-Ray service map.
What the map reveals is that
Our color request first flows through an Envoy proxy for ingress to the gateway service.
Envoy passes the request to the gateway service, which makes a request to a colorteller.
The gateway service makes a request to a colorteller service to fetch a color. Egress traffic also flows through the Envoy proxy, which has been configured by App Mesh to route 100% of traffic for the colorteller to colorteller-blue.
Traffic flows through another Envoy proxy for ingress to the colorteller-blue service.
Envoy passes the request to the colorteller-blue service.
Click on the colorgateway-vn node to display Service details:
Figure 9. Tracing the colorgateway virtual node.
We can see an overview on latency and that 100% of the requests are “OK”.
Click on the “Traces” button:
This provides us with a detailed view about how traffic flowed for the request.
Figure 9. Analyzing a request trace
If we log into the console for AWS App Mesh and drill down into “Virtual routers” for our mesh, we’ll see that currently the HTTP route is configured to send 100% of traffic to the colorteller-blue virtual node.
Figure 10. Routes in the App Mesh console.
Click the “Edit” button to modify the route configuration:
Figure 11. Editing a route.
Click the “Add target” button, choose “colorteller-red-vn”, and set the weight to 1.
Figure 12. Adding another virtual node to a route.
After saving the updated route configuration, you should see:
Figure 13. The updated route for splitting traffic across two virtual nodes.
Now when you fetch a color, you should start to see “red” responses. Over time, the histogram (stats) field will show the distribution approaching 50% for each:
And if you refresh the X-Ray Service map, you should see something like this:
Figure 14. The updated service map with split traffic.
AWS X-Ray is a valuable tool for providing insight into your application request traffic. See the AWS X-Ray docs to learn more instrumenting your own microservice applications to analyze their performance and the effects of traffic shaping with App Mesh.
6. Review
The following is the condensed version of all the steps we performed to run the Color App.
Step #1 Export the following environment variables needed by our deployment scripts. You can use most of the example values below for your own demo, but you will need to modify the last three using your own EC2 key pair and ECR URLs for the color images (see [Deploy gateway and colorteller services]).
In this walkthrough, we stepped through the process of deploying the Color App example with App Mesh. We saw how easy it was to update routes to distribute traffic between different versions of a backend service and to access logs and distributed traces for the app in the AWS Console.
One of the key takeaways is that our control of traffic routing is transparent to the application. The application code for the gateway service that was deployed as an ECS task used the DNS name associated with the virtual service for the colorteller configured in App Mesh (colorteller.demo.local).
App Mesh propagated the configuration updates throughout the mesh that ensured traffic from dependent services to their backends was routed according to the policies we specified using App Mesh configuration, not application configuration.
In this demo, our services ran only on ECS. In the next post in this series, we’ll update the demo and deploy some of the services across different compute environments, including EC2, and see how App Mesh lets us control and monitor our running application managed within the same mesh.
AWS App Mesh Walkthrough was originally published in A Cloud Guru on Medium, where people are continuing the conversation by highlighting and responding to this story.
The tips, tricks, and insider hints for the 2019 AWS re:Invent conference in Las Vegas from December 2–6, 2019
AWS re:Invent is taking place December, 2—6, 2019 in Las Vegas. Yet again, this promises to be the biggest cloud event of the year. Last year’s event was amazing. “Jam packed” doesn’t even begin to describe it — and yet, somehow this year is sure to be even better!
This guide will grow as we get closer to the show. Please check back regularly for updates! Ping me @marknca if you spot a problem or if something is missing.
In the meantime, you can sign up with AWS to receive conference updates — including a notification when registration opens.
No official word on hotel rooms yet this year but last year and for the last couple of years, the conference room rates have been tied to registration. This means if you want to get a well positioned hotel room at the lowest rate possible, you want to register as quickly as possible.
These hotel room blocks disappear quickly. Making sure you have everything lined up for the 21st of May is a smart idea.
The official campus hasn’t been published yet but last year (2018) it was spread across seven properties with room blocks at an additional seven.
The challenge — as with last year — is that you will need to hop between venues throughout the week. The good news? These are all nice properties and it’s hard to go wrong.
Content in 2018 was divided among all of the seven main properties with breakouts from every track hosted in each location. The two big stand outs for events were the Quad—hosted at the Aria—and the Expo—hosted at The Venetian/Palazzo.
This guide will grow as we get closer to the show. Please check back regularly for updates! Ping me @marknca if you spot a problem or if something is missing.
Learning how to build a serverless deployment pipeline on Google Cloud Platform
Several years ago I worked with a team to build a production CI/CD pipeline for an AWS project. At that time, none of the major cloud vendors offered their own CI/CD services, so we then used the open source GO CD server.
Although we were successful, the project was tedious and took several months. For this reason, I’ve been interested to try out integrated alternatives as they become available.
Lately, much of my production work has involved constructing and testing cloud-based data pipelines at scale for bioinformatics research. I have been thinking for awhile now about how best to use new cloud-based CI/CD services in automating these complex pipeline deployments.
Picturing Automated Deployment
As I worked through a well-written tutorial on GCP CI/CD today, I was surprised to be unable to find any sort of diagram of the process. Although I was able to complete the tutorial successfully, I was still left feeling like I was not fully comprehending the solution architecture applications.
GCP cloud-native CI/CD Tutorial Steps
Next, I attempted to draw my own version of what I completed in the tutorial. Quickly sketching considerations from my production experience and trying to combine that with what I had learned from completing the tutorial was unsatisfying. The result is shown below.
My initial thoughts around cloud-native CI/CD
A clear path through the complexity of the steps involved wasn’t yet apparent to me. After sketching, I tried thinking about each service or set of services that would be involved in building this type of pipeline.
Considering GCP Services
I started by trying to understand the GCP Tool Services offerings and how each service would relate to building a CI/CD solution for the type of data pipelines my teams have been building.
There are a number of tools. Below is my summarized understanding of each of the GCP services listed on the Tools menu. Note that all of these services are serverless.
GCP Tools
Cloud Build— to create build pipelines with steps & triggers; outputs artifacts (images…)
Then I watched a few presentations from the recent GCP:Next conference. The most useful to me were the following:
“CI CD Across Multiple Environments” — interesting, but showed combining non GCP tools (Jenkins, HashiCorp Vault…) with GCP tools, so added even more complexity to potential scenarios.
Although these presentations and doing the tutorial were helpful in my understanding of how I might use these services for my customers, I still couldn’t visualize what I would actually build. To that end, I made an attempt to diagram one possible CI/CD architecture for a use case that I’ve been working on for awhile now.
Visualizing CI/CD for GCP Data Pipelines
Here’s my attempt to design a solution for one of my customers. I’ll describe the application architecture first. The use case is bioinformatics research via a specialized machine learning library for genomics, VariantSpark.
Below is a diagram of solution architecture on GCP K8:
CSIRO’s VariantSpark on GCP K8
You’ll note that this is a Data Lake with the K8 cluster processing data from GCS. Managed Jupyter Notebooks are part of the solutions as this is the preferred environment for research work.
Shown below is my first draft of how I imagine my team could use GCP CI/CD services to build a pipeline to deploy a VariantSpark K8 cluster on GCP.
GCP CI/CD Pipeline for VariantSpark K8 Deployment
Because VariantSpark in an open source library, there are continual improvements to it. However, because VariantSpark is a research tool, it uses major and minor version numbers in release so that it can be cited in reproducible bioinformatics research papers.
So the process to build a cluster first requires determining which version of the VariantSpark JAR file is desired for the base container image. The pipeline may needs to perform a maven build on the source GitHub Repo for VariantSpark.
Cloud Build artifacts include Docker Images, Kubernetes Clusters and more
GCP Source Repositories can mirror GitHub Repos. Cloud Build can use triggers on Source Repositories to trigger builds. Cloud Build includes builder (task) for various types of images including maven. Cloud Build builders also include docker and kubectl, so the VariantSpark JAR file could be built as part of the docker image and that image could be used to spawn a GKE cluster.
Additionally, GCP Deployment Manger could serve a YAML-deployment which details other GCP service configurations that are needed for a GKE cluster. It also seems possible to use GCP Private Catalog for research groups who have GCP Organizational Accounts to offer a VariantSpark GKE deployment template.
Next Steps
Now that I’ve learned the fundamentals of these services, I am looking forward to working with research and DevOps teams world-wide to build serverless CI/CD pipelines for bioinformatics.
Lynn Langit is a big data and cloud architect, a Google Cloud Developer Expert, an AWS Community Hero, and a Microsoft MVP for Data Platform.
Cloud-based CI/CD on GCP was originally published in A Cloud Guru on Medium, where people are continuing the conversation by highlighting and responding to this story.
Lean how modern application developers can use the new Data API to create and connect to an Aurora Serverless Database
In this article with screencast videos, we’ll go over the three ways to create an Aurora Serverless Database with Data API. We’ll also cover the four ways to connect to your Data API enabled Aurora Serverless Database.
Let’s quickly go over what Amazon Aurora offers, why a serverless database, and answer, what is the Data API and why should I care?
About Amazon Aurora
Amazon Aurora is a MySQL and PostgreSQL-compatible relational database built for the cloud, that combines the performance and availability of traditional enterprise databases with the simplicity and cost-effectiveness of open source databases.
Why Amazon Aurora Serverless? Amazon Aurora Serverless is an on-demand, auto-scaling configuration for Amazon Aurora (MySQL-compatible edition). The database will automatically start up, shut down, and scale capacity up or down based on your application’s needs. It’s a simple, cost-effective option for infrequent, intermittent, or unpredictable workloads.
What is Amazon Aurora Serverless Data API? The Data API is a new managed API layer on top of an Aurora Serverless database, allowing you to connect directly with your MySQL or PostgreSQL database. It also allows you to execute SQL statements from any application over HTTP without using a MySQL driver, plugin, or need to manage a connection pool or VPC.
Think of the Data API as a fully-managed API for interacting with your relational data. As the Data API is connected to your Serverless Cluster, you also get inherited auto-scaling, available, and backups of your database along with pausing of the database while not in use.
Three Ways to Create an Aurora Serverless Database w/ Data API:
My personal favorite is Option 2 CloudFormation for deployment as you can deploy predictable resources (not missing a step vs manual creation) and you can add additional AWS services and permissions in the CloudFormation template.
Option #1: Amazon RDS Console — Manual The RDS Console is a quick way to deploy your resources and be done in just a few minutes following these instructions:
Create a new Aurora Serverless Cluster:
Launch the Amazon RDS Console.
Select Amazon Aurora engine.
Select MySQL 5.6-compatible (the only option for serverless) and select Next.
Select Serverless for the capacity type and provide a cluster name and master credentials and select Next.
Leave all defaults and choose Create database.
Screencast for creating an Aurora Serverless Cluster via the RDS Console
Enable the Data API (for an existing Aurora Serverless Cluster): We are going to enable the Data API for the newly created Aurora serverless cluster manually via the RDS Management Console. Hopefully, this is temporary until we can enable the Data API via a CloudFormation template using the EnableHTTPEndpointAPI parameter once this puppy is out of BETA. In the meantime …
Select your cluster (even though it says database)
Select the Modify button on the upper right
Select “Data API” under the Network & Security section
Select Continue button
Select “Apply immediately” under Scheduling of modifications
Select “Modify cluster” radio button
Option #2: CloudFormation — Automated CloudFormation is the best way to deploy a consistent environment in an automated way. This solution consists of creating a CloudFormation stack based on the CloudFormation template provided in this GitHub repo here.
The template has all the parameters defined for deploying an Aurora Serverless Database in just a few clicks. In addition to creating a serverless database, the template also creates an AWS Lambda function (written in Node.js 8.10) to access your Data API enabled database using the new AWS RDSDataService API. More on the Lambda function later when we connect to a Data API enabled database in Solution 3 below.
The CloudFormation template will provision the following resources:
Aurora Serverless (MySQL) Cluster
Aurora Serverless (MySQL) Database
AWS Lambda function for connecting to your database via Data API using the AWS RDSDataService API.
IAM Policies for Lambda execution of RDSDataService API, CloudWatch Logs, and read only from AWS Secrets to get database master credentials.
Get Started with CloudFormation Follow the two (2) steps as outlined in this GitHub repo for deploying the resources in your AWS account.
The first step will deploy the resources via a CloudFormation Stack, and the second step walks through enabling the Data APIfor the created Aurora Serverless Database.
Step 1: Deploy CloudFormation Stack via GitHub repo here.
Step 2: Enable Data API following GitHub Step 2 instructions here.
Option #3: AWS CLI or SDK — Scripted The AWS CLI or AWS SDK can be used to create your resources with just a few lines. Here’s a full AWS CLI statement for creating a new Aurora Serverless Cluster:
Once the Aurora Serverless Cluster has been created, follow Step 2: Enable Data API from the previous section and you now have an Aurora Serverless database with Data API enabled.
Four Ways to Connect to Your Database Using the Data API
In the Connect to Database window, select your cluster, provide your master user, master password, and database name (optional).
Select Connect to database.
Note: When you provide the cluster credentials the first time, the service will create an AWS Secret automatically for you, then each subsequent access to this cluster via the Query Editor, the service will use this AWS Secret to pull in the master user credentials for this cluster.
Once in the Query Editor, select the Clear button and type:
use MarketPlace;
select * from Customers;
Select Run.
Here’s the result if you have a Customers table with data:
SQL statement in RDS Console — Query Editor
TIP: The Query Editor is a nice tool to have when you need to verify the contents of the table or just perform some quick SQL statements.
Solution #2: AWS CLI Modify the provided AWS CLI script with your own cluster arn, secrets arn, database name, and the SQS select statement of your choice.
Solution #3: AWS Lambda function Once you deploy an Aurora Serverless Data API enabled database, you can easily connect and execute any SQL statement against the database with a simple AWS Lambda function using the RDSDataService API.
This function does not need to be inside an Amazon VPC and it doesn’t need to have any MySQL drivers or worry about connection pooling. Just make SQL statements as HTTP request and Aurora Serverless takes care of the rest!
Get Started Using a Lambda function to connect via Data API:
You can use the deploy Option #2 — CloudFormation above that provisions a database, a Lambda function, and fills out the environment variables to get you started OR… you can copy this code and deploy to Lambda directly.
If you do this manually, you’ll need to fill-in the Lambda environment variables to match your Aurora Serverless environment like Cluster Arn, DB name, and AWS Secrets arn.
Here’s the entire code to make SQL statement against your Aurora Serverless Data API enabled database. Again, make sure to provide the environment variables and then pass in something like:
{ “sqlStatement”: “SELECT * FROM ” }
The Lambda function will then make a connection to your database using the master credentials pulled from AWS Secrets and return a JSON response. It’s that easy!
Note: As of April 25, 2019 the Data API (beta) is only available to US-EAST-1 region serverless databases. Also, the function uses the RDSDataService API that is not currently available (beta) in the default Node 8 engine for Lambda, so you’ll need to build and deploy the function to Lambda as a zip file.
Solution #4: AWS AppSync When building a GraphQL API via AWS AppSync, you now have direct access to your serverless Data API enabled database as a datasource, and… AppSync will generate a GraphQL schema for you, based on the existing database table design!
For this solution, we’ll use the new add-graphql-datasourcepluginfor the AWS Amplify CLI that automatically takes your serverless database table(s) and creates/updates a GraphQL schema, generates the appropriate mutations, queries, and subscriptions, and sets your database as a GraphQL DataSource to an existing GraphQL API. Yes, please.
Before we use the plugin, we want to make sure our Aurora serverless database has at least one table. If a database doesn’t exist, let’s create those now by using the RDS Query Editor in the RDS Console and then we’ll switch over to the Amplify CLI + add-graphql-datasource plugin.
Here, we are going to use the RDS Query Editor to create a database and sample table to get us started. If you already have a database and table from previous steps, move onto the [Amplify CLI — Installing the CLI] section below.
In the Connect to Database window, select your cluster, provide your master user, master password, and database name (optional).
Select Connect to database.
In the query editor window, run the following commands:
Create database ‘MarketPlace’ if you haven’t already.
CREATE DATABASE MarketPlace;
Create a new Customers table.
USE MarketPlace; CREATE TABLE Customers ( id int(11) NOT NULL PRIMARY KEY, name varchar(50) NOT NULL, phone varchar(50) NOT NULL, email varchar(50) NOT NULL );
Now that we have a database and a Customers table, we can now use the add-graphql-datasource plugin to generate a GraphQL schema, add the database as a datasource, and add mutations, queries, and subscriptions based on the [Customers] table.
Here’s how the add-graphql-datasource plugin works:
Amplify CLI — Installing the CLI If you haven’t installed the AWS Amplify CLI before, here’s a quick shortcut. If you have the AWS CLI installed, the Amplify CLI will utilize those credentials and therefore amplify configure is not necessary.
Amplify CLI — Init Launch Mac Terminal in the root of your iOS project folder. Now, we’ll initialize our AWS backend project using the following Amplify command.
$ amplify init
You will be guided through the process of setting up the project.
Amplify CLI — Add API
$ amplify add api
Amplify CLI — Add GraphQL DataSource
$ amplify api add-graphql-datasource
Now, let’s create a new customer in the Customers table using the Queries tool in the AppSync Console.
AppSync Queries — in action
We can then query the Customers table from the Query Editor in the RDS Console to double check.
Amazon RDS Query Editor — Aurora Serverless Data API SQL Statement
Now that we have the schema, mutations, queries, subscriptions, and our serverless Aurora database as a datasource for our GraphQL API, we can start using a mobile or web client with the generated code to interact with this structured data!
Conclusion
Although the Data API and the RDSDataService API are currently in beta, there’s so much buzz and potential here for modern application developers.
The Data API allows any developer to take advantage of structured collections of data by having more control over the database, less management, no drivers or connection pools, and executing SQL statements as HTTP requests is a game changer.
I’ll update this article as this service feature progresses.
A general critique of Cloud Run relative to FaaS and managed API services — and how this is different from AWS Lambda
This week, at Google Cloud Next, GCP announced an interesting new service: Cloud Run. My thoughts about the new Cloud Run service are a bit more complicated than this Twitter thread, so I’ve expanded on them in this blog and welcome your comments.
Don’t focus too much on the packaging format. Container images are an industry standard for packing bits in a tarball vs Lambda’s use of zip files and layers. https://t.co/LdNCUZixw1
In this article, I’ll compare Google’s Cloud Run with AWS Lambda and API Gateway because I am most familiar with those services and provider. But my thoughts below are a general critique of Cloud Run relative to FaaS including Google Cloud Functions and managed API services in general — regardless of the provider.
What is Cloud Run?
Google’s Cloud Run allows you to hand over a container image with a web server inside, and specify some combination of memory/CPU resources and allowed concurrency. The logic inside your container must be stateless.
Cloud Run then takes care of creating an HTTP endpoint, receiving requests and routing them to containers, and making sure enough containers are running to handle the volume of requests. While your containers are handling requests, you are billed in 100 ms increments.
How is this different than AWS Lambda?
This sounds a lot like Lambda. How is it different? You are handling multiple requests within a single container. At a fundamental level, Cloud Run is just serving as a very fancy load balancer.
All of the web logic is inside your container; auth*, validation, error handling, the lot of it. But instead of just measuring resource utilization or other metrics that are a proxy for load, Cloud Run understands requests and uses that directly as a measure of load to know how to scale and route.
Note: if you’re using GCP IAM and you’re running on the managed version of Cloud Run, the service can do the auth for you. There’s no custom authorizers, nor do I expect there to be in the future, because why not just put it in your web server?
In Lambda, while you can set up API Gateway to do no validation or auth and pass everything through to your Lambda, you’d be missing out on a wealth of managed features that perform these functions for you.
What’s the Good?
So what’s good about Cloud Run?
It’s going to make it very, very simple for people who are running containerized, stateless web servers today to get serverless benefits.
Better scaling and fine-grained billing.
It’s also dead simple to test locally, because inside your container is a fully-featured web server doing all the work.
What’s the Bad?
So what’s bad about Cloud Run? Inside your container is a fully-featured web server doing all the work!
The point of serverless is to focus on business value, and the best way to do that is to use managed services for everything you can — ideally only resorting to custom code for your business logic.
If we try to compare Cloud Run to what’s possible inside a Lambda function execution environment, we’re missing the point.
The point is that the code you put inside Lambda, the code that you are liable for, can be smaller and more focused because so much of the logic can be moved into the services themselves.
The FaaS Model: Handling each request in isolation
API Gateway allows you to use custom authentication. That means that your code, in a Lambda that does nothing else, can reject the request.
You don’t have to even think about that request touching your downstream web handling code.
You don’t have to pay for invocation of the request-handler Lambda.
You aren’t paying for evaluating auth on every request since that custom authentication response is cached by API Gateway.
API Gateway allows you to perform schema validation on incoming requests. If the request fails validation, your Lambda doesn’t get invoked. Your code doesn’t have to worry about malformed requests.
Note: The API Gateway model validation is sadly a little more complicated than I’ve described above. Expect a post from myself and Richard Boyd on this topic in the near future.
The FaaS model is that each request is handled in isolation. Some people complain about this. I’ve even heard someone claim that AWS is pushing Lambda because users’ inability to optimize resource usage across requests is lucrative for them — which is about the most outlandish conspiracy theory I’ve heard this side of flat-earthers.
But the slightly less efficient usage model comes with benefits: you never have to worry about cross-talk effects. In Lambda, I don’t have to think about whether one request might have an impact on another. Everything’s isolated. This makes it easy to reason about, and removes one more thing I need to think about in the development process.
Security is hard, and the ability to scope your code’s involvement with it as small as possible is a huge win. Beyond the security implications, it’s also fewer moving parts that are your responsibility.
Cloud Run is also not the same as Lambda’s custom runtimes. Beyond the fact that custom runtimes should be a last resort, they don’t require running a server. Instead, you only need an HTTP client, which makes it more clear that what your code is doing is not acting as a tiny web server.
Cloud Run is not FaaS
All this is to say that Cloud Run should not be seen as equivalent, or even analogous, to pure FaaS — Cloud Run fundamentally involves significantly more code ownership. Cloud Run is still a valid rung on the serverless ladder, but there are many more above this service.
And that gets to my biggest concern. Cloud Run, and GCP in general, are providing people with a system that is going to make them complacent with traditional architecture, and not push them to gain the immense benefits of shifting (however slowly) to service-full architecture that offloads as many aspects of an application as possible to fully managed services.
Google’s strategy is to push Kubernetes as the solution to cloud architecture. And for good reason: Kubernetes is really good at solving people’s pain points while staying within the familiar architecture paradigm. And Google is doing a great job creating a Kubernetes layer on top of every possible base infrastructure.
But Kubernetes keeps us running servers. It removes the infrastructure notion of server, but encourages us to keep running application servers, like the ones inside Cloud Run containers.
Google’s ability to put Kubernetes on-prem is going to satisfy developers, and this will potentially come at the cost of delaying organizational moves to the public cloud. The difference from an application development perspective will be less apparent and will hide the higher total cost of ownership for being on-prem.
While Cloud Run is going to enable better usage of existing web server infrastructure, it’s also going to provide a safety blanket for developers intimidated by the paradigm shift of FaaS and service-full architecture. This will further delay the shift to the more value-oriented approach to development.
Add a custom domain to an Amplify Console deployment in just a couple of minutes — let’s take a look at how this works!
What is the Amplify Console?
The Amplify console offers hosting for full stack serverless web apps with continuous Git-based deployment. You connect your Github repo, click deploy, and the application is deployed to a live URL.
Built-in atomic deployments eliminate maintenance windows by ensuring that the web app is only updated when the entire deployment has finished.
If you are launching a project with an Amplify backend, the console will also give you the option of deploying and maintaining the Amplify project.
Adding a custom domain
After you’ve deployed an application, the next step is deploying your app to a custom domain purchased through domain registrars such as GoDaddy or Google Domains.
When you initially deploy your web app with the Amplify Console, it is hosted at a location similar to this:
https://branch-name.d1m7bkiki6tdw1.amplifyapp.com
When you use a custom domain, users will be able to easily access your app that is hosted from a vanity URL, such as the following:
https://www.myawesomedomain.com
Let’s learn how to do this!
Launching the app in the Amplify Console
If you already have an application launched in the Amplify Console, you can skip this step and go directly the next step — adding the custom domain.
There is also a ready-made Gatsby Blog to get started quickly. Just click here and then jump ahead to the next step to adding the custom domain.
To deploy an app already in your GitHub account, let’s launch a new application in the Amplify Console. The first step is to direct your browser to https://console.aws.amazon.com/amplify and click GET STARTED under the Deploy section.
Next, connect the Git repository you’d like to launch and select the branch, then click Next. Accept the default build settings, then click Save and deploy.
Now your application is launched and we can move on to setting it up in a custom domain.
Adding the custom domain
In the AWS dashboard, go to Route53 & click on Hosted Zones. Choose Create Hosted Zone. From there, enter your domain name & click Create.
ProTip: Be sure to enter your domain name as is, without www. E.g. myawesomedomain.com
Now in Route53 dashboard you should be given 4 nameservers.
In your hosting account (GoDaddy, Google Domains, etc..), set these custom nameservers in your DNS setting for the domain you’re using.
These nameservers should look something like ns-1355.awsdns-41.org, ns-1625.awsdns-11.co.uk, etc…
Next, in the Amplify Console, click Domain Management in the left menu. Next, click the Add Domain button.
Here, the dropdown menu should show you the domain you have in Route53. Choose this domain & click Configure domain.
This should deploy the app to your domain (this will take between 5–20 minutes). The last thing is to set up redirects. Click on Rewrites & redirects.
That’s it! Once the DNS propagates, you should see your domain live at the URL that you have set up in the above steps.
My Name is Nader Dabit. I am a Developer Advocate at Amazon Web Services working with projects like AWS AppSync and AWS Amplify. I specialize in cross-platform & cloud-enabled application development.
Searching for the nearest or farthest location is easy thanks to DynamoDB and a small geocoding NPM package
Many datasets now include geospatial information — especially if you are working with mobile apps or Google Maps. Geospatial data is a fancy way of describing items containing a latitude and longitude, but handling queries based on the nearest, furthest or within a certain distance has some complexity.
Why is this difficult? Imagine you have 10 retail stores and a customer wants to know the nearest location. The obvious way is to find the location of the customer and then calculate the distance between that location and each of the stores. What’s the problem?
Well, there are two problems —the first is that the previous calculation assumes the two points are on a flat surface and the world isn’t flat (really, it isn’t). You have to factor in the roundness of the rock we live on, which brings us to the Haversine formula.
While the first problem will lead to calculation errors, the second problem is much more problematic. As the number of retail stores increases, the speed of your search slows down — the dreaded O(n) — because you must compare the user’s location against every store. And what if the user is moving so the original location updates frequently?
If you broaden the problem to “find the nearest coffee shop”, our first solution would require us to compare the user’s location to every coffee shop on the planet. So we need a much better way of solving the question, and preferably one that can manage global scale. Basically, something like the way Google Maps does it.
The Power of Geohash
Geohashes are a little complicated — but the key thing to know is that it divides the world into a number of squares in a grid. By doing this, we can eliminate most of the data upfront and only focus on the square our potential targets are in. It’s a pretty elegant solution.
The hash is up to 12 characters in length — the longer the hash, the smaller the square it references.
The first character of the hash identifies one of 32 cells in the grid, roughly 5000km x 5000km on the planet.
The second character identifies one of the 32 squares in the first cell, so the resolution of the two characters combined is now 1250km x 1250km.
This continues until the twelfth character, which can identify an area as small as just a couple of square inches on Earth.
If you use a hash that is too long — in other words, the squares are too small — you will end up searching more squares for the results. If you use a hash that’s too short, you will end up returning too many items in the square and your search won’t be very fast. So you need to know the resolution of your search before selecting the length of the hash.
That might sound difficult — but it’s easier than you’d think. For our example of finding retail stores and coffee shops, it’s not necessary to have a 2-inch grid of the planet. We can resolve 1–10 kilometer searches with 5–7 characters.
The Geo Library for Amazon DynamoDB
There’s an AWS blog post from 2013 that details the Geo Library but it’s a Java-specific solution that needs a server. More recently there’s an NPM package that ports this work across to Node and can be used within a serverless architecture.
To test this out, I found a list of all the Starbucks locations in the US and then discovered a link where someone had turned this into JSON. Following the documentation in the NPM package, I cobbled together this script to configure the DynamoDB table and load it with data.
Let me walk you through the highlights.
First, let’s create the table which spins up a new DynamoDB table with the partitionKey and indexes needed to make the search work.
const ddbGeo = require('dynamodb-geo')
const config = new ddbGeo.GeoDataManagerConfiguration(ddb, 'yourTableName') config.hashKeyLength = 5 const myGeoTableManager = new ddbGeo.GeoDataManager(config)
This loads the Starbucks locations from the json file, creates an array of items to insert into the tables, and uploads into DynamoDB in batches of 25 items. There is a delay introduced between each batch to slow down the insertion process, and reduce the burn on the Write Capacity Units (WCUs).
Once uploaded, you can see all the locations together with their geohash values and hash keys, which were calculated by the library automatically:
With the data loaded, querying is simple — we just supply the latitude and longitude of the search location, together with the radius of the search area:
I built a simple front-end that uses the Starbucks data list to show the nearest Starbucks within 1 kilometer — it defaults to New York City since this looks like the most densely packed location for their stores.
If you click around the map, it fetches the list from DynamoDB based upon the new center of the map:
Overall, the advantage of using DynamoDB for this solution is the extremely steady performance under load. And with on-demand capacity now available for DynamoDB, it can handle spiky traffic without you needing to manage provisioned throughput.
The 90% figure is plucked out of thin air — it’s just me trying to pick a really big number without accurately calculating a percentage
What is Red Nose Day?
Since its launch in 1988, Red Nose Day has become something of a British institution. It’s the day when people across the land can get together and raise money at home, school and work to support vulnerable people and communities in the UK and internationally.
Now that Red Nose Day 2019is over, I want to give you a bit of a run down on my learnings, the tooling that we used and the path taken, on what turned out to be nearly a full cloud migration from a containerised/EC2 ecosystem and into the world of serverless.
Upon starting at Comic Relief in April 2017, I joined a well-developed team that were already working in the cloud-native microservice world, with some legacy products operating on a fleet of EC2 servers (such as the website) and the rest either running on Pivotal Web Services (Cloud Foundry) or outsourced (Donation Platform).
Comic Relief was generally using Concourse CI (awesome tool) to deploy to Cloud Foundry or was using Jenkins for its legacy infrastructure. We frequently used Travis CI to run unit tests and do some code style tests, but have now moved over to CircleCI — thanks to a thorough comparison by Carlos Jimenez.
Nearly all of the code was written in PHP and was either Symfony or Slim Framework. I had quite a bit of experience of NodeJS at my previous role at Zodiac Media, was a big fan of running the same language on the front as on the back and not having to context switch when swapping between the two.
Pretty much the first job that I was tasked with on arrival was to bring the contact form, which was part of the comicrelief.com Drupal site, into the serverless world.
The project was my first introduction into Lambda and Serverless Framework and also my first lesson as to what works well in serverless. I created a serverless backend in NodeJS that accepted a POST request and then forwarded that on to RabbitMQ.
My colleague Andy Phipps (Senior Frontender) created a frontend in React which we dropped into S3 and created a CloudFront distribution in front of it. We then built a Concourse CI pipeline that ran the SLS DEPLOY command and ran some necessary mocha tests against the staging deploy and then deployed to production and did much the same for the frontend.
It was a liberating experience and became the foundation for everything that we would do after.
The contact service backend
The next thing we wanted to do was to send an email from the contact service to our email service provider. I created another Serverless service that accepted messages either from RabbitMQ or via HTTP that again just took the message and formatted for our email service provider and forwarded on.
We then realized that we were using the same mailer code in our Symfony fundraising paying-in application, so we stripped out the code from payin and pointed it to our new mailer service. At this point, I started to realize that the majority of web development was some mix of presenting data and handling forms.
The mailer pipeline
We then halted active development and steamed intoSport Relief 2018; this allowed us to test our assumptions towards Serverless and gain some real-world experience under heavy load.
The revelations were as follows:
Cloudwatch is a pain to debug quickly, but a good final source of truth.
Self-hosted RabbitMQ was OK but took a lot to manage for a small team. Why weren’t we using SQS?
We were duplicating a lot of the boilerplate code across the two projects that we had created.
Our API’s needed to be documented automatically as part of our pipelines and in code.
Serverless Framework was the future.
We then went into a significant business restructure, which meant that a lot of our web-ops team ended up leaving. The result was a remit to simplify the infrastructure that we were using so that a smaller group could manage it, bringing ownership, responsibility and control to the development team. The approach was championed by our Engineering lead Peter Vanhee, without that, where we are now would never have happened.
The next obvious target was our Gift Aid application; the most basic description of it is a form that users submit their details so that we can claim gift aid on SMS submissions. The traffic hitting this application is generally very spikey of the back of a call to action on a BBC broadcast channel, ramping up from 0 to 10’s of thousands of requests in a matter of seconds.
We traditionally had a vast fleet of application and varnish servers to back this (150+ servers on EC2). As one of the most significant revenue sources, this gets a lot of traffic in the 5 hours that we are on mainstream TV and also in the lead up to it, so there was very little wiggle room to get it wrong.
At this point, we diverged from RabbitMQ and started deploying SQS queues from serverless.yml. My colleague Heleen Mol built a React app using create-react-app, and react-router hosted again on S3 with CloudFront in-front of it, this was and is the foundation of every public facing application we make, it can handle copious levels of load and takes zero maintenance.
The Gift Aid Site
At this point, it was apparent that we needed a good way to document our API’s alongside the code. We had previously been using swagger on our giving pages. However, it seemed a bit of a pain to set up, and I wanted something static that could be chucked into S3 and forgotten. We settled on apiDOC as it looked like it would be quick to integrate and was targeted at RESTful JSON API’s.
Comment Block from Payments APIapiDoc documentation from Payments API
The primary donation system was previously outsourced to a company called Armakuni, who had built an ultra-resilient multi-cloud architecture across AWS and Google Cloud Platform.
With our clear remit to consolidate our stack and the successes that we had already had in building a donation system for our Sport Relief app in Symfony that was a fork of our Paying In application,
It really seemed like the next logical step to bring the donation system in-house. This allowed us to share components and styling from our Storybook and Pattern Lab across our products, severely reducing the amount of duplication.
Our Footer in Storybook
It should be noted that at this time we already had a payment service layer that had been built in previous years in Slim Framework which ran the Sport Relief app donation journey, our giving pages and shop.
As Peter (engineering lead) was heading away on paternity leave, the suggestion arose that if there was any time after moving over the giftaid backend to Serverless that I could create a proof of concept for the donation system in Serverless Framework. We agreed that as long I had my main tasks covered, then I would be able to give it a go with any time I had left. I then went about smashing out all of my tasks quicker than I was used to, to get onto the fun stuff ASAP!
After talking to the super knowledgable guys at Armakuni after the wrap up from Sport Relief, it was clear that we needed to recreate the highly redundant and resilient architecture that Armakuni had created, but in a serverless world.
Users would trigger deltas as they passed through the donation steps on the platform, these would go into an SQS queue, and then an SQS fan out on the backend would read the number of messages in the queue and trigger enough lambda’s to consume the message, but most importantly not overwhelm the backend services/database.
The API would load balance the payment service providers (Stripe, Worldpay, Braintree & Paypal), allowing us to gain redundancy and reach the required 150 donations per second that would safely get us through the night of TV (it can handle much more than this).
I initially put in AWS parameter store to store payment service provider configuration, this was free and therefore very attractive in a serverless world, but proved woefully incapable under load and was swapped out for storing configuration in S3.
I then created a basic frontend that would serve up the payment service provider frontend based on which provider the backend. Imported all of the styles over from the Comic Relief Pattern Lab and was good to demo it to Peter and the team on his return.
The donation platform
Upon Peter returning, we went through the system, discussed it’s viability and did some necessary load tests using Serverless Artillery, concluding that we could do what we thought we couldn’t!
A business case was put together by Peterand ourProduct Lead Caroline Rennie, and away we went. At this point, Heleen Mol and Sidney Barrahcame on board and added meat to the bones, getting the system ready to go live and the ever impending night of TV.
Serverless Artillery load testing reporting to InfluxDB and viewed in Grafana
Due to the nature of Red Nose Day, you don’t get many chances to test the system under peak load. We were struggling to get observability of what was going on in our functions using Cloudwatch.
At this point, Peter recommended that we try a tool that he had come across, which was IOPipe. IOPipe gave us unbelievable observability over our functions and how a user is interacting with them; it changed how we used Serverlessand increased our confidence levels substantially.
IOPipe function overview
At this point we also integrated Sentry, which alongside IOPipe gave us the killer one-two punch of being able to get a 360 view of errors within our system, allowing us to quantify bugs for our QA team (lead by Krupa Pammi) and trace the activity that caused them quickly and efficiently. I can’t think of a time where I have been able to have such an overview of everything going wrong, pretty scary, but excellent.
A Sentry bug from the Donation Platform
The next big part of the puzzle was the decision that we were copying way too much code between our Serverless projects. I had a look at Middy based on a recommendation from Peter, but at the time there wasn’t a vast amount of plugins for it, so decided to spin out our own lambda wrapper rather than having to learn and make plugins for a new framework and possibly run into Middy’s limitations (probably none).
I am still not sure yet how bad of an idea this was, however, it seems to work at scale, is easy to develop with and simple to onboard new developers, which is enough for it to stay for the time being.
Lambda wrapper encompasses all of the code to handle API Gateway requests, connect and send messages to SQS queues and a load of other common functionality. Lambda wrapper resulted in a massive code reduction across all of our Serverless projects. It also meant that the integration of Sentry & IOPipe was common and simple across all of our projects.
The donation pipeline in ConcourseCI
To add extra redundancy to the project, we introduced an additional region and created a traffic routing policy based on a health check from a status endpoint. We figured the chance of losing two geographically separate AWS regions was very low.
We also backed up all deltas to S3 on a retention policy of 5 days, to ensure that we could replay all deltas in the event of an SQS or RDS failure. We added timing code to all outbound dependencies using IOPipe and also created a dependency management system so that we could quickly pull out dependencies (such as Google Tag Manager or reCAPTCHA) from external providers at speed.
IOPipe tracing
Based on a suggestion from AWS. We also added a regional AWS Web Application Firewall (WAF) to all of our endpoints, this introduced some basic protections, including stuff we already had covered, but higher up the chain, before API Gateway was even touched.
Another piece of the puzzle was to get decent insights into our delta publishes and processing, this gives us another way to get a good overview of what is happening with our system. We used InfluxDB to do this and consider it as an optional dependency of our system.
It was important for us to understand what our applications critical dependencies were, thus forming our application health check status and whether we would fall over to our backup region. InfluxDB is fantastic, however, is self-hosted. When AWS Timestream comes along, this will be out the door.
Delta processing from InfluxDB, viewed in Grafana
So the night of TV came and went on the 15th of March and the system performed nearly exactly as expected. The one unexpected, but now apparent weak point was the amount of reporting that we were trying to pull from the RDS read replica using Grafanaand our live income reporting, we lowered our reporting requirements and were back on track within no time.
We originally used RDSso that we could achieve compatibility with our legacy payment service layer, in the future we will probably replace this with something more Serverless. Relying on AWS Timestream for more real-time analysis (when it arrives).
API Gateway traffic spike for March 15th
So to sum up this epic and overly long rundown of the journey to 90% Serverless:
Try to get everything Serverless if you can, our highest monetary cost is RDS. It’s still nice to be able to run the SQL queries that we know and love, Athena and S3 are probably a solid replacement.
Try to ingest data and work on it away from user interaction. You can provide the user with an endpoint where they can check on the status of processing. Manage as much state as you can with your frontend. This will hopefully give you redundancy and protection as a default.
Lambda allows you to load test at a significant scale, do it often, make it part of your deployment/feature release strategy. Serverless pushes the load down the line and has a habit of finding weak points in your chain, so make sure you know where your weak points are going to be. Serverless Artillery is the way forward, do better than us and do it as part of your pipeline to production for the win!
Continuously deploy, deploy on a Friday at 5 pm, don’t let fear stop you, create the tooling and automation tests to allow you not to worry. We use Nightwatch, Cypress and Mocha to significant effect. It should be noted that you need decent logging and a fast way to rollback code to be able to do this in a manageable way (Concourse CI).
Serverless infrastructure cost is dependent on usage, so why not deploy your entire infrastructure on a pull request level and run tests in the PR against it. We do this, and it means that developers can be sure that before their code is merged, it works in real life and on our real-world infrastructure.
Don’t host anything if you don’t have to, everything as a service. I am physically averse to calls about infrastructure outages at any time of the day. Also, go multi-region if you can, serverless makes this a doddle, and it reduces the voice of the crowd who will remind you that S3 took out US-EAST-1 in September 2017.
Pick a piece of your architecture, migrate it to Serverless, get comfortable with it, rinse and repeat.
The best system is the one that allows me to be in the pub after 17:30 or be at home with my family not checking my laptop, Serverless for the backend and a React application stored in S3 for the frontend gives you this.
Concourse CI is probably one of the most expensive pieces of infrastructure that we are running, it doesn’t fit in with our fully Serverless headspace. Replacing it would be great. However, the power and flexibility it gives us to deploy reliably and continuously are unmatched. Sometimes in life, you can’t be all one thing, in this case, Serverless. We use Concourse UP to simplify it’s deployment and management, meaning that we don’t have to mess around with bosh.
Don’t try to optimize/abstract your services too early when it comes to Serverless. I remember at my first job where all the servers had names, they were cared for and loved and were then quickly replaced with EC2 when AWS entered the fray. Serverless brings the same down to your code and services; they should perform a function, be replaced with ease and doted over just the right amount. Compose small but relevant services and be ready for the day where you type sls remove on that much-loved service!
The biggest lesson for me is that in my day job, I exist to solve business issues. I think sometimes as technologists we forget this. Serverlessis the fastest way to decouple oneself from rubbish problems, move up the stack and move on to the next issue.
Be sure to watch this presentation by our Engineering Lead Peter Vanhee talking through the current architecture at Serverless Computing London, as well as this presentation featuring our Product Lead Caroline Rennie around the previous donations platform and the problem space.
The point is focus — that is the why of serverless
Functions are not the point If you go serverless because you love Lambda, you’re doing it for the wrong reason. If you go serverless because you love FaaS in general, you’re doing it for the wrong reason. Functions are not the point.
Sure, I love Lambda — but that’s not why I advocate for serverless.
Don’t get me wrong, functions are great. They let you scale transparently, you don’t have to manage the runtime, and they fit naturally with event-driven architectures. These are all fantastic, useful properties.
But functions should end up being a small part of your overall solution. You should use functions as the glue, containing your business logic, between managed services that are providing the heavy lifting that forms the majority of your application.
Managed services are not the point We are fortunate to have such a wide range of managed services for so many different parts of our applications. Databases, identity and access management (so glad I don’t have to own that myself!), analytics, machine learning, content delivery, message queues for all sorts of different patterns.
Managed services provide the functionality you need with less hassle. You’re not patching the servers they run on. You’re not making sure the autoscaling is correctly providing the required throughput without a lot of idle capacity. Managed services lowers your operational burden significantly.
Managed services are great — but … they aren’t the point.
Ops is not the point It’s great to know that you can apply fewer operations resources to keep your applications healthy. It is especially great that the resources you need scales mostly with the number of features you ship — not with traffic volume.
Reduced operations is more efficient — but … it’s not the point.
Cost is not the point Ok, sometimes all the business wants you to do is reduce cost — and that’s all you care about. And serverless will help you do that. But in general, your cloud bill is not the point.
Your cloud bill is only one component of the total cost of your cloud applications. First of all, there’s the operations salaries— and that cost is lower if you have fewer ops resources. There’s also your development costs.
There are a lot of cost advantages — but … none of these are the point.
Code is not the point Not only is code not the point, code is a liability. Code can at best do exactly what you intend it to. Bugs detract from this. You can only lose points through more coding. The more code you own, the more opportunities exist to depart from your intended value. Understanding this is a cultural shift.
Technology has been hard for a long time. It’s taken clever people to create value through technology. So developers started to believe that cleverness was inherent and good. We’ve spent so long crafting Swiss watches that we’ve failed to recognize the advent of the quartz Casio — and impugn the evolution as lacking in elegance.
Instead of applying our cleverness to solving technology problems, we really need to be understanding and solving business problems. And when you have to code — you are solving technology problems.
Technology is not the point The reason that we’re doing this, any of this, is in service of some business goal. The business value that your organization is trying to create is the point.
Now, sometimes, what you’re selling is literally technology. But even if your product is technology, that may not be the value of what you’re selling.
There’s an old adage that people don’t buy drills, they buy holes. When you need a hole in your wall, you don’t care how fancy the drill is — you care how well it creates that hole you need.
At iRobot, we don’t sell robots. We don’t even sell vacuums. We sell clean homes. Roomba gives you time back in your day to focus on the things that matter to you. So if technology isn’t the point, what are we here for?
The point is focus Serverless is a way to focus on business value.
How do functions help you deliver value? They let you focus on writing business logic, not coding supporting infrastructure for your business logic.
Managed services let you focus on writing your functions. Having less operations resources frees up people and money to be applied to creating new value for your customers.
Observability gives you tools to address MTBF and MTTR, both of which are a measure of how often your customers aren’t getting value. Spending less on the cloud means you can spend that money more directly in support of creating value.
Focus is the Why of Serverless
You should go serverless because you want to focus on creating value — and at your company you endeavor to apply technology toward the creation of business value.
Going back to cost, Lyft’s AWS bill, $100 million per year, has been in the news recently. Many people chimed in to say they could do it cheaper — they couldn’t, but that’s beside the point.
Would Lyft’s bill be lower if they switched to Lambda and managed services for everything they possibly could? Probably. But what would that do as they spent time rearchitecting? They would lose focus.
The company is at a stage in its journey where growth is more important than cost control. Eventually, that might change. Public companies are responsible to their shareholders, and so cost reduction can deliver value to them. But for Lyft right now, delivering value to their customers means executing with their current applications and processes. They are making the serverless choice.
What I’m telling you is that serverless has never been about the technology we call serverless. So what does the technology that we call serverless have to do with it?
Serverless is a consequence of a focus on business value Technology is a consequence of how you’re trying to deliver value. Dev and ops teams have traditionally been separated with the notion that they have different focuses. But we’re seeing that trend changing.
The traditional model put the focus on technology — dev tech vs ops tech. But we’re seeing people realize that the focus should be on the value — the feature being delivered, including both how it’s built and how it’s run.
When we take this notion of focusing on business value, and run it to its logical conclusion, we get serverless.
When you want to focus on delivering value, you want to write functions. When your function needs state, you want a database. To get it from someone else, you use DBaaS — and you choose between your options based on how well it lets you focus.
And when you’re choosing managed services, some of them may even be user-facing. If you can use social login instead of owning your own accounts, that’s one less thing you have to manage, and one less piece of the user experience table stakes you need to own.
Now, for everything you are outsourcing, you are still accountable. Your users don’t care if their bad experience is caused by a third party you’re using, it’s still your problem. You need to own outages to your users while accepting that you don’t fully control your destiny there. This is an uncomfortable place to be — but it’s worthwhile.
You can’t win points on these things — but you can lose points. This means that you need to know what “bad” looks like. That requires having enough knowledge about the outsourced pieces of your product and your technology to know that you’re delivering enough quality to your users.
Note that deep expertise in a focus area, and broad but thin knowledge of adjacent areas is exactly analogous to the T-shaped skills concept — applied to organizations and teams.
Serverless is a trait Serverless is a trait of companies. A company is serverless if it decides that it shouldn’t own technology that isn’t core to delivering its business value. Few companies are really totally serverless. But within a company, you can still have parts that are serverless.
If your team decides to focus only on the value it’s delivering, and delegate anything outside that either to another team, or ideally outside — then your team is going serverless. And you can’t always choose to use an outside technology — that’s fine, you can still make the best choice given the constraints.
And with a big enough organization, it can cease to matter. When Amazon.com uses Lambda, that’s fully serverless, even though it’s on-prem in some sense. But what if it’s just you?
What if you’re excited about serverless, but you feel completely alone at your company? What if you’re far removed from actual business value — if you’re patching servers for a team that serves a team that serves a team that creates user-facing content? I want to convince you that you can go serverless today, yourself, in any situation.
Serverless is a direction, not a destination I used to talk about serverless as a spectrum, because I knew there wasn’t a bright line separating serverless technology from non-serverless technology. I mean, there almost never is a bright line separating any given grouping of anything, so I was pretty safe in that assumption.
I talked about how something like Kinesis, where you need to manage shards, is serverless, but less serverless than SQS, where you don’t. How you don’t have to patch instances with RDS, but you do need to choose instance types and number. These technologies are all various shades of serverless.
But recently I’ve come to realize a problem with portraying serverless as a spectrum is that it doesn’t imply movement. Just because you’re using something designated serverless of a sort doesn’t mean you should feel comfortable that you’ve attained serverless — that it’s acceptable to keep using that and think you’ve checked the serverless box.
Climb the serverless ladder I’ve started to think of serverless as a ladder. You’re climbing to some nirvana where you get to deliver pure business value with no overhead. But every rung on the ladder is a valid serverless step.
If you move from on-prem to a public cloud, that’s a rung on the ladder. If you move from VMs to containers, that’s a rung on the ladder. If you move from no container orchestration, or custom orchestration, to Kubernetes, that’s a rung on the ladder. If you move from long-lived servers to self-hosted FaaS, that’s a rung on the ladder. But there’s always a rung above you, and you should always keep climbing.
Climbing the serverless ladder corresponds to using technology further to the right on this graph from Simon Wardley.
One thing the “ladder” doesn’t convey is that it’s not linear. Moving from VMs to containers to Kubernetes while staying on-prem are rungs on the ladder, but so is moving your VMs from on-prem to the cloud. There’s often not a definitive “better” in these cases.
I thought of the metaphor of many paths leading up a mountain, but one thing I like about the ladder is that it can be infinite. There isn’t an end state. I love Lambda, but I am always looking for better ways of delivering code that keep me more focused on value.
Serverless is a State of Mind
Serverless is about how you make decisions — not about your choices. Every decision is made with constraints. But if you know the right direction, even when you can’t move directly that way, you can take the choice that’s most closely aligned, and then you’re moving up another rung. So, how do you adopt this mindset? How do you make serverless choices?
Configuration is your friend I think many developers look down on configuration as “not real programming”. There’s an idolatry of coding today. We’ve been told that “software is eating the world”, and we’ve inaccurately translated that to “coding is eating the world”.
We’ve come to believe that developers are the only important people in an organization, and that our sense of productivity is the only thing that matters. We want to feel in the zone, and that’s what coding provides. Any obstacle to this must be bad for the business. We’ve lost any sense of whether being in the zone is actually producing value faster and better than an alternative route.
Remember: Days of programming can save hours of configuration Constraints are good. Removing options can help you focus. Obviously, not all constraints are good — but in general, the ability to do anything general comes at the cost of it taking longer to do one particular thing. Guard rails may chafe, but you’ll be faster than if you have to constantly watch the edge.
In this way, serverless is about minimalism. Removing distractions. Marie Kondo is big now, and the same advice applies. Find the components of your stack that don’t spark value.
Be afraid of the enormity of the enormity of the possible Possibilities carry with them hidden complexity. For any technology, one of my primarily evaluation metrics is how much capability it has beyond the task at hand. When there’s a lot of extra space, there’s unnecessary complexity to both deal with and learn.
People tout Kubernetes as a single tool to accomplish every cloud need — and it can! But if everything is possible, anything is possible. For a given task, Kubernetes can go wrong because you haven’t accounted for the ways it acts for situations unrelated that task.
On the other hand, when you look at serverless services, you may have to choose between a 80% solution from your main provider, or a 3rd party provider with a service that better fits your needs. But what are the operations needs for that new provider? What’s the auth like? Those are hidden complexities that you’ll pull in — and you’ll need to trade that off with feature differences.
Accept the discomfort of not owning your own destiny When you’re using a managed service, provider outages are stressful. There’s nothing you can do to fix their problem. There is no getting around it — this will always feel awful.
You’ll think, “if I was running my own Kafka cluster instead of using Kinesis, I could find the issue and fix it”. And that may be true, but you should remember two things:
That would be a distraction from creating business value.
You would almost certainly be worse at running it. You’d have more and worse incidents. It’s a service provider’s purpose in life to be good at it — and they have economies of scale you don’t.
Moving past the “I could always build it myself” attitude can be hard. Jared Short recently provided a brilliant set of guidelines for choosing technology.
My thinking on serverless these days in order of consideration. – If the platform has it, use it – If the market has it, buy it – If you can reconsider requirements, do it – If you have to build it, own it
In order, if you’re on a cloud platform, stay within the ecosystem when possible. You’re removing so many possibilities from the equation that way. But if you can’t get what you need on the platform, buy it from somewhere else.
If you can’t buy exactly what you need, can you rethink what you’re doing to fit what you can buy? This one is really important. It gets to the heart of time-to-market.
If you have something you think is valuable, you’ll want to ship it as soon as possible. But it’s better to ship something near to that faster, than to build the exact thing, You don’t know that it’s the right thing yet.
Waiting to build the exact right thing will not only take longer to ship, but your subsequent iterations will be slower — and maintenance of it will take resources that you could apply to shipping more things in the future. This applies even when the technology isn’t serverless: always ask if a tweak to your requirements would enable faster, better, or more focused delivery of value.
Finally, though, if you have to build it, own it. Find a way for it to be a differentiator. Now, this doesn’t mean everything you’ve built already you should turn into a differentiator. Look at only the things you can’t have bought as a service in a perfect world. Imagine what a completely serverless, greenfield implementation would look like, and find what needs to be built there.
Find your part of the business value So fundamentally, you want to find your part of the business value. What is your technology work in service of? Maybe you’re far removed from user-facing product. You may only be contributing a small slice. But it’s there, and you can find it — and focus on that value.
Start with the immediate value you’re providing to others in the organization, and focus on that. And then start to trace the value chain. Make sure all your decisions are oriented around the value you’re creating. Make serverless choices.
Hire the people who will automate themselves out of a job, then just keep giving them jobs.
I love this quote from Jessie Frazelle. You can turn it around; automate yourself out of a job, and keep demanding jobs.
Remember that you are not the tool. For any value that you’re creating — automate that creation. If you manage build servers, find ways to make them self-service, so what you’re delivering is not the builds per se, but the build tooling so teams can deliver the builds themselves.
TL;DR Serverless is a State of Mind
The point is not functions, managed services, operations, cost, code, or technology. The point is focus — that is the why of serverless.
Serverless is a consequence of a focus on business value. It is a trait. It is a direction, not a destination. Climb the never-ending serverless ladder.
Configuration is your friend. Days of programming can save hours of configuration. Be afraid of the enormity of the enormity of the possible. Accept the discomfort of not owning your own destiny
Find your part of the business value, and achieve a serverless state of mind.
Serverless is a State of Mind was originally published in A Cloud Guru on Medium, where people are continuing the conversation by highlighting and responding to this story.
Amazon is collecting a massive treasure trove of data from consumers invoking custom skills that’ll drive further disruption
The consumer data derived from the Alexa voice skills is a gold mine for startups like VoiceLabs
“There’s gold in them thar hills and there’s millions in it.” — M. F. Stephenson
The recently published 2017 Voice Report from Adam Marchick is a solid “state of the union on voice platforms” — and you’ll really appreciate the effort and intellectual rigor that VoiceLabs put into their analysis.
Here are my two takeaways from this report:
Alexa-like voice platforms are about to cross-the-chasm
Amazon is sitting on a gold mine of data
In the new space of voice analytics, VoiceLabs is a standout startup offering services that “provide accurate, real-time data to developers about how consumers use their Amazon Alexa skills and Google Assistant Actions.”
The emerging strategy to collect real-time data was referenced in a recent article which describes how analytics startups like VoiceLabs are Scrambling for Alexa Data from Amazon — and rightfully so.
The private questions that consumers “ask” voice platforms generates an extremely valuable data set. At the very least, the consumer data derived from the voice skills provides can be used to create highly accurate profiles of consumers for marketing purposes.
The Value of Voice Data Analytics
For now, Amazon is keeping the majority of the voice data to themselves. And since Amazon has quickly attracted a massive ecosystem to feed the accumulation of that data set, they have a strategic advantage over competing voice platforms — one they know exactly how to exploit.
The most obvious way for Amazon to leverage the voice analytics is feeding the data into their existing consumer models for targeted pricing and and promotions. For example, if someone asks “Alexa, what are the signs of pregnancy” — the customer should also expect to see diapers as an item on their suggested wish-list the next time they go shopping on Amazon.
But the consumer line of smart-speaker devices and the developers building custom skills are just a pawn in Amazon’s longer-term strategy. Amazon is playing chess and positioning Alexa as the Queen — while everyone else is getting played, or at best, playing checkers.
Amazon can lead a horse to water better than anyone in the industry
Woo the Developers, Then Woo the Enterprise
With over 5M units of Amazon’s smart-speaker devices sold in the past two years, there is an army of users interacting daily with custom-developed Alexa skills. As Amazon builds toward critical mass, it’s hard to bet against their domination of the voice platform — just as Amazon’s AWS has dominated the cloud industry since 2007.
Similar to how Amazon’s cloud strategy evolved from the developer community to the enterprise, AWS will follow a similar pattern from consumer to enterprise dominance with their Alexa voice platform.
When AWS introduced their first set of cloud services, they were geared almost exclusively toward the 180,000 members of their development community. Since 2007, AWS has grown exponentially as it’s strategically cycled iteratively through a series of innovate → leverage → commoditize (ILC) cycles. Each cycle create new services — enabling AWS to move up their value chain and step closer to the needs of enterprise customers.
And with the introduction of Lambda, it’s now possible for AWS to move services even faster through the ILC cycle since underlying functions supporting new services are essentially commoditized as they’re innovated.
After years of moving up the value chain with higher level abstraction of services, AWS is now clearly focused on features that woo the enterprises instead of the developers — such as Snowmobile and Managed Services Program. Nobody should be surprised when the exact same strategy plays out with Alexa.
Amazon ‘s experience with AWS will be a key differentiator in their dominance of voice platforms
This Isn’t Amazon’s First Rodeo
While Amazon harvests usage patterns and strategic insight from their voice platform, the flow through the Innovate-Leverage-Commoditize (ILC) cycle should seem very familiar to the early days of AWS.
Amazon’s Alexa innovative voice platform is first to market (i.e. AWS).
The initial Alexa platform has enough minimal viable features on the development portal to attract an ecosystem of developer (i.e. EC2, S3).
The development community and Alexa Dev Champions are enticed by compelling features, and free T-shirts, to leverage the platform and build custom skills (i.e. web sites).
AWS monitors their usage patterns and leverages the learnings to drive further innovation and commoditization (i.e. Jeff Barr daily blogs).
Rinse and repeat.
Every time an Alexa skill is invoked, Amazon learns — each utterance sharpening the machine learning and artificial intelligence capabilities.
The FUD is Real
With feature-rich IaaS and PasS offerings now available from AWS, migrating a corporations on-premise data centers into their cloud is not a matter of ‘if “ — but “when”. Everyone knows that customer’s don’t give a shit about your data centers.
In similar fashion, AWS will leverage the voice data analytics from consumer Alexa skills to fuel their longer-term strategy — enterprise call centers.
With voice services, their target just moves down the street from the data center — to corporate call centers. The recent AWS releases of the Polly, Lex, and Rekognition services services offer compelling features that are early indicators of a viable strategy.
In the coming years, expect to hear lots of familiar Fear, Uncertainty, and Doubt (FUD) derived from cloud adoption and reapplied to the call center industry as thought leaders declare all the reason why this won’t work — probably starting with security issues. This phase will be followed by nervous sales teams casting doubt on the viability of machine learning and artificial intelligence for call centers.
swardley offers a simple yet effective illustration of how FUD impacts enterprise adoption
Ultimately, we’ll know when the AWS strategy has succeed when incumbent vendors start proposing hybrid call-center solutions. The final sign of success is when all the engineers are arguing about the term callcenterless — and every article starts with “first and foremost, there are still call centers in a callcenterless architecture”.
Let’s just hope the companies we are patronizing have already migrated their call centers to AWS voice platforms at that point. Then we can all finally stop yelling “representative” to get actual service through voice channels — instead we can just ask Alexa.
The innovation in the tech industry paved a new way for infrastructures as a service — bringing a new paradigm shift from infrastructure and hardware ownership to a subscription and capacity on demand model. In minutes, you can now spin up several instances of virtual servers across multiple regions and dramatically reduce your time to market.
While this is still happening right in front of our eyes, a new era of serverless architecture is also rapidly unfolding in parallel taking away the worries of having to even manage servers. With serverless, you no longer even have to know the underlying operating system that is running your application.
Serverless offers developers the ability to quickly experiment with new ideas and fail fast while moving forward — allowing teams to focus on what matters (code) while the undifferentiated heavy lifting is managed by others.
If you are not satisfied with your current architecture or thinking of launching a new idea, here are three reasons serverless might be a good idea for you.
#1 Serverless reduces your time to market
Hiring developers is hard — and finding engineers with the right set of skills for your product needs is even harder. Not only do you have to find developers who can transform your product ideas into code, but also those who can manage the underlying infrastructure that supports your code.
With serverless architecture, your development team can concentrate on writing functions that are tied to your business value, and deploy them in minutes with security, logging and scalability.
#2 Serverless is cheaper
Yes, you read right! As ridiculous as it may sounds, running apps on a serverless environment is a way cheaper than you thought.
The startup phase is characterized by lots of experimentation, followed by continuous iterations designed to test and validate ideas. During these early stages, keeping your costs as low as possible helps to extend your runway.
There are lots of examples of teams migrating to serverless and significantly reducing their costs. No longer do teams have to bear the pain of managing and monitoring servers in development, test, and production environments — or paying for idle EC2 instances.
#3 Serverless scales with your idea
Startups can grow at a rapid pace — and sooner or later scaling becomes a pain point. The last thing a startup wants to get in the way of their growth is infrastructure constraints.
While some startups address the problem by throwing more resources at the underlying infrastructure, it usually requires some reengineering of the existing architecture to effectively scale — with significant costs.
The ability to scale is an area where serverless shines. With the proper architecture, a startup can rapidly scale as your business grows without massive investments in additional infrastructure.
How much does it cost every month to run 70 #lambda per second? $579. @acloudguru@samkroon #ServerlessConf
Like any new technology, there is always a cloud of skepticism around and more often you hear words like Serverless is not ready yet or it’s just another buzz word.
Why Serverless Architecture? was originally published in A Cloud Guru on Medium, where people are continuing the conversation by highlighting and responding to this story.
