Two-Way SMS with Amazon Pinpoint

Learn to implement two-way SMS messaging for a simple approach that results in higher levels of customer engagement

SMS, or text messaging, is the simplest way to reach your users outside of normal customer-facing web or mobile applications. Compared to other communication channels such as email and push notifications, text messaging results in higher engagement.

SMS messaging is extremely convenient — users don’t have to authenticate, download your app, or go to your website. They simply receive your message on their device. When it comes to customer acquisition and retention, it doesn’t get any easier than this.

In this article, I’ll explain what two-way SMS is and how you can quickly and easily start sending personalized, timely, and relevant text messages to your customers with Amazon Pinpoint. We’ll then implement a practical solution for setting up an SMS long code so you can start sending and receiving text messages.

What is two-way SMS?

SMS messages (better known as “text messages”) are typically sent person-to-person, but they are also sent business-to-person. Businesses typically send text messages programmatically, as they would otherwise need a lot of physical phones and dedicated employees manually sending text messages to each customer — which is simply not realistic or scalable.

When a business sends a text message to a customer, it tends to be a one-way SMS. In a one-way communication, the business will send a message in the form of an announcement or one-off notification that typically doesn’t require any further interaction. For example, a restaurant may text a patron when their reserved table is ready. Or, a doctor’s office might send a text to a patient reminding them of their upcoming appointment.

Businesses can also send two-way messages to customers when they want to receive a response that provides them with actionable next steps. For example, going back to the doctor appointment use case, the outbound message to the patient could include an option to respond “1 to ‘confirm’ or 2 to ‘cancel appointment’”. The patient’s text response would come back to the doctor’s office management app, and they can then take action based on the response. With two-way SMS, the possibilities are endless.

How does two-way SMS work?

To engage in programmatic two-way SMS conversations with your customers, you’ll first need to obtain a long code or short code from a messaging provider like Twilio, Nexmo, or Amazon Pinpoint.

Each provider will offer an API for sending and receiving text messages from your customers to your long or short code. These numbers are not associated with a mobile device and are intended as virtual addresses for your applications.

Long Code vs Short Code
A long code is a essentially 10-digit phone numbers used to send and receive SMS and are limited to 1 message per second. A short code is a five-digit or six-digit number that’s meant for high-volume SMS messaging.

Short codes are often used for application-to-person (A2P) messaging, two-factor authentication (2FA), and marketing messages — and you must request a separate short code for each country you support.

Bottom line, you need to be serious about SMS to own a short code as they are more expensive than long codes. You can lease them for around $1,000 USD per month in the USA. Short code also involves a rigorous registration process with wireless carriers.

If your business revolves around SMS as a main form of communication, like bank alerts/transactions, emergency notifications, etc., then a short code is the way to go as it offers high-volume messaging and can reach many different users at once. You can learn more about requesting a dedicated short code for SMS Messaging with Amazon Pinpoint here.

Two-way SMS messaging with Amazon Pinpoint

Amazon Pinpoint enables AWS customers to send both one-way and two-way transactional or promotional text messages via either long or short code. With two-way SMS messaging, Amazon Pinpoint opens up the possibilities for event-driven use cases where customer interaction from human-to-human or human-to-business is now front and center.

Below, I’ll walk through provisioning a long code with Amazon Pinpoint, registering a keyword and canned message, and creating an SNS topic. Don’t worry, I’ll explain how it all works once we set it up.

Get started using Amazon Pinpoint

Setup should take about 5–7 minutes. Please note, this service is not free. For pricing information, check the current rates.

1. Log into the Amazon Pinpoint Console
2. Create a project by giving it a name and select Create a project.

3. Select SMS and voice under project features

4. In General settings, select (a) Enable the SMS channel for this project and (b) select Transactional.

5. Expand the Advanced configurations — optional and Select Request Long Codes

6. Select your Target country or region, quantity of 1. Select Request long codes orange button. If you don’t see a long code for a particular county, you can always request one.

7. Select the newly provisioned phone number

8. Scroll down and ignore the Required keywords and select Enable two-way SMS

When enabling two-way SMS, you are prompted to create or choose an existing SNS topic. Select “Create a new SNS topic” and provide a name for the topic like SMSTwoWayHandler.

9. Select Add another keyword

For keyword, add “Amplify”. Under Response message, add a message that is an automated response to the user when they send “Amplify” to your long code. Select Save.

Done! You now have a dedicated long code for sending & receiving SMS messages. No longer do your customers have to download an app or submit a form online to interact with your business; you can simply set up a phone number (long code) and ask customers to send in votes, register for an event, confirm appointments, or anything else that requires instant feedback and interaction.

SNS Topic, Subscriptions, and Keywords

You just provisioned a long code, created an SNS topic, added a keyword, and provided a default canned response message for the given keyword. Now, let’s talk about how it all works.

Keywords and Canned Messages
For incoming text messages to your new long code, Amazon Pinpoint parses the message and looks for a specific keyword and, if found, immediately responds to the customer via SMS text message with the canned message you provided for that corresponding keyword. This whole cycle is fully automated and managed by Pinpoint.

Handling incoming messages with an SNS topic
In addition to handling the keyword and canned response, the original message is always forwarded to your Amazon Simple Notification Service (SNS) topic for further handling. The SNS topic will receive a JSON message from Pinpoint regardless if the customer provides a keyword or not.

Earlier, we created an SNS topic but didn’t add subscriptions. We need to add a listener (subscription) to the SNS topic in order to capture all inbound messages, otherwise, this two-way SMS thing would be kind of useless.

Amazon Pinpoint sends a JSON message to your SNS topic for all inbound messages sent to your long code. As a developer, you can subscribe one or more protocols (endpoints) to the SNS topic for handling the inbound JSON message.

For subscriptions, you can subscribe any HTTP/s service endpoint, an email address, an Amazon Simple Queue Service (SQS) queue, an AWS Lambda function, a push notification endpoint, or even an SMS phone number. You can subscribe one or many of these endpoints to the same topic.

Incoming JSON Payload From Amazon Pinpoint
When your long code number receives an SMS message, Amazon Pinpoint sends a JSON payload to the Amazon SNS topic that you designated. The JSON payload contains the body of the message and related data, as in the following example:

The value for originationNumber is the number that the message was sent from (that is, your customer’s mobile number). The value for destinationNumber is the number that the message was sent to (your long code). The messageBody is the exact text message that your customer typed into their phone.

Subscribing an Endpoint to Your SNS Topic

As an example, I’ll walk you through subscribing an SMS phone number to the SNS topic. In this case, the flow looks like this: Customer +12064555xxxx sends “Hello, world!” to your long code at +1321517xxxx.

12064555xxxx> Hello, world!

Pinpoint: Checks for keywords and doesn’t find a matched keyword and forwards “JSON message” to SNS topic.

SNS: Publishes JSON msg to subscribed endpoints. In this case, the full JSON message is sent to YOUR SMS number as a user: +12064555xxxx. Notice the message being sent to your phone number from SNS but the incoming phone number is your Amazon Pinpoint two-way SMS long code.

Subscribing Your SMS Phone Number (Testing)

1. Launch the SNS management console
2. Select your topic (we named it SMSTwoWayHandler)
3. Select Create subscription
4. Select SMS for protocol and enter your phone number. Make sure it’s in the right format as shown.

5. Select Create subscription

Done! Now test the subscription by sending a text message to your Amazon Pinpoint long code from any mobile device and you should see the JSON message arrive on your SMS phone number that you just subscribed to the topic. The SMS subscription is just a quick test. A common practice is to subscribe an AWS Lambda function that handles the incoming message, parses it, and performs other business logic and even respond to the customer.

Two-way SMS Limitations

There are limitations to consider when using two-way SMS with Amazon Pinpoint. First, there’s a default spending limit of only $1 per account, which you could reach after sending several dozen text messages. It’s a good idea to increase this limit immediately so that your SMS messaging doesn’t stop without notice.

Secondly, for long codes, SMS transactions have sending rate restrictions as stated in the Amazon Pinpoint docs. Also note, different countries and jurisdictions have different limitations on the permitted uses of SMS — make sure you are complying with your local regulations or you can face fines and have your sending suspended.

My use case for SMS

I created an application about five years ago that monitors the AWS Status Dashboard for issues. When a new posting is made to the RSS feed, I parse the data in an AWS Lambda function and notify about 15 subscribing mobile phone numbers via SMS.

During a large event, the application could send up to 12 messages per hour to those 15 subscribers. That’s about 180 SMS messages delivered per hour and I haven’t observed any issues and have trusted Amazon Pinpoint to deliver those messages successfully and in a timely manner since switching nearly two years ago from Amazon Simple Notification Service (SNS).

Closing Thoughts

SMS messaging is a powerful tool for customer engagement. Using a two-way SMS channel for your business can drive adoption and help retain those users in more ways than email or push notifications. Text messaging takes your application out of the picture and provides immediate interaction and customer feedback. Give Amazon Pinpoint a try and start sending/receiving text messages!

I’m considering some more advanced topics for SMS messaging like phone verification, building a robust Lambda function handler for handling all two-way scenarios, and/or diving deep into the delivery and failure CloudWatch Logs. Please let me know your preference in the comments.

Thanks, and happy texting!

Two-Way SMS with Amazon Pinpoint was originally published in A Cloud Guru on Medium, where people are continuing the conversation by highlighting and responding to this story.

File uploading presents a scalability problem that’s easy to fix with serverless — without taxing your pocketbook

The mechanism for uploading files from a browser has been around since the early days of the Internet. In the server-full environment it’s very easy to use Django, Express, or any other popular framework. It’s not an exciting topic — until you experience the scaling problem.

Imagine this scenario — you have an application that uploads files. All is well until the site suddenly gains popularity. Instead of handling a gigabyte of uploads a month, usage grows to 100Gb an hour for the month leading up to tax day. Afterwards, the usage drops back down again for another year. This is exactly the problem we had to solve.

File uploading at scale gobbles up your resources — network bandwidth, CPU, storage. All this data is ingested through your web server(s), which you then have to scale — if you’re lucky this means auto-scaling in AWS, but if you’re not in the cloud you’ll also have to contend with the physical network bottleneck issues.

You can also face some difficult race conditions if your server fails in the middle of handling the uploaded file. Did the file make to its end destination? What was the state of the processing? It can be very hard to replay the steps to failure or know the state of transactions when the server is overloaded.

Fortunately, this particular problem turns out to be a great use case for serverless — as you can eliminate the scaling issues entirely. For mobile and web apps with unpredictable demand, you can simply allow the application to upload the file directly to S3. This has the added benefit of enabling an https endpoint for the upload, which is critical for keeping the file’s contents secure in transit.

All this sounds great — but how does this work in practice when the server is no longer there to do the authentication and intermediary legwork?

The S3 uploader demo app

I set up a serverless S3 uploader app for a test drive, where you can upload a JPG file and it will push the file from the browser to an S3 bucket. For this exercise, I have limited the file size to 1Mb and the files will be automatically deleted every day. This is a Vue single-page application running from the same bucket. Try it out and then we’ll walk through the solution:

What’s happening behind the scenes is a two-step process — first, the web page calls a Lambda function to request the upload URL, and then it uploads the JPG file directly to S3:

The URL is the critical piece of the process — it contains a key, signature and token in the query parameters authorizing the transfer. Without these, the transfer will fail.

Feel free to clone the Github repo from https://github.com/jbesw/askjames-s3uploader. The public-facing demo app deletes all files within 24 hours and has throttling enabled to prevent abuse, just so you know.

Why use a Lambda function?

It’s possible to eliminate the Lambda function and do everything from the client browser — but there are a number of good reasons to avoid this approach. Apart from adding significantly more code to the web page, the Lambda function allows you to control the process away from the prying eyes of any potential attacker.

For example, what if you want to authorize the user first — maybe only paying subscribers can upload, whereas free trials are read-only? Or maybe you need to add extra hooks in the process to trigger other workflows, logging, or add a breaker in the event there are too many uploads. Or you might not be comfortable revealing bucket names or other information in the client-side code.

The Lambda function requesting the signed URL — the step 1 behind this demo app — is fairly minimal:

const uuidv4 = require('uuid/v4')
const AWS = require('aws-sdk')
AWS.config.update({ region: process.env.REGION || 'us-east-1' })
const s3 = new AWS.S3();

exports.handler = async (event) => {
  return await getUploadURL()
}

const getUploadURL = async () => {
  const actionId = uuidv4()

  const s3Params = {
    Bucket: '<< ENTER YOUR BUCKET NAME HERE >>',
    Key:  `${actionId}.jpg`,
    ContentType: 'image/jpeg',
    ACL: 'public-read',
  }

  return new Promise((resolve, reject) => {
    let uploadURL = s3.getSignedUrl('putObject', s3Params)

    resolve({
      "statusCode": 200,
      "isBase64Encoded": false,
      "headers": { "Access-Control-Allow-Origin": "*" },
      "body": JSON.stringify({
        "uploadURL": uploadURL,
        "photoFilename": `${actionId}.jpg`
      })
    })
  })
})

Although this function is bare bones, it would be easy to add all sorts of logic at this stage before making the request for the Signed URL. Once the function is in place, it’s then just a matter of setting up API Gateway and creating a single GET method to create an endpoint. Alternatively, you can deploy with the Serverless Framework and automate this step.

Building the front end

The project is based on a boilerplate Vue template, but all the important work to demonstrate this functionality happens in s3uploader.vue.

Once again, this is minimal coding with no error handling or niceties to keep the example simple, but you can see the code required to make this work is only a handful of lines. If you open the Chrome developer console (press F12), you can see the console.log output throughout the process.

Set your permissions

Finally, a word about permissions — when you are requesting the signed URLs, the requesting function needs the appropriate IAM permissions for both the request and uploading the file.

Some managed policies include the overly-generous S3:* permission. It’s common to see these functions work in development and fail in production because an IAM role gets switched to a much narrower set of privileges when promoted to production. The IAM role used by the function must be able to write into the bucket, otherwise it won’t work.

Back to scalability

The reason why all this is worthwhile comes back to scalability. In the event that you need to allow large numbers of users to upload files, this approach takes all the network burden away from you (the users upload directly to S3) and the scaling is seamless and automatic. When nobody uses your function, you pay nothing.

Additionally, since there’s no server hop between the user and S3, you can eliminate another point of failure. S3 is famous for its ‘11 9s’ of durability, so you also benefit from the fact that it’s nearly impossible for the file to just disappear.

Overall, given the benefits of the serverless implementation, it seems to be the obvious and easy way to manage any form of file uploading when working with AWS infrastructure.

How to add file upload features to your website with AWS Lambda and S3 was originally published in A Cloud Guru on Medium, where people are continuing the conversation by highlighting and responding to this story.

AWS SQS is a great middleware solution for decoupling message producers from consumers to manage thousands of transactions per second

With over two million users and tens of thousands of daily users playing our Panya Studios live trivia mobile app, we regularly experience huge bursts of activity within a short span of time. Since we decided to store chat data for analytical purposes, we needed a way to manage this process effectively.

In an earlier blog, we demonstrated how our engineering team stores large volumes of chat data in PostgreSQL through dynamic partitions. In this blog, we’ll cover what happens after the data has been sent from the client, but before it reaches the server and use message queue to scale our Chat MicroService.

We could build a large web server and upgrade our database size to handle hundreds of thousands of messages per second, but that would be expensive and difficult to maintain — and we would have to pre-provision our environment for the maximum expected workload and it would not be resilient e.g. database failure or throttling.

If few people were playing, then we’re overpaying for infrastructure; if there’s a traffic spike, then the server could crash and data could be lost.

Instead, we want to find a middleware solution that is highly scalable where it could easily handle 1 message per second or 1 million messages per second.

Rather than bombarding our database directly, we would send requests to this middle layer and then have our microservice pull messages from it as fast as possible to insert into our database.

In other words, what we want is a Message Queue.

Message Queue 101

Message queues enable different applications to have an indirect one-way communication with each other. The sender and receiver of the message do not interact with each other directly, nor do they need to interact with the message queue at the same time. To put it simply, the messenger and the recipients are decoupled.

Amazon offers a bit more formal definition — Message Queue Service is a form of asynchronous service-to-service communication used in serverless and microservices architectures. Messages are stored on the queue until they are processed and deleted. Each message is processed only once, by a single consumer. Message queues can be used to decouple heavyweight processing, to buffer or batch work, and to smooth spiky workloads.

“Message queues enables different applications to have an indirect one-way communication with each other.”

What do you mean by one-way communication?

Good question! It means that information is transferred in one direction only, from producer → consumer. Let’s try to illustrate this by comparing message queues with another popular form of app communication: APIs.

Comparing Message Queues versus APIs

APIs (application program interface) basically allows for different applications to have a two-way communication with each other. In the API world, when you send a request, you expect a response.

On the other hand, in the message queue world, when you send a request, you don’t expect a response. Messages sent get stored in a queue until it is read.

There are pros and cons to both methods — and we use both depending on the situation — but for our current requirement, using message queues made more sense because:

1. Decoupled Architecture — The sender’s job is to send messages to the queue and the receiver’s job is to receive messages from the queue. This separation allows for components to operate independently, while still communicating with each other.
2. Fault tolerant — If the consumer server is down, the messages would still be stored in queue until the server goes back up and retrieves it
3. Scalability —We can push all the messages in the queue and configure the rate at which the consumer server pulls from it. This way, the receiver won’t get overloaded with network requests and can instead maintain a steady stream of data to process.
4. Serverless — AWS SQS is easy to set up. No administrative overhead and infrastructure to support.
5. Single Consumer — we only have one microservice pulling from the queue. Once the message is consumed, it is removed from the queue. This is the reason why we opted for SQS instead of SNS (Simple Notification Service) with Pub/Sub messaging.

Comparing Message Queues versus Pub/Sub

The Pub/Sub (a.k.a. Publisher/Subscriber) pattern is a standard model used in many messaging systems. As you can guess by the name, there are Publishers and Subscribers.

• Publishers are message senders
• Subscribers are message receivers

Like message queues, they are loosely coupled through a communication layer. Publishers do not know which subscribers will see that data or when they will see it and vice versa. The publisher’s job is to send message to the middleware and the receiver’s job is to retrieve message from the middleware.

While similar to message queues (they are siblings), pub/sub messaging allows multiple consumers to receive each message in a topic.

On the other hand, message queues allows one consumer to receive each message. Technically you can have more than one consumer, but once the message is read the other consumer(s) will be unable to read the message.

What are the types of queues?

Amazon SQS offers two queue types for different application requirements:

Standard Queue

FIFO (First In First Out) Queue

As you can see, both queues are used to move data. The main difference between them, however, are messages per second (MPS) and ordering.

Standard queues have unlimited throughput, while FIFO queues support up to 300 MPS. Moreover, standard queues may deliver messages in a different order from which they were sent, while FIFO queues preserves it.

For our requirements, we use standard queues because we can and regularly do get more than 300 MPS.

TLDR; show me the code!

To follow these examples, you would need two Node.js apps and an AWS account.

Step 1: Create Queue
Use Amazon’s UI to create a new message queue.

Step 2: Create a Producer
The following code is directly from Amazon’s documentation

var AWS = require('aws-sdk'); 
AWS.config.update({region: 'REGION'});  

// Create an SQS service object 
var sqs = new AWS.SQS({apiVersion: '2012-11-05'});  
var params = {  
    DelaySeconds: 10,  
    MessageAttributes: {   
        "Medium": {   DataType: "String",     
                      StringValue: "ST wuz Here"    }
        },  
    MessageBody: "Hello world!",  
    QueueUrl: "SQS_QUEUE_URL" 
};  

// send the message
sqs.sendMessage(params, function(err, data) {   
    if (err) {     console.log("Error", err);   } 
    else {     console.log("Success", data.MessageId);   } 
});

Step 3: Create a Consumer
We use sqs-consumer library to Build SQS-based applications on Node.js without the boilerplate code.

const Consumer = require('sqs-consumer');
const AWS = require('aws-sdk');

AWS.config.update({
    region: 'REGION',
    accessKeyId: '...',
    secretAccessKey: '...'
});

const app = Consumer.create({
    queueUrl: "SQS_QUEUE_URL",
    handleMessage: (message, done) => {
// post to db
done();
    },
    sqs: new AWS.SQS()
});

app.on('error', (err) => {
    console.log(err.message);
});

app.start();

With a few lines of code, you can set up a running AWS SQS producer that will send messages to a queue, create a consumer that will listen to any messages from the queue, and then do something with the message such as post it to a database.

Message Queues — middleware for the win

Message Queues are a great middleware for handling huge bursts of data. By decoupling the message producers with our message consumer, we are able to scale our microservice as SQS will handle the majority of the workload. Instead of using the conventional API or Pub/Sub model, message queues worked better for us as a form of communication between microservices.

While there are other message queue providers out there like RabbitMQ and Apache Kafka, we chose SQS because we are already using AWS as our cloud provider for many services (EC2, RDS, S3, etc). Moreover, the fact that it’s serverless made it even easier to quickly set up and start running without worrying about the supporting infrastructure.

Try giving message queues a shot! If you have another way of doing this or you have any problems with examples above, just drop a comment below to let me know.

Thanks for reading — and please follow me here on Medium for more interesting software engineering articles!

Scaling microservices with message queues to handle data bursts was originally published in A Cloud Guru on Medium, where people are continuing the conversation by highlighting and responding to this story.

A step-by-step practical guide to service integration

Amazon Lex enables you to quickly build a chatbot in a few minutes. A few months ago, I tried to integrate a Lex chatbot with Facebook — and spent way too long because of the poor documentation and guidance.

To help save you some time and effort, this blog explains how the integration works and provides a simple and updated step-by-step guide for building your own chatbot using Amazon Lex and Facebook Messenger.

How does the integration work?

The overall integration is fairly straightforward. To get started, a developer simply creates a Facebook Page and a Facebook App. A page visitor communicates to your Facebook App by means of Facebook Messenger.

On the backend, the Facebook App sends and receives message events to Amazon Lex — this enables visitors to have back and forth communication with your Facebook App.

The Facebook App is subscribed to the Facebook Page and listens for events. Whenever a visitor sends a message to the page by means of Facebook Messenger, a webhook event is sent to the Facebook App.

The Facebook App can subscribe to variety of page events. For example, to receive messages the Facebook App must be subscribed to the following page events: messages, messaging_postbacks.

Without Lex integration, the developer needs to build their own webhook implementation to respond to these events. As an example of webhook implementation, developer can use Facebook Starter Project and must to deploy it somewhere.

However, Lex can abstract this level of difficulty away by providing a webhook implementation. To do this, Lex provides an integration channel that developer can use to connect with their Facebook App. With this approach, a developer can focus on building the bot logic — without worrying about creating their own webhook implementation.

Keeping it simple

To integrate Amazon Lex with Facebook, the current Lex documentation redirects users to Facebook’s Quick Start Tutorial. There, you are instructed to create your own webhook implementation and then host it somewhere.

You don’t need to do this. You can integrate Lex and Messenger without deploying a separate application! There are also a few other points in the tutorial which are confusing, and their documentation is very poor.

In this step-by-step tutorial, we’ll walk through all the relevant points with a simple guide to integrating these services for your chatbot.

Step-by-step guide

1. Get started by creating an Amazon Lex bot of your choice. Amazon has some good instructions for creating example bots.

2. Create a page on Facebook. You must do this by using your normal Facebook account that you use daily; otherwise, Facebook might ban your account as a fake account, and you will lose access to all your apps. The process is very straightforward.

For the sake of this tutorial, you can choose “Community or public figure” for your page, and then you can choose page name as well as a narrower page category. We will use this page in later steps.

3. Go to Facebook Developers page and log in with your normal Facebook account, if you already aren’t logged in. Don’t create a separate developer account, otherwise Facebook might consider it as fake and ban your account(s).

4. Create a new App. At the moment, you can do this by clicking on My Apps on the top right corner, and then Add New App.

5. Create a new App ID by providing required information.

6. Once your app is created, Facebook redirects you to the app page and you can choose products that your app can use. If Facebook doesn’t redirect you automatically, you can go to this page by choosing your newly created app in My Apps section on the top navigation bar. Add Messenger product to your app by clicking on “Set Up”

7. Now Facebook redirects you to Messenger Settings page. In Token Generation section, choose the Facebook page that you created in step 2. Then Facebook redirects you to a new page for confirmation.

Once you confirm the prompts, it generates Page Access Token. Copy the Access Token and save it. You will need it soon.

8. While you are in the app page, on the left bar, choose “Settings” and click on “Basic” part. On this page, you can find your App Secret. Copy the value. You will need it soon.

9. Go to Amazon Lex page and choose the bot that you would like to integrate. Choose Channels from top navigation bar, and then choose Facebook on the left column. Along with other required details, fill out Page Access Token and App Secret that you got from previous steps.

Click Activate and if you have done everything correctly, you should be able to see Callback Urls on the bottom of the page. Copy the endpoint url.

To provide some clarity on this process, I have tested from Lex console. You can see the bot reaction on the right side of page, so you can see what I expect to get from my Messenger once connected with Lex. For brevity, I have created a Bot with minimum logic.

10. Go back to your Facebook app page, in Messenger settings. In Webhooks section, click on Setup Webhooks

11. In New Page Subscription page, for Callback URL field, use the Endpoint Url you got from Lex. Also fill out same Verify Token that you defined earlier. In Subscription Fields, mark at least messages, messaging_postbacks, and messaging_optins. Then verify and save it.

12. Now, in Settings page, you should see that your webhook is marked as complete. You also need to subscribe your webhook to your page’s events. Choose the page, and click on “Subscribe”.

13. Now open your page on Facebook. Press “Add a button”. It should be straightforward. You need to create a button enabling page visitors to contact you by sending message through Messenger.

14. Once Send Message button is created, hover on it and test it. As you can see, it works for me. If it doesn’t work for you, check previous steps and make sure your have done everything correctly.

15. At this point, your bot is currently in test phase, and it works only for you. To enable others to test/use the Bot, you must add them manually to your Facebook App and to your page. Go to your App, and choose Roles from left column. Then you can add different people (with various roles ) to your project.

16. Now go to your Facebook Page, then go to Settings.

17. Go to Page Roles on the left column and add project roles. If you want your team members be able to interact with the Bot, you must also add them here.

18. Now your colleague must confirm both of the invitations you sent him/her, to be able to use the bot.

Make your app public

You can bypass the manual invitation process, by submitting your Facebook App for Facebook App Review and making it public. At the moment, Facebook can take some weeks to review your app — however this might vary from some days to some months depending on the permissions you are asking.

For bot-related permissions the review-time is usually relatively short. There is no way currently to speed up the process, so make sure you are in no rush when going live with your bot. You can get updated information about Facebook reviewing process on its documentation.

From December 2018, the permissions are granted at page level, so for each page that you would like to connect with your app, you need to do the review separately. You will also need to verify either a business or pass verification for individual developers which has been available since December 2018.

Acknowledgment

I would like to thank Pinja Kuosmanen for helping me with this post! She helped me as advisor and reviewer.

Don’t waste your time trying to hack me

Before publishing this post, I deleted my Facebook App, Page and Lex app. Therefore, credentials available in the screenshots can’t bring you anywhere. So, you don’t need to waste your time trying to hack me.

How to integrate an Amazon Lex chatbot with Facebook Messenger was originally published in A Cloud Guru on Medium, where people are continuing the conversation by highlighting and responding to this story.

Learn how to get started with Docker on Google Cloud Platform

Kubernetes might be the ultimate choice when deploying heavy workloads on Google Cloud Platform. However, Docker Swarm has always been quite popular among developers who prefer fast deployments and simplicity— and among ops who are learning to get comfortable with an orchestrated environment.

In this post, we will walk through how to deploy a Docker Swarm cluster on GCP using Terraform from scratch. Let’s do it!

Get Started

To get started, sign in to your Google Cloud Platform console and create a service account private key from IAM:

Download the JSON file and store it in a secure folder.

For simplicity, I have divided my Swarm cluster components to multiple template files — each file is responsible for creating a specific Google Compute resource.

1. Setup your swarm managers

In this example, I have defined the Docker Swarm managers based on the CoreOS image:

2. Setup your swarm workers

Similarly, a set of Swarm workers based on CoreOS image, and I have used the resource dependencies feature of Terraform to ensure the Swarm managers are deployed first. Please note the usage of depends_on keyword:

3. Define your network rules

Also, I have defined a network interface with a list of firewall rules that allows inbound traffic for cluster management, raft sync communications, docker overlay network traffic and ssh from anywhere:

4. Automate your inventory with Terraform

In order to take automation to the next level, let’s use Terraform template_file data source to generate a dynamic Ansible inventory from Terraform state file:

The template file has the following format, and it will be replaced by the Swarm managers and workers IP addresses at runtime:

Finally, let’s define Google Cloud to be the default provider:

5. Setup Ansible roles to provision instances

Once the templates are defined, we will use Ansible to provision our instances and turn them to a Swarm cluster. Hence, I created 3 Ansible roles:

• python: as its name implies, it will install Python on the machine. CoreOS ships only with the basics, it’s a minimal linux distribution without much except tools centered around running containers.
• swarm-init: execute the docker swarm init command on the first manager and store the swarm join tokens.
• swarm-join: join the node to the cluster using the token generated previously.

By now, your main playbook will look something like:

6. Test your configuration

To test it out, open a new terminal session and issue terraform init command to download the google provider:

Create an execution plan (dry run) with the terraform plan command. It shows you things that will be created in advance, which is good for debugging and ensuring that you’re not doing anything wrong, as shown in the next screenshot:

You will be able to examine Terraform’s execution plan before you deploy it to GCP. When you’re ready, go ahead and apply the changes by issuing terraform apply command.

The following output will be displayed (some parts were cropped for brevity):

If you head back to Compute Engine Dashboard, your instances should be successfully created:

7. Create your Swarm cluster with Ansible

Now our instances are created, we need to turn them to a Swarm cluster with Ansible. Issue the following command:

ansible-playbook -i inventory main.yml

Next, SSH to the manager instance using it’s public IP address:

If you run docker node ls, you will get a list of nodes in the swarm:

Deploy the visualizer service with the following command:

8. Update your network rules

The service is exposed on port 8080 of the instance. Therefore, we need to allow inbound traffic on that port, you can use Terraform to update the existing firewall rules:

Run terraform apply again to create the new ingress rule, it will detect the changes and ask you to confirm it:

If you point your favorite browser to your http://instance_ip:8080, the following dashboard will be displayed which confirms our cluster is fully setup:

And that’s all it takes! Watch setting up your Swarm cluster in action below:

Deploy Docker Swarm Cluster on GCP with Terraform and Ansible

Stay tuned!

In an upcoming post, we will see how we can take this further by creating a production-ready Swarm cluster on GCP inside a VPC — and how to provision Swarm managers and workers on-demand using instance groups based on increases or decreases in load.

We will also learn how to bake a CoreOS machine image with Python preinstalled with Packer, and how to use Terraform and Jenkins to automate the infrastructure deployment!

Deploy a Docker Swarm cluster on GCP using Terraform in 8 steps was originally published in A Cloud Guru on Medium, where people are continuing the conversation by highlighting and responding to this story.

Anyone willing to challenge the status quo and embrace serverless is somewhat open-minded and forward-thinking

Welcome to “Serverless Superheroes”!
In this space, I chat with the toolmakers, innovators, and developers who are navigating the brave new world of “serverless” cloud applications.

In this edition, I chatted with Farrah Campbell, ecosystems manager at Stackery. The following interview has been edited and condensed for clarity.

Forrest Brazeal: Farrah, we met for the first time at ServerlessConf in San Francisco. You wrote after that event about how much you appreciated the inclusive spirit of the serverless community. How have things been going since then?

Farrah Campbell: Things have been going so well. After ServerlessConf, we had a ServerlessDays event in Portland, and somebody said it was the least “bro-ey” tech gathering of people that they’d ever witnessed. I feel like we’re pioneering a whole new world here.

ServerlessDaysPDX was the least bro-ey gathering of tech people I’ve ever witnessed. That’s awesome.

 — @compurhythm8000

What do you think is driving the diversity in this community? Is it something unique about serverless that draws all kinds of people, or is it a manifestation of something happening in tech as a whole?

I’ve been thinking a lot about this, because it does require a cultural mind shift. If there’s a negative culture around servers and infrastructure, what happens when you remove that infrastructure? What happens when the responsibilities of teams change significantly?

I don’t think that we know exactly how that’s all playing out, but it definitely seems that there’s an opportunity to do things differently — and an opportunity to do things better.

Here’s the thing: anyone willing to do away with the status quo and embrace serverless is at least somewhat open-minded and forward-thinking. They are willing to consider new ways of doing their job (which can be surprisingly rare in tech) and are therefore receptive to new ideas in general.

I believe that this flexibility creates cultural openness as well, ushering in all kinds of new people to start building things. A front-end developer can become a full-stack developer, for instance. People are coming through code school who are able to quickly set up projects and not focus on infrastructure.

In her @Serverlessconf #ThinkFaas Lightning Talk session, @andreapasswater from @goserverless spoke about how a non-developer like herself could use serverless technology to launch her own production application and more. See her session here: https://t.co/iU0SQ2YPh7 #serverless

 — @acloudguru

Moreover, everyone involved in serverless is starting from square one. It’s not just some pleasant, circumstantial thing that we are freely sharing information — it’s intrinsic to the ecosystem.

That’s kind of your story, right? I know you didn’t have a lot of previous background in tech before joining Stackery.

Well, I’d say that my path to serverless has been unique, but that’s the case for most people who are active in this space! I took a leap in 2015 and quit my full-time job as an account manager at an employee benefits consultancy to work as a part-time office administrator.

The energy of it excited me and I knew I would want more than four hours a week. I asked if I could do more administrative things around the office: cleaning, organizing… whatever they needed.

To my surprise, they said no. Instead, the CEO wanted to find more useful things to do with my time. The following week I was invited to do some QA, and and then I got to file JIRA tickets and bug reports.

And then I decided that I could copy our terms and privacy over to our WordPress website. They said: “Are you sure you can do this?” Everything was going live the next morning. I said absolutely!

I quickly learned that evening that copy and paste from the Google Doc does not work! So I started Googling, I got on StackOverflow, and I learned to write HTML that night.

I mean, it was totally crappy and clunky, but I did it, and the site was ready the next morning. My coworkers were like, “Did you know how to write code?” I said nope, I did not, but I learned the basics last night! I felt awesome and knew that I needed and wanted to learn more.

Still working on my imposter syndrome... one day it will go away.

 — @FarrahC32

From there, I started learning Python, doing weekly lunch-and-learns with our CTO, learning everything I could about all the different pieces to our stack and feeling like an essential member of the team early on. I didn’t feel different from anybody else; I was completely engaged at work. They gave me challenges no other company was willing to, and the outcome has been pretty amazing.

And you’ve seen that same inclusive spirit in the broader serverless community?

For sure. I’ve been welcomed from the very start. Even going to the hackathon at ServerlessConf and being completely welcome to join a team, when I’d never had that experience before (in fact, I’d been turned away at other conferences).

Having these opportunities “happen” to me feels like luck, but really it’s not. It’s a combination of hard work, curiosity, raising my hand to help, and being given a chance by the community.

I love my job and working in tech every single day, and it saddens me that other women don’t feel the same, but I really believe that serverless can change this tired and inefficient aspect of tech.

Unfortunately, I think some engineers use their specific technical experiences as a gatekeeping mechanism. They might give off the attitude of, “I don’t consider someone to be ‘legit’ unless they’ve collected all the same stamps in their passport that I’ve had on my technological journey.” Do you ever encounter that attitude, and how do you deal with it?

Definitely. I deal with that on a daily basis, because I’m a woman in tech. My strategy is not to focus on people who aren’t willing to have conversations. If they’ve figured that their way is the only way of doing things, that’s not worth my time. I prefer effective conversations where people are seeking first to understand, then to be understood.

As an ecosystems manager, are you targeting any specific goals with regards to diversity and inclusion in the serverless community?

If we could get, like, half the engineering teams out there to look like Stackery, which is about fifty percent women, that would be amazing. We have representation from a number of communities here, and the more companies we see doing that, the better.

Now, how do we get there? We have to stop assuming a certain person can’t do something, and take on the task of saying ‘Let me help you figure out how to solve this problem, so that it doesn’t stop you and become a blocker.’

And really, that’s the power of serverless, isn’t it? People are trying to solve problems and get past all the the crap that maybe we don’t need. Because we have actual work to do.

Things I care about in a serverless environment - 1) Does my code run? 2) Does it scale? 3) Do you bill me for what my code consumes?

 — @swardley

Things I don't care about in a serverless environment - 1) The servers you use 2) Containers 3) Your electricity provider 4) Nuts and bolts

 — @swardley

So if I’m building an engineering team, what are some concrete steps I can take to become more inclusive and to take advantage of this explosion of opportunity that’s out there?

I’ve been successful in my interactions with all sorts of people because I try hard to come from a place of understanding. I don’t have any preconceived ideas about anyone. But it is true, for example, that women and men tend to communicate differently. So recognizing that, in the middle of an interaction, can be really helpful.

And inclusion goes both ways, too. For example, I don’t automatically assume that anything that is said to me is “mansplaining”. Sometimes I think they’re over-explaining, but I try to operate with giving people the benefit of the doubt. In my experience, they’re often coming from a good place and trying to be helpful, and I can work with that!

Overall, it’s important that people know that they will feel embraced by serverless folks. Personally, I have never felt more welcomed in my career. Before joining this community, I was never entirely comfortable walking up and talking to anybody in a room of tech experts or emailing them with a question or invitation to connect. I do now, which feels really good.

It’s a community that’s entirely eager to learn from its emerging participants. I want more women to feel welcomed in this way. I want that feeling for everybody.

Forrest Brazeal is an AWS Serverless Hero and a cloud architect at Trek10. He writes the ACG Serverless Superheroes series and draws the ‘FaaS and Furious’ cartoon series at A Cloud Guru. If you have a serverless story to tell, please let him know at @forrestbrazeal.

How serverless is breaking down barriers in tech was originally published in A Cloud Guru on Medium, where people are continuing the conversation by highlighting and responding to this story.