Build a serverless Twitter bot using sentiment analysis

Learn how to use AWS Comprehend to modify a Twitter bot to engage with users based on the sentiment of their comments

Social media is an important engagement channel for many individuals and businesses, and some of the more mundane administration can be easily automated by a simple Twitter bot — such as liking positive comments.

When creating a Twitter bot, it’s important that your response takes into consideration the sentiment of the comment to ensure a troll doesn’t earn an undeserved retweet or like from your account.

To solve this problem, we’ll used some basis artificial intelligence offered by the AWS Comprehend service to analyze the comments, assess the sentiment, and trigger the appropriate response.

To try it out, just send a nice tweet to @jbesw — and you’ll get a ‘like’ within 5 minutes. Congratulations, you just interacted with my sentiment-aware Twitter bot!

To build your own Twitter bot with sentiment analysis, simply clone the code from GitHub and fire-up your favorite IDE to follow along.

Thanks Ian! My Twitter bot likes positive tweets!

There are two types of bots in this world

In the serverless world, bots are easy to build and generally have two distinct modes of operation:

  • Webhooks allows a service to ping our code when new content arrives.
  • Polling pings a service every so often to check for new content.

In a perfect world, webhooks are preferred since they allow near-realtime responses to events happening, and remain dormant until triggered. Twitter webhooks requires more complex integration and you only get access with a paid developer accounts plan — while the free level allows us to poll.

In the real world, polling is much more common. Although polling is more wasteful compared to event-triggered webhooks, no complex integration is required and neither is a paid plan.

With polling, it’s important to understand that if your interval is n minutes and the events are randomly distributed, on average it will respond to an event in n/2 minutes. In other words if we check for new tweets every 10 minutes, on average a new tweet will wait 5 minutes for us to respond — a 1 second response is the best-case scenario with a 10 minute delay being the worst-case.

3 Steps to Bot Building

In this article, there are three steps to making our bot work:

  1. Using Twitter’s API
  2. Detecting the Tweet sentiment
  3. Scheduling the Lambda

Step 1: Using Twitter’s API — “There’s an NPM package for that”
As usual, NPM is our first stop for checking if there’s a convenient library that handles all the heavy-lifting. The unpresumingly-titled “Twitter for Node.js” module is all we need.

Before using anything, we’ll need to make a pit-stop at the Twitter Developers homepage and request credentials which simply involves answering a few questions around potential usage. Once finished, you will have two keys and two secrets.

Our search parameters are set to look for recent tweets in English that mention @jbesw, but you should change this to your handle, a hashtag, or anything else described in the API docs:

const params = {
q: '@jbesw',
lang: 'en',
count: 10,
result_type: 'recent'
}
const tweets = await client.get('search/tweets', params)

The search API is limited with the free tier — you can request a list of request tweets but the date-range filter is only available on the paid tier. Instead, we will request the last 10 tweets and then check the timestamps to see which happened in the polling period.

If the polled tweets are new enough, the code will then analyze the sentiment of the text. In this example, we are going to ‘like’ tweets with a positive sentiment — but depending on the use case we could optionally retweet or respond.

Step 2: AWS Comprehend for sentiment analysis
Comprehend provides an extremely simple interface for evaluating whether a given piece of text is positive or negative. You provide the text and it provides a JSON response — as shown in the examples below:

“This is the stupidest thing I have ever heard.”

{ Sentiment: 'NEGATIVE',
SentimentScore:
{ Positive: 0.013419967144727707,
Negative: 0.9612272381782532,
Neutral: 0.015924010425806046,
Mixed: 0.009428821504116058
}
}

“James, this is an amazing idea, the greatest concept ever.”

{ Sentiment: 'POSITIVE',
SentimentScore:
{ Positive: 0.9940874576568604,
Negative: 0.000048249679821310565,
Neutral: 0.0030791808385401964,
Mixed: 0.002785108983516693
}
}

It can sometimes even notice the passive-aggressive:

“I’m surprised — who would expect you to have any good ideas?”

{ Sentiment: ‘NEUTRAL’,
SentimentScore:
{ Positive: 0.23506243526935577,
Negative: 0.2651223838329315,
Neutral: 0.47500795125961304,
Mixed: 0.02480722963809967
}
}

Sentiment analysis becomes harder as the text sample is shorter. Tweets that are short can be deceptively difficult for a bot to detect if the sender is being sarcastic, humorous, or just decidedly human.

In our bot, the code for capturing sentiment analysis is simple:

const getSentiment =  async function(Text) {
console.log(`getSentiment: text=${Text}`)
if (!Text) return null
  var params = {
LanguageCode: "en",
Text
}
  return new Promise((resolve, reject) => {
comprehend.detectSentiment(params, (err, data) => {
if (err) return reject(err)
else return resolve(data)
})
})
}

This example code sets a ‘positivity threshold’ of 0.6 — although this might be too low in real-world applications. It might also be a good idea to flag strongly negative tweets as a notification for human review.

Step 3: Scheduling the Lambda
The polling interval in my example is set to 5 minutes — so it will run around 8,640 times a month — a negligible amount of Lambda usage. You could also set the code to fire every minute, but you will need to ensure you stay within Twitter’s API rate limits.

Using Serverless Framework, setting an event schedule is simply a minor modification to the serverless.yaml configuration:

Some final recommendations

Lambda makes it easy to set up a Twitter bot — and the code required is very small thanks to the existing NPM package. This method for creating the bot is only the beginning — it could be modified to work as a webhook, upgraded to manage Tweets at scale, or set to handle image processing and evaluation.

Once you have deployed the bot, don’t forget to keep a keen eye on CloudWatch logs to make sure everything is running as expected. For scheduled lambdas, I would recommend setting log retention to a limited period (7 days) since otherwise they will accumulate indefinitely.

Once a positive Tweet arrives, you’ll see it in the logs:

This exercise incurs costs for log storage, Lambda activity and Comprehend usage — all offer free tier limits that are more than enough in most cases. For professional usage, check AWS pricing to estimate cost before deployment.

The most important part of this exercise is to ensure you stay within Twitter’s guidelines for bot usage. It’s tempting to think of all sorts of solutions, but you must be guided and bound by their rules — you can lose access to your API and even your Twitter account if you violate their terms. Otherwise, be creative, be positive, and build something that aids communications with your fans and customers!


Build a serverless Twitter bot using sentiment analysis was originally published in A Cloud Guru on Medium, where people are continuing the conversation by highlighting and responding to this story.