Amazon Rekognition Launches Enhanced Face Analysis

Amazon Rekognition provides a comprehensive set of face detection, analysis, and recognition features for image and video analysis. Today, we are launching accuracy enhancements to the face analysis features. This is the fifth model update overall since the service launched. Face analysis generates rich metadata about detected faces in the form of gender, age range, emotions, attributes such as ‘Smile’, ‘Eyeglasses’ and ‘Beard’, face pose, face image quality and face landmarks. With this release, we have improved the accuracy of gender identification, emotion detection (for all 7 supported emotions: ‘Happy’, ‘Sad’, ‘Angry’, ‘Surprised’, ‘Disgusted’, ‘Calm’ and ‘Confused’) and attributes such as ‘EyesOpen’. This release is particularly useful for customers who need to search and categorize large photo collections. For example, using face analysis, customers can easily find all photos that contain smiling people, or all photos with men who have beards and are wearing sunglasses.

Amazon GameLift Realtime Servers Now in Preview

Now available in preview, Amazon GameLift Realtime Servers help developers quickly create affordable game servers. Building a great multiplayer game experience on traditional commercial solutions has barriers that oftentimes deter game developers from building a multiplayer game. It can be time consuming, costly, and requires expertise that not all game developers may have.

Using new GameLift Realtime Servers, you can provide these multiplayer games with a game server that can be customized with just a few lines of JavaScript, and then scale to millions of players for a fraction of a penny per player per month.

To learn more, visit the Amazon GameLift product page or announcement blog post.

If you are interested in participating in the preview, sign up here.

AWS RoboMaker Announces 99.9% Service Level Agreement

AWS has published a service level agreement (SLA) for AWS RoboMaker. The SLA provides availability guarantees for AWS RoboMaker Simulation.

AWS will use commercially reasonable efforts to make AWS RoboMaker available with a Monthly Uptime Percentage, as described below, for each AWS region, during any monthly billing cycle (the “Service Commitment”). In the event AWS RoboMaker does not meet the Service Commitment, you will be eligible to receive a Service Credit as described in the AWS RoboMaker Service Level Agreement.

This SLA is now available in all regions where AWS RoboMaker is available. For more information on where AWS RoboMaker is available, see the AWS region table. Please visit the AWS RoboMaker product page to learn more.

Introducing Amazon Chime Voice Connector

Amazon Chime now lets you place inexpensive, secure telephone calls to over 100 countries from your on-premises phone system, using your internet connection or AWS Direct Connect. You can choose to purchase low-cost inbound calling, outbound calling, or both. With Voice Connector, you can reduce your voice calling costs by up to 50% by eliminating fixed telephone network costs, and simplifying your voice network administration by transitioning it to AWS. You can send some or all of your traffic to Voice Connector, which lets you scale your usage up or down based on your business needs. Voice Connector has no upfront fees or long-term commitments, which means you pay only for the voice minutes and phone numbers you use. There is no charge for simultaneous voice conversations or encryption. Dialing into Chime meetings or placing calls to other Voice Connector customers is free of charge.

You can start using Voice Connector today by logging into the Amazon Chime console to purchase telephone numbers and provision Voice Connectors. The service communicates with your on-premises phone system using the industry-standard Session Initiation Protocol (SIP). Calls are encrypted using Transport Layer Security (TLS) for SIP signaling and secure real-time transport protocol (SRTP) for voice media.

Voice Connector is available today in the US East (N. Virginia) region with support for United States telephone numbers.

To learn more about Voice Connector, visit the Amazon Chime website or to start using Voice Connector, log in to the Amazon Chime console.

 

Introducing Amazon Chime Business Calling

Amazon Chime Business Calling lets you text or call to over 100 countries, directly from the Amazon Chime desktop, mobile, and web applications. Users place calls by dialing from the on-screen dial pad or by clicking on a Chime contact. Incoming calls will ring on all devices where a user is logged into Amazon Chime, and callers can leave a voicemail if there is no reply. There are no up-front payments or long-term commitments with Business Calling; customers pay for phone numbers, voice minutes, and text messages – with no additional seat fees.

Business Calling is easy to set up, and administrators can assign Business Calling features to individual Chime users as needed. Customers can purchase new telephone numbers, or port their existing numbers from their telecommunications provider.

This feature is available today in the US East (N. Virginia) region with support for United States telephone numbers.

To learn more about Business Calling, visit the Amazon Chime website. To start using Business Calling, log in to the Amazon Chime Console.

 

How Google Cloud helped Multiplay power a record-breaking Apex Legends Launch

Can you take a wild guess how many players a new multiplayer game typically attracts in its first day of availability? Would you say thousands, tens of thousands or even hundreds of thousands?

Without any pre-launch marketing or promotional pushes, the free-to-play battle royale game Apex Legends, from Respawn Entertainment, reached a whopping one million unique players during the first eight hours of its debut on Monday, February 4, 2019. In the first 72 hours after its initial launch, Apex Legends reached 10-million players and has now reached 50 million unique players after just one month.

Managing such high levels of engagement can be nerve-racking and intense. If players experience connectivity issues at launch, the game may never recover. So much rides on a game’s launch, including its reputation, revenue, and longevity; it’s no surprise that it requires a robust infrastructure for an optimal multiplayer experience.

Apex Legends was developed by Respawn Entertainment and published by Electronic Arts, using the game server hosting specialists on Unity’s Multiplay team to facilitate the game’s availability across most major platforms. With a state-of-the-art cloud server orchestration framework and 24/7/365 professional services team, Multiplay is able to fully support ongoing game growth. The orchestration layer leverages Google Cloud to help deliver seamless global-scale game server hosting for Apex Legends in ten regions spanning the Americas, Europe, and Asia.

Predicting the capacity required for a free-play title, from such a prominent studio, is impossible. Multiplay’s Hybrid Scaling technology scaled the majority of the demand for Apex Legends with Google Cloud while utilizing its global network of bare metal data centers.  Google Compute Engine, an Infrastructure-as-a-Service that delivers virtual machines running in Google’s data centers and global network, provides the core computing services. Compute Engine enables Multiplay to effortlessly ramp up to match user spikes — a critical requirement for many games, especially since Apex Legends received 1M downloads in eight hours after its initial debut. Compute Engine virtual machines can also spin down quickly, correlated to player demand, helping to optimize costs when fewer game servers are needed.

Google Cloud’s global private network is also an important infrastructure component for Multiplay. Fast connections, low latency and the ability for game servers to crunch through updates as quickly as possible together ensure the best experience for players.

Multiplay, a division of Unity Technologies, creator of the world’s most widely used real-time 3D development platform, has had a long-standing relationship with Google Cloud.

“After working with Google Cloud on Respawn’s Titanfall 2, Google Cloud was the logical option for Apex Legends. With its reliable cloud infrastructure and impressive performance during our testing phase, it was clear we made the right choice,” Paul Manuel, Managing Director for Multiplay, recently shared. “Throughout launch, Google Cloud has been a great partner. We greatly appreciated the level of dedication the team demonstrated during the simulated game launch, and for making sure we had the necessary number of cores worldwide to support this launch.”

You can learn more about how game developers and platforms turn to Google Cloud for game server hosting, platform services, and global scale and reach in this blog post. And for more information about game development on Google Cloud, visit our website.

Analyzing 3024 rice genomes characterized by DeepVariant

Rice is an ideal candidate for study in genomics, not only because it’s one of the world’s most important food crops, but also because centuries of agricultural cross-breeding have created unique, geographically-induced differences. With the potential for global population growth and climate change to impact crop yields, the study of this genome has important social considerations.

This post explores how to identify and analyze different rice genome mutations with a tool calledDeepVariant. To do this, we performed a re-analysis of the Rice 3K dataset and have made the data publicly available as part of the Google Cloud Public Dataset Program pre-publication and under the terms of the Toronto Statement.

We aim to show how AI can improve food security by accelerating genetic enhancement to increase rice crop yield. According to the Food and Agriculture Organization of the United Nations, crop improvements will reduce the negative impact of climate change and loss of arable land on rice yields, as well as support an estimated 25% increase in rice demand by 2030.

Why catalog genetic variation for rice on Google Cloud?

In March 2018, Google AI showed that deep convolutional neural networks can identify genetic variation in aligned DNA sequence data. This approach, called DeepVariant, outperforms existing methods on human data, and we showed that the approach to call variants on a human can be used to call variants on other animal species. This blog post demonstrates that DeepVariant is also effective at calling variants on a plant, thus demonstrating the effectiveness of deep neural network transfer learning in genomics.

In April 2018, three research institutions—the Chinese Academy of Agricultural Sciences (CAAS), the Beijing Genomics Institute (BGI) Shenzhen, and the International Rice Research Institute (IRRI)published the results of a collaboration to sequence and characterize the genomic variation of the Rice 3K dataset, which consists of genomes from 3,024 varieties of rice from 89 countries. Variant calls used in this publication were identified against a Nipponbare reference genome using best practices and are available from the SNP-Seek database (Mansueto et al, 2017).

We recharacterized the genomic variation of the Rice 3K dataset with DeepVariant. Preliminary results indicate a larger number of variants discovered at a similar or lower error rate than those detected by conventional best practice, i.e. GATK.

In total the Rice3K DeepVariant datasetcontains ~12 billion variants at ~74 million genomic locations (SNPs and Indels). These are available in a 1.5 terabyte (TB) table that uses the BigQuery Variants Schema.

Even at this size, you can still run interactive analyses, thanks to the scalable design of BigQuery. The queries we present below run on the order of a few seconds to a few minutes. Speed matters, because genomic data are often being interlinked with data generated by other precision agriculture technologies.

Illustrative queries and analyses

Below, we present some example queries and visualizations of how to query and analyze the Rice 3K dataset. Our analyses focus on two topics:

  • The distribution of genome variant positions, across 3024 rice varieties.
  • The distribution of allele frequencies across the rice genome.

For a step-by-step tutorial on how to work with variant data in BigQuery using the Rice 3K data or another variant dataset of your choosing, consider trying out the Analyzing variants with BigQuery codelab.

Analysis 1: Genetic variants are not uniformly distributed

Genomic locations with very high or very low levels of variation can indicate regions of the genome that are under unusually high or low selective pressure.

In the case of these rice varieties, high selective pressure (which corresponds to low genetic variation) indicates regions of the genome under high artificial selective pressure (i.e. domestication). Moreover, these regions contain genes responsible for traits that regulate important cultivational or nutritional properties of the plant.

We can measure the magnitude of the regional pressure by calculating at each position the Z statistic of each individual variety vs. all varieties. Here’s the query we used to produce the heatmap below, which shows the distribution of genetic variation across all 1Mbase-sized regions across all 12 chromosomes as columns (labeled by the top colored row), vs. all 3024 rice varieties as rows. Red indicates very low variant density relative to other samples within a particular genomic region, while pale yellow indicates very high variant density within a particular genomic region. The dendrogram below shows the similarity among samples (branch length) and groups similar rice varieties together:

rice_genomes_plot.png

A high resolution PDF of this plot is available, as well as the R script used to generate it.

Some interesting details of the dataset are highlighted (in yellow) in the heatmap above:

  1. Closer inspection of chromosome 5 (cyan columns, 1Mbase blocks 9-12) shows that the distinct distribution of Z scores across samples likely occurs due to two factors:

    1. this region includes many centromeric satellites resulting in a high false-positive rate of variants detected, and

    2. a genomic introgression present in some of the rice varieties multiplies this effect (yellow rows).

  2. Nearly all of the 3024 rice varieties included in the Rice 3K dataset are from rice species Oryza sativa. However, 5 Oryza glaberrima varieties were also included. These have a high level of detected genetic variation because they are from a different species, and are revealed as a bright yellow band at the top of the heatmap.

  3. The majority of samples can be partitioned into one group with high variant density and another group with low variant density. This partition fits with previously used methods for classification by admixture. For example, the bottom rows that are mostly red correspond to rice varieties in the japonica and circum-basmati (aromatic) groups that are similar to the Nipponbare reference genome we used.

Analysis 2: Some specific regions are under selective pressure

According to the Hardy-Weinberg Principle, the expected proportion of genotype frequencies within a randomly mating population, in the absence of selective evolutionary pressure, can be calculated from the component allele frequencies. For a bi-allelic position having alleles P and Q and corresponding population frequencies p and q, the expected genotype proportions for PP, PQ, and QQ can be calculated with the formula p2 + 2pq + q2 = 1. However we need to modify this formula by adding an inbreeding coefficient F to reflect the population structure (see: Wahlund effect) and the self-pollination tendency of rice: PP=p2+Fpq ; PQ=2(1-F)pq ; QQ=q2+Fpq where F=0.95.

The significance of genomic positions deviating from the expected genotype distribution follows χ2 , allowing a p-value to be derived and thus identification of positions that are either under significant selective pressure or neutral. In short, this analysis, highlights the fact that rice is highly inbred.

Below you can find a plot of 10-kilobase genome regions from the Oryza sativa genome, colored according to the proportion of variant positions that are significantly (p<0.05) out of (inbreeding modified) Hardy-Weinberg equilibrium, with white regions corresponding to those under low selective pressure and red regions corresponding to those under high selective pressure:

Oryza sativa genome plot.png

The data shown above were retrieved using this query and plotted using this R script. The query used to make this figure was adapted to the BigQuery Variants Schema from one of a number of quality control metrics found in the Google Genomics Cookbook.

Note that selective pressure on the genome is not uniformly distributed, indicated by the clumps of red visible in the plot. Interestingly, there is little correspondence between the prevalence of variants within a region (previous figure) and the proportion of variants within that same region that are under significant selective pressure. The bin size (10 kilobases) used in this visualization is on the order of the average Oryza sativa gene size (3 kilobases) and, given the low correlation between high selective pressure and variant density, it may be useful to guide a gene hunting expedition aimed at identifying genomic loci associated with phenotypes of interest (i.e. those that affect caloric areal yield, nutritive value, and drought- and pest-resistance).

Data availability and conclusion

Genome sequencer reads in FastQ format from Sequence Read Archive Project PRJEB6180, were aligned to the Oryza sativa Os-Nipponbare-Reference-IRGSP-1.0 reference genome using the Burrow-Wheeler Aligner (BWA), producing a set of aligned read files in BAM format.

Subsequently, the BAM files were processed with the Cloud DeepVariant Pipeline, a Cloud TPU-enabled, managed service that executes the DeepVariant open-source software. The pipeline produced a list of variants detected in the aligned reads, and these variants were written out to storage as a set of variant call files in VCF format.

Finally, all VCF files were processed with the Variant Transforms Cloud Dataflow Pipeline, which wrote records to a BigQuery Public Dataset table in the BigQuery Variants Schema format.

For additional guidance on how to use DeepVariant and BigQuery to analyze your own data on Google Cloud, please check out the following resources:

Acknowledgments

We’d like to thank our collaborators and their organizations—both within and outside Google—for making this post possible:

  • Allen Day, Google Cloud

  • Ryan Poplin, Google AI

  • Ken McNally, IRRI

  • Dmytro Chebotarov, IRRI

  • Ramil Mauleon, IRRI

Making game development more flexible and open with Google Cloud

The gaming industry is entering a period of tremendous growth.There are more than two billion players across the world, from competitive gamers to casual enthusiasts, and they enjoy games across a variety of platforms. Whether it’s mobile, console, PC, AR or VR—anyone can play, from anywhere, on any device. 

But they are not playing alone. Advances in global connectivity have powered the rise of real-time multiplayer games that offer shared experiences to players from all over the world.

As a result, these global smash hits are more than just games—increasingly they are becoming platforms themselves, with complex game economies and growing live viewing and esports communities.

For game developers of all sizes, these trends have incredible implications for the underlying cloud infrastructure powering their games. To operate a global game, it’s critical to have reliable, scalable infrastructure. Game services, such as matchmaking, need to be flexible enough to support cross platform gaming. And finally, data, analytics, and machine learning are essential tools for optimizing player engagement, segmentation and monetization, especially with the prevalence of free-to-play models.

Google Cloud is already powering many of the world’s top AAA and mobile games and developers, helping build better player experiences. Our infrastructure has 18 regions and a presence in over 200 countries and territories, connected by our private fiber optic network, to ensure that game servers and players are as close to each other as possible. 

If your game requires working with bare metal or multi-cloud deployments, we provide that flexibility as well. Through Kubernetes, we empower you to simply run your backend services wherever it makes sense, and open source Kubernetes services like Agones—co-founded with Ubisoft—helping to make hosting and scaling dedicated game servers easy and flexible. To make it even easier for developers to take advantage of Agones, we’ve now made it available in the Cloud Marketplace, which makes installation and management available in just a few clicks.

We want to give game developers the freedom to build without being constrained by inflexible off-the-shelf solutions that put constraints on their vision—and that starts with building a stronger open source community for games. 

Open Match, our open source matchmaking framework co-founded with Unity, lets developers re-use their matchmakers instead of building them from scratch for every game. It’s designed for flexibility, allowing you to bring your own match logic, so you can build your game your way, across all platforms. Open Match was used to help create Google’s first multiplayer Doodle, which scaled to a peak of 500,000 concurrent players.

Finally, Google Cloud’s leading analytics and machine learning capabilities can help developers store, manage, and analyze the petabytes of data generated by hit games, and generate insights and predictions that can help grow your game. King, makers of the Candy Crush Saga, transitioned their data warehouse from Hadoop to leverage the scalability, flexibility and reliability of BigQuery in 2018, and created hundreds of virtual players, trained using our Cloud Machine Learning Engine (CMLE), to quickly gather insights that were used to optimize the game design.

If you’re attending GDC March 18-22 at San Francisco’s Moscone Center, please stop by our booth and say hello. Don’t miss our Cloud Developer Day on Wednesday, March 20 or our ongoing booth sessions at the conference to hear from Google Cloud experts as well as companies we collaborate with like DeNA, FACEIT, Improbable, Multiplay, Pocket Gems, Square Enix, SuperSolid, Ubisoft, Unity and others. They’ll share how they’re using Google Cloud to make great games. Can’t make it? No worries. Our sessions will also be live streamed and recorded, viewable here.  

If you attend GDC, you’ll also hear from other Google teams such as Google Play, Google Maps Platform, Assistant, and Android on how we’re working with developers to create great games, connect with players, and scale their business.

Let’s take your game to the next level, together.

Serverless is a State of Mind

The point is focus — that is the why of serverless

Functions are not the point
If you go serverless because you love Lambda, you’re doing it for the wrong reason. If you go serverless because you love FaaS in general, you’re doing it for the wrong reason. Functions are not the point.

Sure, I love Lambda — but that’s not why I advocate for serverless.

Don’t get me wrong, functions are great. They let you scale transparently, you don’t have to manage the runtime, and they fit naturally with event-driven architectures. These are all fantastic, useful properties.

But functions should end up being a small part of your overall solution. You should use functions as the glue, containing your business logic, between managed services that are providing the heavy lifting that forms the majority of your application.

Managed services are not the point
We are fortunate to have such a wide range of managed services for so many different parts of our applications. Databases, identity and access management (so glad I don’t have to own that myself!), analytics, machine learning, content delivery, message queues for all sorts of different patterns.

Managed services provide the functionality you need with less hassle. You’re not patching the servers they run on. You’re not making sure the autoscaling is correctly providing the required throughput without a lot of idle capacity. Managed services lowers your operational burden significantly.

Managed services are great — but … they aren’t the point.

Ops is not the point
It’s great to know that you can apply fewer operations resources to keep your applications healthy. It is especially great that the resources you need scales mostly with the number of features you ship — not with traffic volume.

Reduced operations is more efficient — but … it’s not the point.

Cost is not the point
Ok, sometimes all the business wants you to do is reduce cost — and that’s all you care about. And serverless will help you do that. But in general, your cloud bill is not the point.

Your cloud bill is only one component of the total cost of your cloud applications. First of all, there’s the operations salaries— and that cost is lower if you have fewer ops resources. There’s also your development costs.

There are a lot of cost advantages — but … none of these are the point.

Code is not the point
Not only is code not the point, code is a liability. Code can at best do exactly what you intend it to. Bugs detract from this. You can only lose points through more coding. The more code you own, the more opportunities exist to depart from your intended value. Understanding this is a cultural shift.

Technology has been hard for a long time. It’s taken clever people to create value through technology. So developers started to believe that cleverness was inherent and good. We’ve spent so long crafting Swiss watches that we’ve failed to recognize the advent of the quartz Casio — and impugn the evolution as lacking in elegance.

Instead of applying our cleverness to solving technology problems, we really need to be understanding and solving business problems. And when you have to code — you are solving technology problems.

Technology is not the point
The reason that we’re doing this, any of this, is in service of some business goal. The business value that your organization is trying to create is the point.

Now, sometimes, what you’re selling is literally technology. But even if your product is technology, that may not be the value of what you’re selling.

There’s an old adage that people don’t buy drills, they buy holes. When you need a hole in your wall, you don’t care how fancy the drill is — you care how well it creates that hole you need.

At iRobot, we don’t sell robots. We don’t even sell vacuums. We sell clean homes. Roomba gives you time back in your day to focus on the things that matter to you. So if technology isn’t the point, what are we here for?

The point is focus
Serverless is a way to focus on business value.

How do functions help you deliver value? They let you focus on writing business logic, not coding supporting infrastructure for your business logic.

Managed services let you focus on writing your functions. Having less operations resources frees up people and money to be applied to creating new value for your customers.

Observability gives you tools to address MTBF and MTTR, both of which are a measure of how often your customers aren’t getting value. Spending less on the cloud means you can spend that money more directly in support of creating value.

Focus is the Why of Serverless

You should go serverless because you want to focus on creating value — and at your company you endeavor to apply technology toward the creation of business value.

Going back to cost, Lyft’s AWS bill, $100 million per year, has been in the news recently. Many people chimed in to say they could do it cheaper — they couldn’t, but that’s beside the point.

Would Lyft’s bill be lower if they switched to Lambda and managed services for everything they possibly could? Probably. But what would that do as they spent time rearchitecting? They would lose focus.

The company is at a stage in its journey where growth is more important than cost control. Eventually, that might change. Public companies are responsible to their shareholders, and so cost reduction can deliver value to them. But for Lyft right now, delivering value to their customers means executing with their current applications and processes. They are making the serverless choice.

What I’m telling you is that serverless has never been about the technology we call serverless. So what does the technology that we call serverless have to do with it?

Serverless is a consequence of a focus on business value
Technology is a consequence of how you’re trying to deliver value. Dev and ops teams have traditionally been separated with the notion that they have different focuses. But we’re seeing that trend changing.

The traditional model put the focus on technology — dev tech vs ops tech. But we’re seeing people realize that the focus should be on the value — the feature being delivered, including both how it’s built and how it’s run.

When we take this notion of focusing on business value, and run it to its logical conclusion, we get serverless.

When you want to focus on delivering value, you want to write functions. When your function needs state, you want a database. To get it from someone else, you use DBaaS — and you choose between your options based on how well it lets you focus.

And when you’re choosing managed services, some of them may even be user-facing. If you can use social login instead of owning your own accounts, that’s one less thing you have to manage, and one less piece of the user experience table stakes you need to own.

Now, for everything you are outsourcing, you are still accountable. Your users don’t care if their bad experience is caused by a third party you’re using, it’s still your problem. You need to own outages to your users while accepting that you don’t fully control your destiny there. This is an uncomfortable place to be — but it’s worthwhile.

You can’t win points on these things — but you can lose points. This means that you need to know what “bad” looks like. That requires having enough knowledge about the outsourced pieces of your product and your technology to know that you’re delivering enough quality to your users.

Note that deep expertise in a focus area, and broad but thin knowledge of adjacent areas is exactly analogous to the T-shaped skills concept — applied to organizations and teams.

Serverless is a trait
Serverless is a trait of companies. A company is serverless if it decides that it shouldn’t own technology that isn’t core to delivering its business value. Few companies are really totally serverless. But within a company, you can still have parts that are serverless.

If your team decides to focus only on the value it’s delivering, and delegate anything outside that either to another team, or ideally outside — then your team is going serverless. And you can’t always choose to use an outside technology — that’s fine, you can still make the best choice given the constraints.

And with a big enough organization, it can cease to matter. When Amazon.com uses Lambda, that’s fully serverless, even though it’s on-prem in some sense. But what if it’s just you?

What if you’re excited about serverless, but you feel completely alone at your company? What if you’re far removed from actual business value — if you’re patching servers for a team that serves a team that serves a team that creates user-facing content? I want to convince you that you can go serverless today, yourself, in any situation.

Serverless is a direction, not a destination
I used to talk about serverless as a spectrum, because I knew there wasn’t a bright line separating serverless technology from non-serverless technology. I mean, there almost never is a bright line separating any given grouping of anything, so I was pretty safe in that assumption.

I talked about how something like Kinesis, where you need to manage shards, is serverless, but less serverless than SQS, where you don’t. How you don’t have to patch instances with RDS, but you do need to choose instance types and number. These technologies are all various shades of serverless.

But recently I’ve come to realize a problem with portraying serverless as a spectrum is that it doesn’t imply movement. Just because you’re using something designated serverless of a sort doesn’t mean you should feel comfortable that you’ve attained serverless — that it’s acceptable to keep using that and think you’ve checked the serverless box.

Climb the serverless ladder
I’ve started to think of serverless as a ladder. You’re climbing to some nirvana where you get to deliver pure business value with no overhead. But every rung on the ladder is a valid serverless step.

If you move from on-prem to a public cloud, that’s a rung on the ladder. If you move from VMs to containers, that’s a rung on the ladder. If you move from no container orchestration, or custom orchestration, to Kubernetes, that’s a rung on the ladder. If you move from long-lived servers to self-hosted FaaS, that’s a rung on the ladder. But there’s always a rung above you, and you should always keep climbing.

Climbing the serverless ladder corresponds to using technology further to the right on this graph from Simon Wardley.

One thing the “ladder” doesn’t convey is that it’s not linear. Moving from VMs to containers to Kubernetes while staying on-prem are rungs on the ladder, but so is moving your VMs from on-prem to the cloud. There’s often not a definitive “better” in these cases.

I thought of the metaphor of many paths leading up a mountain, but one thing I like about the ladder is that it can be infinite. There isn’t an end state. I love Lambda, but I am always looking for better ways of delivering code that keep me more focused on value.

Serverless is a State of Mind

Serverless is about how you make decisions — not about your choices. Every decision is made with constraints. But if you know the right direction, even when you can’t move directly that way, you can take the choice that’s most closely aligned, and then you’re moving up another rung. So, how do you adopt this mindset? How do you make serverless choices?

Configuration is your friend
I think many developers look down on configuration as “not real programming”. There’s an idolatry of coding today. We’ve been told that “software is eating the world”, and we’ve inaccurately translated that to “coding is eating the world”.

We’ve come to believe that developers are the only important people in an organization, and that our sense of productivity is the only thing that matters. We want to feel in the zone, and that’s what coding provides. Any obstacle to this must be bad for the business. We’ve lost any sense of whether being in the zone is actually producing value faster and better than an alternative route.

Remember: Days of programming can save hours of configuration
Constraints are good. Removing options can help you focus. Obviously, not all constraints are good — but in general, the ability to do anything general comes at the cost of it taking longer to do one particular thing. Guard rails may chafe, but you’ll be faster than if you have to constantly watch the edge.

In this way, serverless is about minimalism. Removing distractions. Marie Kondo is big now, and the same advice applies. Find the components of your stack that don’t spark value.

Be afraid of the enormity of the enormity of the possible
Possibilities carry with them hidden complexity. For any technology, one of my primarily evaluation metrics is how much capability it has beyond the task at hand. When there’s a lot of extra space, there’s unnecessary complexity to both deal with and learn.

People tout Kubernetes as a single tool to accomplish every cloud need — and it can! But if everything is possible, anything is possible. For a given task, Kubernetes can go wrong because you haven’t accounted for the ways it acts for situations unrelated that task.

On the other hand, when you look at serverless services, you may have to choose between a 80% solution from your main provider, or a 3rd party provider with a service that better fits your needs. But what are the operations needs for that new provider? What’s the auth like? Those are hidden complexities that you’ll pull in — and you’ll need to trade that off with feature differences.

Accept the discomfort of not owning your own destiny
When you’re using a managed service, provider outages are stressful. There’s nothing you can do to fix their problem. There is no getting around it — this will always feel awful.

You’ll think, “if I was running my own Kafka cluster instead of using Kinesis, I could find the issue and fix it”. And that may be true, but you should remember two things:

  1. That would be a distraction from creating business value.
  2. You would almost certainly be worse at running it. You’d have more and worse incidents. It’s a service provider’s purpose in life to be good at it — and they have economies of scale you don’t.

Moving past the “I could always build it myself” attitude can be hard. Jared Short recently provided a brilliant set of guidelines for choosing technology.

In order, if you’re on a cloud platform, stay within the ecosystem when possible. You’re removing so many possibilities from the equation that way. But if you can’t get what you need on the platform, buy it from somewhere else.

If you can’t buy exactly what you need, can you rethink what you’re doing to fit what you can buy? This one is really important. It gets to the heart of time-to-market.

If you have something you think is valuable, you’ll want to ship it as soon as possible. But it’s better to ship something near to that faster, than to build the exact thing, You don’t know that it’s the right thing yet.

Waiting to build the exact right thing will not only take longer to ship, but your subsequent iterations will be slower — and maintenance of it will take resources that you could apply to shipping more things in the future. This applies even when the technology isn’t serverless: always ask if a tweak to your requirements would enable faster, better, or more focused delivery of value.

Finally, though, if you have to build it, own it. Find a way for it to be a differentiator. Now, this doesn’t mean everything you’ve built already you should turn into a differentiator. Look at only the things you can’t have bought as a service in a perfect world. Imagine what a completely serverless, greenfield implementation would look like, and find what needs to be built there.

Find your part of the business value
So fundamentally, you want to find your part of the business value. What is your technology work in service of? Maybe you’re far removed from user-facing product. You may only be contributing a small slice. But it’s there, and you can find it — and focus on that value.

Start with the immediate value you’re providing to others in the organization, and focus on that. And then start to trace the value chain. Make sure all your decisions are oriented around the value you’re creating. Make serverless choices.

I love this quote from Jessie Frazelle. You can turn it around; automate yourself out of a job, and keep demanding jobs.

Remember that you are not the tool. For any value that you’re creating — automate that creation. If you manage build servers, find ways to make them self-service, so what you’re delivering is not the builds per se, but the build tooling so teams can deliver the builds themselves.

TL;DR Serverless is a State of Mind

The point is not functions, managed services, operations, cost, code, or technology. The point is focus — that is the why of serverless.

Serverless is a consequence of a focus on business value. It is a trait. It is a direction, not a destination. Climb the never-ending serverless ladder.

Configuration is your friend. Days of programming can save hours of configuration. Be afraid of the enormity of the enormity of the possible. Accept the discomfort of not owning your own destiny

Find your part of the business value, and achieve a serverless state of mind.


Serverless is a State of Mind was originally published in A Cloud Guru on Medium, where people are continuing the conversation by highlighting and responding to this story.

Microsoft Azure Government is First Commercial Cloud to Achieve DoD Impact Level 5 Provisional Authorization, General Availability of DoD Regions

Furthering our commitment to be the most trusted cloud for Government, today Microsoft is proud to announce two milestone achievements in support of the US Department of Defense.

Information Impact Level 5 DoD Provisional Authorization by the Defense Information Systems Agency

Azure Government is the first commercial cloud service to be awarded an Information Impact Level 5 DoD Provisional Authorization by the Defense Information Systems Agency. This provisional authorization allows all US Department of Defense (DoD) customers to leverage Azure Government for the most sensitive controlled unclassified information (CUI), including CUI of National Security Systems. 

DoD Authorizing Officials can use this Provisional Authorization as a baseline for input into their authorization decisions on behalf of mission owner systems using the Azure Government cloud DOD Region. 

This achievement is the result of the collective efforts of Microsoft, DISA and its mission partners to work through requirements pertaining to the adoption of cloud computing for infrastructure, platform and productivity across the DoD enterprise.

General Availability of DoD Regions

Information Impact Level 5 requires processing in dedicated infrastructure that ensures physical separation of DoD customers from non-DoD customers. Over the past few months, we ran a preview program with more than 50 customers across the Department of Defense, including all branches of the military, unified combatant commands and defense agencies.

We are thrilled to announce the general availability of the DOD Region to all validated DoD customers. Key services covering compute, storage, networking and database are available today with full service level agreements and dedicated Azure Government support.

Dave Milton, Chief Technology Officer for Permuta Technologies, a leading provider of business solutions tailored for the military affirmed the significance of the general availability of the Azure DoD regions, saying:

“Azure Government DOD Regions has given us the ability to deploy our SaaS offering, DefenseReady Cloud, to the US Department of Defense in a scalable, secure, and cost-effective environment. The mission-critical nature of DefenseReady Cloud requires high availability, compliance with DoD’s SRG Impact Level 5 requirements, and scalability to support our customers changing demand, with a flexible pricing structure that allow us to offer capability to large enterprises as well as local commands. With Azure Government DOD Region, we are now able to onboard a customer in weeks, not months, allowing for a time-to-value that is unparalleled when compared with on-premises or other government-sponsored options. Through our partnership, Microsoft provided direct access to product group engineers, compliance support, training, and other resources needed to bring our SaaS solution to DoD.”

These accomplishments and the commentary of our customers and partners further reinforce our commitment to, and the strength of, our long-standing partnership with the US Department of Defense. For more information on Microsoft Cloud for Government services with Information Impact Level 5 provision authorization visit the Microsoft in Government blog, and for more detail on the Information Impact Level 5 Provision authorization (including in-scope services), please visit our Microsoft Trust Center.

To get started today, customers and mission partners may request access to our Azure Government Trial program.

Join Microsoft at the NVIDIA GPU Technology Conference

The world of computing goes deep and wide on working on issues related to our environment, economy, energy, and public health systems. These needs require modern, advanced solutions that were traditionally limited to a few organizations, are hard to scale, and take a long time to deliver. Microsoft Azure delivers High Performance Computing (HPC) capability and tools to power solutions that address these challenges integrated into a global-scale cloud platform. 

Whether it’s a manufacturer running advanced simulations, an energy company optimizing drilling through real-time well monitoring, or a financial services company using AI to navigate market risk  Microsoft’s partnership with NVIDIA makes access to NVIDIA GPUs easier than ever.

Join us in San Jose next week at NVIDIA’s GPU Technology Conference to learn how Azure customers combine the flexibility and elasticity of the cloud with the capability of NVIDIA GPUs. We will share examples of work we’ve done in oil & gas, automotive, artificial intelligence, and much more. Also, be on the lookout for new and exciting integrations between Azure AI and NVIDIA that bring GPU acceleration to more developers.

Microsoft sessions at the conference include:

If you are participating in any of the many NVIDIA DLI training classes, you will get a chance to experience firsthand the breadth of Azure GPU compute options through the interactive classes which are now Azure GPU powered.

Please come by and say “hello” at our Microsoft Booth – 1122 – where Microsoft and partners (including Teradici and Workspot) will have demos of customer use cases and we will have experts on hand to talk about how Azure is the cloud for any GPU workload.  Additionally, we will be demoing Microsoft Bing – which uses the power of NVIDIA GPUs on Azure to execute a variety of tasks such as generating instant answers to complex questions and analyzing images to help you find similar looking items or products.

As you can see, NVIDIA GPUs are a key part of the Microsoft High Performance Computing strategy that Azure customers rely on to drive innovation.

We’re looking forward to talking to you next week.