Application Development Archives

10 Oct 2019
By editor
Categories: Application Development, GCP, Google Cloud Platform, Identity & Security

Best practices for password management, 2019 edition

It is hard to imagine life today without passwords. They come in many forms, from your email credentials to your debit card PIN number, and they’re all secrets you use to help prove your identity. But traditional password best practices are no match for today’s sophisticated, and often automated, cybersecurity threats. With the all-too-often news of massive data breaches, leaked passwords, and phishing attacks, internet users must adapt to protect their valuable information.

While passwords are far from perfect, they aren’t going away in the foreseeable future. Google’s automatic protections prevent the vast majority of account takeover attacks—even when an attacker knows the username and password—but there are also measures that users and IT professionals can take to further enhance account security. In the spirit of October being National Cybersecurity Awareness Month, we’ve released two new whitepapers to help you navigate password security.

Modern password security for users provides pragmatic and human-centric advice for end users to help improve your authentication security habits. We go in-depth with tips on improving the security of the passwords you use today, advice on how to answer security questions, and explanations of why certain practices should be avoided.

Modern password security for system designers is the first paper’s technical counterpart, outlining the latest advice on password interfaces and data handling. It provides technical guidance on how to handle UTF-8 characters, advice on sessions, and best practices for building a secure authentication system that can stand up to modern threats.

Our aim is to promote an open and secure internet where users are equipped to protect their personal information and online systems are designed to prevent credential loss, even if those systems are compromised. We hope these whitepapers—available in PDF form at the links above—help you in your quest to better protect your environment.

7 Oct 2019
By editor
Categories: Application Development, DevOps & SRE, GCP, Google Cloud Platform

Push configuration with zero downtime using Cloud Pub/Sub and Spring Framework

As application configuration grows more complex, treating it with the same care we treat code—applying best practices for code review, and rolling it out gradually—makes for more stable, predictable application behavior. But deploying application configuration together with the application code takes away a lot of the flexibility that having separate configuration offers in the first place. Compared with application code, configuration data has different:

Granularity – per server or per region, rather than (unsurprisingly) per application.
Lifecycle – more frequent if you don’t deploy your application very often; less frequent, perhaps, if you embrace continuous deployment for code.

This leads us to an important best practice for software development teams: separating code from configuration when deploying applications. More recently, DevOps teams have started to practice “configuration as code”—storing configuration in version-tracked repositories.

But if you update your configuration data separately, how will your code learn about it and use it? It’s possible, of course, to push new settings and restart all application instances to pick up the updates, but that could result in unnecessary downtime.

If you’re a Java developer and use the Spring Framework, there’s good news. Spring Cloud Config lets applications monitor a variety of sources (source control, database etc.) for configuration changes. It then notifies all subscriber applications that changes are available using Spring Cloud Bus and the messaging technology of your choice.

If you’re running on Google Cloud, one great messaging option is Cloud Pub/Sub. In the remainder of this blog post, you’ll learn how to configure Spring Cloud Config and Spring Cloud Bus with Cloud Pub/Sub, so you can enjoy the benefits of configuration maintained as code and propagated to environments automatically.

Setting up the server and the client

Imagine you want to store your application configuration data in a GitHub repository. You’ll need to set up a dedicated configuration server (to monitor and fetch configuration data from its true source), as well as a configuration client embedded in the application that contains your business logic. In a real world scenario, you’d have many business applications or microservices, each of which has an embedded configuration client talking to the server and retrieving the latest configuration from it. You can find the full source code for all the examples in this post in this Spring Cloud GCP sample app.

Configuration server setup

To take advantage of the power of distributed configuration, it’s common to set up a dedicated configuration server. You configure a GitHub webhook to notify it whenever there are changes, and the configuration server, in turn, notifies all the interested applications that run the business logic that new configuration is available to be picked up.

The configuration server has the following three dependencies (we recommend using the Spring Cloud GCP Bill Of Materials for setting up dependency versions):

pom.xml

The first dependency, spring-cloud-gcp-starter-bus-pubsub, ensures that Cloud Pub/Sub is the Spring Cloud Bus implementation that powers all the messaging functionality.

The other two dependencies make this application act as a Spring Cloud Config server capable of being notified of changes by the configuration source (Github) on the /monitor HTTP endpoint it sets up.

The config server application also needs to be told where to find the updated configuration; we use a standard Spring application properties file to point it to the GitHub repository containing the configuration:

application.properties

You’ll need to customize the port if you are running the example locally. Like all Spring Boot applications, the configuration server normally runs on port 8080 by default, but that port is used by the business application we are about to configure, so an override is needed.

The last piece you need to run a configuration server is the Java code!

PubSubConfigGitHubServerApplication.java

As is typical for Spring Boot applications, the boilerplate code is minimal—all the functionality is driven by a single annotation, @EnableConfigServer. This annotation, combined with the dependencies and configuration, gives you a fully functional configuration server capable of being notified when a new configuration arrives by way of the /monitor endpoint. Then, in turn, the configuration server notifies all the client applications through a Cloud Pub/Sub topic.

Speaking of the Cloud Pub/Sub topic, if you run just the server application, you’ll notice in the Google Cloud Console that a topic named springCloudBus was created for you automatically, along with a single anonymous subscription (a bit of trivia: every configuration server is capable of receiving the configuration it broadcasts, but configuration updates are suppressed on the server by default).

Configuration client setup

Now that you have a configuration server, you’re ready to create an application that subscribes to that server’s vast (well… not that vast) knowledge of configuration.

The client application dependencies are as follows:

pom.xml

The client needs a dependency on spring-cloud-gcp-starter-bus-pubsub, just as the server did. This dependency enables the client application to subscribe to configuration change notifications arriving over Cloud Pub/Sub. The notifications do not contain the configuration changes; the client applications will pull those over HTTP.

Notice that the client application only has one Spring Cloud Config dependency: spring-cloud-config-client. This application doesn’t need to know how the server finds out about configuration changes, hence the simple dependency.

For this demo, we made a web application, but client applications can be any type of application that you need. They don’t even need to be Java applications, as long as they know how to subscribe to a Cloud Pub/Sub topic and retrieve content from an HTTP endpoint!

Nor do you need any special application configuration for a client application. By default, all configuration clients look for a configuration server on local port 8888 and subscribe to a topic named springCloudBus. To customize the configuration server location for a real-world deployment, simply configure the spring.cloud.config.uri property in the bootstrap.properties file, which is read before the regular application initialization. To customize the topic name, add the spring.cloud.bus.destination property to the regular application.properties file, making sure that the config server and all client applications have the same value.

And now, it’s time to add the client application’s code:

PubSubConfigApplication.java

ExampleController.java

Again, the boilerplate here is minimal—PubSubConfigApplication starts up a Spring Boot application, and ExampleController sets up a single HTTP endpoint /message. If no configuration server is available, the endpoint serves the default message of “none”. If a configuration server is found on the default localhost:8888 URL, the configuration found there at client startup time will be served. The @RefreshScope annotation ensures that the message property gets a new value whenever a configuration refresh event is received.

The code is now complete! You can use the mvn spring-boot:run command to start up the config server and client in different terminals and try it out.

To test that configuration changes propagate from GitHub to the client application, update configuration in your GitHub repository, and then manually invoke the /monitor endpoint of your config server (you would configure this to be done automatically through a GitHub webhook for a deployed config server):

After running the above command, the /message endpoint serves the most recent value retrieved from GitHub.

And that’s all that’s required for a basic Spring Cloud Config with Cloud Pub/Sub-enabled bus server/client combination. In the real world, you’ll most likely serve different configurations to different environments (dev, QA etc.). Because Spring Cloud Config supports hierarchical representation of configuration, it can grow to adapt to any environment setup.

For more information, visit the Spring Cloud GCP documentation and sample.

27 Sep 2019
By editor
Categories: Application Development, GCP, serverless

6 strategies for scaling your serverless applications

A core promise of a serverless compute platform like Cloud Functions is that you don’t need to worry about infrastructure: write your code, deploy it and watch your service scale automatically. It’s a beautiful thing.

That works great when your whole stack auto-scales. But what if your service depends on an APIs or databases with rate or connection limits? A spike of traffic might cause your service to scale (yay!) and quickly overrun those limits (ouch!). In this post, we’ll show you features of Cloud Functions, Google Cloud’s event-driven serverless compute service, and products like Cloud Tasks that can help serverless services play nice with the rest of your stack.

Serverless scaling basics

GCP Serverless Scale.png — *Serverless scaling patterns*

Lets review the basic way in which serverless functions scale as you take a function from your laptop to the cloud.

At a basic level, a function takes input, and provides an output response.
That function can be repeated with many inputs, providing many outputs.
A serverless platform like Cloud Functions manages elastic, horizontal scaling of function instances.
Because Google Cloud can provide near-infinite scale, that can have consequences for other systems with which your serverless function interacts.

Most scale-related problems are the result of limits on infrastructure resources and time. Not all things scale the same way, and not all serverless workloads have the same expected behaviors in terms of how they get work done. For example, whether or not the result of a function is returned to the caller or is only directed elsewhere, can change how you handle increasing scale in your function. Different situations may call for one or more different strategies to manage challenges scale can introduce.

Luckily, you have lots of different tools and techniques at your disposal to help ensure that your serverless applications scale effectively. Let’s take a look.

1. Use Max Instances to manage connection limits
Because serverless compute products like Cloud Functions and Cloud Run are stateless, many functions use a database like Cloud SQL for stateful data. But this database might only be able to handle 100 concurrent connections. Under modest load (e.g., fewer than 100 queries per second), this works fine. But a sudden spike can result in hundreds of concurrent connections from your functions, leading to degraded performance or outages.

One way to mitigate this is to configure instance scaling limits on your functions. Cloud Functions offers the max instances setting. This feature limits how many concurrent instances of your function are running and attempting to establish database connections. So if your database can only handle 100 concurrent connections, you might set max instances to a lower value, say 75. Since each instance of a function can only handle a single request at a time, this effectively means that you can only handle 75 concurrent requests at any given time.

2. Use Cloud Tasks to limit the rate of work done
Sometimes the limit you are worried about isn’t the number of concurrent connections, but the rate at which work is performed. For example, imagine you need to call an external API for which you have a limited per-minute quota. Cloud Tasks gives you options in managing the way in which work gets done. It allows you to perform the work outside of the serverless handler in one or more work queues. Cloud Tasks supports rate and concurrency limits, making sure that regardless of the rate work arrives, it is performed with rates applied.

3. Use stateful storage to defer results from long-running operations
Sometimes you want your function to be capable of deferring the requested work until after you provide an initial response. But you still want to make the result of the work available to the caller eventually. For example, it may not make sense to try to encode a large video file inside a serverless instance. You could use Cloud Tasks if the caller of your workload only needs to know that the request was submitted. But if you want the caller to be able to retrieve some status or eventual result, you need an additional stateful system to track the job. In Google APIs this pattern is referred to as a long-running operation. There are several ways you can achieve this with serverless infrastructure on Google Cloud, such as using a combination of Cloud Functions, Cloud Pub/Sub, and Firestore.

4. Use Redis to rate limit usage
Sometimes you need to perform rate-limiting in the context of the HTTP request. This may be because you are performing per-user rate limits, or need to provide a back-pressure signal to the caller of your serverless workload. Because each serverless instance is stateless and has no knowledge of how many other instances may also be serving requests, you need a high-performance shared counter mechanism. Redis is a common choice for rate-limiting implementations. Read more about rate limiting and GCP, and see this tutorial for how to use serverless VPC access to reach a private Redis instance and perform rate limiting for serverless instances.

5. Use Cloud Pub/Sub to process work in batches
When dealing with a large number of messages, you may not want to process every message individually. A common pattern is to wait until a sufficient number of messages have accumulated before handling all of them in one batch. Cloud Functions integrates seamlessly with Cloud Pub/Sub as a trigger source, but serverless workloads can also use Cloud Pub/Sub as a place to accumulate batches of work, as the service will store messages for up to seven days.

Then, you can use Cloud Scheduler to handle these accumulated items on a regular schedule, triggering a function that processes all the accumulated messages in one batch run.

You can also trigger the batch process more dynamically based on the number and age of accumulated messages. Check out this tutorial, which uses Cloud Pub/Sub, Stackdriver Alerting and Cloud Functions to process a batch of messages.

6. Use Cloud Run for heavily I/O-bound work
One of the more expensive components of many infrastructure products is compute cycles. This is reflected in the pricing of many managed services which include how many time-units of CPU you use. When your serverless workload is just waiting around for a remote API call it may make to return, or waiting for a file to read, these are moments where you are not using the CPU, but are still “occupying it” so will be billed. Cloud Run, which lets your run fully managed serverless containers, allows your workload to specify how many concurrent requests it can handle. This can lead to significant increases in efficiency for I/O bound workloads.

For example, if the work being done spends most of its time waiting for replies from slow remote API calls, Cloud Run supports up to 80 requests concurrently on the same serverless instance which shares the use of the same CPU allocation. Learn more about tuning this capability for your service.

When to use which strategy

After reading the above, it may be clear which strategy might help your current project. But if you are looking a little more guidance, here’s a handy flow-chart.

GCP Serverless-scale-flow.png — *click to enlarge*

Of course you might choose to use more than one strategy together if you are facing multiple challenges.

Just let it scale

Even if you don’t have any scaling problems with your serverless workload, you may still be uneasy, especially if this is your first time building software in a serverless environment—what if you’re about to hit some limit, for example? Rest easy, the default limits for Google Cloud serverless infrastructure are high enough to accomodate most workloads without having to do anything. And if you do find yourself approaching those limits, we are happy to work with you to keep things running at any scale. When your serverless workload is doing something useful, more instances is a good thing!

Serverless compute solutions like Cloud Functions and Cloud Run are a great way to build highly scalable applications—even ones that depend on external services. To get started, visit cloud.google.com/serverless to learn more.

20 Sep 2019
By editor
Categories: Application Development, DevOps & SRE, GCP, Inside Google Cloud

Cloud Build named a Leader for Continuous Integration in the Forrester Wave

Today, we are honored to share that Cloud Build, Google Cloud’s continuous integration (CI) and continuous delivery (CD) platform, was named a Leader in The Forrester Wave™: Cloud-Native Continuous Integration Tools, Q3 2019. The report identifies the 10 CI providers that matter most for continuous integration (CI) and how they stack up on 27 criterias. Cloud Build received the highest score in both the current offering and strategy categories.

“Google Cloud Build comes out swinging, matching up well with other cloud giants. Google Cloud Build is relatively new when compared to the other public cloud CI offerings; this vendor had a lot to prove, and it did…. Customer references are happy to trade the cost of paying for the operation and management of build servers for moving their operations to Google’s pay-per-compute system. One reference explained that it’s in the middle of a cloud shift and is currently executing up to 650 builds per day. With that proved out, this customer plans to move an additional 250 repositories to the cloud” – Forrester Wave™ report

Top score in the Current Offering category: Among all 10 CI providers evaluated in the Wave, Cloud Build got the highest score in the Current Offering category. The Current Offering score is based on Cloud Build’s strength in the developer experience, build speed and scale, enterprise security and compliance, and enterprise support, amongst other criteria.

Top score in the Strategy category: Along with top score in the current offering category, Cloud Build also received the highest score in the Strategy category. In particular, Cloud Build’s scores in the partner ecosystem, commercial model, enterprise strategy and vision, and product roadmap criteria contributed to the score.

CI plays an increasingly important role in DevOps, allowing enterprises to drive quality from the start of their development cycle. The report mentions “that cloud-native CI is the secret development sauce that enterprises need to be fast, responsive, and ready to take on incumbents and would-be digital disruptors.”

At Google, we’ve seen first-hand how cloud-based serverless CI tool can help drive quality and security at scale, and the lessons we’ve learned in that process manifest directly in Cloud Build.

Today, organizations of all sizes use Cloud Build to drive productivity improvements via automated, repeatable CI processes. Some customers include Zendesk, Shopify, Snap, Lyft, and Vendasta, who chose Cloud Build for its:

Fully serverless platform: Cloud Build scales up and scale down in response to load with no need to pre-provision servers or pay in advance for additional capacity.
Flexibility: With custom build steps and pre-created extensions to third party apps, enterprises can easily tie their legacy or home-grown tools as a part of their build process.
Security and compliance features: Developers have an ability to perform deep security scans within the CI/CD pipeline and ensure only trusted container images are deployed to production.

We’re thrilled by Forrester’s recognition for Cloud Build. You can download a copy of the report here.

3 Sep 2019
By editor
Categories: Application Development, Chrome Enterprise, GCP, Google Cloud Platform

Build a dev workflow with Cloud Code on a Pixelbook

Can you use a Pixelbook for serious software development? Do you want a workflow that is simple, doesn’t slow you down, and is portable to other platforms? And do you need support for Google Cloud Platform SDK, Kubernetes and Docker? I switched to a Pixelbook for development, and I love it!

Pixelbooks are slim, light, ergonomic, and provide great performance. Chrome OS is simple to use. It brings many advantages over traditional operating systems:

frictionless updates
enhanced security
extended battery life

And the most compelling feature for me: almost instant coming to life after sleep. This is great when hopping between meetings and on the road.

A little about me – I’m a Developer Programs Engineer. I work on Google Cloud and contribute to many open source projects. I need to accomplish repeatable development tasks: working with Github, build, debug, deploy and observe. Running and testing the code on multiple platforms is also of high importance. I can assure you, the workflow below built on Pixelbook satisfies all the following:

Simple, repeatable development workflow with emphasis on developer productivity
Portable to other platforms (Linux, MacOS, Windows)—“create once, use everywhere”
Support for Google Cloud Platform SDK, Github, Kubernetes and Docker.

Let’s dive into how you can set up a development environment on Pixelbook that meets all those requirements using Cloud Code for Visual Studio Code, remote extensions, and several other handy tools. If you are new to the world of Chromebooks and switching from a PC, check out this post to get started.

Step 1: Enable Linux apps on Pixelbook

Linux for Chromebooks (aka Crostini) is a project to let developers do everything they need locally on a Chromebook, with an emphasis on web and Android app development. It adds Linux support.

On your Pixelbook:

1. Go to Settings (chrome://settings) in the built-in Chrome browser.
2. Scroll down to the “Linux (Beta) ” section (see screenshot below).

3. Click “Turn on” and follow the prompts. It may take up to 10 minutes depending on your Wi-Fi connection.
4. At the end, a new Terminal window should automatically open to a shell within the container. We’re all set to continue to the next step – installing developer tools!

Pin the terminal window to your program bar for convenience.

Configure Pixelbook keyboard to respect Function keys
Folks coming from Windows or MacOS backgrounds are used to using Function keys for development productivity. On Chrome OS, they are replaced by default to a group of shortcuts.

However, we can bring them back:

Navigate to chrome://settings. Now, pick “Device” on the left menu, then pick “keyboard”. Toggle “treat top-row keys as function keys”:

Configure Pixelbook keyboard to respect Function key.png

Step 2: Install development tools

For Kubernetes development on GCP, we need to install tools like Docker, Google Cloud SDK and kubectl. Pixelbook Linux is Debian Stretch, so we will install prerequisites for docker and gcloud using instructions for Debian Stretch distribution.

Install and configure Google Cloud SDK (gcloud):
Run these commands from gcloud Debian quickstart to install gcloud sdk:

Troubleshooting
You might run into this error:

Your keyrings are out of date. Run the following commands and try the Cloud SDK commands again:

Add gcloud to PATH

Installing Docker CE for Linux:
Follow these instructions.

And then add your user to the docker group:

NOTE: This allows running docker commands without sudo.

Install kubectl

Installing Visual Studio Code

Go to VSCode linux install instructions page.

Download the.deb package (64bit) from the link on the page.
After the download is complete, install the deb file using “Install app with Linux (beta)”:

Troubleshooting
If you don’t see “Install with Linux” as an option for the deb file, double check that you switched to the beta channel.

Now let’s install a few extensions that I find helpful when working on a remote container using VS Code:

Docker – managing docker images, autocompletion for docker files, and more.

Remote Containers – use a docker container as a full-featured development environment.

These two, along with Cloud Code, are key extensions in our solution.

Step 3: Configuring Github access

Configure github with SSH key

Now copy and past the key into Github.

NOTE:If facing permissions error doing ssh-add, run sudo chown $USER .ssh and re-run all the steps for github setup again.

Set the username and email of github:

Step 4: Remote development

Now that we have the tools installed and Github access configured, let’s configure our development workflow. In order to create a solution that is portable to other platforms, we will use remote containers extension. We will create a container that will be used to build, deploy and debug applications that we create. This is how it will work:

We will open our codebase in a remote container. This will let VS Code think that it is open in isolated Linux environment, so everything we do (build, deploy, debug, file operations) will be interpreted as if we were working on a dedicated Linux VM with its own file system: every command we execute on VS Code will be sent for execution on our remote container. This way we achieve the goal of portability—remote Linux container can run on both MacOS and Windows just like we do it on Pixelbook with Chrome OS that supports Linux.

Dev Container settings for each repo

Here’s how to set up a dev container for an existing project. You can find the full source code in the Cloud Code templates repo. This Github repo includes templates for getting started with repeatable Kubernetes development in five programming languages—Node.js, Go, Java, Python and .NET. Each template includes configuration for debugging and deploying the template to Kubernetes cluster using Cloud Code for VS Code and IntelliJ. For simplicity, we work with a HelloWorld template that just serves “Hello World” message from a simple web server in a single container.

To enable remote container development, we need to add a .devcontainer folder with two files:

Dockerfile — defines container image that holds all developer tools we need installed in a remote development container
Devcontainer.json — Instructs VS Code Remote Tools extension how to run remote development container.

Creating a container image for remote development
Our remote container needs to have the SDK we use for development in the programming language of our choice. In addition, it needs tools that enable Cloud Code and Kubernetes workflows on Google Cloud. Therefore in the Dockerfile we install:

Google Cloud SDK
Skaffold — tool Cloud Code uses for handling the workflow for building, pushing and deploying apps in containers
Docker CLI

In addition, container images are immutable. Every time we open the code in a remote container, we’ll get a clean state—no extra settings will be persisted between remote container reloads by default (kubernetes clusters to work with, gcloud project configuration, github ssh keys). To address that, we mount our host folders as drives in the container (see this part later in devcontainer.json) and copy its content to the folder in the container file system where dev tools expect to find these files.

Example from Dockerfile of kubeconfig, gcloud and ssh keys sync between host and remote container:

devcontainer.json
This file tells Remote Container extension which ports to expose in the container, how to mount drives, which extensions to install in the remote container, and more.

A few notable configurations:

runArgs contains command line arguments remote extension passes to docker when remote container is launched. This is where we set environment variables and mount external drives in a container. This helps to eliminate authorizations and specifies the kubernetes clusters we want to work with in Cloud Code.

In the extensions section, we add a few VS Code extensions for enhanced productivity in the development container. These will be installed on a dev container but not on the host, so you can tailor this choice to the codebase you plan to work on in the dev container. In this case I am setting up for nodejs development.

Cloud Code for VS Code — Google’s extension that helps to write, deploy and debug cloud-native applications quickly and easily. It allows deploying code to kubernetes and supports 5 programming languages.
Npm support for VS Code
Code Spell Checker
Markdownlint — Improves the quality of markdown files.
Gitlens — Shows the history of code commits along with other relevant useful information.
Output colorizer — Colors the output of various commands. Helpful when observing application logs and other info in the IDE.
Vscode-icons — Changes icons to known file extensions for better visibility and discoverability of the files.
Docker — Manages docker images, autocompletion for docker files and more
TSLint — Linting for typescript (optional)
Bracket pair colorizer (optional)
Npm intellisense (optional)
ESLint Javascript (optional)

Hello World in Dev Container on Pixelbook

Let’s try to build, debug and deploy the sample Hello World nodejs app on Pixelbook using the remote dev container setup we just created:

Initialize gcloud by running gcloud init in a command line of your Pixelbook and following the steps. As part of our earlier setup, when we open the code in a remote container, Gcloud settings will be sync’ed into a dev container, so you won’t need to re-initialize every time.
Connect to a GKE cluster using the command below. We will use it to deploy our app. This also can be done outside of the dev container and will be sync’ed using our earlier setup in .devsettings.

Open the code in dev container: In VS Code command palette, type: Remote-Containers: Open Folder in Container… and select your code location. The code will open in dev container, pre-configured with all the toolset and ready to go!

Build and deploy the code to GKE using Cloud Code: In VS Code Command Palette, type: Cloud Code: Deploy and follow the instructions. Cloud Code will build the code, package it into container image, push it into container registry, then deploy it into GKE cluster we initialized earlier—all from the dev container on a Pixelbook!

Though slick and small, the Pixelbook might just fit your developer needs. With VS Code, Remote development extension, Docker, Kubernetes and Cloud Code you can lift your development setup to the next level, where there is no need to worry about machine-specific or platform-specific differences affecting your productivity. By sharing dev container setup on Github, developers that clone your code will be able to reopen it in a container (assuming they have the Remote – Containers extension installed).

Once done, developers will get an isolated environment with all dependencies baked in — just start coding!

If you have a Pixelbook — or if you don’t, and just want to try out Cloud Code — the Hello World app and all config files are available on GitHub. Let me know how it went and what your favorite setup for developer productivity is.

What is least privilege?

The principle of least privilege states that a resource should only have access to the exact resource(s) it needs in order to function. For example, if a service is performing an automated database backup, the service should be restricted to read-only permissions on exactly one database. Similarly, if a service is only responsible for encrypting data, it should not have permissions for decrypting data. Providing too few permissions prohibits the service from completing its task, but providing too many permissions can have rippling security ramifications.

If an attacker is able to gain access to a service that doesn’t follow the principles of least privilege, they may be able to force the service to behave nefariously—for example access customer data, delete critical infrastructure, or steal confidential business intelligence.

How do we achieve least privilege in Cloud Functions?

By default, all Cloud Functions in a Google Cloud project share the same runtime service account. This service account is bound to the function, and is used to generate credentials for accessing Cloud APIs and services. This default service account has the Editor role, which includes all read permissions, plus permissions for actions that modify state, such as changing existing resources. This enables a seamless development experience, but may include overly broad permissions for your functions, since most functions only need to access a subset of resources.

To practice the principle of least privilege in Cloud Functions, you can create and bind a unique service account to each function, granting the service account only the most minimal set of permissions required to execute the function.

Calling GCP services
Consider the following example function, which is triggered when a file is uploaded to a Cloud Storage bucket. The function reads the contents of the file, transforms it, and then writes the transformed file back to the same Cloud Storage bucket.

Reviewing the Cloud Storage IAM permissions, this function needs the following permissions on the Cloud Storage bucket:

storage.objects.get
storage.objects.create

We will use the ability to set a service account on an individual Cloud Function, giving each function its own service account with unique permissions. To do this:

1. Create a new service account. The service account name must be unique within the project. For more information, please see the managing service accounts documentation.

2. Grant the service account minimal IAM permissions. By default, service accounts have very minimal permissions. To use the service account with a function, we need to add bindings for the service account to the resources it needs to access.

3. Deploy a function that uses the new service account. When deploying our function, we use the --service-account flag to specify that our function should run as our custom service account instead of the default account. Since the function executes as the service account, our function inherits the permissions granted to the service account.

The Cloud Storage Object Admin role includes the following permissions:

storage.objects.create
storage.objects.update
storage.objects.delete
storage.objects.get
storage.objects.list
storage.objects.getIamPolicy
storage.objects.setIamPolicy

Permissions are very fine-grained access control rules. One or more permissions are usually combined to form a role. There are pre-built roles (like the Object Admin role above), and you also have the ability to generate custom roles with very specific sets of permissions.

If you recall, our function only needs the create and get permissions, but the role we picked includes five additional permissions that are not needed. While we have gotten closer, we are still not fully practicing the principle of least privilege.

There are no pre-built roles that includes only the two permissions we need, so we need to create a custom role in our project and grant that role to the service account on the bucket:

1. Create a custom role with exactly the two permissions needed.

2. Grant the service account access to the custom role on the bucket:

3. Deploy the function bound to that service account:

Calling other functions
In addition to calling a Google Cloud service like Cloud Storage, you may want to a function to call (“invoke”) another function. The concept of least privilege also applies to restricting which functions or users can invoke your function. You can achieve this by using the Cloud IAM roles/cloudfunctions.invoker role. Set IAM policies on each function to enforce that only certain users, functions, or services can invoke the function.

A good first step is to ensure that a function cannot be invoked by the public. For example, remove the special allUsers member from the roles/cloudfunctions.invoker role associated with the function:

This makes your function private and restricts the ability to invoke the function unless the caller has cloudfunctions.invoker permissions. If a caller does not have this permission, the request is rejected, your function is not invoked and you avoid billing charges.

Once a Cloud Function is private, you will need to add authentication when invoking it. Specifically, the caller needs a Google-signed identity token (a JSON Web Token) in the Authorization header of the outbound HTTP request. The audience (aud field) must be set to the URL of the function you are calling.

One of the easiest ways to get such a token is by querying the compute metadata server:

For example, suppose we have two functions – myFunction and otherFunction, where myFunction needs permission to invoke otherFunction. To accomplish this while also following the principle of least privilege we would:

1. Create a new, dedicated service account:

2. Grant the service account permissions to invoke otherFunction (this assumes that otherFunction is already running and deployed):

3. Deploy myFunction bound to the service account which has permission to invoke otherFunction:

Cloud Run and App Engine (when using IAP) can also perform similar validation.

When calling other services
If you are calling a compute service that you control that does not have Cloud IAM policies restricting access (like a Compute Engine VM), you can follow the same steps to generate the token and then validate the Google-signed identity token yourself.

Next steps

We hope this post illustrates the importance of the principle of least privilege and provides concrete steps you can take to improve the security of your serverless functions. If you want to learn more about Cloud Functions security, you can watch Serverless Security made Simple from Cloud Next 2019. If you want to learn more about how Google Cloud is enabling organizations to improve their security, including adopting the principle of least privilege, sign up for the Policy Intelligence alpha.

23 Jul 2019
By editor
Categories: Application Development, GCP, GCP Developer Pro Tips, Google Cloud Platform, Solutions and How-to's

GCP developer pro-tips: How to schedule a recurring Python script on GCP

So, you find yourself executing the same Python script each day. Maybe you’re executing a query in BigQuery and dumping the results in BigTable each morning to run an analysis. Or perhaps you need to update the data in a Pivot Table in Google Sheets to create a really pretty histogram to display your billing data. Regardless, no one likes doing the same thing every day if technology can do it for them. Behold the magic ofCloud Scheduler, Cloud Functions, and PubSub!

Cloud Scheduler is a managed Google Cloud Platform (GCP) product that lets you specify a frequency in order to schedule a recurring job. In a nutshell, it is a lightweight managed task scheduler. This task can be an ad hoc batch job, big data processing job, infrastructure automation tooling—you name it. The nice part is that Cloud Scheduler handles all the heavy lifting for you: It retries in the event of failure and even lets you run something at 4 AM, so that you don’t need to wake up in the middle of the night to run a workload at otherwise off-peak timing.

When setting up the job, you determine what exactly you will “trigger” at runtime. This can be a PubSub topic, HTTP endpoint, or an App Engine application. In this example, we will publish a message to a PubSub topic.

Our PubSub topic exists purely to connect the two ends of our pipeline: It is an intermediary mechanism for connecting the Cloud Scheduler job and the Cloud Function, which holds the actual Python script that we will run. Essentially, the PubSub topic acts like a telephone line, providing the connection that allows the Cloud Scheduler job to talk, and the Cloud Function to listen. This is because the Cloud Scheduler job publishes a message to the topic. The Cloud Function subscribes to this topic. This means that it is alerted whenever a new message is published. When it is alerted, it then executes the Python script.

The Code

SQL
For this example, I’ll show you a simple Python script that I want to run daily at 8 AM ET and 8 PM ET. The script is basic: it executes a SQL query in BigQuery to find popular Github repositories. We will specifically be looking for which owners created repositories with the most amount of forks and in which year they were created. We will use data from the public dataset bigquery-public-data:sample, which holds data about repositories created between 2007 and 2012. Our SQL query looks like this:

Python
We will soon paste this query in our github_query.sql file. This will be called in our main.py file, which calls a main function that executes the query in Python by using the Python Client Library for BigQuery.

Step 1: Ensure that you have Python 3 and install and initialize the Cloud SDK. The following will walk you through how to create the GCP environment. If you wish to test it locally, ensure that you have followed the instructions for setting up Python 3 on GCP first.

Step 2: Create a file called requirements.txt and copy and paste the following:

Step 3: Create a file called github_query.sql and paste in the SQL query from above.

Step 4: Create a file called config.py and edit with your values for the following variables.

You may use an existing dataset for this or pick an ID of a new dataset that you will create, just remember the id as you will need it for granting permissions later on.

Step 4:Create a file called main.py which references the previous two files.

In order to deploy the function on GCP, you can run the following gcloud commands. This specifies using a runtime of Python 3.7, creating a PubSub topic with a name of your choosing, and specifying that this function is triggered whenever a new message is published to this topic. I have also set the timeout to the maximum that GCP offers of 540 seconds, or nine minutes.

Make sure you first cd into the directory where the files are located before deploying, or else the following will not work.

You specify the frequency of how often your Cloud Function will run in UNIX cron time when setting up Cloud Scheduler with the schedule flag. This means that it will publish a message to the PubSub topic every 12 hours in the UTC timezone, as seen below:

where [JOB_NAME] is a unique name for a job, [SCHEDULE] is the frequency for the job in UNIX cron, such as “0 */12 * * *” to run every 12 hours, [TOPIC_NAME] is the name of the topic created in the step above when you deployed the Cloud Function, and [MESSAGE_BODY] is any string. An example command would be:

Our Python code does not use the actual message This is a job that I run twice per day!“” published in the topic because we are just executing a query in BigQuery, but it is worth noting that you could retrieve this message and act on it, such as for logging purposes or otherwise.

Grant permissions
Finally, open up the BigQuery UI and click ‘Create Dataset’ in the project that you referenced above.

By creating the Cloud Function, you created a service account with the email in the format [PROJECT_ID]@appspot.gserviceaccount.com. Copy this email for the next step.

Hover over the plus icon for this new dataset.
Click “Share Dataset”.
In the pop-up, enter the service account email. Give it permission “Can Edit”.

Run the job:
You can test the workflow above by running the project now, instead of waiting for the scheduled UNIX time. To do this:

Open up the Cloud Scheduler page in the console.
Click the ‘Run Now’ button.
Open up BigQuery in the console.
Under your output dataset, look for your [output_table_name], this will contain the data.

To learn more, read our documentation on setting up Cloud Scheduler with PubSub trigger, and try it out using one of our BigQuery public datasets.

22 Jul 2019
By editor
Categories: Application Development, DevOps & SRE, GCP, Google Cloud Platform

Introducing Spinnaker for Google Cloud Platform—continuous delivery made easy

Development teams want to adopt continuous integration (CI) and continuous delivery (CD), to identify and correct problems early in the development process, and to make the release process safe, low-risk, and quick. However, with CI/CD, developers often spend more time setting up and maintaining end-to-end pipelines and crafting deployment scripts than writing code.

Spinnaker, developed jointly by Google and Netflix, is an open-source multi-cloud continuous delivery platform. Companies such as Box, Cisco, and Samsung use Spinnaker to create fast, safe, repeatable deployments. Today, we are excited to introduce the Spinnaker for Google Cloud Platform solution, which lets you install Spinnaker in Google Cloud Platform (GCP) with a couple of clicks, and start creating pipelines for continuous delivery.

Spinnaker for GCP comes with built-in deployment best practices that can be leveraged whether teams’ resources (source code, artifacts, other build dependencies) are on-premises or in the cloud. Teams get the flexibility of building, testing, and deploying to Google-managed runtimes such as Google Kubernetes Engine (GKE), Google Compute Engine (GCE), or Google App Engine (GAE), as well as other clouds or on-prem deployment targets for hybrid and multi-cloud CD.

Spinnaker for GCP integrates Spinnaker with other Google Cloud services, allowing you to extend your CI/CD pipeline and integrate security and compliance in the process. For instance, Cloud Build gives you the flexibility to create Docker containers or non-container artifacts.

Likewise, integration with Container Registry vulnerability scanning helps to automatically scan images, and Binary Authorization ensures that you only deploy trusted container images. Then, for monitoring hybrid deployments, you can use Stackdriver to gain insights into visibility into the performance, uptime, and overall health of your application, and of Spinnaker itself.

Google’s Chrome Ops Developer Experience team uses Spinnaker to deploy some of their services:

“Getting a new Spinnaker instance up and running with Spinnaker for GCP was really simple,” says Ola Karlsson, SRE on the Chrome Ops Developer Experience team. “The solution takes care of the details of managing Spinnaker and still gives us the flexibility we need. We’re now using it to manage our production and test Spinnaker installations.”

Spinnaker for GCP lets you add sample pipelines and applications to Spinnaker that demonstrate best practices for deployments to Kubernetes, VMs and more. DevOps teams can use these as starting points to provide “golden path” deployment pipelines tailored to their company’s requirements.

“We want to make sure that the solution is great both for developers and DevOps or SRE teams,“ says Matt Duftler, Tech Lead for Google’s Spinnaker effort. “Developers want to get moving fast with the minimum of overhead. Platform teams can allow them to do that safely by encoding their recommended practice into Spinnaker, using Spinnaker for GCP to get up and running quickly and start onboard development teams.”

The Spinnaker for GCP advantage
The availability of Spinnaker for GCP gives customers a fast and easy way to set up Spinnaker in a production-ready configuration, optimized for GCP. Some other benefits include:

Secure installation: Spinnaker for GCP supports one-click HTTPS configuration with Cloud Identity Aware Proxy (IAP), letting you control who can access the Spinnaker installation.
Automatic backups: The configuration of your Spinnaker installation is automatically backed up securely, for auditing and fast recovery.
Integrated auditing and monitoring: Spinnaker for GCP integrates Spinnaker with Stackdriver for simplified monitoring, troubleshooting and auditing of changes and deployments.
Simplified maintenance: Spinnaker for GCP includes many helpers to simplify and automate maintenance of your Spinnaker installations, including configuring Spinnaker to deploy to new GKE clusters and GCE or GAE in other GCP projects.

Existing Spinnaker users can migrate to Spinnaker for GCP today if they’re already using Spinnaker’s Halyard tool to manage their Spinnaker installations.

22 Jul 2019
By editor
Categories: Application Development, GCP, Google Cloud Platform, serverless

A dozen reasons why Cloud Run complies with the Twelve-Factor App methodology

With the recent release of Cloud Run, it’s now even easier to deploy serverless applications on Google Cloud Platform (GCP) that are automatically provisioned, scaled up, and scaled down. But in a serverless world, being able to ensure your service meets the twelve factors is paramount.

The Twelve-Factor App denotes a paradigm that, when followed, should make it frictionless for you to scale, port, and maintain web-based software as a service. The more factors your environment has, the better.

So, on a scale of 1 to 12, just how twelve-factor compatible is Cloud Run? Let’s take the factors, one by one.

The Twelve Factors

I. CODEBASE

One codebase tracked in revision control, many deploys

Each service you intend to deploy on Cloud Run should live in its own repository, whatever your choice of source control software. When you want to deploy your service, you need to build the container image, then deploy it.

For building your container image, you can use a third-party container registry, or Cloud Build, GCP’s own build system.

You can even supercharge your deployment story by integrating Build Triggers, so any time you, say, merge to master, your service builds, pushes, and deploys to production.

You can also deploy an existing container image as long as it listens on a PORT, or find one of the many sporting a shiny Deploy on Cloud Run button.

II. DEPENDENCIES

Explicitly declare and isolate dependencies

Since Cloud Run is a Bring-Your-Own container environment, you can declare whatever you want in this container, and the container encapsulates the entire environment. Nothing escapes, so two containers won’t conflict with each other.

When you need to declare dependencies, these can be captured using environment variables, keeping your service stateless.

It is important to note that there are some limitations to what you can put into a Cloud Run container due to the environment sandboxing, and what ports can be used (which we’ll cover later in Section VII.)

III. CONFIG

Store config in the environment

Yes, Cloud Run supports stored configuration in the environment by default. And it’s mandatory. You must listen for requests on PORT, otherwise your service will fail to start.

To be truly stateless, your code goes in your container, and configurations are decoupled by way of environment variables. These can be declared when you create the service, in the Optional Settings.

Don’t worry if you miss this setting when you declare your service. You can always edit it again by clicking “+ Deploy New Revision” when viewing your service, or by using the –update-env-vars flag in gcloud beta run deploy

Each revision you deploy is not editable, which means revisions are reproducible, as the configuration is frozen. To make changes you must deploy a new revision.

For bonus points, consider using berglas, which leverages Cloud KMS and Cloud Storage to secure your environment variables. It works out of the box with Cloud Run (and the repo even comes with multiple language examples).

IV. BACKING SERVICES

Treat backing services as attached resources

Much like you would connect to any external database in a containerized environment, you can connect to a plethora of different hosts in the GCP universe.

And since your service cannot have any internal state, to have any state you must use a backing service.

V. BUILD, RELEASE, RUN

Strictly separate build and run stages

Having separate build and run stages is how you deploy in Cloud Run land! If you set up your Continuous Deployment back in Section I, then you’ve already automated that step.

If you haven’t, building a new version of your Cloud Run service is as easy as building your container image:

gcloud builds submit --tag gcr.io/YOUR_PROJECT/YOUR_IMAGE .

to take advantage of Cloud Build, and deploying the built container image:

gcloud beta run deploy --image gcr.io/YOUR_PROJECT/YOUR_IMAGE YOUR SERVICE

Cloud Run creates a new revision of the service, ensures the container starts, and then re-routes traffic to this new revision for you. If for any reason your container image encounters an error, the service is still active with the old version, and no downtime occurs.

You can also create continuous deployment by configuring Cloud Run automations using Cloud Build triggers, further streamlining your build, release, and run process.

VI. PROCESSES

Execute the app as one or more stateless processes

Each Cloud Run service runs its own container, and each container should have one process. If you need multiple concurrent processes, separate those out into different services, and use a stateful backing service (Section IV) to communicate between them.

VII. PORT BINDING

Export services via port binding

Cloud Run follows the modern architecture best practices and each Service must expose themselves on a port number, specified by the PORT environment variable.

This is the fundamental design of Cloud Run: any container you want, as long as it listens on port 8080.

Cloud Run does support outbound gRPC and WebSockets, but does not currently work with these protocols inbound.

VIII. CONCURRENCY

Scale out via the process model

Concurrency is a first-class factor in Cloud Run. You declare what the maximum number of concurrent requests your container can receive. If the incoming concurrent request count exceeds this number, Cloud Run will automatically scale by adding more container instances to handle all incoming requests.

IX. DISPOSABILITY

Maximize robustness with fast startup and graceful shutdown

Since Cloud Run handles scaling for you, it’s in your best interest to ensure your services are the most efficient they can be. The faster they are to startup, the more seamless scaling can be.

There are a number of tips around how to write effective services, so be sure to consider the size of your containers, the time they take to startup, and how gracefully they handle errors without terminating.

X. DEV/PROD PARITY

Keep development, staging, and production as similar as possible

A container-based development workflow means that your local machine can be the development environment, and Cloud Run can be your production environment! Even if you’re running on a non-Linux environment, a local Docker container should behave in the same way as the same container running elsewhere. It’s always a good idea to test your container locally when developing. Testing locally helps you achieve a more efficient iterative development strategy, allowing you to work more effectively. To ensure that you get the same port-binding behaviour as Cloud Run in production, make sure you run with a port flag:

PORT=8080 && docker run -p 8080:${PORT} -e PORT=${PORT} gcr.io/[PROJECT_ID]/[IMAGE]

When testing locally, consider if you’re using any GCP external services, and ensure you point Docker to the authentication credentials.

Once you’ve confirmed your service is sound, you can deploy the same container to a staging environment, and after confirming it’s working as intended there, to a production environment.

A GCP Project can host many services, so it’s recommended that your staging and production (or green and blue, or however you wish to call your isolated environments) are separate projects. This also ensures isolation between databases across environments.

XI. LOGS

Treat logs as event streams

Cloud Run uses Stackdriver Logging out of the box. The “Logs” tab on your Cloud Run service view will show you what’s going on under the covers, including log aggregation across all dynamically created instances. Stackdriver Logging automatically captures stdout and stderr, and there may also be a native client for Logging in your preferred programming language.

In addition, since logs are captured in Stackdriver Logging, you can then use the tools available for StackDriver logging to further work with your logs; for example, exporting to Big Query.

XII. ADMIN PROCESSES

Run admin/management tasks as one-off processes

Administration tasks are outside the scope of Cloud Run. If you need to do any project configuration, database administration, or other management changes, you can perform these tasks using the GCP Console, gcloud CLI, or Cloud Shell.

A near-perfect score, as a matter of fact(or)

With the exception of one factor being outside of scope, Cloud Run maps near perfectly with Twelve-Factor, which means it will map well to scalable, manageable infrastructure for your next serverless deployment. To learn more about Cloud Run, check out this quickstart.