Introducing AWS Application Composer (Preview)

AWS Application Composer helps developers simplify and accelerate architecting, configuring, and building serverless applications. You can drag, drop, and connect AWS services into an application architecture by using AWS Application Composer’s browser-based visual canvas. AWS Application Composer helps you focus on building by maintaining deployment-ready infrastructure as code (IaC) definitions, complete with integration configuration for each service.

Los experimentos del piloto automático de Amazon SageMaker se ejecutan con el modo de entrenamiento de conjunto y brindan visibilidad y métricas adicionales del flujo de trabajo de AutoML

El piloto automático de Amazon SageMaker ahora brinda más información sobre el flujo de trabajo subyacente de cada prueba dentro de un experimento del piloto automático de SageMaker puesto en marcha con el modo de entrenamiento de conjunto. El piloto automático de SageMaker clasifica una lista de modelos de machine learning por latencia de inferencia (es decir, el tiempo que se debe esperar para obtener el resultado de la predicción de un punto de conexión en tiempo real en el que se implementa el modelo) y por métricas objetivas, como la exactitud, la precisión, la recuperación y el área bajo la curva en la tabla de clasificación de modelos. El piloto automático de SageMaker crea, entrena y ajusta automáticamente los mejores modelos de machine learning en función de sus datos, al mismo tiempo que le permite mantener un control y una visibilidad totales. 

Amazon EMR en EKS incorpora la compatibilidad con la configuración de las propiedades de Spark dentro de los cuadernos de Jupyter en EMR Studio

Nos complace anunciar la compatibilidad con la configuración de las propiedades de Spark dentro de las sesiones de cuadernos de Jupyter en EMR Studio para las cargas de trabajo interactivas de Spark. Amazon EMR en EKS permite que los clientes ejecuten marcos de macrodatos de código abierto, como Apache Spark en Amazon EKS. Los clientes de Amazon EMR en EKS configuran y utilizan un punto de conexión administrado (disponible en la versión preliminar) para ejecutar cargas de trabajo interactivas mediante integrated development environments (IDE, entornos de desarrollo integrado) como EMR Studio.

Administre los metadatos de las tablas en el catálogo de datos de AWS Glue cuando ejecute cargas de trabajo de Flink en Amazon EMR

Los clientes de Amazon EMR ahora pueden utilizar el catálogo de datos de AWS Glue de sus flujos de trabajo de SQL de lote y de streaming en Flink. El catálogo de datos de AWS Glue es un catálogo compatible con la tienda de metadatos de Apache Hive. Puede configurar sus trabajos de Flink en Amazon EMR de modo tal que el catálogo de datos se utilice como una tienda de metadatos de Apache Hive externa. Con este lanzamiento, puede ejecutar directamente las consultas de SQL de Flink sobre las tablas almacenadas en el catálogo de datos.

Compatibilidad con la lectura y escritura de datos en Amazon DynamoDB y el acceso entre cuentas de Amazon S3 con Amazon EMR sin servidor

Amazon EMR sin servidor presenta la compatibilidad con la lectura y escritura de datos en Amazon DynamoDB con sus flujos de trabajo de Spark y Hive. Ahora, puede exportar, importar, consultar y unir tablas de Amazon DynamoDB directamente desde sus aplicaciones Spark o Hive de EMR sin servidor. Amazon DynamoDB es una base datos NoSQL completamente administrada que cumple con los requisitos de latencia y rendimiento de las aplicaciones de alta demanda, ya que proporciona una latencia de milisegundos de un solo dígito y un perfecto rendimiento predecible con escalabilidad de almacenamiento. 

How to use App Engine pull tasks (Module 18)

Posted by Wesley Chun (@wescpy), Developer Advocate, Google Cloud

Introduction and background

The Serverless Migration Station mini-series helps App Engine developers modernize their apps to the latest language runtimes, such as from Python 2 to 3 or Java 8 to 17, or to sister serverless platforms Cloud Functions and Cloud Run. Another goal of this series is to demonstrate how to move away from App Engine’s original APIs (now referred to as legacy bundled services) to Cloud standalone replacement services. Once no longer dependent on these proprietary services, apps become much more portable, making them flexible enough to:

App Engine’s Task Queue service provides infrastructure for executing tasks outside of the standard request-response workflow. Tasks may consist of workloads exceeding request timeouts or periodic tangential work. The Task Queue service provides two different queue types, push and pull, for developers to perform auxiliary work.

Push queues are covered in Migration Modules 7-9, demonstrating how to add use of push tasks to an existing baseline app followed by steps to migrate that functionality to Cloud Tasks, the standalone successor to the Task Queues push service. We turn to pull queues in today’s video where Module 18 demonstrates how to add use of pull tasks to the same baseline sample app. Module 19 follows, showing how to migrate that usage to Cloud Pub/Sub.

Adding use of pull queues

In addition to registering page visits, the sample app needs to be modified to track visitors. Visits are comprised of a timestamp and visitor information such as the IP address and user agent. We’ll modify the app to use the IP address and track how many visits come from each address seen. The home page is modified to show the top visitors in addition to the most recent visits:

Screen grab of the sample app's updated home page tracking visits and visitors
The sample app’s updated home page tracking visits and visitors

When visits are registered, pull tasks are created to track the visitors. The pull tasks sit patiently in the queue until they are processed in aggregate periodically. Until that happens, the top visitors table stays static. These tasks can be processed in a number of ways: periodically by a cron or Cloud Scheduler job, a separate App Engine backend service, explicitly by a user (via browser or command-line HTTP request), event-triggered Cloud Function, etc. In the tutorial, we issue a curl request to the app’s endpoint to process the enqueued tasks. When all tasks have completed, the table then reflects any changes to the current top visitors and their visit counts:

Screen grab of processed pull tasks updated in the top visitors table
Processed pull tasks update the top visitors table

Below is some pseudocode representing the core part of the app that was altered to add Task Queue pull task usage, namely a new data model class, VisitorCount, to track visitor counts, enqueuing a (pull) task to update visitor counts when registering individual visits in store_visit(), and most importantly, a new function fetch_counts(), accessible via /log, to process enqueued tasks and update overall visitor counts. The bolded lines represent the new or altered code.

Adding App Engine Task Queue pull task usage to sample app showing 'Before'[Module 1] on the left and 'After' [Module 18] with altered code on the right
Adding App Engine Task Queue pull task usage to sample app


This “migration” is comprised of adding Task Queue pull task usage to support tracking visitor counts to the Module 1 baseline app and arrives at the finish line with the Module 18 app. To get hands-on experience doing it yourself, do the codelab by hand and follow along with the video. Then you’ll be ready to upgrade to Cloud Pub/Sub should you choose to do so.

In Fall 2021, the App Engine team extended support of many of the bundled services to 2nd generation runtimes (that have a 1st generation runtime), meaning you are no longer required to migrate pull tasks to Pub/Sub when porting your app to Python 3. You can continue using Task Queue in your Python 3 app so long as you retrofit the code to access bundled services from next-generation runtimes.

If you do want to move to Pub/Sub, see Module 19, including its codelab. All Serverless Migration Station content (codelabs, videos, and source code) are available at its open source repo. While we’re initially focusing on Python users, the Cloud team is covering other runtimes soon, so stay tuned. Also check out other videos in the broader Serverless Expeditions series.

Introducing the Amazon EC2 Spot Ready Software Products

The new Amazon EC2 Spot Ready specialization helps customers identify validated AWS Partner software products that support Amazon EC2 Spot Instances, a compute purchase option that allows customers to utilize spare EC2 capacity at a discounted price from on demand (up to 90%). Amazon EC2 Spot Ready ensures that customers have a well-architected and cost-optimized solution to help them benefit from EC2 Spot savings for their workloads.

Amazon GuardDuty RDS Protection now in preview

Amazon GuardDuty now offers threat detection for Amazon Aurora to identify potential threats to data stored in Aurora databases. Amazon GuardDuty RDS Protection profiles and monitors access activity to existing and new databases in your account, and uses tailored machine learning models to accurately detect suspicious logins to Aurora databases. Once a potential threat is detected, GuardDuty generates a security finding that includes database details and rich contextual information on the suspicious activity, is integrated with Aurora for direct access to database events without requiring you to modify your databases, and is designed to not affect database performance.

Amazon Athena now supports Apache Spark

Amazon Athena now supports Apache Spark, a popular open-source distributed processing system that is optimized for fast analytics workloads against data of any size. Athena is an interactive query service that helps you query petabytes of data wherever it lives, such as in data lakes, databases, or other data stores. With Amazon Athena for Apache Spark, you get the streamlined, interactive, serverless experience of Athena with Spark, in addition to SQL.