Azure Archives - KenkoGeek

20 Sep 2019
By editor
Categories: Azure, Database

Three ways to leverage composite indexes in Azure Cosmos DB

Composite indexes were introduced in Azure Cosmos DB at Microsoft Build 2019. With our latest service update, additional query types can now leverage composite indexes. In this post, we’ll explore composite indexes and highlight common use cases.

Index types in Azure Cosmos DB

Azure Cosmos DB currently has the following index types that are used for the following types of queries:

Range indexes:

Equality queries
Range queries
ORDER BY queries on a single property
JOIN queries

Spatial indexes:

Geospatial functions

Composite indexes:

ORDER BY queries on multiple properties
Queries with a filter as well as an ORDER BY clause
Queries with a filter on two or more properties

Composite index use cases

By default, Azure Cosmos DB will create a range index on every property. For many workloads, these indexes are enough, and no further optimizations are necessary. Composite indexes can be added in addition to the default range indexes. Composite indexes have both a path and order (ASC or DESC) defined for each property within the composite index.

ORDER BY queries on multiple properties

If a query has an ORDER BY clause with two or more properties, a composite index is required. For example, the following query requires a composite index defined on age and name (age ASC, name ASC):

SELECT * FROM c WHERE c.age ASC, c.name ASC

This query will sort all results in ascending order by the value of the age property. If two documents have the same age value, the query will sort the documents by name.

Queries with a filter as well as an ORDER BY clause

If a query has a filter as well as an ORDER BY clause on different properties, a composite index will improve performance. For example, the following query will require fewer request units (RU’s) if a composite index on name and age is defined and the query is updated to include the name in the ORDER BY clause:

Original query utilizing range index:

SELECT * FROM c WHERE c.name = “Tim” ORDER BY c.age ASC

Revised query utilizing a composite index on name and age:

SELECT * FROM c WHERE c.name = “Tim” ORDER BY c.name ASC, c.age ASC

While a composite index will significantly improve query performance, you can still run the original query successfully without a composite index. When you run the revised query with a composite index, it will sort documents by the age property. Since all documents matching the filter have the same name value, the query will return them in ascending order by age.

Queries with a filter on multiple properties

If a query has a filter with two or more properties, adding a composite index will improve performance.

Consider the following query:

SELECT * FROM c WHERE c.name = “Tim” and c.age > 18

In the absence of a composite index on (name ASC, and age ASC), we will utilize a range index for this query. We can improve the efficiency of this query by creating a composite index for name and age.

Queries with multiple equality filters and a maximum of one range filter (such as >,<, <=, >=, !=) will utilize the composite index. In some cases, if a query can’t fully utilize a composite index, it will use a combination of the defined composite indexes and range indexes. For more information, reference our indexing policy documentation.

Composite index performance benefits

We can run some sample queries to highlight the performance benefits of composite indexes. We will use a nutrition dataset that is used in Azure Cosmos DB labs.

In this example, we will optimize a query that has a filter as well as an ORDER BY clause. We will start with the default indexing policy which indexes all properties with a range index. Executing the following query as referenced in the image below in the Azure Portal, we observe the query metrics:

Query metrics:

This query, with the default indexing policy, required 21.8 RU’s.

Adding a composite index on foodGroup and _ts and updating the query text to include foodGroup in the ORDER BY clause significantly reduced the query’s RU charge.

Query metrics:

After adding a composite index, the query’s RU charge decreased from 21.8 RU’s to only 4.07 RU’s. This query optimization will be particularly impactful as the total data size increases. The benefits of a composite index are significant when the properties in the ORDER BY clause have a high cardinality.

Creating composite indexes

You can learn more about creating composite indexes in this documentation. It’s simple to update the indexing policy directly through the Azure Portal. While creating a composite index for data that’s already in Azure Cosmos DB, the index update will utilize the RU’s leftover from normal operations. After the new indexing policy is defined, Azure Cosmos DB will automatically index properties with a composite index as they’re written.

Explore whether composite indexes will improve RU utilization for your existing workloads on Azure Cosmos DB.

19 Sep 2019
By editor
Categories: Announcements, Azure, Storage, Backup & Recovery, Updates

Azure Files premium tier gets zone redundant storage

Azure Files premium tier is now zone redundant!

We’re excited to announce the general availability of zone redundant storage (ZRS) for Azure Files premium tier. Azure Files premium tier with ZRS replication enables highly performant, highly available file services, that are built on solid-state drives (SSD).

Azure Files ZRS premium tier should be considered for managed file services where performance and regional availability are critical for the business. ZRS provides high availability by synchronously writing three replicas of your data across three different Azure Availability Zones, thereby protecting your data from cluster, datacenter, or entire zone outage. Zonal redundancy enables you to read and write data even if one of the availability zones is unavailable.

With the release of the ZRS for Azure Files premium tier, premium tier now offers two sets of durability options to meet your storage needs, zone redundant storage (ZRS) for intra-region high availability and locally-redundant storage (LRS) for lower-cost single region durable storage.

Getting started

You can create ZRS Azure premium files account through Azure Portal, Azure CLI, or Azure PowerShell.

Azure Files premium tier requires FileStorage as the account kind. To create a ZRS account in the Azure Portal, set the following properties:

Currently, ZRS option for Azure Files premium tier is available in West Europe and we will be gradually expanding the regional coverage. Stay up to date on the premium tier ZRS region availability through the Azure documentation.

Migration from LRS premium files account to ZRS premium files account requires manual copy or movement of data from an existing LRS account to a new ZRS account. Live account migration on request is not supported yet. Please check the migration documentation for the latest information.

Refer to the pricing page for the latest pricing information.

To learn more about premium tier, visit Azure Files premium tier documentation. Give it a try and share your feedback on the Azure Storage forum or email us at [email protected].

Happy sharing!

18 Sep 2019
By editor
Categories: Azure, Big Data, Developer

HDInsight support in Azure CLI now out of preview

We are pleased to share that support for HDInsight in Azure CLI is now generally available. The addition of the az hdinsight command group allows you to easily manage your HDInsight clusters using simple commands while taking advantage of all that Azure CLI has to offer, such as cross-platform support and tab completion.

Key Features

Cluster CRUD: Create, delete, list, resize, show properties, and update tags for your HDInsight clusters.
Script actions: Execute script actions, list and delete persistent script actions, promote ad-hoc script executions to persistent script actions, and show the execution history of script action runs.
Manage Azure Monitor integration: Enable, disable, and show the status of Azure Monitor integration on HDInsight clusters.
Applications: Create, delete, list, and show properties for applications on your HDInsight clusters.
Core usage: View available core counts by region before deploying large clusters.

Create an HDInsight cluster using a single, simple Azure CLI command

Azure CLI benefits

Cross platform: Use Azure CLI on Windows, macOS, Linux, or the Azure Cloud Shell in a browser to manage your HDInsight clusters with the same commands and syntax across platforms.
Tab completion and interactive mode: Autocomplete command and parameter names as well as subscription-specific details like resource group names, cluster names, and storage account names. Don’t remember your 88-character storage account key off the top of your head? Azure CLI can autocomplete that as well!
Customize output: Make use of Azure CLI’s globally available arguments to show verbose or debug output, filter output using the JMESPath query language, and change the output format between json, tab-separated values, or ASCII tables, and more.

Getting started

You can get up and running to start managing your HDInsight clusters using Azure CLI with 3 easy steps.

Install Azure CLI for Windows, macOS, or Linux. Alternatively, you can use Azure Cloud Shell to use Azure CLI in a browser.
Log in using the az login command.
Take a look at our reference documentation, “az hdinsight” or run az hdinsight -h to see a full list of supported HDInsight commands and descriptions and start using Azure CLI to manage your HDInsight clusters.

About HDInsight

Azure HDInsight is an easy, cost-effective, enterprise-grade service for open source analytics that enables customers to easily run popular open source frameworks, such as Apache Hadoop, Spark, Kafka, and more. The service is available in 28 public regions and Azure Government Clouds in the US, Germany, and China. Azure HDInsight powers mission-critical applications in a wide variety of sectors and enables a wide range of use cases including ETL, streaming, and interactive querying.

18 Sep 2019
By editor
Categories: Azure, Security

SAP on Azure Architecture – Designing for security

This blog post was contributed to by Chin Lai The, Technical Specialist, SAP on Azure.

This is the first in a four-part blog series on designing a great SAP on Azure Architecture, and will focus on designing for security.

Great SAP on Azure Architectures are built on the pillars of security, performance and scalability, availability and recoverability, and efficiency and operations.

Microsoft investments in Azure Security

Microsoft invests $1 billion annually on security research and development and has 3,500 security professionals employed across the company. Advanced AI is leveraged to analyze 6.5 trillion global signals from the Microsoft cloud platforms and detect and respond to threats. Enterprise-grade security and privacy are built into the Azure platform including enduring, rigorous validation by real world tests, such as the Red Team exercises. These tests enable Microsoft to test breach detection and response as well as accurately measure readiness and impacts of real-world attacks, and are just one of the many operational processes that provide best-in-class security for Azure.

Azure is the platform of trust, with 90 compliance certifications spanning nations, regions, and specific industries such as health, finance, government, and manufacturing. Moreover, Azure Security and Compliance Blueprints can be used to easily create, deploy, and update your compliant environments.

Security – a shared responsibility

It’s important to understand the shared responsibility model between you as a customer and Microsoft. The division of responsibility is dependent on the cloud model used – SaaS, PaaS, or IaaS. As a customer, you are always responsible for your data, endpoints, account/access management, irrespective of the chosen cloud deployment.

SAP on Azure is delivered using the IaaS cloud model, which means security protections are built into the service by Microsoft at the physical datacenter, physical network, and physical hosts. However, for all areas beyond the Azure hypervisor i.e. the operating systems and applications, customers need to ensure their enterprise security controls are implemented.

Key security considerations for deploying SAP on Azure

Resource based access control & resource locking

Role-based access control (RBAC) is an authorization system which provides fine-grained access for the management of Azure resources. RBAC can be used to limit access and control permissions on Azure resources for the various teams within your IT operations.

For example, the SAP basis team members can be permissioned to deploy virtual machines (VMs) into Azure virtual networks (VNets). However, the SAP basis team can be restricted from creating or configuring VNets. On the flip side, members of the networking team can create and configure VNets, however, they are prohibited from deploying or configuring VMs in VNets where SAP applications are running.

We recommend validating and testing the RBAC design early during the lifecycle of your SAP on Azure project.

Another important consideration is Azure resource locking which can be used to prevent accidental deletion or modification of Azure resources such as VMs and disks. It is recommended to create the required Azure resources at the start of your SAP project. When all additons, moves, and changes are finished, and the SAP on Azure deployment is operational all resources can be locked. Following, only a super administrator can unlock a resource and permit the resource (such as a VM) to be modified.

Secure authentication

Single-sign-on (SSO) provides the foundation for integrating SAP and Microsoft products, and for years Kerberos tokens from Microsoft Active Directory have been enabling this capability for both SAP GUI and web-browser based applications when combined with third party security products.

When a user logs onto their workstation and successfully authenticates against Microsoft Active Directory they are issued a Kerberos token. The Kerberos token can then be used by a 3rd party security product to handle the authentication to the SAP application without the user having to re-authenticate. Additionally, data in transit from the users front-end towards the SAP application can also be encrypted by integrating the security product with secure network communications (SNC) for DIAG (SAP GUI), RFC and SPNEGO for HTTPS.

Azure Active Directory (Azure AD) with SAML 2.0 can also be used to provide SSO to a range of SAP applications and platforms such as SAP NetWeaver, SAP HANA and the SAP Cloud Platform.

This video demonstrates the end-to-end enablement of SSO between Azure AD and SAP NetWeaver

Protecting your application and data from network vulnerabilities

Network security groups (NSG) contain a list of security rules that allow or deny network traffic to resources within your Azure VNet. NSGs can be associated to subnets or individual network interfaces attached to VMs. Security rules can be configured based on source/destination, port, and protocol.
NSG’s influence network traffic for the SAP system. In the diagram below, three subnets are implemented, each having an NSG assigned – FE (Front-End), App and DB.

A public internet user can reach the SAP Web-Dispatcher over port 443
The SAP Web-Dispatcher can reach the SAP Application server over port 443
The App Subnet accepts traffic on port 443 from 10.0.0.0/24
The SAP Application server sends traffic on port 30015 to the SAP DB server
The DB subnet accepts traffic on port 30015 from 10.0.1.0/24.
Public Internet Access is blocked on both App Subnet and DB Subnet.

SAP deployments using the Azure virtual Ddatacenter architecture will be implemented using a hub and spoke model. The hub VNet is the central point for connectivity where an Azure Firewall or other type of network virtual appliances (NVA) is implemented to inspect and control the routing of traffic to the spoke VNet where your SAP applications reside.

Within your SAP on Azure project, it is recommended to validate that that inspection devices and NSG security rules are working as desired, this will ensure that your SAP resources are shielded appropriately against network vulnerabilities.

Maintaining data integrity through encryption methods

Azure Storage service encryption is enabled by default on your Azure Storage account where it cannot be disabled. Therefore, customer data at rest on Azure Storage is secured by default where data is encrypted/decrypted transparently using 256-bit AES. The encrypt/decrypt process has no impact on Azure Storage performance and is cost free. You have the option of Microsoft managing the encryption keys or you can manage your own keys with Azure Key Vault. Azure Key Vault can be used to manage your SSL/TLS certificates which are used to secure interfaces and internal communications within the SAP system.

Azure also offers virtual machine disk encryption using BitLocker for Windows and DM-Crypt for Linux to provide volume encryption for virtual machine operating system and data disks. Disk encryption is not enabled by default.

Our recommended approach to encrypting your SAP data at rest is as follows:

Azure Disk Encryption for SAP Application servers – operating system disk and data disks.
Azure Disk Encryption for SAP Database servers – operating system disks and those data disk not used by the DBMS.
SAP Database servers – leverage Transparent Data Encryption offered by the DBMS provider to secure your data and log files and to ensure the backups are also encrypted.

Hardening the operating system

Security is a shared responsibility between Microsoft and you as a customer where your customer specific security controls need to be applied to the operating system, database, and the SAP application layer. For example, you need to ensure the operating system is hardened to eradicate vulnerabilities which could lead to attacks on the SAP database.

Windows, SUSE Linux, RedHat Linux and others are supported for running SAP applications on Azure and various images of these operating systems are available within the Azure Marketplace. You can further harden these images to comply with the security policies of your enterprise and within the guidance from the Center of Internet Security (CIS)- Microsoft Azure foundations benchmark.

Enterprises generally have operational processes in place for updating and patching of their IT software including the operating system. Once an operating system vulnerability has been exposed, it is published in security advisories and usually remediated quickly. The operating system vendor regularly provides security updates and patches. You can use the Update Management solution in Azure Automation to manage operating system updates for your Windows and Linux VMs in Azure. A best practice approach is a selective installation of security updates for the operating system on a regular cadence and installation of other updates such as new features during maintenance windows.

Learn more

Within this blog we have touched upon a selection of security topics as they relate to deploying SAP on Azure. Incorporating solid security practices will lead to a secure SAP deployment on Azure.

Azure Security Center is the place to learn about the best practices for securing and monitoring your Azure deployments. Also, please read the Azure Security Center technical documentation along with Azure Sentinel to understand how to detect vulnerabilities, generate alerts when exploitations have occurred and provide guidance on remediation.

In blog number two in our series we will cover designing for performance and scalability.

17 Sep 2019
By editor
Categories: Azure, Storage, Backup & Recovery

Announcing user delegation SAS tokens preview for Azure Storage Blobs

Cloud storage often serves as a content source for browser and mobile applications. This is typically achieved using application-issued, pre-authorized URLs which provide time-limited access directly to specific content without requiring a service to proxy this access.

Azure Storage supports this pattern through the use of shared access signature tokens (SAS tokens). These tokens grant specific, time-limited access to storage objects by signing an authorization statement using the storage account access key, which is controlled by account administrators. While this approach provides the required limited access to clients, it sometimes represents an over-provisioning of access to these token-issuing services since this provides full control over the entire account where they may only require read access to specific content.

Today we are launching a preview for a new kind of SAS token, user delegation SAS tokens. By extending the recent release of Azure AD and Azure role-based access control (RBAC) for Azure Storage, lower-privileged users and services can now delegate subsets of their access to clients, using this new type of pre-authorized URL. Clients retrieve a user delegation key tied to their Azure Active Directory (AD) account, and then use it to create SAS tokens granting a subset of their own access rights.

This means, for example, that an application component with only read access to end-user content could be configured to issue short lived read-only URLs to clients without the risks involved with storing and using the powerful account access key. Azure Storage access logs will also reflect client use of these SAS tokens as associated with the Azure AD principal of this application component.

During this preview, you can generate user delegation SAS tokens with your own code or use Azure PowerShell or Azure CLI. Remember, you will first need to grant RBAC permissions to access data to the user account that will generate the SAS token. Learn more about granting RBAC access to your blob data in our documentation here.

How to create a user delegation SAS token

The preview capability is available now for everyone, for non-production use. No registration is required.

For developers, here is an example using .NET code to generate a user-delegation SAS token. It also shows our new .NET client libraries for Storage and integrated Azure.Identity libraries.

client = new BlobServiceClient(accountUri, new DefaultAzureCredential());

//define permission set to read blob, valid from low for a specified number of minutes
BlobSasBuilder builder = new BlobSasBuilder()
{
    ContainerName = containerName,
    BlobName = blobName,
    Permissions = "r",
    Resource = "b",
    StartTime = DateTimeOffset.UtcNow,
    ExpiryTime = DateTimeOffset.UtcNow.AddMinutes(tokenLifetime)
};

//refresh user-delegation key as necessary, valid for up to a maximum of 7 days
if (currentUdk == null || currentUdk.SignedExpiry <= builder.ExpiryTime)
{
    currentUdk = client.GetUserDelegationKey(DateTimeOffset.UtcNow, DateTimeOffset.UtcNow.AddDays(7)).Value;
}

//generate a signed SAS token using the user-delegation key
string sasToken = builder.ToSasQueryParameters(currentUdk, accountName).ToString();

You can find a complete working example in the Azure Storage documentation.

Users can also use our command line tool previews to generate user-delegation SAS tokens. Here is an example using Azure CLI to generate a read-only SAS based on the user’s credentials:

$ az login
$ az storage blob generate-sas 
> --account-name myaccount 
> --container-name container 
> --name file.txt 
> --permissions r 
> --expiry 2019-08-30 
> --auth-mode login 
> --as-user 
> --full-uri
Argument '--as-user' is in preview. It may be changed/removed in a future release.

https://myaccount.blob.core.windows.net/container/file.txt?se=2019-08-30&sp=r&sv=2018-11-09&sr=b&skoid=8c93ed4c-3e11-43f4-9307-3664c9c16554&sktid=9341f370-b982-47de-b7c1-8dbe61328559&skt=2019-08-28T22%3A57%3A38Z&ske=2019-08-30T00%3A00%3A00Z&sks=b&skv=2018-11-09&sig=7trGEakY86Uj5rXsH2ApiyCZfxFgNnh6NFy4wcnmfmQ%3D

Check out our documentation for more detailed examples in Azure PowerShell and Azure CLI.

User delegation SAS tokens for Azure Blobs allow for issuing pre-authorized URLs from lower privileged identities, and are available in preview in all Azure clouds and locations. Please read more about them in our documentation, and give them a try. We would love to hear your feedback at [email protected].

13 Sep 2019
By editor
Categories: Azure, Media Services & CDN

Azure Media Services’ new AI-powered innovation

Animated character recognition, multilingual speech transcription and more now available

At Microsoft, our mission is to empower every person and organization on the planet to achieve more. The media industry exemplifies this mission. We live in an age where more content is being created and consumed in more ways and on more devices than ever. At IBC 2019, we’re delighted to share the latest innovations we’ve been working on and how they can help transform your media workflows. Read on to learn more, or join our product teams and partners at Hall 1 Booth C27 at the RAI in Amsterdam from September 13th to 17th.

Video Indexer adds support for animation and multilingual content

We made our award winning Azure Media Services Video Indexer generally available at IBC last year, and this year it’s getting even better. Video Indexer automatically extracts insights and metadata such as spoken words, faces, emotions, topics and brands from media files, without you needing to be a machine learning expert. Our latest announcements include previews for two highly requested and differentiated capabilities for animated character recognition and multilingual speech transcription, as well as several additions to existing models available today in Video Indexer.

Animated character recognition

Animated content or cartoons are one of the most popular content types, but standard AI vision models built for human faces do not work well with them, especially if the content has characters without human features. In this new preview solution, Video Indexer joins forces with Microsoft’s Azure Custom Vision service to provide a new set of models that automatically detect and group animated characters and allow customers to then tag and recognize them easily via integrated custom vision models. These models are integrated into a single pipeline, which allows anyone to use the service without any previous machine learning skills. The results are available through the no-code Video Indexer portal or the REST API for easy integration into your own applications.

We built these animated character models in collaboration with select customers who contributed real animated content for training and testing. The value of the new functionality is well articulated by Andy Gutteridge, Senior Director, Studio & Post-Production Technology at Viacom International Media Networks, which was one of the data contributors: “The addition of reliable AI-based animated detection will enable us to discover and catalogue character metadata from our content library quickly and efficiently. Most importantly, it will give our creative teams the power to find the content they want instantly, minimize time spent on media management and allow them to focus on the creative.”

To get started with animated character recognition, please visit our documentation page.

Multilingual identification and transcription

Some media assets like news, current affairs, and interviews contain audio with speakers using different languages. Most existing speech-to-text capabilities require the audio recognition language to be specified in advance, which is an obstacle to transcribing multilingual videos. Our new automatic spoken language identification for multiple content feature leverages machine learning technology to identify the different languages used in a media asset. Once detected, each language segment undergoes an automatic transcription process in the language identified, and all segments are integrated back together into one transcription file consisting of multiple languages.

The resulting transcription is available both as part of Video Indexer JSON output and as closed-caption files. The output transcript is also integrated with Azure Search, allowing you to immediately search across videos for the different language segments. Furthermore, the multi-language transcription is available as part of the Video Indexer portal experience so you can view the transcript and identified language by time, or jump to the specific places in the video for each language and see the multi-language transcription as captions as a video is played. You can also translate the output back-and-forth into 54 different languages via the portal and API.

Read more about the new multilingual option and how to use it in Video Indexer in our documentation.

Additional updated and improved models

We are also adding new and improving existing models within Video Indexer, including:

Extraction of people and locations entities

We’ve extended our current brand detection capabilities to also incorporate well-known names and locations, such as the Eiffel Tower in Paris or Big Ben in London. When these appear in the generated transcript or on-screen via optical character recognition (OCR), a specific insight is created. With this new capability, you can review and search by all people, locations and brands that appeared in the video, along with their timeframes, description, and a link to our Bing search engine for more information.

Editorial shot detection model

This new feature adds a set of “tags” in the metadata attached to an individual shot in the insights JSON to represent its editorial type (such as wide shot, medium shot, close up, extreme close up, two shot, multiple people, outdoor and indoor, etc.). These shot-type characteristics come in handy when editing videos into clips and trailers as well as when searching for a specific style of shots for artistic purposes.

Explore and read more about editorial shot type detection in Video Indexer.

Expanded granularity of IPTC mapping

Our topic inferencing model determines the topic of videos based on transcription, optical character recognition (OCR), and detected celebrities even if the topic is not explicitly stated. We map these inferred topics to four different taxonomies: Wikipedia, Bing, IPTC, and IAB. With this enhancement, we now include level-2 IPTC taxonomy.

Tanking advantage of these enhancements is as easy as re-indexing your current Video Indexer library.

New live streaming functionality

We are also introducing two new live-streaming capabilities in preview to Azure Media Services.

Live transcription supercharges your live events with AI

Using Azure Media Services to stream a live event, you can now get an output stream that includes an automatically generated text track in addition to the video and audio content. This text track is created using AI-based live transcription of the audio of the contribution feed. Custom methods are applied before and after speech-to-text conversion in order to improve the end-user experience. The text track is packaged into IMSC1, TTML, or WebVTT, depending on whether you are delivering in DASH, HLS CMAF, or HLS TS.

Live linear encoding for 24/7 over-the-top (OTT) channels

Using our v3 APIs, you can create, manage, and stream live channels for OTT services and take advantage of all the other features of Azure Media Services like live to video on demand (VOD), packaging, and digital rights management (DRM).

To try these preview features, please visit the Azure Media Services Community page.

New packaging features

Support for audio description tracks

Broadcast content frequently has an audio track that contains verbal explanations of on-screen action in addition to the normal program audio. This makes programming more accessible for vision-impaired viewers, especially if the content is highly visual. The new audio description feature enables a customer to annotate one of the audio tracks to be the audio description (AD) track, which in turn can be used by players to make the AD track discoverable by viewers.

ID3 metadata insertion

In order to signal the insertion of advertisements or custom metadata events on a client player, broadcasters often make use of timed metadata embedded within the video. In addition to SCTE-35 signaling modes, we now also support ID3v2 or other custom schemas defined by an application developer for use by the client application.

Microsoft Azure partners demonstrate end-to-end solutions

Bitmovin is debuting its Bitmovin Video Encoding and Bitmovin Video Player on Microsoft Azure. Customers can now use these encoding and player solutions on Azure and leverage advanced functionality such as 3-pass encoding, AV1/VVC codec support, multi-language closed captions, and pre-integrated video analytics for QoS, ad, and video tracking.

Evergent is showing its User Lifecycle Management Platform on Azure. As a leading provider of revenue and customer lifecycle management solutions, Evergent leverages Azure AI to enable premium entertainment service providers to improve customer acquisition and retention by generating targeted packages and offers at critical points in the customer lifecycle.

Haivision will showcase its intelligent media routing cloud service, SRT Hub, that helps customers transform end-to-end workflows starting with ingest using Azure Data Box Edge and media workflow transformation using Hublets from Avid, Telestream, Wowza and Cinegy, and Make.tv.

SES has developed a suite of broadcast-grade media services on Azure for its satellite connectivity and managed media services customers. SES will show solutions for fully managed playout services, including master playout, localized playout and ad detection and replacement, and 24×7 high-quality multichannel live encoding on Azure.

SyncWords is making its caption automation technology and user-friendly cloud-based tools available on Azure. These offerings will make it easier for media organizations to add automated closed captioning and foreign language subtitling capabilities to their real-time and offline video processing workflows on Azure.

Global design and technology services company Tata Elxsi has integrated TEPlay, its OTT platform SaaS, with Azure Media Services to deliver OTT content from the cloud. Tata Elxsi has also brought FalconEye, its quality of experience (QoE) monitoring solution that focuses on actionable metrics and analytics, to Microsoft Azure.

Verizon Media is making its streaming platform available in beta on Azure. Verizon Media Platform is an enterprise-grade managed OTT solution including DRM, ad insertion, one-to-one personalized sessions, dynamic content replacement, and video delivery. The integration brings simplified workflows, global support and scale, and access to a range of unique capabilities available on Azure.

Many of our partners will also be presenting in the theater at our booth, so make sure you stop by to catch them!

Short distance, big impact

We are proud to support the 4K 4Charity Fun Run as a gold sponsor. This is a running and walking event held at various media industry events since 2014, and it raises awareness and financial support for non-profits focused on increased diversity and inclusion. Register and come join us on Saturday, September 14th, at 7:30am at the Amstelpark in Amsterdam.

Don’t miss out

There’s a lot more going on at the Microsoft booth this IBC. To learn more, read about how the community of our customers and partners are innovating on Azure in media and entertainment, or better yet come and join us in Hall 1 Booth C27. If you won’t be there, we’re sorry we’ll miss you, but you can try Video Indexer and Azure Media Services for yourself by following the links.

13 Sep 2019
By editor
Categories: Azure, Azure Maps, Internet of Things

Expanded Azure Maps coverage, preview of Azure Maps feedback site, and more

This blog post was co-authored by Ricky Brundritt, Principal Technical Program Manager, Azure Maps.

Azure Maps services continue to expand our support for Microsoft enterprise customers’ needs in Azure. And, we’ve been busy expanding our capabilities. Today we’re announcing Azure Maps is now available in Argentina, India, Morocco, and Pakistan. We have also launched a new Azure Maps data feedback site that is now in preview. In addition, we’re also introducing several enhancements that are available via our Representational state transfer (REST) services and Azure Maps web and Android SDKs.

Here is a run-down of the new features:

Azure Maps is available in new countries and regions

Azure Maps is now available in Argentina, India, Morocco, and Pakistan and these regions require specific consideration for using maps. Azure Maps will now empower our customers to use the appropriate map views in these regions. To learn more about how to request data via our REST services and SDKs for the new regions and countries listed above, please see our Azure Maps localization page.

Introducing preview of Azure Maps data feedback site

To serve the freshest map data as possible to our customers and as an easy way to provide map data feedback, we’re introducing the Azure Maps data feedback site. The new site empowers our customers to provide direct data feedback, especially on business points of interest and residential addresses. The feedback goes directly to our data providers and their map editors who can quickly evaluate and incorporate feedback into our mapping products. To learn how to provide different types of feedback using the Azure Maps feedback site, please see our How-to guide.

REST service enhancements

Point of interest data updates

When requesting point of interest data, you might want to restrict the results to specific brands. For example, your scenario is to only show gas stations under a specific brand to your end users. To support this, we’ve added the capability to include one or multiple brands in your request to limit the search results. To learn more, please see our How-to Guide article where we share useful tips to call data via Azure Maps search services.

In addition, Azure Maps now returns hours of operation for points of interest like business listings. We return the opening hours for the next week, starting with the current day in the local time of the point of interest. This information can be used to better optimize your planned routes, and for example, show end users store locations that are open during a specific timeframe.

Sunset and sunrise times

According to a recent report from the Global Alliance for Buildings and Construction, buildings construction and operations account for 36 percent of global final energy use and nearly 40 percent of energy-related carbon dioxide emissions when upstream power generation is considered. To create impact with IoT and help to combat climate change and optimize buildings for energy efficiency, Get Timezone by Coordinates API now returns sunset and sunrise times for a given coordinate location. Developers can automate device messages in their IoT solutions, for example, by building rules to schedule heating and cooling by using sunrise and sunset times combined with telemetry messages from a variety of devices and sensors.

Cartography and styling updates

Point of interest data rendering

To provide richer and more informative map data content, we’ve pushed up certain point of interest data so that certain categories appear at higher levels. As a result, airport icons are rendered at zoom levels 10 to 22.

Point of interest icons for important tourist attractions like museums, and railway and metro stations are displayed on zoom levels 12 to 22. In addition, universities, colleges, and schools are shown on zoom levels 13 to 22.

State boundaries and abbreviated state names

To improve usability and give more detailed views, state boundaries are pushed up in the data so that they appear already at zoom level 3. Abbreviated state names are also now shown in zoom level.

Blank map styles in web SDK

Often it is useful to be able to visualize data on top of a blank canvas or to replace the base maps with custom tile layers. With this in the mind the Azure Maps web SDK now supports two new map styles; blank and blank_accessible. The blank map style will not render any base map data, nor will it update the screen reader on where the map is centered over. The blank_ accessible style will continue to provide screen reader updates with location details of where the map is located, even though the base map is not displayed. Please note, you can change the background color of web SDK by using the CSS background-color style of the map DIV element.

Web SDK enhancements

The Azure Maps team has made many additions and improvements to the web SDK. Below is a closer look at some of the key improvements.

Cluster aggregates

Clustering of point data based on zoom level can be done to reduce the visual clutter on the map and make it easier to make sense of the data. Often clusters are represented using a symbol with the number of points that are within the cluster, however sometimes you may want to further customize the style of clusters based on a metric like the total revenue of all points within a cluster. With cluster aggregates, custom properties can be created and populated using an aggregate expression. To learn more please see our Azure Maps documentation.

Aggregating data in clusters

Image templates

The Azure Maps web SDK uses WebGL for rendering most data on the map. Symbol layers can be used to render points on the map with an image, line layers can have images rendered along it, and polygon layers can be rendered with a fill pattern image. In order to ensure good performance, these images need to be loaded into the map image sprite resource before rendering. The web SDK already provides a couple of images of markers in a handful of colors, however, there is an infinite number of color combinations that developers may want to use. With this in mind we have ported the SVG template functionality for HTML markers over to the image sprite and have added 42 image templates, 27 symbol icons, and 15 polygon fill patterns. You can easily define a primary and secondary color as well as a scale for each template when loading it into the map image sprite. These templates can also be used with HTML markers as well. Check out our documentation and see our Try it now tool to learn more.

Images can be used HTML markers and various layers within the Azure Maps Web SDK

Additional notable improvements to the web SDK:

Accessibility improvements – The team has spent a lot of time improving accessibility in the web SDK and ensuring that every user is able to use the map. A major part of this consisted of leveraging the vector tiles of the base map so that we can provide highly accurate descriptions of what the map is rendering.
Limit spinning of the globe – By default the map mimics a globe by allowing the user to infinitely scroll the map west or east. When the user is zoomed out, sometimes the map will render additional copies of the globe to fill in the blank space. This is great for most scenarios, but some developers prefer having a single copy of the globe that doesn’t scroll infinitely. Now this can be configured using the new renderWorldCopies map option.
Easily show all map styles in style picker – Up until now, if you wanted to show all map styles in the style picker control you had to list them all in an array in the mapStyles option. Now you simply set this option to “all.”
Image overlay georeferencing tools – When georeferencing an image to overlay on the map, sometimes all you have is some reference points (i.e. pixels to positions) which might not be the corners of the image. We added some functions which can be used to correctly georeference the image. We also added tools for reprojecting between pixels and positions relative to the image. For example, if you have an image of a floor plan displayed on the map, you can take any map position and determine its pixel coordinate on the original image and vice versa.
New spatial math functions – Several new spatial math functions have been added. One of the new spatial math functions we added will calculate the closest point to a location that falls on the edge of another geometry object. This has a lot of use cases, such as basic snapping of points to lines or simply knowing how far off the path something is.
Pitch touch support – You can now pitch the map using touch, with two-finger drag up/down.
Popup customizations – Up until now you could only have a popup with a white background and pointer arrow. Now you can set the color of the popup and optionally hide the pointer arrow. Popups can also be made draggable now too!
Shape and Data source events – New events for tracking changes to shapes and data sources.

Tile layers in the Android SDK

The Azure Maps team released an Android SDK into preview earlier this year. It is able to render point, line, and polygon data. The team has now added support for rendering tile layers. Tile layers are a great way to visualize large data sets on the map. Not only can a tile layer be generated from an image, but vector data can also be rendered as a tile layer too. By rendering vector data as a tile layer, the map control only needs to load the tiles which can be much smaller in file size than the vector data they represent. This technique is used by many who need to render millions of rows of data on the map.

Rendering tile layers within the Azure Maps Android SDK

We want to hear from you!

We are always working to grow and improve the Azure Maps platform and want to hear from you. We’re here to help and want to make sure you get the most out of the Azure Maps platform.

Have a feature request? Add it or vote up the request on our feedback site.
Having an issue getting your code to work? Have a topic you would like us to cover on the Azure blog? Ask us on the Azure Maps forums.
Looking for code samples or wrote a great one you want to share? Join us on GitHub.
To learn more, read the Azure Maps documentation.

12 Sep 2019
By editor
Categories: Azure, Internet of Things

Five best practices for unlocking IoT value

Accenture and Avanade won the 2019 Microsoft Internet of Things Partner of the Year award this past spring. At the Microsoft Inspire partner conference in July, Brendan Mislin, Managing Director, Industry X.0 IoT Lead at Accenture, shared some insights and best practices that have helped this award-winning partner unlock the value of Azure IoT for our mutual customers.

Figure 1 Petrofac, a leading oilfield services company, leverages Accenture’s Connected Construction solution

Five of the best practices Mislin shared are below. For the complete discussion, view the interview.

1. Expect IoT opportunities in all industry segments

Partners should expect to use IoT technology in every industry from retail and manufacturing to energy and health. “Accenture doesn’t have a typical IoT project,” Brendan stated. “In the last 12 months, we’ve seen IoT technology used in many of the 40 industries in which we deliver business transformation solutions.”

Connected operations is the highest growth IoT segment for Accenture. “If I had to identify where IoT is really crossing the chasm and where Accenture has seen the most growth of its IoT implementations, it’s in connected operations,” Mislin shared. Accenture defines connected operations as connecting equipment within a factory or other industrial environment to the Cloud – or connecting multiple factories within the same company to the Cloud and then providing dashboards & advanced analytics that bring to life specific insights for various roles within each factory.

2. Market and sell business transformation—not IoT

Many IoT customers get stuck at the proof of concept stage. Observing this pattern resulted in Accenture shifting its approach.
“A few years ago, we made the conscious decision to stop selling IoT,” Mislin explained. “To move our clients forward, we realized we needed to help customers transform their businesses.” As Accenture delivered business transformation solutions, they implemented the best technology from IoT to artificial intelligence (AI), blockchain, cybersecurity, or augmented reality (AR/XR).

Real-life example: Accenture enabled PT SERA, an auto rental and leasing provider in Indonesia, to transform their rental services and be positioned as a digital visionary and leader in mobility through Accenture’s Connected Fleet solution. While the solution is IoT-based, Accenture leveraged technologies like AI and machine learning (ML) to maximize business value from the platform. “We’re using advanced analytics to build in native use cases like driver scoring (safety and eco index), driver fuel optimization, route optimization, vehicle predictive maintenance, geofencing and route violation, and more,” stated Mislin.

3. Understand and articulate project value

“In the early days of IoT, customers focused on running proofs of concept to test out IoT technology. In the early days of IoT, we thought it was great to get data in real-time from a factory and see it on our phones,” noted Mislin. “However, these projects seldom progressed to production.”

In today’s market, Accenture starts by articulating business value. Accenture first determines what insights can make positive changes to the way a client operates. For example, there might be several pieces of information that factory operators could leverage to increase the number of widgets produced on each shift.

Then Accenture looks at a client’s business to make sure the IoT improvements translate to increased revenue or profit. Just because the client can now produce more widgets, does not mean the client will automatically make more money. As more widgets are produced, for example, Accenture may work with a client to revamp its supply chain.

4. Leverage the Microsoft Azure IoT partner ecosystem

When it comes to IoT projects, one partner cannot do it alone. This is where a vibrant partner ecosystem is key. Mislin used the example of Accenture’s Connected Construction work at Petrofac, a leading oilfield services company, to illustrate the importance of a partner ecosystem.

“On the Petrofac Connected Construction project, Accenture worked with a wide variety of partners,” shared Mislin. “We had multiple hardware partners for tracking people and equipment indoors, outdoors, on sunny days, on cloudy days, near big steel objects, and out in the open desert. We had a partner providing the LoRa base stations, a partner for fiber optic cable transformations, and more.”

5. Innovate with Microsoft

“The pace of innovation from Microsoft has enabled us to deliver solutions that would have been previously unthinkable,” offered Mislin. In the Petrofac project, for example, Microsoft’s advances in Azure IoT Edge and Azure Stream Analytics gave Accenture the technology needed to build out a Connected Construction solution that would have been impossible last year.

Summary

These are five of the best practices shared by Brendan Mislin, Managing Director, Industry X.0 IoT Lead at Accenture, during the recent Inspire partner conference. For more, view the complete interview.

11 Sep 2019
By editor
Categories: Azure, Developer, DevOps, Management

Building cloud-native applications with Azure and HashiCorp

With each passing year, more and more developers are building cloud-native applications. As developers build more complex applications they are looking to innovators like Microsoft Azure and HashiCorp to reduce the complexity of building and operating these applications. HashiCorp and Azure have worked together on a myriad of innovations. Examples of this innovation include tools that connect cloud-native applications to legacy infrastructure and tools that secure and automate the continuous deployment of customer applications and infrastructure. Azure is deeply committed to being the best platform for open source software developers like HashiCorp to deliver their tools to their customers in an easy-to use, integrated way. Azure innovation like the managed applications platform that power HashiCorp’s Consul Service on Azure are great examples of this commitment to collaboration and a vibrant open source startup ecosystem. We’re also committed to the development of open standards that help these ecosystems move forward and we’re thrilled to have been able to collaborate with HashiCorp on both the CNAB (Cloud Native Application Bundle) and SMI (Service Mesh Interface) specifications.

Last year at HashiConf 2018, I had the opportunity to share how we had started to integrate Terraform and Packer into the Azure platform. I’m incredibly excited to get the opportunity to return this year to share how these integrations are progressing and to share a new collaboration on cloud native networking. With this new work we now have collaborations that help customers connect and operate their applications on Azure using HashiCorp technology.

Connect — HashiCorp Consul Service on Azure

After containers and Kubernetes, one of the most important innovations in microservices has been the development of the concept of a service mesh. Earlier this year we partnered with HashiCorp and others to announce the release of Service Mesh Interface, a collaborative, implementation agnostic API for the configuration and deployment of service mesh technology. We collaborated with HashiCorp to produce a control rules implementation of the traffic access control (TAC) using Consul Connect. Today we’re excited that Azure customers can take advantage of HashiCorp Consul Services on Azure powered by the Azure Managed Applications platform. HashiCorp Consul provides a solution to simplify and secure service networking and with this new managed offering, our joint customers can focus on the value of Consul while confident that the experts at HashiCorp are taking care of the management of the service. Reducing complexity for customers and enabling them to focus on cloud native innovation.

Provision — HashiCorp Terraform on Azure

HashiCorp Terraform is a great tool for doing declarative deployment to Azure. We’re seeing great momentum with adoption of HashiCorp Terraform on Azure as the number of customers has doubled since the beginning of the year – customers are using Terraform to automate Azure infrastructure deployment and operation in a variety of scenarios.

The momentum is fantastic on the contribution front as well with nearly 180 unique contributors to the Terraform provider for Azure Resource Manager. The involvement from the community with our increased 3-week cadence of releases (currently at version 1.32) ensures more coverage of Azure services by Terraform. Additionally, after customer and community feedback regarding the need for additional Terraform modules for Azure, we’ve been working hard at adding high quality modules and now have doubled the number of Azure modules in the terraform registry, bringing it to over 120 modules.

We believe all these additional integrations enable customers to manage infrastructure as code more easily and simplify managing their cloud environments. Learn more about Terraform on Azure.

Microsoft and HashiCorp are working together to provide integrated support for Terraform on Azure. Customers using Terraform on Microsoft’s Azure cloud are mutual customers, and both companies are united to provide troubleshooting and support services. This joint entitlement process provides collaborative support across companies and platforms while delivering a seamless customer experience. Customers using Terraform Provider for Azure can file support tickets to Microsoft support. Customers using Terraform on Azure support can file support tickets to Microsoft or HashiCorp.

Deploy — Collaborating on Cloud Native Application Bundles specification

One of the critical problems solved by containers is the hermetic packaging of a binary into a package that is easy to share and deploy around the world. But a cloud-native application is more than a binary, and this is what led to the co-development, with HashiCorp and others, of the Coud Native Application Bundle (CNAB) specification. CNABs allow you to package images alongside configuration tools like Terraform and other artifacts to allow a user to seamlessly deploy an application from a single package. I’ve been excited to see the community work together to build the specification to a 1.0 release that shows CNAB is ready for all of the world’s deployment needs. Congratulations to the team on the work and the fantastic partnership.

If you want to learn more about the ways in which Azure and HashiCorp collaborate to make cloud-native development easier, please check out the links below:

HashiCorp announcement regarding the HashiCorp Consul Service offering on Azure
Terraform on Azure
Cloud Native application bundles (CNAB)

11 Sep 2019
By editor
Categories: Azure, Big Data

Monitoring on Azure HDInsight part 4: Workload metrics and logs

This is the fourth blog post in a four-part series on monitoring on Azure HDInsight. Monitoring on Azure HDInsight Part 1: An Overview discusses the three main monitoring categories: cluster health and availability, resource utilization and performance, and job status and logs. Part 2 is centered on the first topic, monitoring cluster health and availability. Part 3 discussed monitoring performance and resource utilization. This blog covers the third of those topics, workload metrics and logs, in more depth.

During normal operations when your Azure HDInsight clusters are healthy and performing optimally, you will likely focus your attention on monitoring the workloads running on your clusters and viewing relevant logs to assist with debugging. Azure HDInsight offers two tools that can be used to monitor cluster workloads: Apache Ambari and integration with Azure Monitor logs. Apache Ambari is included with all Azure HDInsight clusters and provides an easy-to-use web user interface that can be used to monitor the cluster and perform configuration changes. Azure Monitor collects metrics and logs from multiple resources such as HDInsight clusters, into an Azure Monitor Log Analytics workspace. An Azure Monitor Log Analytics workspace presents your metrics and logs as structured, queryable tables that can be used to configure custom alerts. Azure Monitor logs provide an excellent overall experience for monitoring workloads and interacting with logs, especially if you have multiple clusters.

Azure Monitor logs

Azure Monitor logs enable data generated by multiple resources such as HDInsight clusters to be collected and aggregated in one place to achieve a unified monitoring experience. As a prerequisite, you will need a Log Analytics workspace to store the collected data. If you have not already created one, you can follow these instructions for creating an Azure Monitor Log Analytics workspace. You can then easily configure an HDInsight cluster to send a host of logs and metrics to Azure Monitor Log Analytics.

HDInsight monitoring solutions

Azure HDInsight offers pre-made, monitoring dashboards in the form of solutions that can be used to monitor the workloads running on your clusters. There are solutions for Apache Spark, Hadoop, Apache Kafka, live long and process (LLAP), Apache HBase, and Apache Storm available in the Azure Marketplace. Please see our documentation to learn how to install a monitoring solution. These solutions are workload-specific, allowing you to monitor metrics like central processing unit (CPU) time, available YARN memory, and logical disk writes across multiple clusters of a given type. Selecting a graph takes you to the query used to generate it, shown in the logs view.

The HDInsight Spark monitoring solutions provide a simple pre-made dashboard where you can monitor workload-specific metrics for multiple clusters on a single pane of glass.

The HDInsight Kafka monitoring solution enables you to monitor all of your Kafka clusters on a single pane of glass.

Query using the logs blade

You can also use the logs view in your Log Analytics workspace to query the metrics and tables directly.

HDInsight clusters emit several workload-specific tables of logs, such as log_resourcemanager_CL, log_spark_CL, log_kafkaserver_CL, log_jupyter_CL, log_regionserver_CL, and log_hmaster_CL.

On the metrics side, clusters emit several metrics tables, including metrics_sparkapps_CL, metrics_resourcemanager_queue_root_CL, metrics_kafka_CL, and metrics_hmaster_CL. For more information, please see our documentation, Query Azure Monitor logs to monitor HDInsight clusters.

The Logs blade in a Log Analytics workspace lets you query collected metrics and logs across many clusters.

Azure Monitor alerts

You can also set up Azure Monitor alerts that will trigger when the value of a metric or the results of a query meet certain conditions. You can condition on a query returning a record with a value that is greater than or less than a certain threshold, or even on the number of results returned by a query. For example, you could create an alert to send an email if a Spark job fails or if a Kafka disk usage becomes over 90 percent full.

There are several types of actions you can choose to trigger when your alert fires such as an email, SMS, push notification, voice, an Azure Function, an Azure LogicApp, a webhook, an IT service management (ITSM), or an automation runbook. You can set multiple actions for a single alert, and find more information about these different types of actions by visiting our documentation, Create and manage action groups in the Azure Portal.

Finally, you can specify a severity for the alert in addition to the name. The ability to specify severity is a powerful tool that can be used when creating multiple alerts. For example, you could create an alert to raise a Sev 1 warning alert if a single head node becomes unavailable and another alert that raises a Sev 0 critical alert in the unlikely event that both head nodes go down. Alerts can be grouped by severity when viewed later.

Apache Ambari

The Apache Ambari dashboard provides links to several different views for monitoring workloads on your cluster.

ResourceManager user interface

The ResourceManager user interface provides several views to monitor jobs on a YARN-based cluster. Here, you can see multiple views, including an overview of finished or running apps and their resource usage, a view of scheduled jobs by queue, and a list of job execution history and the status of each. You can click on an individual application ID to view more details about that job.

Spark History Server

The Apache Spark History Server shows detailed information for completed Spark jobs, allowing for easy monitoring and debugging. In addition to the traditional tabs across the top (jobs, stages, executors, etc.), you will find additional d ata, g raph, and d iagnostic tabs to help with further debugging.

Cluster logs

YARN log files are available on HDInsight clusters and can be accessed through the ResourceManager logs link in Apache Ambari. For more information about cluster logs, please see our documentation, Manage logs for an HDInsight cluster.

Next steps

If you haven’t read the other blogs in this series, you can check them out below:

About Azure HDInsight

Azure HDInsight is an easy, cost-effective, enterprise-grade service for open source analytics that enables customers to easily run popular open source frameworks including Apache Hadoop, Spark, Kafka, and others. The service is available in 36 regions and Azure Government and national clouds. Azure HDInsight powers mission-critical applications in a wide variety of sectors and enables a wide range of use cases including extract, transform, and load (ETL), streaming, and interactive querying.

10 Sep 2019
By editor
Categories: Azure, Big Data, Cloud Strategy

Azure HPC Cache: Reducing latency between Azure and on-premises storage

Today we’re previewing the Azure HPC Cache service, a new Azure offering that empowers organizations to more easily run large, complex high-performance computing (HPC) workloads in Azure. Azure HPC Cache reduces latency for applications where data may be tethered to existing data center infrastructure because of dataset sizes and operational scale.

Scale your HPC pipeline using data stored on-premises or in Azure. Azure HPC Cache delivers the performant data access you need to be able to run your most demanding, file-based HPC workloads in Azure, without moving petabytes of data, writing new code, or modifying existing applications.

For users familiar with the Avere vFXT for Azure application available through the Microsoft Azure Marketplace, Azure HPC Cache offers similar functionality in a more seamless experience—meaning even easier data access and simpler management via the Azure Portal and API tools. The service can be driven with Azure APIs and is proactively monitored on the back end by the Azure HPC Cache support team and maintained by Azure service engineers. What is the net benefit? The Azure HPC Cache service delivers all the performance benefits of the Avere vFXT caching technology at an even lower total cost of ownership.

Azure HPC Cache works by automatically caching active data in Azure that is located both on-premises and in Azure, effectively hiding latency to on-premises network-attached storage (NAS), Azure-based NAS environments using Azure NetApp Files or Azure Blob Storage. The cache delivers high-performance seamless network file system (NFSv3) access to files in the Portable Operating System Interface (POSIX) compliant directory structures. The cache can also aggregate multiple data sources into an aggregated name space to present a single directory structure to clients. Azure compute clients can then access data as though it all originated on a single NAS filer.

Ideal for cloud-bursting applications or hybrid NAS environments, Azure HPC Cache lets you keep your data on existing datacenter-resident Azure NetApp or Dell EMC Isilon arrays. Whether you need to store data on premises while you develop your cloud strategy for security and compliance reasons, or because you simply have so much data on-premises that you don’t want to move it, you can still take full advantage of Azure compute services and do it sooner, rather than later. Once you are ready or able to shift data to Azure Storage resources, you can still run file-based workloads with ease. Azure HPC Cache provides the performance you need to lift and shift your pipeline.

To the cloud in days, not months

Combined with other Azure services such as the Azure HB- and HC-series virtual machines (VMs) for HPC and the Azure CycleCloud HPC workload manager, Azure HPC Cache lets you quickly reproduce your on-premise environment in the cloud and access on-premise data without committing to a large-scale migration. You can also expect to run your HPC workloads in Azure at performance levels similar to your on-premises infrastructure.

Azure HPC Cache service is easy to initiate and manage from the Azure Portal. Once your network has been set up and your on-premises environment has IP connectivity to Azure, you can typically turn on Azure HPC Cache service in about ten minutes. Imagine being able to do HPC jobs in days rather than waiting for months while your IT team fine-tunes data migration strategies and completes all required data moves and synchronization processes.

From burst to all-in: Your choice, your pace

The high-performance Azure HPC Cache delivers the scale-out file access required by HPC applications across an array of industries, from finance to government, life sciences, manufacturing, media, and oil and gas. The service is ideally suited for read-heavy workloads running on 1,000 to 50,000 compute cores. Because Azure HPC Cache is a metered service with usage charges included on your Azure bill, you can turn it off—and stop the meter—when you’re done.

Azure HPC Cache helps HPC users access Azure resources more simply and economically. You can deliver exactly the performance needed for computationally intensive workloads, in time to meet demand. Start by using Azure capacity for short-term demand, and enabling a hybrid NAS environment, or go all-cloud and make Azure your permanent IT infrastructure. Azure HPC Cache provides the seamless data access you need to leverage cloud resources in a manner and at a pace that suits your unique business needs and use cases.

Proven technology maintained by Azure experts

Azure HPC Cache service is the latest innovation in a continuum of high-performance caching solutions built on Avere Systems FXT Edge Filer foundational technology. Who uses this technology? A diverse, global community that includes post-production studio artists in the UK, weather researchers in Poland, animators in Toronto, investment bankers in New York City, bioinformaticists in Cambridge and Switzerland, and many, many more of the world’s most demanding HPC users. Azure HPC Cache combines this most sought-after technology with the technical expertise and deep-bench support of the Microsoft Azure team.

Can’t wait to try it?

Ready to get off the sidelines and start running your HPC workloads in Azure? We have a few opportunities for customers to preview Azure HPC Cache. Just complete a short survey, and we’ll review your submission for suitability.

The Azure HPC Cache team is committed to helping deliver on Microsoft’s “Cloud for all” mission and will work with you to design a cloud that you can use to quickly turn your ideas into solutions. Have questions? Email them to [email protected].

10 Sep 2019
By editor
Categories: Azure, Cloud Strategy

Microsoft Azure available from new cloud regions in Germany

Deutsche Bank, Deutsche Telekom, SAP, and others trust Microsoft for their digital transformations

Today, we’re announcing the availability of Azure in our new cloud regions in Germany. These new regions and our ongoing global expansion are in response to customer demand as more industry leaders choose Microsoft’s cloud services to further their digital transformations. As we enter new markets, we work to address scenarios where data residency is of critical importance, especially for highly regulated industries seeking the compliance standards and extensive security offered by Azure.

Additionally, Office 365—the world’s leading cloud-based productivity solution—and Dynamics 365 and Power Platform, the next generation of intelligent business applications and tools, will be offered from these new cloud regions to advance even more customers on their cloud journeys.

Trusted Microsoft cloud services

Microsoft cloud services delivered from a given geography, such as our new regions in Germany, offer scalable, highly available, and resilient cloud services while helping enterprises and organizations meet their data residency, security, and compliance needs. We have deep expertise protecting data and empowering customers around the globe to meet extensive security and privacy requirements by offering the broadest set of compliance certifications and attestations in the industry. We also have a history of collaborating with customers to navigate evolving business needs, including delivering innovative strategies to help customers accelerate their European Union General Data Protection Regulation (GDPR) compliance.

Addressing the evolving needs of German customers

In Germany, companies across industries are adopting cloud technology amidst a changing regulatory framework that includes GDPR and a need for in-country data residency. Cloud services are becoming a key driver of product development, business model creation, and international stage competition. Responding to these changes, we’ve evolved our cloud strategy to better enable the digital transformation of our German customers.

Azure is now available from our new cloud datacenter regions in Germany to provide customers and partners with greater flexibility, the latest intelligent cloud services, full connectivity to our global cloud network, and data residency within Germany. The new regions with German-specific compliance, including Cloud Computing Compliance Controls Catalogue (C5) attestation, and will remove barriers so in-country companies can benefit from the latest solutions such as containers, IoT, and AI. These customers include:

Deutsche Bank, Germany’s leading bank, is leveraging our cloud services to accelerate the innovation of financial products and services while maintaining high-quality service and data security. With our collaboration, Deutsche Bank has developed a data platform that meets both international and local regulatory requirements while offering customers secure and cost-efficient services.
Deutsche Telekom, one of the world’s leading integrated telecommunications companies, will play an integral role in onboarding customers to our new cloud regions in Germany.
SAP, the market leader in enterprise application software, will combine Microsoft Azure and SAP HANA Enterprise Cloud to provide solutions directly from Germany—for the “Intelligent Enterprise in the Intelligent Cloud.”
Arvato Systems, a global IT specialist and multi-cloud service provider, is now able to offer their customers fully integrated Azure services with data retention in Germany, empowering the digital transformation of German medium-sized companies.

These investments help us deliver on our continued commitment to serve our customers, reach new ones, and elevate their businesses through the transformative capabilities of the Microsoft Azure cloud platform.

Please contact your Microsoft representative to learn more about opportunities in Germany or follow this link to learn about Microsoft Azure.