Category Archives: Artificial Intelligence

This post was co-authored by Hadas Bitran, Group Manager, Microsoft Healthcare Israel.

Every day, healthcare organizations are beginning their digital transformation journey with the Microsoft Healthcare Bot Service built on Azure. The Healthcare Bot service empowers healthcare organizations to build and deploy an Artificial Intelligence (AI) powered, compliant, conversational healthcare experience at scale. The service combines built-in medical intelligence with natural language capabilities, extensibility tools, and compliance constructs, allowing healthcare organizations such as providers, payers, pharma, HMOs, and telehealth to give people access to trusted and relevant healthcare services and information.

Healthcare organizations can leverage the Healthcare Bot Service on their digital transformation journey today, as we announced in our blog Microsoft Healthcare Bot brings conversational AI to healthcare. That’s why we are so happy to share more information on the Healthcare Bot Service partner program. Our Healthcare Bot certified partners empower healthcare organizations to successfully deploy virtual assistants on the Microsoft Healthcare Bot service. Working with an official partner, healthcare organizations can achieve the full potential of the Microsoft Healthcare Bot by leveraging the expertise and experience of partners who understand the business needs and challenges in healthcare.

This new program is open to existing Microsoft partners that support organizations in the healthcare domain, and delivers the training and resources required to support customers with end to end solutions using Microsoft’s Healthcare Bot Service. The program is designed to support partner success and enable partners to provide tailored solutions using the Healthcare Bot service as a foundation.

With the power of the cloud and a platform that is uniquely built for healthcare conversational intelligence, partners can quickly demonstrate value and iterate on solutions for customers. Official partners have access to partner-only resources and benefits that will enable them to provide customers with differentiated and value-added offerings such as:

• Partner listing in the Healthcare Bot partner directory.
• Preferential messaging tiers.
• Free demonstration and proof of concept Healthcare Bot Instances.
• Direct support channel from the product team.
• Partner resources including sales materials, product updates and release notes.

The Microsoft Healthcare Bot service helps partners bring conversational AI to innovative healthcare organizations. Partners can support healthcare organizations to deploy customized conversational experiences at scale, reducing costs and improving outcomes for their patients with virtual assistants built to complement their healthcare services.

The Healthcare Bot provides partners with a comprehensive platform to automate healthcare engagements and provides patients with instant access to the services they need. The service facilitates multi-channel healthcare conversations such as chat bots or handoff to live nurses over Microsoft Teams. Partners can build differentiated offerings and create unique conversational healthcare experiences that support the type of digital interaction required by the patient.

Next steps

Partners interested in certification should submit a request to [email protected]. Healthcare organizations seeking certified Healthcare Bot partners can find more information in the official partner directory.

Through integration with Cognitive Services APIs, Azure Search has long had the ability to extract text and structure from images and unstructured content. Until recently, this capability was used exclusively in full text search scenarios, exemplified in demos like the JFK files which analyzes diverse content in JPEGs and makes it available for online search. The journey from visual unstructured content, to searchable structured content is enabled by a feature called cognitive search. This capability in Azure Search is now extended with the addition of a knowledge store that saves enrichments for further exploration and analysis beyond search itself.

The knowledge store feature of Azure Search, available in preview, refers to a persistence layer in cognitive search that describes a physical expression of documents created through AI enrichments. Enriched documents are projected into tables or hierarchical JSON, which you can explore using any client app that is able to access Azure Storage. In Azure Search itself, you define the physical expression or shape of the projections in the knowledge store settings within your skillset.

Customers are using a knowledge store (preview) in diverse ways, such as to validate the structure and accuracy of enrichments, generate training data for AI models, and ad-hoc analysis of their data.

For example, the Metropolitan Museum of Art opened access to all images of public domain works in its collection. Enriching the artworks with cognitive search and the knowledge store allowed us to explore the latent relationships within the artworks on different dimensions like time and geography. Questions like how have images of family groups changed over time, or when were domestic animals included in paintings, are now answerable when you are able to identify, extract, and save the information in a knowledge store (preview).

With the knowledge store, anyone with an Azure subscription can apply AI to find patterns, insights, or create dashboards over previously inaccessible content.

What is the knowledge store (preview)?

Cognitive search is the enrichment of documents with AI skills before they are added to your search index. The knowledge store allows you to project the already enriched documents as objects (blobs) in JSON format or tabular data in table storage.

As part of your projection, you can shape the enriched document to meet your needs. This ensures that the projected data aligns with your intended use.

When using tabular projections, a knowledge store (preview) can project your documents to multiple tables while preserving the relationships between the data projected across tables. The knowledge store has several other features like allowing you to save multiple unrelated projections of your data. You can find more information about a knowledge store (preview) in the overview documentation.

Data visualization and analytics

Search enables you to find relevant documents, but when you’re looking to explore your data for corpus wide aggregations or want to visualize changes over time you need your data represented in a form other than a search index.

Leveraging Power BI’s integration with Azure tables, gets your dashboard started with only a few clicks. To identify insights from the enriched documents over dimensions like time or space, simply project your enriched documents into tables, validate that Power BI recognizes the relationships and you should now have your data in a format that is ready to consume within the visuals.

When you create a visual, any filters work, even when your data spans related tables. As an example, the art dashboard was created on the open access data from the MET in the knowledge store and the Art Explorer site uses the search index generated from the same set of enrichments.

The art explorer site allows you to find art works and related works while the Power BI report gives you a visual representation of the corpus and allows you to slice your data along different dimensions. You now can answer questions like “How does body armor evolve over time?”

In this example, a knowledge store (preview) enabled us to analyze the data ad-hoc. In another example, we may for instance enrich invoices or business forms, project the structured data to a knowledge store (preview), and then create a business-critical report.

Improving AI models

A knowledge store (preview) can also help improve the cognitive search experience itself as a data source for training AI models deployed as a custom skill within the enrichment pipeline. Customers deploying an AI model as a custom skill can project a slice of the enriched data shaped to be the source of their machine learning (ML) pipelines. A knowledge store (preview) now serves as a validator of the custom skill as well as a source of new data that can be manually labeled to retrain the model. While the enrichment pipeline operates on each document individually, corpus level skills like clustering require a set of documents to act on. A knowledge store (preview) can operate on the entire corpus to further enrich documents with skills like clustering and save the results back in a knowledge store (preview) or update the documents in the index.

Getting started

To start using a knowledge store (preview) you will need to:

1. Add a knowledge store (preview) configuration to your skillset.
2. Optionally, add a shaper skill to the skillset to define the shape of the projected enrichment.
3. Add a projection for tables, objects, or both to a knowledge store (preview). You may project the output of the shaper skill, or elements from the enriched document directly.

A knowledge store (preview) enables the use of your enriched data in new or improved models, visualizing and exploring the data in tools like Power BI and app based experiences merging the raw and enriched data. We will continue to add more capabilities and updates over the coming months.

For a detailed walkthrough, see the knowledge store (preview) getting started guide.

This blog post was co-authored by Parashar Shah, Senior Program Manager, Applied AI Developer COGS.

When data scientists work on building a machine learning model, their experimentation often produces lots of metadata: metrics of models you tested, actual model files, as well as artifacts such as plots or log files. They often try different models and parameters, for example random forests of varying depth, linear models with different regularization rates, or deep learning models with different architectures trained using different learning rates. With all the bookkeeping involved, it is easy to miss a test case, or waste time by repeating an experiment unnecessarily. After they finalize the model that they want to use for predictions, they have to do multiple things in order to create a deployment environment and then create a webservice (http endpoint) from their model.

For small proof-of-concept machine learning projects, data scientists might be able to keep track of their project using manual bookkeeping with spreadsheets and versioned copies of training scripts. However, as their project grows, and they work together with other data scientists as a team, they’ll need a better tracking solution that allows them to:

• Analyze the performance of the model while tuning parameters.
• Query the history of experimentation to find the best models to take to production.
• Revisit and follow up on promising threads of experimentation.
• Automatically link training runs with related metadata.
• View snapshots and audit previous training runs.

Once a data scientist has created a model, a model management, and model deployment solution is needed. This solution should allow for:

• Storing multiple models and multiple versions of the same model in a common workspace.
• Easy deployment and creation of a scalable web service.
• Monitoring the deployed web service.

Azure Machine Learning and MLflow

Azure Machine Learning service provides data scientists and developers with the functionality to track their experimentation, deploy the model as a webservice, and monitor the webservice through existing Python SDK, CLI, and Azure Portal interfaces.

MLflow is an open source project that enables data scientists and developers to instrument their machine learning code to track metrics and artifacts. Integration with MLflow is ideal for keeping training code cloud-agnostic while Azure Machine Learning service provides the scalable compute and centralized, secure management and tracking of runs and artifacts.

Data scientists and developers can take their existing code, instrumented using MLflow, and simply submit it as a training run to Azure Machine Learning. Behind the scenes, Azure Machine Learning plug-in for MLflow recognizes they’re within a managed training run and connects MLflow tracking to their Azure Machine Learning Workspace. Once the training run has completed, they can view the metrics and artifacts from the run at Azure Portal. Later, they can query the history of your experimentation to compare the models and find the best ones.

Once the best model has been identified it can be deployed to a Kubernetes cluster (Azure Kubernetes service), from within the same environment using MLflow.

Example use case

At Spark + AI Summit 2019, our team presented an example of training and deploying an image classification model using MLflow integrated with Azure Machine Learning. We used the PyTorch deep learning library to train a digit classification model against MNIST data, while tracking the metrics using MLflow and monitoring them in Azure Machine Learning Workspace. We then saved the model using MLflow’s framework-aware API for PyTorch and deployed it to Azure Container Instance using Azure Machine Learning Model Management APIs.

Where can I use MLflow with Azure Machine Learning?

One of the benefits of Azure Machine Learning service is that it lets data scientists and developers scale up and scale out their training by using compute resources on Azure cloud. They can use MLflow with Azure Machine Learning to track runs on:

• Their local computer
• Azure Databricks
• Machine Learning Compute cluster
• CPU or GPU enabled virtual machine

MLflow can be used with Azure Machine Learning to deploy models to:

• Azure Container Instance (ACI)
• Azure Kubernetes Service (AKS)

Getting started

The following resources give instructions and examples how to get started:

• Microsoft documentation “How to use MLflow with Azure Machine Learning service (preview).”
• MLflow documentation including a quickstart guide, tutorials, and other key concept information.
• Example Jupyter Notebooks from our GitHub repository.