Design For Business

Working with clients of all sizes, across all geographies to manage your software locally or remotely, in a multi-cloud environment performing the best practices and design patterns.

Migration For Blast

Through our expertise with software and cloud insight into how it can transform your business on speed, innovation and global reach that is required today with optimization of your IT investment.

Cloud Adoption

Moving workloads to the cloud provide your business with greater flexibility, efficiency, productivity, and scalability, as well as lower costs compared to traditional IT infrastructures.

Services

Flexible software delivery approaches are required to meet the challenges of merging technology strategies and commercial and compliance needs. We Delivery Management Methodology offers cloud deployments to meet business needs wherever they operate.

Web Development

We develop fast, dynamics and fault tolerant web sites for your business using the best coding patterns and cloud practices.

Cloud Design Patterns

Design patterns to build reliable, scalable, secure applications in the cloud based on your business requirement.

Business Software Solution

Whether you need software for any business model we offer a software solutions to meet your needs.

Digital Marketing

The best digital marketing using a clear picture of how each campaign supports their overarching goals.

News

Building a document understanding pipeline with Google Cloud

Document understanding is the practice of using AI and machine learning to extract data and insights from text and paper sources such as emails, PDFs, scanned documents, and more.In the past, capturing this unstructured or “dark data” has been an expensive, time-consuming, and error-prone process requiring manual data entry. Today, AI and machine learning have made great advances towards automating this process, enabling businesses to derive insights from and take advantage of this data that had been previously untapped. 

In a nutshell, document understanding allows you to:

  • Organize documents 
  • Extract knowledge from documents 
  • Increase processing speed

At Google Cloud we provide a solution, Document AI, which enterprises can leverage in collaboration with partners to implement document understanding. However, many developers have both the desire and the technical expertise to build their own document understanding pipelines on Google Cloud Platform (GCP)—without working with a partner—using the individual Document AI products.

If that sounds like you, this post will take you step-by-step through a complete document understanding pipeline. The Overview section explains how the pipeline works, and the step-by-step directions below walk you through running the code.

Overview

In order to automate an entire document understanding process, multiple machine learning models need to be trained and then daisy-chained together alongside processing steps into an end-to-end pipeline. This can be a daunting process, so we have provided sample code for a complete document understanding system mirroring a data entry workflow capturing structured data from documents.

Our example end-to-end document understanding pipeline consists of two components:

  1. A training pipeline which formats the training data and uses AutoML to buildImage Classification,Entity Extraction,Text Classification, andObject Detection models.
  2. A prediction pipeline which takes PDF documents from a specified Cloud Storage Bucket, uses the AutoML models to extract the relevant data from the documents, and stores the extracted data in BigQuery for further analysis.

Training Data
The training data for this example pipeline is from a public dataset containing PDFs of U.S. and European patent title pages with a corresponding BigQuery table of manually entered data from the title pages. The dataset is hosted by the Google Public Datasets Project.

Part 1: The Training Pipeline

 The training pipeline consists of the following steps:

  • Training data is pulled from the BigQuery public dataset. The training BigQuery table includes links to PDF files in Google Cloud Storage of patents from the United States and European Union.

  • The PDF files are converted to PNG files and uploaded to a new Cloud Storage bucket in your own project. The PNG files will be used to train the AutoML Vision models.

  • The PNG files are run through the Cloud Vision API to create TXT files containing the raw text from the converted PDFs. These TXT files are used to train the AutoML Natural Language models.

  • The links to the PNG or TXT files are combined with the labels and features from the BigQuery table into a CSV file in the training data format required by AutoML. This CSV is then uploaded to a Cloud Storage bucket. Note: This format is different for each type of AutoML model.

  • This CSV is used to create an AutoML dataset and model. Both are named in the format patent_demo_data_%m%d%Y_%H%M%S. Note that some AutoML models can sometimes take hours to train.

Part 2: The Prediction Pipeline

This pipeline uses the AutoML models previously trained by the pipeline above. For predictions, the following steps occur:

  • The patent PDFs are collected from the prescribed bucket and converted to PNG and TXT files with the Cloud Vision API (just as in the training pipeline).

  • The AutoML Image Classification model is called on the PNG files to classify each patent as either a US or EU patent. The results are uploaded to a BigQuery table.

  • The AutoML Object Detection model is called on the PNG files to determine the location of any figures on the patent document. The resulting relative x, y coordinates of the bounding box are then uploaded to a BigQuery table.

  • The AutoML Text Classification model is called on the TXT files to classify the topic of the patent content as medical technology, computer vision, cryptocurrency or other. The results are then uploaded to a BigQuery table.

  • The AutoML Entity Extraction model is called to extract predetermined entities from the patent. The extracted entities are applicant, application number, international classification, filing date, inventor, number, publication date and title. These entities are then uploaded to a BigQuery table.

  • Finally, the BigQuery tables above are joined to produce a final results table with all the properties above.

Step-by-Step Directions

For the developers out there, here’s how you can build the document understanding pipeline of your dreams. You can find all the code in our GitHub Repository.

Before you begin:

  • You’ll need a Google Cloud project to run this demo. We recommend creating a new project.

  • We recommend running these instructions in Google Cloud Shell(Quickstart). Other environments will work but you may have to debug issues specific to your environment. A Compute Engine VM would also be a suitable environment.

1. Git clone the repo and navigate to the patents example.

2. Install the necessary system dependencies. Note that in Cloud Shell system-wide changes do not persist between sessions, so if you step away while working through these step-by-step instructions you will need to rerun these commands after restarting Cloud Shell.

3. Create a virtual environment and activate it.

When your virtual environment is active you’ll see patents-demo-env in your command prompt. Note: If your Cloud Shell session ends, you’ll need to reactivate the virtual environment by running the second command again.

For the remaining steps, make sure you’re in the correct directory in the professional-services repo: /examples/cloudml-document-ai-patents/.

4. Install the necessary libraries into the virtual environment.

5. Activate the necessary APIs.

Note: We use the gcloud SDK to interact with various GCP services. If you are not working in Cloud Shell you’ll have to set your GCP project and authenticate by running gcloud init.

6. Edit the config.yaml file. Look for the configuration parameter project_id (under pipeline_project) and set the value to the project id where you want to build and run the pipeline. Make sure to enclose the value in single quotes, e.g. project_id: ‘my-cool-project’.

Note: if you’re not used to working in a shell, run nano to open a simple text editor.

7. Also in config.yaml, look for the configuration parameter creator_user_id (under service_acct) and set the value to the email account you use to log in to Google Cloud. Make sure to enclose the value in single quotes, e.g. creator_user_id: ‘[email protected].

8. Create and download a service account key with the necessary permissions to be used by the training and prediction pipelines.

9. Run the training pipeline. This may take 3–4 hours, though some models will finish more quickly.

Note: If Cloud Shell Closes while the script is still downloading, converting, or uploading the PDFs, you will need to reactivate the virtual environment, navigate to the directory, and rerun the pipeline script. The image processing should take about 15-20 minutes, so make sure Cloud Shell doesn’t close during that time.

10. After training the models (wait about 4 hours), your Cloud Shell session has probably disconnected. Reconnect to Cloud Shell and run the following commands to reinstall dependencies, reactivate the environment, and navigate to the document understanding example code.

11. Next, you need to use the AutoML UIs to deploy the object detection and entity extraction models, and to find the ids of your models (which you will enter into config.yaml). In the UIs, you can also view relevant evaluation metrics about the model and see explicit examples where your model got it right (and wrong).

Note: Some AutoML products are currently in beta; the look and function of the UIs may change in the future.

Go to the AutoML Image Classification models UI, and make sure you are in same project you ran the training pipeline in (top right dropdown). Note the ID of your trained image classification model. Also note the green check mark to the right of the model—if this is not a check mark, or if there is nothing listed, it means model training is still in progress and you need to wait.

1.png

In Cloud Shell, edit config.yaml. Look for the line model_imgclassifier: and on the line below you’ll see model_id:. Put the image classification model id after model_id: in single quotes, so the line looks something like model_id: 'ABC1234567890'

Go to the AutoML Natural Language models UI, and similarly, make sure you are in the correct project, make sure your model is trained (green check mark) and note the model id of the trained text classification model.

2.png

In Cloud Shell, edit config.yaml. Look for the line model_textclassifier: and on the line below you’ll see model_id:. Put the text classification model id after model_id: in single quotes, so the line looks something like ” model_id: 'ABC1234567890'

Go to the AutoML Object Detection models UI, again making sure your project is correct and your model is trained. In this UI, the model id is below the model name. 

You also need to deploy the model, by opening the menu under the three dots at the far right of the model entry and selecting “Deploy model”. A UI popup will ask how many nodes to deploy to, 1 Node is fine for this demo. You need to wait for the model to deploy before running prediction (about 10-15 minutes), the deployment status is in the UI under “Deployed” and it will be “Yes” when the model is deployed.

3.png

In Cloud Shell, edit config.yaml. Look for the line model_objdetect: and on the line below you’ll see model_id:. Put the object detection model id after model_id: in single quotes, so the line looks something like model_id: 'ABC1234567890'

Go to the AutoML Entity Extraction models UI, and make sure the project and region are correct. Verify the model training is complete by checking to make sure the Precision and Recall metrics are listed. Note the id as well.

4.png

To deploy the model, click the model name and a new UI view will open. Click “DEPLOY MODEL” near the top right and confirm. You need to wait for the model to deploy before running prediction (about 10-15 minutes), the UI will update when deployment is complete.

5.png

In Cloud Shell, edit config.yaml. Look for the line model_ner: and on the line below you’ll see model_id:. Put the entity extraction model id after model_id: in single quotes, so the line looks something like model_id: 'ABC1234567890'.

Before going further, make sure the model deployments are complete.

12. To run the prediction pipeline, you’ll need some PDFs of patent first pages in a folder in Cloud Storage. 

You can provide your own PDFs in your own Cloud Storage location, or for demonstration purposes, you can run the following commands, which create a bucket with the same name as your project id (if it doesn’t already exist) and copy a small set of five patent first pages into a folder in the bucket called patent_sample.

13. In the config.yaml file, fill in the configuration parameter labeled demo_sample_data (under pipeline_project) with the Cloud Storage location of the patents you want to process (in single quotes). If you’re following the example above, the parameter value is 'gs:///patent_sample', substituting your project id.

Also, fill in the parameter labeled demo_dataset_id, just under demo_sample_data. With the BigQuery dataset (in single quotes) in your project where the predicted entities will be collected. For example, you can use 'patent_demo'. Note that this dataset must not exist already; the code will attempt to create the dataset and will stop running if the dataset already exists.

Now, you are ready to make predictions!

14. Run the prediction pipeline. This will process all the patent PDF first pages in the Cloud Storage folder specified in the demo_sample_data parameter, and upload predictions to (and create) BigQuery tables in the dataset specified by the  demo_dataset_id parameter.

python3 run_predict.py

15. Finally, go check out the dataset in the BigQuery UI and examine all the tables that have been created. There is a table called "final_view" which collects all the results in a single table. You should see something like this:

6.png

Note, for this, or any of the other intermediate tables that were created during the prediction pipeline, you may need to query the table to see the results since it is so new. For example:

If you are new to BigQuery, you can open a querying interface in the UI prepopulated with the table name by clicking the “QUERY TABLE” button visible when you select a table to view.

Conclusion

Congratulations! You now have a fully functional document understanding pipeline that can be modified for use with any PDF documents. To modify this example to work with your own documents, the data collection and training stages will need to be modified to pull documents from a local machine or another Cloud Storage bucket rather than a public BigQuery dataset. The training data will also need to be created manually for the specific type of documents you will be using. 

Now, if only we could create a robot that could scan paper documents to PDFs…

For more information about the general process of document understanding, see this blog post from Nitin Aggarwal at Google Cloud. And for more information about the business use cases of Document Understanding on Google Cloud, check out this session from Google Cloud Next ’19.

From Code to Community: Why developers call DevFest home
DevFest Banner

Ricardo on the left with fellow GDG lead planning DevFest Coimbra

Ricardo Costeira is a Software Engineer from Coimbra, Portugal. For the first time last year, he attended DevFest, the largest developer community-led movement hosted by Google Developer Groups across the world. To celebrate DevFest 2019, we want to share with you Ricardo’s story and how he went from writing code to finding community.

Ricardo (left) and a fellow GDG Lead plan DevFest Coimbra.

1. How did you first hear about DevFest? What inspired you to join?

In 2018, after living in Coimbra for 3 years, I didn’t have any friends outside of work that were software developers. I longed to fill my life with more people that understood my passion and decided it was time to make a change. So I took to social media to see how I could connect with more like minded thinkers. Eventually, DevFest showed up on my feed. Out of nowhere, I saw this crazy event in Coimbra packed with bold leaders, energizing speakers, and profoundly creative exercises. I never expected that being with a community would get me so excited. I got a ticket on the spot.

2. Can you tell us about your first experience at DevFest?

It was exhilarating – out of this world. When I first walked in, everyone talked to me as if we had known each other for years. Big smiles, loud laughs, and deep kindness were all around me. As someone who is relatively shy and a loner by nature, I was stunned when I felt myself saying, “I belong here, I’m home.” That very same night, I looked up the next event I could attend. Since then, I have attended 2 other events, signed up for 6 more, and have become a GDG Lead. In other words, I’m hooked.

Ricardo

So how did this all change me? To be honest, DevFest brought forward a shift in my personality. I now want to be part of a community – that is a new feeling in my life.

“As someone who is relatively shy and a loner by nature, I was stunned when I felt myself saying, “I belong here, I’m home.”

3. What from DevFest 2018 are you looking forward to seeing again this year?

The booths. DevFest Coimbra has booths where you can talk with different companies. It’s exciting to learn about all the opportunities to grow so close to home. In my case, it was thrilling to see just how quickly Portugal is scaling and how so many companies come to DevFest eager to talk with the best talent. Forming these relationships is what can make the difference when finding the right opportunity for you.

4. Leading up to DevFest 2019, what are you most excited for?

Lighting a spark in new attendees. I recently joined the organizational staff and I’m excited to give new attendees that feeling I had when I first walked into DevFest. I’ve found such meaning in working with my fellow GDG Leads to bring together attendees in a sense of shared awe.

5. Any advice for those attending DevFest 2019?

Just say hi. You will be surprised with how far it will take you. DevFest is not only about the talks or workshops, it’s also about the people. This community knows the extrovert and understands the introvert, and warmly welcomes both. That is to say, no matter who you are or how you code, there is a place for you here.

“DevFest is not only about the talks or workshops, it’s also about the people.”

Want to find a DevFest event near you? Check out devfest.withgoogle.com to join our community, meet other developers and learn about Cloud, Android, Flutter, Machine Learning and more.

#DevFest #Community
Actualizaciones de AWS CloudFormation para Amazon EC2, Amazon ECS, Amazon ElastiCache, Amazon ElasticSearch y más

Ahora puede usar plantillas de CloudFormation a fin de configurar y aprovisionar características adicionales para Amazon Elastic Compute Cloud (Amazon EC2), Amazon Elastic Container Service (Amazon ECS), Amazon ElastiCache, Amazon ElasticSearch Service (Amazon ES) y más recursos de AWS. La compatibilidad de CloudFormation se amplía regularmente para facilitar a los desarrolladores las tareas de configuración y aprovisionamiento de servicios de AWS.