Software Architect / Microsoft MVP (AI) and Technical Author

Cognitive Services, Machine Learning, Prototyping

AI for Good: Using Cognitive Services Form Recognizer API to understand extreme weather.

Recently I caught sight of a Tweet where there was a request for someone with experience in artificial intelligence / machine learning.

There is was a shout out to see if anyone in the tech world had ideas using AI or ML that could help contribute to climate science and understanding extreme weather.

Cognitive Services Form Recognizer API

Form Recognizer API came to mind.  This is a newer API that lives in the Cognitive Services ecosystem. With just 5 sample documents, Form Recognizer can ingest form data from JPG, PDF or PNG files and then output structured data.

Another bonus is the structured data returned by Form Recognizer includes the original relationships that were present in the uploaded file.

(image source: Microsoft)

High Level Plan

I’ve been fortunate enough to have been granted access to a set of PDFs which include thousands of data tables from 1873-1921 which contain a valuable source of undigitized weather data.  A rough plan could consist of the following:

  1. Identify sample dataset
  2. Provision Form Recognizer in Azure
  3. Create custom code to convert sample dataset to byte arrays
  4. Push binary to form recognizer
  5. Process the response!

Other Ideas

I had some other thoughts…

  • data could be pushed into an Azure DB
  • create a REST API as an interface into the data (like initiatives)

Going to see what’s possible with the API for this use case in the coming weeks.

Get the latest content and code from the blog posts!
I respect your privacy. No spam. Ever.


  1. Deighton

    How did this go?

    • Comment by post author


      Hi Deighton,

      Thanks for asking about this. I have prototype code but need wont be resuming this until early January I’m afraid. Other items in the todo list took priority. I can let you know when this is done if you’re still interested?

Leave a Reply