Keeping worksites safe: How Resco and AI enhance safety inspections

We’ve explored how Resco Inspections+ and Microsoft Azure Custom Vision integrate together. Find out how they create a solution for easier and faster inspections in the oil & gas or construction industry.

In today’s digital age, technology provides multiple ways of automating and simplifying work tasks.

And the latest accelerator of frontline work transformation is AI.

A recent study shows that 65% of frontline workers believe that AI is their ally. They envision a future where technology can lighten their load and amplify their impact.

The collaboration between Resco and Microsoft exemplifies the potential of such digital integration.

This article looks at how Resco Inspections+ and AI in Azure Custom Vision can digitize health and safety processes for the construction industry. And the resulting solution provides benefits in terms of speed and accuracy of safety inspections.

The use case: Safety inspections on worksites

Personal Protective Equipment (PPE) Inspections are a common and important part of several industries. Helmets, goggles, or protective clothing are all part of mandatory equipment in oil & gas or construction industry.

It helps workers prevent injuries and mitigate safety hazards. But it is also important for organizations, as it helps them ensure safety and comply with regulations.

For example, one of the biggest OSHA fines in 2022 included penalty for failing to provide eye and head protection on a construction site. The fine totaled $321,956. The risk of work injuries on oil & gas rigs is even higher, making it critical to use new tools for safety improvement.

PPE inspections are often done with pen and paper, which creates a risk of losing or damaging the reports. Some field workers use spreadsheets that aren’t integrated with the core backend systems of organizations. This creates extra manual work for workers and it’s a complex task for organizations to keep all documents in order for years.

The solution: Mobile forms and AI that automate tasks

Mobile forms in Inspections+ enable inspectors to collect and evaluate data digitally. Inspectors can take notes, photos, or videos on site, providing visual proof of health and safety measures. 

The native connection to Dataverse, Dynamics, or Power Platform allows inspectors to sync data in real-time with their organization’s main database. Doing so mitigates data silos and allows organizations to set up a single source of truth.

The AI can automate and optimize related processes. For example, it can make evaluation of workers wearing protective equipment quick and error-free.

The integration: Technical part of integrating Inspections+ and Azure Custom Vision

The integration allows users to classify images in questionnaires or entity forms using Azure’s Custom Vision service. It requires an Azure Custom Vision subscription and at least one model for classifying images.

It is necessary to pre-train this model prior to use in the mobile app. The main steps for setting up Custom Vision include creating a new project, selecting project types and classification types, and obtaining the prediction URL.

You can read more about the process in the technical documentation.

The connection to Azure services itself is set up in Woodford. You can do so by adding a new model definition with all the necessary information in the ‘Settings: AI Image recognition’ section.

Once the training and setup are complete, users can utilize image recognition in questions, views, and forms. It allows them to scan images and match them to specific fields or records.

The entity where image recognition is to be used, must include a string field with barcode formatting on. For example, you can add a row button to the form that uses the ‘Scan Image’ step. Then, simply fill in the value or find the record you are looking for in a lookup view.

Another use case of how you can leverage the custom vision service is to answer questions in questionnaires directly. To do so, you can use the ‘image recognition’ question type.

If the captured image matches the tag in the pretrained model (with a probability above a customizable threshold), the tag’s name serves as the question output. You can apply this result to the other questions (e.g., string fields or lookups) and build your questionnaire business logic based on the tags.

POV: A health & safety inspector with Resco and AI at hand

Let’s have a look at one real-life scenario where technicians cannot start their duties until the safety standards are met.

The legal requirement states that until the protective equipment is not demonstrably present and documented, the mobile worker can’t begin the inspection on a worksite.

You can easily achieve this by integrating Inspections+ and AI in Custom Vision service. The upcoming release will enable this option even for Power Platform.

A technician simply takes a picture of his or her protective equipment at the beginning of the work order process.

The picture will be automatically attached to the documentation and well represented in the overall report. Only after taking the picture and detecting all the required safety protection (e.g., helmet in this case), the operative can proceed with the inspection.

Why it matters:

Real-time safety: Instant safety evaluation means workers can start their work faster and without a burden of administrative tasks. Organizations can also ensure compliance according to OSHA policy.

Audit trail and compliance: Digital data collection and the combination of photographic documentation, signed reports, and cloud storage ensure data integrity and easier document storage. 

Digital data collection: Real-time evaluation, inspections scoring, and elimination of data rewriting will help save inspectors’ time. 

No-code integration: A fast solution for ISV partners, consultants, and internal IT teams to incorporate the benefits of Inspections+, Power Platform, and AI technology without lengthy and expensive development.

Conclusion

That’s the showcase on Resco and Custom Vision joining forces. Quick photos, AI analysis, improved health and safety – all for a safer oil & gas rigs or construction sites. It’s tech that works for the frontline workforce, without the hassle.