HoloLens puts fire prevention into a new perspective

The Firestarter Hackathon, an initiative of Vias, EY and Netwerk Brandweer, took place for the very first time on 17 and 18 October 2019. Among the more than one hundred participants were AE and Mensura, who joined forces to develop an innovative solution in the field of smart firefighting. Mensura was determined to design a tool that would make company tours, an important part of fire prevention, run more efficiently, while AE was eager to apply Microsoft's HoloLens – we love a challenge, after all. Read on to learn all about the exciting result!

Five experts

Mensura and AE arrived at the Firestarter Hackathon with a diverse team of experts. Johan and Koen both work as a fire prevention adviser at Mensura, and Florian is a firefighter and data scientist at AE. Axel and Gertjan are Technical Consultants at AE. This was Mensura’s very first Hackathon, but Gertjan and Axel, who are both well-seasoned players at the game, were more than happy to teach them the ropes. Gertjan:

“It’s always a great experience participating in a Hackathon in collaboration with one of our customers. The atmosphere at these kinds of events is invigorating, to say the least. In this particular case, we contributed to the future of our public services. That made it even more worth the while for me.”

Axel, too, very much enjoyed collaborating with Mensura:

“Coming from such different backgrounds, it sometimes took us a while to operate on the same wavelength. But the diversity of our team certainly had its advantages as well. Mensura knows exactly what the common stumbling blocks are for this kind of company tour, and we’ve made smart use of that insider knowledge.”

Koen and Johan from Mensura are equally pleased:

“This Hackathon proved a very successful exercise for us, not in the least thanks to the infectious enthusiasm of AE’s young technicians, Ruben’s professional preparation and Florian’s very convincing sales techniques. We have high hopes for this project to lead to an amazing product!” – Koen Mercken, Occupational Safety Prevention Expert for Mensura.

The problem with company tours

Carrying out a company tour (which is an annual requirement) is no easy feat, according to Johan and Koen from Mensura. As fire-related risks are detected and indexed manually, company tours are prone to human error and spaces are not rarely overlooked. “An additional problem is that fire brigades during an intervention should have an up-to-date plan of the premises at their disposal, but that simply isn’t the case right now,” adds fireman Florian.

Fire prevention via AR

To pinpoint fire risks in the best possible manner, the team started from a value proposition canvas to work out the idea in more concrete terms. They also checked with the Garaeg whether working with a HoloLens – something a lot of AE people have been dreaming of for several years now – was in fact an option. And ... they got the green light! A really big thank you to Microsoft for lending us the HoloLens and making this project possible! 

It took quite a bit of developing, but the result was impressive indeed. The AR application created by the team enables inspectors to walk around a building and effortlessly detect, analyze and locate fire doors, chairs and fire extinguishers. By combining the annual tour with a digital intervention plan, fire hazards can also be mapped automatically. Finally, our data intelligence expertise also came in handy to make automatic recognition possible by means of image recognition and machine learning.

All these efforts combined ultimately yielded the following benefits:

• Company tours are concluded much quicker, leaving more time to focus on other prevention aspects.
• Differences (new hazards) in comparison to previous tours are easily spotted.
• Fire departments always receive an up-to-date version of the premises.
• Different data sources can be linked.
• The data generated can be shared with external parties (inspection organizations, insurers, government agencies).

Working with a HoloLens? Ho there!

Axel and Gertjan enthusiastically got started working with the HoloLens, but soon found out their plan was easier said than done. Despite Axel’s thorough preparations, the Hackathon’s Wavespace network turned out to pose some problems. The (overly) strict network policy made it impossible to find other devices on the network, so debugging became a challenge. And that wasn’t even the only hurdle. Axel and Gertjan advise anyone who considers working with a HoloLens to take the following information into account.

Tips & tricks

• Brace yourself, because Google will provide you with very few manuals and tutorials to help you develop mixed-reality applications for the HoloLens. For the time being, the official Microsoft website remains your best starting point. Axel also recommends this Mixed-Reality Academy tutorial and this interesting guide to HoloLens video recording.
  
  
• Many tutorials use Azure services. Creating your Azure account in advance will therefore save you valuable time. The free trial version will get you more than far enough, at least for a hackathon.
  
  
• Know that it takes a considerable amount of time to first get started with a HoloLens. Install as many tools as possible before the hackathon takes place: Windows 10 SDK, Visual studio, Unity, Mixed-Reality Toolkit (MRTK) and the HoloLens Emulator are all on the list.
  
  
• If you’re already familiar with Unity, the environment and workflow will likely hold little or no surprises for you. However, a lot of boiler plating is required before you start working on a mixed-reality application. It certainly pays to set up a number of pre-configured projects in advance.
  
  
• Using the Emulator is not a given either. A relatively ‘beefy’ system is recommended as standard (I7 6^th generation (6 core) + GTX980 / 1060 + 16GB RAM). But there is good news: you don't actually need the Emulator. Thanks to the Unity Play Mode, you can ‘stream’ your project to the HoloLens in real time. First install the Holographic Remoting Player app on the HoloLens and then connect to the HoloLens' ip over the local network.
  
  

Catchy pitch

A Hackathon is, of course, not complete without a pitch, and the Firestarter Hackathon is no exception. Our team successfully demonstrated the HoloLens and their pitch provoked lots of positive reactions to boot. Unfortunately, there was no victory in it for them this time, but Axel is nonetheless proud of his team’s accomplishment:

“It was a very intense two days, but I learned a lot and met some interesting new people. The story certainly doesn’t end here for us. We’ve been contacted by various public institutions who would very much like to further develop our project. At Mensura, too, they’re already warming up to get cracking at it thoroughly.”

Florian does not feel as if he has returned home empty-handed either:

“Pitching, dealing with stakeholders, making contacts, ... This Hackathon was so much more than twenty-four hours of programming. It was a unique chance for me to expand my network and explore new technology. Whether you’re just starting out or have been an expert for years, I advise everyone to not miss out on opportunities like this.”

Creating a platform

The team realized during the Hackathon that their original idea had the characteristics of a platform: joined by a common theme, multiple parties connect with each other and exchange valuable information. On the one hand, Mensura in this example acts as a producer on the platform. They take tours of the buildings, thus capturing the digital information. But other companies who, due to the nature of their activities, are regularly and structurally present in company buildings (e.g. cleaning companies or security firms) can be producers just as well.

On the other hand, fire brigades, but possibly also insurers, governments, ... will be able to improve their services based on the captured and possibly enriched data. Partners such as technology players who offer their services to support this capturing and exchanging of data can connect to the platform as well. This insight enables us to better and further explore ways to make our idea take shape. The first step will be creating a platform value canvas through which we will map out the added value provided and gained by each stakeholder. To be continued!