The large scope and complexity of R&D projects encompassing topics such as digital solutions, smart cities, smart health, etc., are often difficult to understand from the public. Similarly, at FrostBit Software Lab, we are constantly working on multidisciplinary projects that blend game engines, XR solutions, web development, and data systems. To give some concrete examples of our daily tasks, the ongoing project of ‘MiGaEL’ aims to improve education in the mining industry through digital game-based learning, whereas the ‘VR Slaughterhouse’ project will create virtual reality training and simulation for reindeer husbandry. In one hand, FrostBit is also running the ‘Towards Sustainable Tendering’ project that will develop a web-based tool to enhance the process of tendering of acquisition that will lead to a long-term economic and environmental impact in the Lapland region.
Executing R&D projects is complex and requires comprehensive planning. How does this work in practice at the FrostBit Software Lab? When developing new tech solutions, we employ an iterative process based on researching, applying, and testing. I will try to break down the workflow by focusing on the case of the ongoing WinterSIM project: In brief, WinterSIM which is funded by Business Finland aims at turning Lapland and Snowbox testing facilities into the European ‘hub’ of winter testing and simulations for autonomous vehicles. The role of FrostBit in the WinterSIM project is to create virtual simulations for autonomous driving vehicles using game technologies such as Unreal and the open-source CARLA simulator. This will be done using sensors, LIDAR’s and cameras, which will gather real-world data, for example, snowfall and ice, to create the virtual simulation environment. The data will then be exported into the CARLA simulator and combined with urban and rural road winter conditions.
Designing the measuring station – data collection point
FrostBit engineers have designed a tailored measuring station which will be used to collect the data from the real-world simulations. The measuring station will be located at the Lapland UAS. The measuring station includes two LIDAR sensors, a depth-sensing camera, a radar, and a weather station. The measuring station will be controlled remotely supported by a wireless connection. Currently, the engineers are testing the station and preparing for real data collection. Additional measuring devices will be used for data collection in the later stages of the project.
Utilizing CARLA open-source simulator
The CARLA simulator is an open-source simulator developed with Unreal Engine 4 that will be used to develop the winter layer (conditions) simulation for autonomous driving vehicles. FrostBit Software Lab will be the first one to create the winter layer for the CARLA simulator. The expected outcome is a validated virtual winter simulation in CARLA, which combines data from the measuring station, a digital twin of the Aurora Intelligent Road, and 3D surroundings of the Aurora Road.
RoadRunner to create a virtual model of the Aurora Intelligent Road
The FrostBit engineers use real map data in RoadRunner to create a virtual model of the Aurora Intelligent Road. RoadRunner is fully compatible with Unreal Engine 4 and enables altitude model, point cloud, and aerial view. By using RoadRunner we ensure that the virtual model represents the real road as accurately as possible.
Data collection through 360 degrees video in Muonio, Lapland
Since there was not sufficient material available, the FrostBit team went to the town of Muonio, Lapland, and took 360 degrees video in order to use the material as an accurate source for 3D modeling of the surroundings of the Aurora Intelligent Road. We will use Blender (open-source) to model a virtual environment. The 360 camera was mounted on top of the car, which drove 40 km/h alongside a 10 km long road.
Early-phase prototype
In the video below, you can see an early phase prototype of the working simulator. We are ready to integrate the dataset into the simulation as well as 3D models of the Aurora Intelligent Road. The aim is to create a virtual simulation environment in which the visual winter condition can be simulated according to the data. For instance, users can simulate heavy snow or very icy road conditions. The first results of the visual winter simulations are promising although the project is in the early stages.
The project is a joint collaboration between companies and universities such as Lapland Proving Ground, Ramboll Finland, Lapland University of Applied Sciences, Roadscanners, Sensible 4, Toptester, and Aurora Snowbox.
If you would like to know more about the project, you can contact Pertti Rauhala at (firstname.lastname@lapinamk.fi).
The FrostBit team working in the project include Pertti Rauhala, Tuomas Herranen, Reko Paananen, Samuli Valkama, Miko Piitsalo, Aleksi Narkilahti, Juha Petäjäjärvi and Joel Koutonen.
Teachers working in the WinterSIM project: Minna Korhonen, Tauno Tepsa and Petri Hannula.
Written by Erson Halili | 01/12/2020