Creating a Digital Twin VR-Environment

DUKE is a collaboration project between Lapland University of Applied Sciences (Lapland UAS) and Rovaniemi Municipal Federation of Education (REDU). The idea of DUKE is to examine how VR solutions can be implemented in educational purposes while leveraging renewable energy. Our main pilot is a digital twin of an educational district heating plant located in Rovaniemi and our secondary pilot is a digital twin of a customer grade heat pump. Digital twins allow us to visualize the environment with great immersion, this enables us to implement accurate functionality to various devices and systems. High immersion helps users to subconsciously connect the virtual object to the real-life object, thus seeing a virtual lever would have the same relationship as seeing a real lever; it should be interactable.

DUKE’s customer grade heat pump was made in cooperation with a project partner. Our partner wanted to incorporate VR solutions alongside their user manual. This way, the user can visualize more accurately basic procedures and maintenance work for the heat pump. Our DUKE engineers tackled this project by working closely with the heat pump specialists. The specialists taught us how to do maintenance on the heat pump and we, in turn, translated that knowledge into a VR simulation. The development process fundamentally revolved around an iterative workflow, where we had assessment and review meetings back and forth. In doing so, we ensure that the 3D-recreation was accurate and at a satisfactory quality.

DUKE’s official logo

Creating the digital twin VR-environment

A digital twin is a 3D virtual recreation of an object or a real-world environment. The definition ‘digital twin’ can be perceived as an instruction regarding how closely or realistically the virtual recreation represents its real-world counterpart. Here in Frostbit Software Lab, we aim to create our digital twins to the extent where the 3D-recreation is highly accurate and unmistakable when comparing the 3D-version to its reference.

Side-by-side image of the rendered 3D-scene and the raw 3D-scene.

The workflow and subsequently the challenges we face depends largely on the use case scenario. The VR simulation hosts a physics simulation system in addition to the high-fidelity 3D-visual representation of the environment. For this reason, we chose to use Unity for its lightweight and adaptable nature to implement our real time VR simulation. Unity allows us to easily manipulate and optimize the VR simulation to work with customer grade computers and high-end computers for a higher fidelity experience.

The developer’s viewport inside Unity game-engine. The screenshot is taken from the work-in-progress district heating plant.

We kicked off the visual implementation process by taking reference images on-site. The images are captured from multiple angles in order to get an understanding of the object’s shape and contour. Moreover, it is exceedingly useful in figuring out the object’s material properties, such as how shiny it is, what type of metal it is and what kind of reflections it produces.

A reference image of the district heating plant.

The 3D-modeling process uses blueprints in tandem with the reference images to get accurate measurements for a high-detail recreation. A keen observer may still find some discrepancies when comparing the screenshot image to the reference image. This is due to readjustments, some parts of the district heating plant were deemed obsolete for the VR simulation and therefore had to be cut or modified.

The texturing process uses the multi-angle images to figure out what type of material would be the most accurate to its real-world version. We use HDRi-images to create realistic lighting to the 3D-scene. HDRi’s are essentially 360-degree panoramic images that contain a vast amount of lighting data. Ultimately, we are able to create believable environments by following real-world lighting conditions while incorporating physical based rendering techniques.

A screenshot of the disctrict heating plant digital twin.

VR-Implementation in a game-engine

The first step our developers need to make is to choose an ecosystem, in other words, which development pipeline to use. Locking into a development pipeline is essential in guaranteeing efficient feature implementation down the line. The ecosystem in this context consists of choosing which manufacturers VR-gear to use, what game-engine and development toolkit to use. Our DUKE engineers ended up on using Unity game-engine for its lightweight and adaptable nature.

A test-player trying out the playable demo of the DUKE VR-simulation

Our DUKE engineers chose to use SteamVR software development kit for its wide VR-gear support. This SDK provides accessible script blueprints, that we can tailor for our own usage. For example, the SDK provides a basic functionality for object interaction and player movement. This way, more time can be allocated to polish more advanced features such as the dynamic interaction between different objects while a mathematic simulation is running in the background.

The mathematic simulation is built with modularity in mind. Each part of the simulation is broken down to individual pieces. This grants us the customizability to assign the simulation scripts to the 3D-objects that would logically handle the same physics in the real world. For example, the scripts that handle pressure loss are assigned to the 3D-objects that look like valves or pipes. One of the challenges we faced when working with a VR solution was that real-life dimensions did not translate well in a VR environment. EU-standard sized doors seemed too narrow, standardized knobs, bolts and valves seemed too small and ultimately play-testers reported having a difficult time maneuvering in the VR environment overall. A silver lining needed to be struck in order to satisfy the digital twin -nature of the project and the ease-of-use side for the players.

The current DUKE heat pump build contains four scenarios: “Tutorial”, “Water filter”, “Pressure adjustment” and “Water leakage”. Each scenario provides information on how to operate the customer grade heat pump and how to perform basic maintenance.

Tutorial -scenario

Tutorial scenario explains thoroughly all the functions available in the project. This scenario also includes information covering the main parts of the heating system. This is usually the first scenario the player should launch, since it also teaches the player how to use the VR-gear.

Water filter -scenario

The user can learn how to clean the water filter using this scenario.

Pressure adjustment -scenario

The user can learn how to adjust the heat pump to a correct pressure level in the system and how to use the safety valve to release the pressure from the system.

Water leakage -scenario

Water leakage -scenario provides a good guideline on how to store the heat-pump system. We show the most common issues that can be encountered in such a space.

The total budget of the project is 761 732 euros, of which Regional Council of Lapland has granted European Regional Development Fund (ERDF) and state funding 609 385 euros. This project will be executed in the timeframe of 1.1.2020-31.12.2022.

This article has been evaluated by the FrostBit’s publishing committee, which includes Heikki Konttaniemi, Toni Westerlund, Jarkko Piippo, Tuomas Valtanen, Pertti Rauhala and Tuuli Nivala. The article publications can be found on the FrostBit publication blog.


Written by
Onni Li, Bachelor of Engineering (Lapland UAS), Project Engineer, FrostBit, Lapland UAS
Severi Kangas, Bachelor of Engineering (Lapland UAS), Project Engineer, FrostBit, Lapland UAS

A new era of educational games – learning through digital solutions and gamification

Advances in technology have made it possible to create more advanced games for educational purposes. It is no longer just “point and click” style computer games for children in the late 90s, but developments have given us the keys to assist in more complex learning. The wildest dreams of digital twins that utilize virtual reality have already come true and being constantly developed in the FrostBit laboratory. However, for the consumer, the virtual world viewed through glasses can still be foreign.

At the same time as the spearhead of technological development is far ahead of the consumer, we have also reached a different milestone in the world of educational games: each of us has access to a digital terminal that opens the door to learning in increasingly diverse ways. In practice, therefore, everyone can use educational games in their daily learning via a smartphone or computer. The quality and diverse content of games have also improved in the same proportion, and in recent years various device manufacturers also seem to be slowly waking up to the opportunities brought by digitalization. Virtual reality will reach us all at one point, but at this point we must at least leave behind the old moldy cereal-pack games.

Virtual learning in healthcare

In co-operation with the social and healthcare section, we are currently implementing a simple project as part of the Nursing RDI activity, the aim of which is to create a tutorial that teaches how to use an infusion pump. This Braun Infusomat Space is an everyday tool for the nurse and learning how to use it is one small but important part of nursing studies. Traditionally, the infusion pump is learned to be used in the classroom without the patient and in theory it is possible for the student to go through the operation of the device as much as they wish. In practice, however, there are often many students, limited time, and personal learning difficulties that can get in the way of learning, and because of these aspects, traditional classroom instruction does not always guarantee that all students will be able to use the device enough. A nursing student focuses mainly in learning to take care of the patient, and the use of technical equipment is not necessarily their primary focus. Infusomat is a reasonably valuable and even a little “threatening” device, and when practicing it, the student often learns certain procedures and tries to avoid mistakes. Thus, learning to use a device does not actually involve deep learning, but mainly the ability to repeat what has been learned. Such learning procedures lead to the expected result, but as an unfortunate side effect, the student does not actually get to know the device but merely uses it in predetermined situations. In case of problem situations or in the event of a disturbance, the ability to react and act on the situation is not the best, and the student may not feel confident when using the device.

The functions of an infusion pump (alpha)

Gamified learning can enhance the students’ deep learning. Instead of learning how to use an infusion pump only in a controlled way in the classroom, the student can continue to “play with the device” in the virtual world. Once the operation of the entire device has been 3D-modelled accurately in the game and the interface is suitably simple, the player will be able to try out different functions and resolve problem situations without worrying about accidentally breaking the pump. By utilizing the game, the student achieves confidence in using the device and then the operation of the correct infusion pump feels much more familiar as well as easier to learn in practice.

How are the functions of the infusion pump transferred to the game?

The most important aspects of making this game are the following:

  • User interface
  • Device operation
  • Correspondence of the 3D model
The 3D-model of the infusion pump (alpha)

What is essential, of course, is that the developer not only thoroughly understands the operation of the device, but also models it with the accuracy required for its operation. A big factor in particular is user orientation: a game can be useless to its target audience if the target audience is unable to use it. The developer must assume that the user of the device has little or no experience in playing different games and therefore must understand to develop functionalities so that no user experience difficulties will alter the learning process. In practice, this means that features very familiar to many digital games, such as first-person character movement and camera control, must be omitted or made so simple that they don’t interfere with the gaming experience. It is unnecessary to force the player to learn the seemingly complicated interface that they even get the opportunity to learn how to use the device himself.

The development phase has two stages: device modelling and construction of the game mechanics. Blender-3D Software is used to model the infusion pump and game mechanics are created with the Unity game engine. In modelling the device, it is essential to model all those external features that matter either in terms of game mechanics (buttons, screens, power cords) or visual equivalence (round arc of the door, aspect ratio). In game mechanics, an accurate representation of the infusion pump operation is important. For example, when you press the start button on the device, the infusion pump emits a small beep, the lights turn on, and then a text appears on the display to indicate that it has started. All of those functions are essential to make it easy for a player to transfer what they have learned from the digital to their real world. However, it is not essential that the internal operating logic should be exactly the same as that of the device being modelled.

A real infusion pump

The end result can be considered as the crucial point: the game must be able to convince the player that it works in the same way as the real-world device it represents. The choices made in game development must therefore be viewed from the perspective of transferring what has been learned to the real world: if the function is important for using the right physical device, then it must also be implemented in the digital version. Thus, a game made of an infusion pump does not have to be a perfect digital twin, but mainly a model that simulates its operation as accurately as possible.

The project is still ongoing, but looks very promising. Combining game development with the development of learning outcomes is an interesting and important direction that will certainly provide interesting data as the game develops. The tutorial is scheduled to be ready in June 2021. The game will run on a regular computer, mobile device, and in addition, the possibility of virtual reality will also be explored.

‘TKI-toiminta digiaikaan’ -project is funded by Lapin Liitto with EAKR-funding.

Written by Samuli Valkama | 29/04/2021

Historical accuracy versus playability – Designing the Struve Geodetic Arc mobile game

If a game, game-like learning environment or any other software using game technologies is based on real events, the relationship between realism and playability has to be taken into consideration during the development process. Imitating the real world precisely with all its limitations set by physics, biology or the passage of time doesn’t necessarily serve the original purpose or goal of the game or software. It doesn’t matter if the goal is to entertain or to educate, both can suffer from total realistic accuracy.

Making a game replicate its real-life counterpart as precisely as possible is sensible and sometimes absolutely necessary when a game is focused on a very limited subject, such as controlling a specific vehicle. In the case of vehicle simulators, the physical controllers are also often made to look and feel like the real thing. On the other hand, when the subject of the game is broader, decisions have to be made on which things to replicate and which things to change to enable e.g. playability, entertainment or better understanding of the bigger picture. The intended platform of the game also plays into these decisions. A keyboard-mouse combination, regular commercial game controller or a smartphone screen are all very generic controllers and as such can’t exactly replicate the feel and functions of any specific tools or items. If the chosen platform is mobile, physical size and performance of the devices as well as the inaccuracy of gesture controls set extra limitations.

This balancing between realism, hardware limitations and the goal of the game has been and will continue to be an integral part of the design process of Struve Geodetic Arc mobile game. The game is being produced as a part of The Northern parts of the World Heritage Struve Geodetic Arc project (, partially financed by EU’s Interreg Nord programme (ERDF).

The background story: What is the Struve Geodetic Arc?

Friedrich Georg Wilhelm von Struve (1793-1864) was an astronomer who also had an interest in geodetic surveying.  As a part of his research he organized a triangulation survey with a triangle chain reaching from Black Sea to Hammerfest. The goal of this survey was to gain a better understanding of the shape of the Earth near its poles. The whole chain includes 258 main triangles and 265 main station points that stretch over 10 different countries. A total of 34 of the main station points are part of UNESCO World Heritage. (

Struve Geodetic Arc was accepted into the UNESCO World Heritage list in 2005, but it isn’t particularly well known in the Nordics. The Northern parts of the World Heritage Struve Geodetic Arc project aims to improve the accessibility and knowledge of the Struve Geodetic Arc. The purpose of the mobile game is to attract new audience for the World Heritage site by means of entertainment.

The events and environment of the game are based on real events in what is currently Finland, Sweden and Norway, then Sweden-Norway and the Russian Empire, during the 19th century. However, the intent is not to make a “real” educational game or replicate historical events and circumstances as precisely as possible, even though it would be possible in theory. Struve Geodetic Arc mobile game has been intentionally designed to handle its real-life backstory with a lighthearted and entertaining touch. The reasons behind these decisions can roughly be divided into three parts: reaching casual gamers, creating a better flow for the story and limitations of the mobile platform. These reasons and historical accuracy aren’t necessarily mutually exclusive, but in this project the choice was made to favor entertainment and casual approach.

Handling time and order of events

When a several hundred pages long novel is adapted into a movie, the events of the story are shortened and perhaps rearranged to some degree, in order to make all the important things fit in a more compact format and to keep the viewer engaged for the entire duration of the movie. Time and order of events have been approached somewhat similarly in the development process of the Struve Geodetic Arc mobile game for the exact same reasons: limiting the total length of the game and making it easy for the players to follow the events and keep them interested in the game from start to finish.

In reality, measurements across the entire triangle chain spanned several decades. The focus of the game is mainly on the section going over Lapland where measurements also took years. Exactly replicating all events from a time period this long would inevitably make the game longer despite the way time is handled inside the game. A typical mobile game session isn’t very long: GameAnalytics, a company selling tools for analyzing game usage, states in their 2019 report that the average session length for a mobile adventure game was less than 15 minutes (GameAnalytics Mobile Gaming Benchmarks Report: H1 2019, p.13). If the total length of a game is measured in several dozen hours like in some current AAA-titles for PC or console, it would take a much larger amount of these average length sessions to finish the game compared to a situation where the total length is just a few hours. In the Struve Geodetic Arc mobile game one of the goals is to get the players to finish the game and go all the way to the northern endpoint of the triangle chain in Hammerfest. So, if the game in total is very long and following that the required amount of game sessions needed to finish the game is large, it is possible the players feel that the game is too long and won’t ever finish it. Keeping the players engaged until the end of the game is a good enough reason to give up accuracy.

It was never intended that the actual historical timeline would be completely removed from the game, however. At the current development phase (spring 2021) the game is being designed to have calendar years pass according to the game’s internal time and to give the player a goal to reach the endpoint the same year the actual measuring crew got there. Some known places of stay might also be matched with the correct years.

Much like handling time inside the game flexibly, it was decided that the order of events would be presented in a more linear fashion compared to reality. In the game the player’s party follows a route along the triangle chain in one direction, whereas in reality the measuring crew did not move linearly from point A to point B. During the measurements the crew stayed longer in some places than others and measurements in a certain area could be done as separate trips from one place of stay. In the early stages of design, it would have been possible to choose game mechanics that supported moving in a way more closely resembling the actual measurements, but the development team decided to make movement on the map more linear. The team felt it would be more motivating for the player to follow a clear route from one place to another compared to e.g. separate trips from one place and returning there every time. One factor in this decision was that in the earliest design stages the team didn’t have the information on the more exact routes, it became available at a later phase. Lacking this information, it was easier to design the game to have a linear route.

Handling time and events in a flexible manner sacrifices much of the historical accuracy. The reasoning behind this sacrifice is making the game more approachable. No background information or prior interest towards the subject of the game is required from the player. The game also doesn’t aim at educating players by force. The thought is first and foremost to offer a good experience. Then on the side tell the player about a subject that was possibly previously unknown to them and get them interested enough to find out more through other means.

Effects of mobile platform

Regardless of genre and content of the game, limitations set by the platform must be taken into consideration when developing a game for mobile devices. Even though the physical size and resolution of screens on mobile devices have been growing, the screens are still many times smaller compared to a typical laptop or desktop screen. This sets limitations to the number of menus or other elements that can be fit on the screen at one time while still maintaining legibility. In addition to this, mobile devices are typically controlled with finger input, which is considerably less accurate way of navigation compared to a game controller or a mouse. This in turn limits the smallest possible size e.g. buttons can be.

When better accuracy is required, mobile devices can be controlled with a stylus. A small number of stylus-controlled games have also been made. Controllers made specifically for mobile gaming are also being made and sold, these closely resemble console controllers. Both styluses and controllers are mostly sold separately, so when the aim is to reach as large audience as possible and make using the game easy, it is not reasonable to require the user to have any kind of special controller devices.

In terms of game design, screen size affects (among other things) the amount of information the player can be shown at one time. This amount can be regulated for example by showing only the information that is required in that precise menu or at that precise moment of the game. Another option is dividing the information into smaller sections and having separate menus or menu levels for each section. Neither of these options are without problems. If the total amount of the information the player needs is large and most of it is contextual and not shown to player at all times, the limits of short-term memory might prevent the player from keeping all required things in mind. Then again, multi-leveled menus can be hard to navigate. This means the only solution isn’t just dividing and categorizing information or changing the way it is presented. The game itself must be designed to be simple enough so that the amount of required information isn’t too great.

Poor accuracy of finger input means that the basic functions of the game can’t require precise or complex actions from the player. The controls of the game must be simplified to a few most important buttons and simple gestures so that the learning curve isn’t too steep and continued use is effortless.

The need for simplification means more adaptations and loss of historical accuracy. Actions that in reality are complex or demanding have to be made easier and more straightforward. In the case of Struve Geodetic Arc mobile game one example of this is the triangle measurement itself. The aim is to give the player factual information on the measuring process, but using the measurement devices required skill and precision and the actions won’t be precisely replicated in-game. The realistic measurement actions will be replaced with simplified versions or with an entirely another set of related activities. When the basics of the game are easy to learn and playing doesn’t require complex actions, more casual players or people who haven’t really played at all, can easily start playing Struve Geodetic Arc game.

What historical things are included in the game?

Despite putting game mechanics and the requirements set by the chosen platform first in the development process of Struve Geodetic Arc mobile game, the intention is not to make a generic adventure game and just have it wear a cosmetic skin with some historical influences. The project team has gathered information on e.g. the progression of the actual measurements, known places of stay and the day-to-day -life of people in the 19th century. The game development team has then incorporated this information into their designs. The game will have player to encounter problems similar to those the real measuring crew encountered. The remoteness and challenging weather conditions of Lapland affected e.g. the ways measuring equipment could be transported and when measurements could be performed in the first place. One example of these conditions is that during the 19th century the actual road network in Finnish Lapland, excluding winter roads, reached Kittilä, Kolari and Sodankylä, so when the player moves further north they can’t choose to use a road and have to advance by other means.

One goal of the game is also to have the player understand, how significant the measurements were to the society at the time. Using the information gathered from the measurements, new more correct maps could be drawn and the significance of good maps to trade and warfare for example was and still is great.

Visual elements are one of the ways used to convey information and feeling of the time period to the player. Even though the art style of the game is somewhat comic-like, the appearance of e.g. clothes, tools, buildings and measuring equipment is modelled after their real-life counterparts. The intent is to show events and life from nearly 200 years ago as lively and interesting despite deciding to give up true historical accuracy in many ways.

More on The Northern parts of the World Heritage Struve Geodetic Arc project:

More on the Struve Geodetic Arc in general:

Game Analytics report of 2019:

Written by Sanni Mustonen | 31/03/2021

UX and Service Design in the FrostBit lab

Besides being an intern in the FrostBit Software Lab, I am also a PHD student major in service design in the University of Lapland. I am very honoured to be an intern student in FrostBit. I got a chance to apply to UX and service design in different projects, and I am happy to tell you about my experiences.

What is UX and service design?

Recently, UX design and service design have been very topical: you maybe have heard about many products and software applied with UX design and service design principles. However, what exactly is UX design and service design?

UX means User Experience. Normally, in terms of design software, UX designers focus more on the logic in software. Through UX design, users will not feel confused or get lost in the 2D (applications, website and etc.) or 3D (VR or AR technologies) worlds. In most situations UX design links closely together with UI design (User Interface design).

Until now, there is still no single official definition to point out what is service design. From my own understanding, the key point in service design is “empathy”, which means that service designer must think of the “service process” from all stakeholders’ perspectives. For example, if you are developing a B2C websites, customers and staff who used the websites, as well as even some third parties are included as stakeholders. Furthermore, service design involves the whole process given the process design solution. Let me demonstrate you an example with a restaurant: when you first time see the logo and face of the restaurant, the service design already begins. The dining environment, taste of food, personal service and even the feedback you give yourself – all of these can be part of service design processes.

From my perspective, service design is more like a strategic design: in addition to the product itself, service design provides a more efficient and valuable design process solving through the “pain points”. Service design has a broader scope, which does not only include UX design, but also graphic design, product design, information design and so on.

Below, you can see a basic design process that I always follow:

How did I end up in UX and service design in the FrostBit Software Lab?

During my intern period in FrostBit Software Lab, there are two main things I have done: one is re-designing the FrostBit websites and the other one is doing interior design to the lobby outside the lab office. You could think that the website design is connected to the UX design and service design – which is true. In my design process, I implemented animations for the website, which could catch customers’ glance immediately when they arrive at the website, and push them to pay attention on the important information on the site. Moreover, some icons are combined with texts: in this way, the info could be transferred to readers in a very short time. Although there are common tips designers always apply in the web design, what I want to talk about is the way of thinking: thinking for readers and information provider at the same time. When you do it this way, you can define the core of design solution, balance the requirements from both two sides, and understand what is the “empathy ” I mentioned before.

The interior design for the lobby has the same thinking process: the process is determining the target group as well as determining the aim and main functions as well as the design elements (for each group) in the lobby. From my experience, I used UX and service design to plan the “big picture” of the lobby, and then use the interior design as a method to apply and show my ideas and design solutions.

During my internship, I did not only increase my basic design skills, but also practiced the key points of service design in the FrostBit software lab. In my opinion, there are three certain key things that you need to keep in mind during design process:

  • How to increase the value
  • How to use co-design
  • What is the new idea or solution

What about the future of service and UX design in FrostBit?

I am definitely sure the service design and UX design have a bright future in the FrostBit Software Lab. No matter how the world is changing, every progress that Frostbit software made could change or instruct people’s behaviour. This is why we use service design to see the whole picture and give “correct ” directions for our projects. At FrostBit, we are not only applying the “real” design skills, but also the way of “empathy” thinking in our projects and we are continuously working towards ways to establish this. If you want to see how we use UX and service design in our project to create inspiring outcomes, stay tuned for updates on our websites!

Written by Li Nan | 10/03/2021

Game Dev team working in FrostBit

We already have about 40 employees working in the lab, as well as, of course, our trainees, exchange students and students working on their thesis. Much of this is happening in the Game Development team amongst various game projects. Our number of employees has grown tremendously in recent years and a lot of new different skills have come along. We are no longer just a team of engineers coding in a dark lab with the power of ‘Jolt Cola’. Professionals in 3D-modelling, graphic design, service design, audiovisual and game pedagogy, among others, have joined the team. This has brought along everyone’s personal skills to the team. Our team has really strong skills to establish holistic projects from the beginning to the end. We constantly work closely together with projects on the Mobile and Web team, since almost every one of our projects needs at least a website or back-end system. Building a complex background system is the core competency of the lab’s Mobile and Web team.

Getting to know the mining processes in the Mantovaara open mine

What do we actually do in the Game Dev team? Our main tools are different game engines: we mainly use Unity3D or Unreal game engine according to the project. While our team name is Game Dev Team, we do not only make games in the “traditional format”. We utilize game technology, for example, for visualizations, simulations, learning environments and marketing. This makes the work really versatile; we get to work with different substance industries and we have made various implementations utilizing game technology ranging from particle physics to medical care. However, we don’t only cooperate with different material industries – we also often get to get to know and study what it is like to work in the industries, such as mining. A good example is the KaiVi project, during which the programmers and modelers from the lab went on internship to the mine to get acquainted with the mining operations. Such action is paramount when it comes to building environments that reflect reality. It is also important for the development team to have some understanding of the substance industry and thus be able to communicate effectively with professionals.

Multidisciplinary skills and learning

The full use of gaming technologies requires the cooperation of several professionals. Each project begins with designing and defining, involving substance experts, designers, programmers as well as artists. At this stage, pedagogical solutions to learning environments are also considered and service design tools are included. The actual implementations involve different stages depending on the project, but typically the project starts with conception and technology tests or prototypes, from which the transition to the actual product development takes place. Depending on the project, suitable tools are selected for the project, as well as the necessary methods for carrying out the project. We use the Scrum Method in several projects: the method enables agile development, and works very well in the current situation, where work takes place largely remotely.

We use new technological equipment and solutions in several projects. For example, we get to implement different virtual reality environments and also look for new solutions to take advantage of virtual reality. However, virtual reality is just one of the new technologies we take advantage of. In addition to VR, we can utilize artificial intelligence, machine learning, sensors, motion platforms and controllers in our projects, amongst other things. We are always striving to find new ways to leverage technology in our operations to get the best results.

While we each have our job roles in the lab, that doesn’t mean the programmer just needs to code, or the modeler to model. Depending on your own skills and state of mind, you can participate in various tasks in a variety of ways. There are project planning, articles, presentations, development assignments, internship guidance, technical leadership, workshops, webinars, and many other tasks important to our operations. Cliché, but true; no workday is similar and no project is similar.

Some of our staff also work as part-time teachers in ICT training. It is rewarding to be able to share our know-how with future engineers, and additionally, through it be a laboratory that makes its own contribution to the development of the area. In the upcoming summer 2021, we are once again involved in arranging the Summer Game Studies in the Lapland UAS. The planning phase is ongoing and the greatest Summer Game studies are on the way! Be sure to follow the FrostBit publications, as we will definitely report the mood from the SGS during summer.

Go check out our projects in the lab Portfolio

Written by Toni Westerlund | 26/02/2021

Visibility for mining industry with gamification

There are as many as 46 mines in Finland, with more than 5,000 employees in 2018. Especially in northern Finland, the mining industry is an important employer for many and is an source of livelihood in areas where diverse employment and welfare development can be otherwise challenging ( There is also a wide range of education possibilities of mining industry throughout Finland, which is why the reform of degree structures and the development of modern learning methods are topical due to the growing demand of the field.

Since mining is such a practical field, how could its education be modernized? The Migael-project answers this question with gamification and modern technologies. The aim of the project is to develop mining education by creating a virtual learning environment with a variety of scenario-based exercises. By the beginning of 2021, three exercises have been completed in the project and the fourth exercise will be finalized during February. Each exercise focuses on different “scenarios” in the open-pit-mine as well as in the underground mine. For each exercise, different platforms and technologies have been utilized to achieve the learning goals:

In the first exercise, the player will be able to practice the charging of explosives in 2D view, as well as view the simulation of charging and blasting in both 3D- and Oculus Guest virtual views. In the exercise, the player gets to simulate charges of different sizes and see what their blasting effect would look like.

The second exercise was also created to be played with Oculus Guest’s VR-glasses. In this exercise, the player will be able to do an occupational safety card inspection at an underground mine. The player is able to move around the underground mine and observe the necessary safety measures and fill in the occupational safety card accordingly.

Driving departure
The third exercise was created to be played on both Oculus Rift VR-glasses and Android phones, and this exercise will go through a mining vehicle run-in inspection. The player must check the worker’s safety equipment and the condition and usability of the mining vehicle before it can departure.

PRE- and POST-blasting safety measures
In the fourth exercise, the player performs the pre-blast and post-blast safety measures. The exercise was made into a more traditional 3D-desktop “serious game”, since it is possible to make more use of textual teaching material on a wide range of topics. The player gets to observe objects in the open-pit-mine environment to secure the area before and after the blast.

Different technical implementations enable different ways of presenting or simulating the subject to be taught. VR-technology is able to take a player or learner to a realistic learning situation that responds as much as possible to a real life situation. In this way, for example, training situations can be usefully simulated when the implementations in real life would be demanding, costly and often dangerous. Implementation on a mobile platform establishes easier learning regardless of time and place, and this can help to reach a wider group of users. Therefore, in the third exercise, for example, the driving departure was implemented for both VR glasses and mobile.

However, virtual glasses create their own challenge if the material and subject area is extensive and requires more than observation and “tangible measures.” While instructions for “floating texts” can be established on virtual glasses, interactive texts combined to player movement and other activity can create an unnecessary challenge in the VR-world. For this reason, the fourth exercise of Migael was created as a 3D-desktop game, as the topic required a lot of text in the form of teaching material and other information. In this exercise, it was necessary to plan particularly carefully which operations should and could be “gamified”, as there are several safety measures related to blasting. Some of the most generic and difficult operations were implemented in the form of cinematic transitions and instructions given by the game’s characters:

Thus, different technological implementations can enable different ways of learning and achieve certain learning goals. Some technologies allow for a more realistic simulation of exercises, while others allow for a broader presentation of the content and the possibility to reach a larger target audience. The exercises produced in Migael form a gameful learning environment where you can experience the benefits of many new technologies and learn about the mining industry in an inspiring and safe way. The aim of the project is to make the mining industry more visible and interesting, especially for students, and to establish important and safe training regardless of time and place.

In addition of the fourth exercise being completed soon, the early part of the year 2021 is eventful for the project: a Teams-webinar titled as ‘Utilization of modern technologies in the mining sector’ will be held on 25.01.2021 as part of the project. In addition to the FrostBit lab, the webinar will feature speakers from VTT and Kajaani University of Applied Sciences, who will present, amongst other things, the possibilities of gamification, VR / AR technologies and new sensor technologies in the mining industry. Mining staff, project staff and students of Lapland University of Applied Sciences will be invited to join and listen to the webinar.

The project’s activities will be updated on its official website, which includes a blog and the downloadable exercises:

In addition, see the FrostBit portfolio-page of Migael:

The northen Ostrobothnia of Centre for Economic Development, Transport and the Environment has granted 337 502 € from the European Social Fund (ESF) and state funding for the Migael-project. The total cost of the project is 450,002 €.


Written by Tuuli Nivala | 15/01/2021

Cross-platform mobile development, the Flutter experience

FrostBit Software Lab has followed the development of the Flutter platform (created by Google) with great interest for the last few years. Our laboratory has also actively applied Flutter in various projects to enhance the development of our mobile applications. The need for an efficient cross-platform technology has been great, since producing two separate native mobile applications (Android + iOS) has proven to be too expensive and inefficient for our purposes, whether considering the development or the maintenance phase.

The strength of native mobile applications lies in the possibility of total customization and the native access to the features of the mobile phone (camera, sensors etc.). On the other hand, the weaknesses of the native mobile application approach are the need for a sizable human resource and the maintenance challenges provided by the rapidly evolving mobile phone ecosystems.

Previously our laboratory has created applications by using the PhoneGap –platform, but the limitations it provides to mobile application design proved to be too great for the technical needs of the FrostBit Software Lab.

Google released the first version of Flutter in the year 2017, and it immediately sparked some interest within the developers at FrostBit Software Lab. At that time, we had two sizable mobile projects upcoming in our project calendar, so we bravely decided to try Flutter out in both projects to avoid the challenges that come with the development of two separate native mobile applications.

Going towards the year 2021, we are tentatively yet extremely positively surprised and optimistic when it comes the possibilities of using Flutter in creating cross-platform applications. We are certainly going to use it further in upcoming projects as much as we can.  Then again, nothing in this world is perfect, and Flutter also shares its weaknesses, despite all its strengths. For this purpose we have gathered our own experiences on Flutter from the past few years, and here’s our conclusion of what we think of Flutter so far:

Flutter strengths / pros:

  • Development is easier when compared to PhoneGap or Xamarin, for example, since Flutter needs less platform-specific code
  • Flutter is cross-platform; it’s possible to develop applications for Android, iOS, Windows, Linux, MacOS, web etc. at the same time
  • It’s fast to develop applications on Flutter
  • Flutter allows the developer to create complex and fully customized UI components, since everything related to layout can be altered
  • Flutter has good documentation and a great number of examples are available. Flutter’s userbase also grows rapidly all the time
  • Flutter is powered by a very performance efficient 2D-UI engine (sky_engine), which works well on a 120Hz display as well
  • Flutter uses the Dart-programming language, which is an easy-to-learn OOP language
  • Google is a huge organization, which has the resources to develop Flutter efficiently. Google also uses Flutter in some of its own products
  • Flutter-code will eventually compile into native application code on different platforms and devices
  • Flutter already has good development tools, for example, for Android Studio and Visual Studio Code
  • Flutter is rapidly being updated

Flutter weaknesses / cons:

  • Complex user interfaces can be tedious to develop
  • Since Flutter is updated frequently, a great number of changes are integrated into the platform continuously, which can make large projects difficult
  • Both Dart-language and Flutter are relatively new technologies, because of which they are also rapidly being updated. This problem will fix itself in the future though.
  • Many of the best features of Flutter are still in development stage (null safety, web and desktop application support etc. )
  • If Flutter does not have a plugin for a certain platform specific feature (Android or iOS), you will need to make platform specific code
  • Because of Flutter’s popularity and its relative easiness for application development, there are numerous third party plugins which can have questionable quality
  • Since Flutter is a new technology, so called “best practices” and recommended architectural designs haven’t been really formed or standardized

In conclusion, we think Flutter has way more strengths than weaknesses, most of which are related to the fact that Flutter is still a new technology.

If we also take a look into a few internet articles related to this subject, we can conclude that many other developers in the world have similar thoughts with us (e.g. Rozwadowski 2020;  Costa 2019; Sannacode 2020;  Powalowski 2019). According to these articles, Flutter weaknesses also include the large size of the mobile application as well as certain compromises that are related to the mobile application’s layout and user interface design recommendations of different platforms (e.g. material design on Android or iOS). Here at FrostBit Software Lab, we do not consider these weaknesses to be too problematic in our projects for now, however.

We have used Flutter in two major projects, first being “Arktori” and the other being “DWELL”. In the Arktori project, we are developing a mobile application, with which the rectors of the northern part of Finland can network with each other, as well as enhance their professional knowledge and also mentor each other in their daily work. In the DWELL project, we are developing a mobile application, which promotes and endorses communal living in apartment buildings. The pilot apartment building in DWELL project is the DAS Kelo –student dormitory. We have been quite happy with Flutter in both projects when it comes to mobile application development.

Some screenshots from the Arktori and DWELL mobile applications:

Arktori mobile application (Flutter), development version, December 2020
Arktori mobile application (Flutter), development version, December 2020
DWELL mobile application (Flutter), development version, December 2020
DWELL mobile application (Flutter), development version, December 2020

Flutter has proven to be a powerful tool to create cross-platform mobile applications. However, it remains to be seen, how flexible and efficient Flutter will be in the future when it comes to web and desktop applications. If Flutter will be truly competitive outside mobile applications, it will be a realistic scenario (from the software developer’s point of view) to concentrate mostly on Flutter when developing applications, only to be supported by other technologies in cases, where Flutter is not an optimal approach.

For now, we are not completely certain here at FrostBit if Flutter is the all-powering efficient application development platform for all kinds of applications, but no one can deny that they surely are trying hard to do so!

In any case, we are going to continue watching the development of Flutter with great interest, and are always ready to try out Flutter in new projects and use cases in the future!

Written by Tuomas Valtanen | 11/12/2020

Referenced Flutter-articles:

Info about the Arktori project (in Finnish)

Info about the DWELL project (in Finnish)

A glimpse of R&D projects at FrostBit – WinterSIM project kicks off

1. FrostBit engineers testing the measuring station (Credits: FrostBit Software Lab)

The large scope and complexity of R&D projects encompassing topics such as digital solutions, smart cities, smart health, etc., are often difficult to understand from the public. Similarly, at FrostBit Software Lab, we are constantly working on multidisciplinary projects that blend game engines, XR solutions, web development, and data systems. To give some concrete examples of our daily tasks, the ongoing project of ‘MiGaEL’ aims to improve education in the mining industry through digital game-based learning, whereas the ‘VR Slaughterhouse’ project will create virtual reality training and simulation for reindeer husbandry. In one hand, FrostBit is also running the ‘Towards Sustainable Tendering’ project that will develop a web-based tool to enhance the process of tendering of acquisition that will lead to a long-term economic and environmental impact in the Lapland region.

Executing R&D projects is complex and requires comprehensive planning. How does this work in practice at the FrostBit Software Lab? When developing new tech solutions, we employ an iterative process based on researching, applying, and testing. I will try to break down the workflow by focusing on the case of the ongoing WinterSIM project: In brief, WinterSIM which is funded by Business Finland aims at turning Lapland and Snowbox testing facilities into the European ‘hub’ of winter testing and simulations for autonomous vehicles. The role of FrostBit in the WinterSIM project is to create virtual simulations for autonomous driving vehicles using game technologies such as Unreal and the open-source CARLA simulator. This will be done using sensors, LIDAR’s and cameras, which will gather real-world data, for example, snowfall and ice, to create the virtual simulation environment. The data will then be exported into the CARLA simulator and combined with urban and rural road winter conditions.

Designing the measuring station – data collection point

FrostBit engineers have designed a tailored measuring station which will be used to collect the data from the real-world simulations. The measuring station will be located at the Lapland UAS. The measuring station includes two LIDAR sensors, a depth-sensing camera, a radar, and a weather station. The measuring station will be controlled remotely supported by a wireless connection. Currently, the engineers are testing the station and preparing for real data collection. Additional measuring devices will be used for data collection in the later stages of the project.

2. The 3D visualization of the measuring station and the components (Credits: FrostBit Software Lab)

Utilizing CARLA open-source simulator

The CARLA simulator is an open-source simulator developed with Unreal Engine 4 that will be used to develop the winter layer (conditions) simulation for autonomous driving vehicles. FrostBit Software Lab will be the first one to create the winter layer for the CARLA simulator. The expected outcome is a validated virtual winter simulation in CARLA, which combines data from the measuring station, a digital twin of the Aurora Intelligent Road, and 3D surroundings of the Aurora Road.

3. Unreal 4 Engine

RoadRunner to create a virtual model of the Aurora Intelligent Road

The FrostBit engineers use real map data in RoadRunner to create a virtual model of the Aurora Intelligent Road. RoadRunner is fully compatible with Unreal Engine 4 and enables altitude model, point cloud, and aerial view. By using RoadRunner we ensure that the virtual model represents the real road as accurately as possible.

Data collection through 360 degrees video in Muonio, Lapland

Since there was not sufficient material available, the FrostBit team went to the town of Muonio, Lapland, and took 360 degrees video in order to use the material as an accurate source for 3D modeling of the surroundings of the Aurora Intelligent Road. We will use Blender (open-source) to model a virtual environment. The 360 camera was mounted on top of the car, which drove 40 km/h alongside a 10 km long road.

4. Filming the Aurora Intelligent Road in Muonio, Lapland using a 360 camera mounted on top of the car (Credits: FrostBit Software Lab)

Early-phase prototype

In the video below, you can see an early phase prototype of the working simulator. We are ready to integrate the dataset into the simulation as well as 3D models of the Aurora Intelligent Road. The aim is to create a virtual simulation environment in which the visual winter condition can be simulated according to the data. For instance, users can simulate heavy snow or very icy road conditions. The first results of the visual winter simulations are promising although the project is in the early stages.

5. A prototype version of winter condition simulations (Credits: FrostBit Software Lab)

The project is a joint collaboration between companies and universities such as Lapland Proving Ground, Ramboll Finland, Lapland University of Applied Sciences, Roadscanners, Sensible 4, Toptester, and Aurora Snowbox.

If you would like to know more about the project, you can contact Pertti Rauhala at (

The FrostBit team working in the project include Pertti Rauhala, Tuomas Herranen, Reko Paananen, Samuli Valkama, Miko Piitsalo, Aleksi Narkilahti, Juha Petäjäjärvi and Joel Koutonen.

Teachers working in the WinterSIM project: Minna Korhonen, Tauno Tepsa and Petri Hannula.  

Written by Erson Halili | 01/12/2020

How can immersive technologies (VR/AR) foster meaningful education and travel experiences during the pandemic?

1. VR in teaching at Lapland University of Applied Sciences (Photo credits: FrostBit Software Lab)

Often, challenging times bring resilience and inspiration for creative solutions. Now more than ever, digitalization and technology have kept us going and connected; nevertheless, we all have experienced the effects that a long day of working or teaching with digital tools has on our mental and physical health. For numerous reasons, working and connecting through a flat-screen is simply not enough.  As humans, we need more; we need to connect in a meaningful way. These challenges and difficult times have fostered the rise of creative digital solutions, especially immersive technologies (virtual and augmented reality, VR/AR) since they enable us to travel without the need to leave our home or perform tasks that we will not be able to complete otherwise. Is this the right time for VR/AR to be the next digital tool and tech disruption that will elevate the way we work, connect, and travel? All the odds are in favor, as well as the hype.

How can VR/AR add value to education?

Education is definitely one of the areas that VR/AR technologies can create a significant impact based on the fact that most of the teaching and training is conducted remotely at the moment. The current teaching and learning methods through online lectures and classes lack interactivity and versatility; hence, immersive technologies can enhance the way we teach and learn. Think about the possibility that a teacher would put a VR headset and ‘teleport’ worldwide during the geography classes using Google Earth. Similarly with other school subjects, students can have a virtual visit, which is immersive and engaging, for instance, to the Colosseum of Rome during the history classes. A recent Stanford University study conducted with middle school students examined the differences in cognitive learning by comparing the usage of desktop and VR videos to teach about crystal reefs and ocean acidity. The study showed that the students who watched the VR videos had higher scores in learning compared to the students who watched the desktop videos. In addition, the students who experienced VR videos about the coral reef showed higher scores on the perception of self-efficacy compared to the group who watched the desktop videos.

2. Virtual visit of Florence, Italy through Google Earth (Photo credits: Google Earth VR, Steam)

One major drawback is the costly prices of such technologies, although the devices are becoming more affordable and of higher quality, as time passes. Besides, teachers in the universities and especially high schools, have limited access to the technology and VR/AR devices because specific R&D or research groups mostly use them. An additional significant factor that has prevented the widespread of VR/AR technologies in teaching relies on the fact that teachers are often scared to implement new technologies in teaching, ensuring that they complement the curriculum goals. What is the starting point then? Being curious and embracing innovation in teaching! In the following steps, have a general understanding of the technology and how you could utilize it in teaching. Ideally, if your school has a dedicated lab or environment for VR/AR, that is the first place to go. Ask for a general introduction and possibly request to borrow one device that you can explore by yourself. If that is not possible, then there is a lot of information about the role of VR/AR in education online. Moreover, if your organization does not have professional devices, you can start by experimenting with more affordable options in the market, for example, Samsung Gear VR. Entering the world of immersive technologies is still a confusing pathway but fortunately, the accessibility to the VR/AR information and technology is increasing day by day.

3. Student painting in VR

Which are the subjects that VR/AR can play a critical role at the moment?


Due to the complexity of the situation regarding the physical distancing, healthcare is one of the sectors that need solutions that would enable remote collaboration and simulations. Several studies have proven the effectiveness of VR/AR in healthcare training and simulations; however, the employment of immersive technologies in healthcare is in the very early stages. According to the Gartner Hype Cycle for Digital Care Delivery including Telemedicine and Virtual Care, VR/AR for care delivery are ranked as ‘on the rise’ technologies, indicating that the real potential will evolve simultaneously with the technologies and applications. Therefore, the XR (extended reality, VR/AR/MR) simulations in the healthcare sector are estimated to have a market value of around 850 million euros by 2025, followed by a significant improvement of the technical and market value chains globally.

Considering the importance of immersive technologies in enhancing learning and safety in the healthcare sector, Lapland University of Applied Sciences has applied for international funding for a project which will create cutting-edge VR/AR solutions for healthcare education and medical staff. Our goal is to make healthcare training and simulations accessible to the students and professionals, enable hands-on experience which is safe and enhances learning. Another recent study indicates that when medical students train in VR, they scored better in all categories compared to the traditionally trained group, especially regarding information retention. The total test score showed an overall improvement of 230%. The future of healthcare training and simulations is surely based on immersive technologies.

Industry Training and Simulations

Similarly, with the healthcare sector, enterprise training and simulations is another critical sector where immersive technologies can be a solution for the disruptions caused by the pandemic. Long-term remote work has called for solutions for remote training and simulations. XR training and simulations for different sectors of industry is one of the key areas of expertise at the FrostBit Software Lab. We have developed VR training and simulations for the mining industry, renewable energy production, reindeer herding, real estate, and forestry.

4. Creating the digital twin of the district heating power plant (Credits: DUKE project)

The on-going DUKE project will develop a digital twin (digital representation) of the district heating power plant in Jänkätie, Rovaniemi, which will give students and new operators hands-on experience on the operation of the plant without the need to physically be there. A cost-effective and safe solution, which will make learning more accessible and effective for the students.


The tourism sector in Lapland has been massively impacted by the pandemic and the negative effects are predicted to be long-term. While hoping for a better future, very few companies have shifted their attention towards innovative and creative solutions to sell their experiences. That might be due to the lack of awareness about the possibilities on the market, technology readiness, or the costly technological solutions. This is a time when the travel industry should adapt and look for creative and innovative solutions to reach their customers. XR solutions will not solve all the problems, but they will give hope and new markets. How can we create an immersive experience of Santa Claus Village, Northern Lights, or Lapland Landscapes without the need to physically be here? Surely, it is a complex task but the technology is here. We have to step up and implement it. The Amazon Explore platform was launched to provide people with virtual traveling experiences. You can pay to have a tour with a private guide in different cities of the world, such as having a virtual walking tour through Mexico’s city urban art scene. The experiences are video and desktop-based; thus, not immersive. Traveling experiences are immersive and VR leads to immersion.

Fostering distance collaboration through immersive technologies – the Arctic perspective on XR at VR Days 2020

This year, we were invited to share our XR expertise and use-cases at the most comprehensive VR/AR event in Europe which this year was held remotely. VR Days blends different fields in which immersive technologies are applied such as business, art, training and simulation, education, hardware, funding etc. This was a major step for FrostBit Software Lab and Lapland University of Applied Sciences to feature among the most influential individuals and companies of the XR industry such as Oculus, HTC, Facebook, Google and much more. As the first Finnish VR laboratory, FrostBit has a long history of solving-real life challenges with immersive technologies. For instance, creating a virtual graveyard experience for Salla’ museum of War and Reconstruction which would allow the visitors to access the German soldier graveyard located in the Finland-Russia border area. The VR experience enables an authentic graveyard visit without the need to go through the border control between the two countries. Check out the speech for VR Days 2020 below:

Erson Halili speaking at VR Days 2020 New Horizons (Credits: FrostBit Software Lab)

What to consider when planning and creating immersive experiences?

The quality and the content of the VR/AR experience can vary on the desired outcome of the specific experience; however, there are certain key steps that are critical when planning a VR/AR project. Most importantly, the combination of engineering, psychology and education are essential in creating meaningful VR/AR experiences. My perspective when planning XR experiences: combining cognitive psychology, media education and user-experiences. What makes a VR experience meaningful? Consider these tips:

  1. The XR experience should solve real-life problems. Although that might not always be the case, technology will solve a problem that is not possible to be solved otherwise.
  2. Employing user-centric design. The XR experiences should emerge from the users and carefully designed with them.
  3. Strive for meaningful experiences. Meaningful experiences engage the user and there is a clear intended outcome at the end of the experience.
  4.  Make sure you combine multi-disciplinary teams and skills. As stated above, combining multi-disciplinary teams will possibly ensure that the three other points are considered. Multi-disciplinary teams and skills lead to holistic user-centric experiences. That is a core strength we have here at FrostBit, where we proudly co-work with a diverse team of engineers, educational specialists, designers, artists, etc.
6. Visitors experiencing VR at FrostBit Software Lab

We, here at FrostBit Software Lab (Lapland UAS) are on a mission to make technological solutions accessible to the community. Therefore, we are organizing info sessions with teachers, educational specialists, decision-makers, and companies in Rovaniemi and Lapland region on how to utilize immersive technologies during the pandemic. We want to give our support to overcome the barriers caused by the pandemic and we believe that immersive technologies are a powerful assisting tool. Are you ready to explore VR/AR technologies and understand how it can be of help in your subject or business? Send us a message in advance to have a personalized meeting and demo. For teachers at the Lapland UAS and the University of Lapland, you can pop-up anytime at the FrostBit lab facilities.

Written by Erson Halili | 17/11/2020


Hakkennes, Sh., Craft, L., Jones, M. (2020). Hype Cycle for Digital Care Delivery Including Telemedicine and Virtual Care. Retrieved on November 15 from:

Muller Queiroz, Anna Carolina & Nascimento, Alexandre & Tori, Romero & da Silva Leme, Maria. (2018). Using HMD-Based Immersive Virtual Environments in Primary/K-12 Education. 10.1007/978-3-319-93596-6_11.

Pottle J. 2019. Virtual reality and the transformation of medical education. Retrieved on November 12 from

Digital Twins with Game Engines

A digital twin (Digital Twin) is a virtual model of a product, process, or service. It can be used to visualize the operation of the object and its related information. Digital Twins for Leveraging Renewable Energy (DUKE) project, funded by Lapin Liitto and executed by the Lapland University of Applied Sciences, has its first functional pilot done in the Lapland Education Centre REDU’s educational heating plant in Jänkätie, Rovaniemi. It will be implemented with the Unity 3D game engine.

The game engine is powerful

The strength of game engines is the efficient and fast implementation of the visual environments. They are particularly well suited for 3D modelling. In addition to the visual model, the digital twin contains functionalities that can also be found in the real-world counterpart. The most straightforward functionalities to be modeled are, for example, details related to the mechanical properties of the system. In the visual model of a heating plant, mechanical objects to be modeled include, for example, modeling of building doors, inspection hatches, manually adjustable valve handles and electrical switches. Modeling doors and inspection hatches involves a visual effect that the observer sees after some input information triggering the door opening has been provided. The input information can be a virtual operation of the user to open the door or some movement of the cursor by the user in the vicinity of a suitable visual key element.

The functional model can be implemented with a game engine

There are numerous different options for implementing a key element or measure. Behind the door, a view opens to another space and through the inspection hatch, a view inside, for example, a boiler. Depending on the operating mode of the system, the boiler can be passive or produce heating power. When the heating is active, the visual effect after opening the door can be, for example, a visible combustion reaction inside, i.e. flames of fire. In the real world, opening the inspection door may also be accompanied by an emergency alert or other similar process-related action. In a real-world situation, a cross-effect, for example, on a malfunction of a vacuum that takes care of the pressure difference in the furnace, can also cause smoke flaking out of the inspection hatch. As a combination of the functional scenarios of the digital model, a description of the process is created, which in the case of game engines has to be modeled and programmed with the accuracy and to the extent required in the model.

In addition to the visual effect, sufficient realism may also require the creation of sounds that corresponds to reality. The sound associated with the opening of the door itself and the sound world carried from another space enhances the immersion of the model, i.e. the sense of presence or realism felt by the user. The concept of immersion comes from the gaming industry and especially from experiencing virtual reality.

Producing a functional model requires modeling the process

Producing a functional model of a complex process requires an understanding of the basics of process modeling. Systems combining mechanical, thermodynamic, hydraulic, and electrical subsystems, for example, have typically been modeled by describing the subsystems individually or with tools suitable for process modeling, such as Matlab Simulink, Labview, or Ptolemy II. In these simulation programs, it is possible to implement a time model required by the processes.

The functional model of the process describes the progress of the process in real time

In the time model, the time of the process progresses in real time or even faster than the actual rate of occurrence. The subprocesses take place in order and by mimicking the chain of events of the actual process so that the dependencies, i.e. causality, are realistic. In this case, for example, the fuel entering the boiler burns and, as a result of the combustion, thermal energy is transferred to the liquid circulating in the system. In a district heating plant, the circulating liquid is water and, for example, in ground source heat pumps, a water-ethanol mixture. Modeling of the time model and process progression can be implemented in its simplest way by thinking of a stable operating point for the system, where all the functional elements to be modeled in the system have settled into a stable state.

For example, in the modelling of a heat plant, the steady state is one in which the temperature of the circulating liquid, the volume flow of the circulating water in the circuit, and the pressure differences in the piping and boiler systems are in a stable state. The heating plant model has two separate water circuits connected by a heat exchanger. Through the heat exchanger, the thermal energy of the primary water, i.e. the water heated in the boiler circuit, is transferred to the secondary circuit, i.e. the district heating network.

Progressing the process over time is worth doing simply

The functionality of the sample heating plant was modeled so that at the start of the simulation, all variables, i.e., temperatures, pressure differences, and flows, have zero initial values. The calculation of the stable operating mode starts from the pump of the primary or boiler circuit and first the pressure difference produced by this power is calculated on the basis of the power set on the pump. The calculation of the pressure difference is started from the numerical value zero and the pressure differences of all elements of the circuit are calculated in small steps and at each cycle the pressure is raised to a state where the pressure difference in the pump and the set pump power theoretically correspond to each other. The pressure information produced by the pump is transferred in each revolution to the next element in the primary circuit which is the pipe. In the simplest model, the pipes transfer the pressure difference directly forward to the next model as is, so there are no pressure losses in the pipes. The pressure drop for the pipes can of course be set as a parameter, but in the simplest model it is not necessary.

First, the static state of the process is determined

The calculation proceeds element by element and the pressure difference moves forward for each circuit element. The elements form a system in which, viewed in the direction of rotation, they have a pressure difference as input data and a pressure difference after the element as output data. If the element is a valve, the numerical value of the valve position as input data is checked and based on this the pressure change produced by the valve is calculated and passed to the next element as input data. When the entire circuit is computationally rotated element by element, the calculation proceeds to the second stage, where the flow and temperature losses of the water flowing in the circuit are calculated on the basis of the pressure difference, if they have been defined for that circuit element. The temperature of the liquid circulating in the circuit is determined on the basis of the boiler power data and the volume flow of the flowing water. The pressure differences, flows and temperatures of the elements of the secondary circuit are calculated in a similar way. In the secondary circuit, the calculation is started in the same way as in the primary circuit for the pump producing the pressure difference. There is no heating boiler in the secondary or district heating circuit, but it corresponds to the output connection of the heat exchanger, which produces the heating power of the primary circuit to the secondary circuit.

The static model is sufficient for some of the simplest modeling. A more realistic model also includes modeling the dynamic properties of the system. Dynamic properties are the change phenomena that occur in the system. Such dynamic properties include, for example, increases and decreases in boiler power and flow changes in the secondary or district heating circuit. Modeling the dynamic properties leads to solving the differential equations describing the system. The simplest model of the differential equation is the so-called first-order differential equation, the mathematical solution of which is an exponentially descending or ascending graph. For example, the boost circuit boost can be modeled with a first-order differential equation if, as a result of the boost, the circulating water temperature rises to some new temperature value without the temperature behaving unstably. For example, temperature oscillation is an example of instability. A first-order differential equation is often a sufficient model for the dynamic properties of a system.

The phenomena of change, i.e. dynamic properties, are modeled with differential equations

The functionality of a simple dynamic model can be determined, for example, by determining the time during which the temperature rise occurs, i.e. the time constant of the temperature rise rate, by means of a step response test. By determining the time constant, the temperature rise corresponding to the power increase and the delay in the system, i.e. the so-called dead time, a dynamic model sufficient for most practical cases can be implemented. The dynamic properties of the sample heating plant were modeled by performing the test runs of the heating plant on 25.5. – 27.5.2020. During the test times, e.g. stepwise boiler circuit power increases and district heating circuit flow changes. During the changes, changes in the temperatures and flows of the primary or boiler circuit and the secondary or district heating circuit, as well as pressure differences, were measured. Based on the test measurements, the dynamic properties of the heating center were determined using regression analysis methods. The models of dynamic properties calculated on the basis of test runs and measurements are transferred to a functional model describing the properties of the system by programming them in the game engine into the program code of the model. The programming language is the Unity 3D game engine C#.

The visual and functional model combines into a digital twin

The final digital twin combines a visual model and a functional model describing the functionality of the heating plant process into a single entity. The goal is to produce the desired functionalities feature by feature. Each increase in functionality develops the system in a more usable and real-world direction. The digital twin, in its more advanced form, can be in real-time communication with its physical counterpart, so that operating operations through the digital twin can even control its real-world counterpart or alternatively the real-world thermal power plant functional change is realistically reflected in the digital twin. For example, so that the popping of smoke in the real world is heard and seen in the virtual world. The smell of smoke in the virtual model is still a fictitious feature for the time being, but in the future, perhaps that too can be implemented.

Official name of the project: Digital twins for leveraging renewable energy
Project timetable: 01/01/2020 – 31/12/2022
Total budget: 761 732 €
Funding: EAKR 2014-2020
Contact person: Tauno Tepsa (+358 40 821 6865)

Written by Tauno Tepsa | 16/10/2020