If a fusion of AI and digital twins is at least a convenient abstraction to use describing the relationship, does a metaverse figure in? If so, how? Could it be that what I’ve called the “metaverse of things” or MoT, is what the AI/digital-twin fusion is, and if not, what is MoT in this picture? Very few enterprises from my recent sample of 76 had any comment on these points, but these early views are important.
One of the most important points enterprises have made to me about IoT, AI, digital twins, and real-world productivity enhancement is that they don’t see AI agents operating autonomously as often as they see them cooperating with workers. Enterprises who see large-scale operations enhanced by technology, using models based on digital twins, AI, or a combination of the two, see those models necessarily embracing some processes that remain fully human-controlled. Thus, there is a need to consider how these models interact with humans.
In simple automation, a model element that needs human assistance can display or print a message that will then be read and actioned by a worker. As things get more complicated, it may be necessary to provide workers with a more sophisticated visual representation of the model process and their own world, meaning some virtual reality picture. That’s where metaverses in general, and the MoT concept in particular, could come into play.
Metaverses are virtual worlds, and their single most distinguishing characteristics are that they have inhabitants and can be visualized in some way. Given that, it’s tempting to say that an MoT concept, as an extension of an AI/digital-twin partnership, would make sense in applications where one, or preferably both, those characteristics are present. However, deciding when that’s the case isn’t easy, say the four enterprises who made comments.
The four enterprises don’t feel that every digital twin, AI or not, that feeds a human system necessarily has an inhabitant or requires visualization in a metaverse sense. I think that the view comes down to this; if a human’s interaction is observational, then they’re not a participant in or inhabitant of the model system, and it’s not necessary to visualize the system as a virtual reality. If there is a human inhabitant of the system, one that is modeled by it and thus a participant, that also doesn’t mean the system is a virtual reality.
A better test, say these enterprises, is the nature of the control inherent in a digital model of the real world. If you visualize an IoT industrial process that’s autonomous, meaning that the digital twin system (with or without AI) decides what to do and then in some way commands devices to do it, then there’s no need for virtual-reality visualization. They do point out that “autonomous” here has to mean that not only is a human not regularly controlling the system, but also that they are not supervising it. If the system facilitates human control or is subject to human supervision, then the model has to present a visualization that supports the human role.
How about situations where the human is part of the process being twinned, but perhaps acting as a kind of sensor or effector, not actually running or supervising? This is where our four enterprises see complication, and to show why we need an illustration.
Let’s presume that we have a business process called “pick and ship goods”. It’s a warehouse-centric process that ties in goods to be identified and moved, a place where the goods are stored, a vehicle onto which the goods are placed for shipment, and something that moves between place and vehicle. Enterprises generally agree that this pick-and-ship model is usually inefficient and often opens them up to theft of goods, so it’s a regular target for automation of some sort. Whether that includes a virtual-reality, MoT, approach depends to a great extent on what that “something” is.
Suppose it’s a bunch of human workers. We’d expect a system to deliver a manifest for the vehicle, and workers would use this to grab the goods from the shelf and load them. All the workers need is the manifest, and if there’s no “supervision” process to validate that the right stuff is being loaded, there’d be little benefit to creating a VR representation of the process, so no MoT.
Now let’s suppose that the goods are all tagged with barcodes or RFIDs or something similar, and that these tags are read by a sensor at the point where the vehicle is loaded, to prevent something from being loaded that’s not supposed to be there, and to ensure the entire manifest of goods is picked and loaded. There might be a light set above the truck bay, with “red” indicating something not on the manifest was being loaded, “yellow” that the manifest wasn’t yet complete, and “green” that loading was correct. What we’ve done here is introduce supervision.
Worker walks up to truck with package. “Yellow” light, load away. Red light? Put the box aside and go back to the bins to pick again. Green light? Close up the vehicle and dispatch it, then go on to the next pick and load. The problem is that you now have three new problems to supervise. One, did the worker load anyway? Two, where did the wrong item get placed, and how will it get put back? Three, will the worker pick the correct item next time?
For the first problem, you could expect that the worker load the wrong item into a “put back” bin, with its own sensor set, and so you’d expect to see the item go there and not on the vehicle, but did the worker pick two items, put one back, and the other on the vehicle? You could add in sensors to try to home in on what’s happening, or let a warehouse supervisor monitor things. In the latter case, you may want to give the supervisor a VR representation of the process to facilitate that.
The notion of supervision, and perhaps some extra sensors, could address the other two points. Workers could be detailed to move things from the put-back bin to the shelves, and it would be possible to give workers a portable device to scan the tag on each package to get put-back directions and also to scan a tag on the location where the item was put to verify placement. If the pick and load process workers had the same device, they could validate each pick against the manifest, and that could reduce the need for supervision and for VR-type visualization.
The extent to which human interaction is essential in a modeled process, then, is the real measure of the value of an MoT extension into visualization, and the thing that likely differentiates a digital-twin model from an MoT model. What I find most interesting in reviewing the comments of my four leading-edge enterprises is that they suggest that a MoT conception of process automation may be important as we try to evolve to broader use of IoT and process automation, and not a later step to be added to refine the process. If that’s true, then we need to think about these points quickly.
