Visual collaboration needs more than just pixels and pointers

More than a cursor

The first thing to consider is a more direct and tangible way of manipulating data. User interaction made a huge leap forward to the mouse pointer with the advent of windowing user interfaces. However, the model of television screen and typewriter keyboard persisted probably until Apple released the iPad into the world and brought users closer to their content.

But although the mouse virtualised allowing users to touch screens, most of the interaction is still based around a tiny point of connection to the data behind the screen – a blinking cursor or an arrow.

Multi-touch screens allow for more actions, using more than one finger to simply point, but this requires proximity to the screen, which may not work well in a group collaborative environment.

More sophisticated gesture-based interfaces are required to allow even closer connection to the data, but at a distance that allows human interaction. Those who have seen the music conductor-like fictional interfaces of the Iron Man films or Minority Report will have wondered if elements of such spectacular approaches could be brought into their offices and workplaces.

Some of the ideas are already in use in gaming systems and interfaces, with the use of cameras in particular for picking up and tracking gestures. For entertainment these work well, but they may not fit as effectively into the workplace, where cameras may not be the best solution for allowing several people to participate and collaborate when precision and assurance of choices made are still required.

So, physical and direct connections still have an important role to play and remote point-and-click – either controlled by specialised wands, such as in augmented reality applications, or collaboration tools, such as Oblong’s Mezzanine – might be more practical in the short term. The plethora of sensors in mobile phones may mean that, over time, specialised wands will be replaced by the ubiquitous general-purpose mobile devices that everyone already carries.

Immersive and mixed reality

Audio-visual output is a huge part of ISE and, over the years, advancements in presenting information in a more immersive way has generally resulted in more and bigger screens. Available and accessible screen real estate has improved massively, with clearer, higher-definition and brighter, more readily deployable screen technology.

However, the key for collaboration is not necessarily just to see more data, but to see through it more clearly to the information behind. The mechanisms for the presentation of information need to evolve so that masses of information presented can be digested and understood more readily and easily.

This problem escalates when, as in group visual interaction and conferencing, multiple different streams of information are in play at the same time. Overlapping or resizing windows is no longer enough – additional visual context needs to be applied.

Some of this will need to be automatic. A common feature of videoconferencing is to automatically flip to, or enlarge, the camera where sound is emanating, so that there is more emphasis on the person speaking. But too much automation could lead to confusion – sometimes there is a need for a chairperson or coordinator to ensure good flow in the meeting.

Adding a third dimension immediately creates visual context. Content that is “close” appears more important than that which is distant and might be easier to manipulate – just see how well Tony Stark manages in Iron Man.

One way to accomplish the third dimension in display without the Hollywood-inspired hologram would be to use virtual reality (VR). Three-dimensional spaces projected inside headsets can also allow multiple people to collaborate in the same virtual place even if not based in the same locations.

This is worth investigating further when planning future collaboration strategies, and progressive organisations might consider attending the XR (multiple reality) Summit at a side venue during ISE 2019. This one-day conference is focused on the business use of VR, augmented reality (AR) and mixed reality (MR) technology.

This might be a step too far for many organisations today, but there are other ways to bring three-dimensional context into display technology. Filling a meeting space with more and larger screens, whether through telepresence immersive videoconferencing or other multi-display systems, gives an effective illusion of three-dimensional content.

If this is physically sited around the participants so they can engage together, it will create an immersive experience. Finally, if perspective can be accurately applied to information or video streams to move them “further away” when less important, visual context will allow more critical information to be seen more readily.

Screens may appear to be a glass barrier that hold users back from the data, and sometimes, when used for presentation by a lone individual, they can be perceived as a mechanism of controlling access. But with immersive display and interaction tools that give individuals direct manipulation of data, they can be powerful collaborative environments.