Apple’s iPad arrived five years ago. It is a device that changed the way we think about computing, marking a seismic shift from keyboard and mouse to direct manipulation with our fingers. The iPad wasn’t the first tablet computer – it wasn’t even Apple’s first tablet computer – but it was the first to capture the world’s imagination and sell tens of millions of devices.
Nowhere is this more obvious than in the hands of children, who these days will walk up to any screen and expect to be able to interact with and shift content with the prod of a finger. This style of interaction has even followed us to our workstations where, despite their questionable use, touchscreens now frequently come as standard or are common options when buying a personal computer.
Touchscreens bring the user’s fingers into direct contact with the virtual objects onscreen, but still fundamentally present data representing a 3D visual environment through the medium of a flat 2D screen. Fully comprehending the interface relies almost entirely on our own visual sense, rather than exploiting our other, well-trained sense of touch.
From the pixel to the physical
Touchscreen tablets free us from the constraints of working at a desk and are more liberating due to their smaller size and weight. But, to make better use of all our highly-tuned senses, the next generation of displays will not be 2D and flat, but will have self-actuated, physically re-configurable surfaces. Flat screens will be able to deform themselves into other shapes. These interfaces will change the shape of their display surface to better represent on-screen content and provide additional means to pass on information by touch rather than vision alone.
Dynamic physical geometry – tablets with interfaces that morph in three, real dimensions, rather than simply displaying 2D representations of them – will fundamentally change the way we approach computer interaction. Displays with pixels that can physically protrude from the surface will allow developers to enhance familiar applications such as architecture, design, terrain modelling and photography by rendering computer-generated 3D scenes in three dimensions in the real world. This will opens all sorts of opportunities for novel applications in team collaboration, tangible entertainment and ways to make computing more accessible to those with disabilities.
Devices will be able to change their form and function: a mobile phone that mutates into a TV remote control, and then into a videogame controller, re-configuring itself to provide appropriate interfaces. Apps will not only be able to modify a visual display, but also dynamically change the physical properties of the device.
This display revolution is closer than we think: commercial ventures such as Tactus Technology’s Phorm already provide a way to generate fixed-position buttons that protrude from the screen by filling small pockets with liquid on command.
Building a physical screen
In our lab, we’ve begun to explore the implications of users interacting with shape-changing displays. We’ve created a 10×10 interactive bar chart with which to represent common data visualisation tasks such as displaying data, filtering data, organising it into different rows and columns, navigating between large datasets, and making annotations. We’ve found that the physical nature of dynamic bars encouraged users to directly manipulate data points for annotation and comparison-style tasks and that traditional touch-based controls work well for navigation and organisation tasks.
Certainly, constructing these shape-changing displays requires expert electronic and mechanical knowledge. There’s a need to involve people with a wide range of interaction design skills to drive forward early prototype design, so we developed a tool that allows non-technical researchers to experiment with shape-changing displays.
ShapeClip is a tool to transform any computer screen from a flat viewing surface to a 3D one, transforming light from the screen into movement through coordinates in physical space above it. By adding a z-axis to the screen’s x- and y-axes, designers can produce dynamic physical content by adding ShapeClip tools to screens. ShapeClip displays are portable, scaleable and can be re-arranged to suit need. They are also fault-tolerant. Users need no knowledge of electronics or programming and can develop motion designs with presentation software, image editors, or web sites.
The iPad shifted our approach from pressing buttons to pressing with our fingers. Future displays will not be flat glass screens we prod, but physically dynamic surfaces capable of reconfiguring themselves in order to better present information to the user through a rich tactile experience that offers more to our senses.
This article was originally published on The Conversation.
Nowhere is this more obvious than in the hands of children, who these days will walk up to any screen and expect to be able to interact with and shift content with the prod of a finger. This style of interaction has even followed us to our workstations where, despite their questionable use, touchscreens now frequently come as standard or are common options when buying a personal computer.
Touchscreens bring the user’s fingers into direct contact with the virtual objects onscreen, but still fundamentally present data representing a 3D visual environment through the medium of a flat 2D screen. Fully comprehending the interface relies almost entirely on our own visual sense, rather than exploiting our other, well-trained sense of touch.
From the pixel to the physical
Touchscreen tablets free us from the constraints of working at a desk and are more liberating due to their smaller size and weight. But, to make better use of all our highly-tuned senses, the next generation of displays will not be 2D and flat, but will have self-actuated, physically re-configurable surfaces. Flat screens will be able to deform themselves into other shapes. These interfaces will change the shape of their display surface to better represent on-screen content and provide additional means to pass on information by touch rather than vision alone.
Dynamic physical geometry – tablets with interfaces that morph in three, real dimensions, rather than simply displaying 2D representations of them – will fundamentally change the way we approach computer interaction. Displays with pixels that can physically protrude from the surface will allow developers to enhance familiar applications such as architecture, design, terrain modelling and photography by rendering computer-generated 3D scenes in three dimensions in the real world. This will opens all sorts of opportunities for novel applications in team collaboration, tangible entertainment and ways to make computing more accessible to those with disabilities.
Devices will be able to change their form and function: a mobile phone that mutates into a TV remote control, and then into a videogame controller, re-configuring itself to provide appropriate interfaces. Apps will not only be able to modify a visual display, but also dynamically change the physical properties of the device.
This display revolution is closer than we think: commercial ventures such as Tactus Technology’s Phorm already provide a way to generate fixed-position buttons that protrude from the screen by filling small pockets with liquid on command.
Building a physical screen
In our lab, we’ve begun to explore the implications of users interacting with shape-changing displays. We’ve created a 10×10 interactive bar chart with which to represent common data visualisation tasks such as displaying data, filtering data, organising it into different rows and columns, navigating between large datasets, and making annotations. We’ve found that the physical nature of dynamic bars encouraged users to directly manipulate data points for annotation and comparison-style tasks and that traditional touch-based controls work well for navigation and organisation tasks.
Certainly, constructing these shape-changing displays requires expert electronic and mechanical knowledge. There’s a need to involve people with a wide range of interaction design skills to drive forward early prototype design, so we developed a tool that allows non-technical researchers to experiment with shape-changing displays.
ShapeClip is a tool to transform any computer screen from a flat viewing surface to a 3D one, transforming light from the screen into movement through coordinates in physical space above it. By adding a z-axis to the screen’s x- and y-axes, designers can produce dynamic physical content by adding ShapeClip tools to screens. ShapeClip displays are portable, scaleable and can be re-arranged to suit need. They are also fault-tolerant. Users need no knowledge of electronics or programming and can develop motion designs with presentation software, image editors, or web sites.
The iPad shifted our approach from pressing buttons to pressing with our fingers. Future displays will not be flat glass screens we prod, but physically dynamic surfaces capable of reconfiguring themselves in order to better present information to the user through a rich tactile experience that offers more to our senses.
This article was originally published on The Conversation.
Limited-time offer: Big stories, small price. Keep independent media alive. Become a Scroll member today!
Our journalism is for everyone. But you can get special privileges by buying an annual Scroll Membership. Sign up today!