The
Touch Screen technology can be traced back at least to the University of Illinois at the time of the 1970s, when scientists installed the latest results in the lab on the PLATO IV computer for classroom-assisted instruction, and the touch screens was officially entered the laboratory for the first time. The actual application phase. However, the technology of
Touch Screen Panel is not mature, and commercial value has not been discovered for a long time. In the 1990s, the touch screen appeared in high-end PDAs (older readers must remember the fast translation and Wenquxing), but unfortunately still not warm
On the one hand, the market size of PDAs is not large. On the Other hand, due to the limited precision of touch screens, most PDA users prefer to operate directly with physical keyboards. In 2004, Nintendo's handheld game console NDS was sold globally. Its biggest selling point is that NDS has two screens and one of them is a touch screen, so the touch screen has entered more people's field of vision with the popularity of NDS.
However, until then, the use of touch screens was limited to a specific group of people. Really let the touch screen into the thousands of households or thanks to Apple's Joe. In 2007, the first iPhone was released, which redefines the smartphone. Before the iPhone was released, the definition of a smart phone was “a mobile phone that users can install software on their own”. After the iPhone was released, the definition of the smart phone became “a mobile phone with a screen that can slide.” The touch screen is not a symbol for the iPhone. Or missing modules. In fact, carrying a touch screen on the mobile phone is not the first, but the Apple iPhone uses the touch screen to completely update the operation mode of the mobile phone, and introduces dragging, zooming and other gesture operations in the iPhone, which greatly improves the user's operating experience. As iPhone sales continue to rise, Samsung, HTC and other companies are not to be outdone, have launched a smart phone with a touch screen and operating similar to the iPhone. Overnight, the phone's button sound could no longer be heard on the subway, because everyone started using the touch screen to "slide" the phone.
The touch screen first appeared in the PLATO IV computer (top left, and later appeared in the fast translation (upper right), NDS (bottom left) and other devices, and finally Apple launched the iPhone (bottom right) so that the touch screen really enters thousands of households.
After the iPhone was introduced, the touch screen did not stop updating.
The capacitive screen used in the Apple iPhone greatly enhances the operational feel and almost makes the sliding phone screen a pleasure. On the other hand, in order to make the phone lighter and thinner, the screen manufacturing technology has also experienced out-of-cell to on-cell up to the thinnest in-cell. Compared to the touch screen ten years ago, today's touch screens are thinner and have a better handling feel.
Break through the shackles of two-way foil: 3D touch
However, the exploration of touch screen interaction methods has not ended here. Here it is a good idea to review the evolution of the way mobile phones interact with users.
Initially, the user can only operate the phone by pressing a button that is fixed at several locations on the phone. This can be called a one-dimensional operation. After Apple introduced the touch screen and gesture operation, the user can move freely in the two-dimensional space of the touch screen of the mobile phone, so it can be called a two-dimensional operation mode. If our touch screen is no longer evolving, then the user-phone interaction dimension will be limited to two-dimensional, just like the civilization in the three-body foil hit by the two-way foil.
Naturally, we will think: Since humans live in a three-dimensional space, why is the user's interaction with the device only in a two-dimensional space? Is it possible to implement a three-dimensional interaction method?
To achieve three-dimensional interaction, it is necessary to capture the coordinates of the user's hand in three-dimensional space in real time, and respond accordingly according to the three-dimensional coordinates (and their changes) of the user's hand.
Fortunately, scientists and engineers have begun to develop 3D touch to achieve human-computer interaction beyond two-dimensional. Before specifically analyzing the technology, let us first look at the revolutionary application of the 3D human-computer interaction method.
1, the game (including VR)
When it comes to 3D touch, the first thing that people think of is the application in the game. Indeed, games are the most demanding way of interacting in all applications. Using a dedicated interactive device that matches the game, players can fully appreciate the charm of the game (don't believe you try to play the game with the keyboard), which is why some games need to develop special peripherals (such as the dance mat of the early "Dance Dance Revolution". "Guitar Hero" guitar, steering wheel of various racing games, etc.).
When the touch screen was just popular on the mobile phone, a group of games that perfectly utilized the interactive features of the touch screen, such as Angry Birds and Fruit Ninja, also won the favor of everyone. When the touch screen can capture the movement of the human hand in three dimensions, many new gameplays can be implemented in the game. A very promising direction is to combine with VR technology. For example, a game like "Fruit Ninja" can be played in the three-dimensional space by the player's gestures. The game of "Street Fighter" and other fighting games can be triggered by the player's real gesture. The player's sense of substitution is greatly increased.
3D touch screen combined with VR technology can greatly increase game playability
2. Augmented Reality (AR)
The 3D touch screen has the potential to become the basic technology for human-computer interaction in AR applications.
In the AR technology, the special glasses worn by the user are used as display screens, and the image interface of the AR is organically combined with the real world through computer vision technology and projected onto the retina of the user's eyes. When the user's hand makes an action, the AR device must be able to accurately capture the real-time position of the hand and respond accordingly according to the action of the user's hand. The 3D touch screen captures the precise position of the hand and becomes the basic technology for AR human-computer interaction. After the popularity of AR technology, not only mobile phones and computers will use 3D touch screens, but even the surface of everyday furniture (such as tables and cabinets) may need to support 3D touch technology, so that users can use AR anytime, anywhere.
Augmented reality combines computer graphics with reality (left), AR interacts with the user to accurately capture the three-dimensional position of the user's hand (right)
3D touch screen candidate technology
At present, the three-dimensional touch screen technology is still in the exploration stage. There are two most promising commercially available 3D touch screen technologies, one based on millimeter wave radar and the other based on capacitive sensing.
The 3D touch technology based on millimeter wave radar technology is represented by Google's Project Soli. In May of this year, Google officially released a 3D touch module codenamed Project Soli.
So, how does Project Soli's millimeter-wave radar implement 3D touch? First we have to understand the principle of radar.
Everyone must have seen the searchlight: in the dark sky, the position of the object in the direction of the beam of the searchlight can be seen clearly. The searchlight changes the direction of the beam by continuously rotating, so that the position of the object in all directions in the entire sky can be detected one by one. The radar is the same, but the radar emits not the beam visible to the naked eye, but the beam of the electromagnetic wave, and the detection circuit is used to detect the position of the object in the beam direction. Obviously, radar can also be used on 3D touch: the hand is the object that needs to be detected. Through the radar, we can monitor the position of the hand in space in real time and let the device react accordingly to realize three-dimensional human-computer interaction. This is also Project. The principle of Soli.
The searchlight detects the target by changing the direction of the beam (top left). The radar scans the target by changing the beam direction (upper right). Project Soli uses the principle of radar to detect the position of the hand to achieve 3D touch (bottom).
So what is a millimeter wave radar?
What is the difference between it and the huge radar that appears on TV? Originally, the resolution of the radar is related to the wavelength of the electromagnetic wave it emits. The shorter the wavelength of the emitted electromagnetic wave, the better the resolution, that is, the more accurate the position of the object is detected. However, the shorter the wavelength of the electromagnetic wave, the greater the attenuation in the air, so if the object is far from the radar, it will not be detected.
Therefore, object detection accuracy and detection distance are a pair of contradictions.
Traditional military and police radars use the microwave band, because the objects that traditional radars need to detect are usually very large in size, and the resolution of the microwave band can be about 10 cm. The traditional radar needs sufficient detection. Distance can meet the needs of use. However, the resolution accuracy of the 10cm level is not enough for 3D touch.
On the other hand, the distance required for 3D touch detection is very short, and the distance between the hand and the touch screen is usually no more than 20 cm.
Finally, the volume of the 3D touch module must be small enough.
Therefore, Project Soli uses a millimeter-wave radar with a wavelength of the order of millimeters, which theoretically achieves millimeter-level resolution accuracy. The radar can be integrated into coin-sized chips for installation on a wide range of devices.
The picture below shows the millimeter wave radar sensor chip used by Project Soli.
The chip size is about 8mm x 10mm, and the small white dot above should be the solder bump used to fix the chip to the Motherboard. There is also an antenna array (within the green frame) on the chip for beamforming. According to the size of the antenna, we can estimate that the millimeter wave radar used by Project Soli has a wavelength of about 2mm-5mm.
Millimeter wave radar can achieve high precision for 3D touch. However, its disadvantage is that the power consumption is too large. Even the most advanced millimeter-wave radar chips require at least 100mW of power, so using them on mobile devices can cause the battery to run out quickly. In this way, the millimeter wave radar touch is more suitable for use on a device where the power source is not a problem, such as a 3D touch on a large game machine or a television.
Another very promising 3D touch technology is capacitive sensing technology. Millimeter wave radar technology utilizes dynamic electromagnetic waves, while capacitive sensing technology utilizes electrostatic fields. Capacitive sensing type 3D touch technology is an enhanced version of the current Capacitive Touch Screen: the capacitive touch screen can sense the position of the hand that is in contact with the screen, while the capacitive inductive 3D touch technology enhances the sensing range when the hand has not touched the screen. It can sense the three-dimensional position of the hand in space, thus realizing three-dimensional touch.
In order to understand the principle of capacitive sensing 3D touch, we can imagine that there are many arrays of thermal sensors, and there is a flame (heat source) above the sensor array. Depending on the relative temperature distribution of the sensor (ie where the temperature is higher and where the temperature is lower) we can know which sensor the flame is above (ie the two-dimensional position of the flame), depending on the absolute temperature of the sensor (ie the absolute temperature detected by the sensor) How high) We can know how far the flame is from the sensor (ie the third dimension of the flame in space). Combining these two pieces of information, we can get the three-dimensional position of the flame in space.
The thermal sensor array can determine the position of the flame in three-dimensional space by relative temperature distribution and absolute temperature.
The principle of capacitive sensing 3D touch is also the same, except that the temperature change caused by the flame is not detected here but the electrostatic field change brought by the finger. By detecting which capacitive sensor detects the largest change in the electrostatic field, we can sense the two-dimensional position of the finger, and by detecting the absolute intensity of the electrostatic field change by the capacitive sensor, we can sense the third-dimensional coordinates of the finger, thereby realizing the three-dimensional touch. .
The advantage of capacitive sensing 3D touch is that the power consumption of the sensor can be much smaller than that of the millimeter wave radar (about one tenth or less of the millimeter wave radar), so it can be installed on mobile devices that are sensitive to power consumption. on. However, capacitive sensing also has its own problems to be solved, that is, mutual interference between sensors. We also use the thermal sensor to sense the position of the flame for analogy. Now we assume that the heat sensor will heat itself in addition to the heat that the flame will heat up. In this way, if the flame is far away from the heat sensor, the temperature change caused by it may be negligible relative to the heat of the heat sensor itself, so that the heat sensor requires a relatively high detection accuracy to detect the position of the flame according to the temperature change.
The same is true for capacitive sensing 3D touch: the electric fields between the capacitive sensors are coupled to each other to form a large capacitance, so the electrostatic field changes caused by the fingers need to be detected by a very high precision detector. Fortunately, with the development of circuit technology, even small changes can be detected by high-precision analog amplifiers, so the prospect of capacitive sensing 3D touch is very bright in the future.
At present, capacitive sensing type 3D touch has appeared in Microsoft's pre-touch screen demo, which can realize 3D touch closer to the screen (1-2cm).
On the other hand, many top universities' labs have also demonstrated 3D touch modules based on the principle of capacitive sensing.
For example, the team led by Professor Naveen Verma at Princeton University successfully demonstrated 3D touch based on thin-film electronics (currently established by SpaceTouch) and has the opportunity to be used on future flexible screens.
In addition, UCLA's Airtouch team, led by Professors Frank Chang and Dr. Li Du, uses traditional low-cost CMOS technology to match the normal mobile phone touch screen to achieve 3D touch within 10cm of the screen. The chip was first released at the International Solid State Semiconductor Conference in 2015 (ISSCC, the world's highest specification conference in the chip field, known as the Olympic event in the chip industry), after which the team used the hot iron to combine deep learning with 3D touch chips. High-precision 3D gesture recognition and was invited to present the latest results in the 2016 Auto Design Conference (DAC, one of the highest-profile conferences in the global electronic design field).
The Airtouch chip consumes only 2 mW (far less than Google's millimeter-wave radar touch solution) and is compatible with common touch screens. It can be widely used in 3D touch of mobile devices such as mobile phones in the future.
Conclusion
Touch technology has experienced decades of development, and today it has been able to enter the 3D touch field beyond traditional 2D touch. 3D touch brings innovation in human-computer interaction, and can be used in games, AR/VR and other applications. Currently, the more promising commercial 3D touch solutions include millimeter wave radar (represented by Google Project Soli) and capacitive sensing (represented by UCLA Airtouch). We hope to see 3D touch entering thousands of households in the near future and become the basic way of human-computer interaction.
