Univ. Iowa-developed 3-D teleconference system projects subject images on glass[/caption] "Help me, Obi-Wan Kenobi. You're my only hope." -- Hologram in original Star Wars movie that was too small for real teleconferencing. Despite the obvious success of that hologram SOS message in the original Star Wars movie (and of the movie itself), high-resolution video conferencing has been slow to catch on as a mainstream business-communications option, and even slower to fold out from 2D to 3D. Leaving aside the difficulty of acquiring an R2 unit in good enough condition to act as a display at the other end of a teleconference, it has been too difficult to create and transmit three-dimensional images of acceptable quality quickly enough to allow full-video teleconferencing in real time. Video, especially 3-D video, creates very large image files that transmit across wireless or wired networks too slowly to keep up with the words that go along with the pictures. "I was originally worried about transmission," according to Song Zhang, professor of mechanical engineering at the University of Iowa and leader of the 3-D teleconference project announced Oct. 31. Instead "it turned out we had to worry about all three" of the big technical problems involved in teleconferencing, Zhang added: capturing 3-D images, transmitting them efficiently and displaying them at the other end. Zhang and collaborator Nik Karpinsky, a Ph.D. candidate in human Computer Interaction, got around the networking problem by converting 3-D images to 2-D before transmission, then converting them back to 3-D on the receiving end. A PC's GPU provides enough power to run the system in full 3-D at each end, Zhang said. The data stream in between still added up to 700Mbit/sec, however, so the two added to the workload of the single PC on which the teleconference app runs by having it compress the video feed from 700Mbit/sec to less than 14Mbit/sec. Zhang has researched three-dimensional data compression across networks and published several papers about novel ways to accomplish it. That level of compression would allow high-res, full 3-D teleconference imaging even across wireless connections, according to Zhang, who predicts 3-D teleconferencing will be built into smartphones within a few years: "These phones are [already] powerful enough to do all the computation." The business end of the system is a projector that illuminates the person taking part in a teleconference. Cameras sitting to the left and right of the projector record a binocular image of the teleconferencer by reading the light from the projector as it bounces off the person. The images are combined into a single 3-D one that is flattened, compressed, transmitted, uncompressed and filled out into three dimensions quickly enough (30 frames per second) to allow real-time conversations without too much delay. The demonstration teleconference in a video posted here; it's transmitted only from the second floor of one UI engineering building to a room on the first floor of the same building. There may be greater problems sending 3-D images around the world, but not any that are significant compared to what Zhang and Karpinsky have already figured out how to do, Zhang said. The speed of networks and power of even ordinary PCs has improved quickly enough during the past decade that it can overcome most of the computational problems, as long as the imaging and networking are designed well enough to not introduce much additional latency to the link. “When Nik first proposed this idea to me," Zhang said. "I never believed we could reach this level by now." A shorter version of the explanation of the system Zhang and Karpinsky developed is on Karpinsky's personal page. The two published an initial description of the project, which uses the Holovideo codec to compress and decompress data, in Science Direct in 2011. Zhang will continue to develop the system. Karpinsky is preparing to graduate this year, after which he will go to work for Microsoft and will probably start a company of his own to develop and market the teleconferencing technology, he said. Image:Univ. Iowa/Nik Karpinsky Smartphones Soon to Get Real-Time 3-D Video Conferencing: Researcher
[caption id="attachment_13650" align="aligncenter" width="500"] Univ. Iowa-developed 3-D teleconference system projects subject images on glass[/caption] "Help me, Obi-Wan Kenobi. You're my only hope." -- Hologram in original Star Wars movie that was too small for real teleconferencing. Despite the obvious success of that hologram SOS message in the original Star Wars movie (and of the movie itself), high-resolution video conferencing has been slow to catch on as a mainstream business-communications option, and even slower to fold out from 2D to 3D. Leaving aside the difficulty of acquiring an R2 unit in good enough condition to act as a display at the other end of a teleconference, it has been too difficult to create and transmit three-dimensional images of acceptable quality quickly enough to allow full-video teleconferencing in real time. Video, especially 3-D video, creates very large image files that transmit across wireless or wired networks too slowly to keep up with the words that go along with the pictures. "I was originally worried about transmission," according to Song Zhang, professor of mechanical engineering at the University of Iowa and leader of the 3-D teleconference project announced Oct. 31. Instead "it turned out we had to worry about all three" of the big technical problems involved in teleconferencing, Zhang added: capturing 3-D images, transmitting them efficiently and displaying them at the other end. Zhang and collaborator Nik Karpinsky, a Ph.D. candidate in human Computer Interaction, got around the networking problem by converting 3-D images to 2-D before transmission, then converting them back to 3-D on the receiving end. A PC's GPU provides enough power to run the system in full 3-D at each end, Zhang said. The data stream in between still added up to 700Mbit/sec, however, so the two added to the workload of the single PC on which the teleconference app runs by having it compress the video feed from 700Mbit/sec to less than 14Mbit/sec. Zhang has researched three-dimensional data compression across networks and published several papers about novel ways to accomplish it. That level of compression would allow high-res, full 3-D teleconference imaging even across wireless connections, according to Zhang, who predicts 3-D teleconferencing will be built into smartphones within a few years: "These phones are [already] powerful enough to do all the computation." The business end of the system is a projector that illuminates the person taking part in a teleconference. Cameras sitting to the left and right of the projector record a binocular image of the teleconferencer by reading the light from the projector as it bounces off the person. The images are combined into a single 3-D one that is flattened, compressed, transmitted, uncompressed and filled out into three dimensions quickly enough (30 frames per second) to allow real-time conversations without too much delay. The demonstration teleconference in a video posted here; it's transmitted only from the second floor of one UI engineering building to a room on the first floor of the same building. There may be greater problems sending 3-D images around the world, but not any that are significant compared to what Zhang and Karpinsky have already figured out how to do, Zhang said. The speed of networks and power of even ordinary PCs has improved quickly enough during the past decade that it can overcome most of the computational problems, as long as the imaging and networking are designed well enough to not introduce much additional latency to the link. “When Nik first proposed this idea to me," Zhang said. "I never believed we could reach this level by now." A shorter version of the explanation of the system Zhang and Karpinsky developed is on Karpinsky's personal page. The two published an initial description of the project, which uses the Holovideo codec to compress and decompress data, in Science Direct in 2011. Zhang will continue to develop the system. Karpinsky is preparing to graduate this year, after which he will go to work for Microsoft and will probably start a company of his own to develop and market the teleconferencing technology, he said. Image:Univ. Iowa/Nik Karpinsky
Univ. Iowa-developed 3-D teleconference system projects subject images on glass[/caption] "Help me, Obi-Wan Kenobi. You're my only hope." -- Hologram in original Star Wars movie that was too small for real teleconferencing. Despite the obvious success of that hologram SOS message in the original Star Wars movie (and of the movie itself), high-resolution video conferencing has been slow to catch on as a mainstream business-communications option, and even slower to fold out from 2D to 3D. Leaving aside the difficulty of acquiring an R2 unit in good enough condition to act as a display at the other end of a teleconference, it has been too difficult to create and transmit three-dimensional images of acceptable quality quickly enough to allow full-video teleconferencing in real time. Video, especially 3-D video, creates very large image files that transmit across wireless or wired networks too slowly to keep up with the words that go along with the pictures. "I was originally worried about transmission," according to Song Zhang, professor of mechanical engineering at the University of Iowa and leader of the 3-D teleconference project announced Oct. 31. Instead "it turned out we had to worry about all three" of the big technical problems involved in teleconferencing, Zhang added: capturing 3-D images, transmitting them efficiently and displaying them at the other end. Zhang and collaborator Nik Karpinsky, a Ph.D. candidate in human Computer Interaction, got around the networking problem by converting 3-D images to 2-D before transmission, then converting them back to 3-D on the receiving end. A PC's GPU provides enough power to run the system in full 3-D at each end, Zhang said. The data stream in between still added up to 700Mbit/sec, however, so the two added to the workload of the single PC on which the teleconference app runs by having it compress the video feed from 700Mbit/sec to less than 14Mbit/sec. Zhang has researched three-dimensional data compression across networks and published several papers about novel ways to accomplish it. That level of compression would allow high-res, full 3-D teleconference imaging even across wireless connections, according to Zhang, who predicts 3-D teleconferencing will be built into smartphones within a few years: "These phones are [already] powerful enough to do all the computation." The business end of the system is a projector that illuminates the person taking part in a teleconference. Cameras sitting to the left and right of the projector record a binocular image of the teleconferencer by reading the light from the projector as it bounces off the person. The images are combined into a single 3-D one that is flattened, compressed, transmitted, uncompressed and filled out into three dimensions quickly enough (30 frames per second) to allow real-time conversations without too much delay. The demonstration teleconference in a video posted here; it's transmitted only from the second floor of one UI engineering building to a room on the first floor of the same building. There may be greater problems sending 3-D images around the world, but not any that are significant compared to what Zhang and Karpinsky have already figured out how to do, Zhang said. The speed of networks and power of even ordinary PCs has improved quickly enough during the past decade that it can overcome most of the computational problems, as long as the imaging and networking are designed well enough to not introduce much additional latency to the link. “When Nik first proposed this idea to me," Zhang said. "I never believed we could reach this level by now." A shorter version of the explanation of the system Zhang and Karpinsky developed is on Karpinsky's personal page. The two published an initial description of the project, which uses the Holovideo codec to compress and decompress data, in Science Direct in 2011. Zhang will continue to develop the system. Karpinsky is preparing to graduate this year, after which he will go to work for Microsoft and will probably start a company of his own to develop and market the teleconferencing technology, he said. Image:Univ. Iowa/Nik Karpinsky 
