What's the next step beyond Zoom and FaceTime calls? How about streaming from one device to another in real time using augmented reality? That's the premise behind a startup called beem announcing its first customer app, with a $4 million seed round, and its long-term plan to become a communications technology for the Augmented Reality (AR) glasses of the future.
Customers are introduced to AR technology through interactions with Snapchat Lenses and TikTok Effects, through mobile games like Pokémon Go, and by viewing products they are considering buying, such as furniture to be placed in their room or makeup being tried on virtually through an AR filter. But using AR for telepresence, as intended beem, is not a common use case.
The idea for the startup comes from a founder who grew up without much access to technology. The executive director of beemJanosch Amstutz describes his parents as "hippies" who lived in a small Australian seaside town, Byron Bay, where they raised cows and chickens, used solar energy, collected rainwater, and didn't even have a telephone, just a two-way radio. that the community would share.
“We put a lot of emphasis on face-to-face communication,” he explains of the lack of modern technology in his environment.
Meanwhile, communications continued to evolve from home phones to cell phones to Skype. "But then we stopped," says Amstutz. “The premise of beem is that there will be an inevitable next step in the way humans communicate digitally that is more believable and more immersive than making a video call.”
Amstutz initially rebelled against his upbringing by working in the physical commodity trade in the steel industry. But he did not find the job satisfying. He eventually realized that he wanted to do something else and, specifically, he wanted to address the problem of the evolution of modern communication.
The founder assembled a team of computer scientists and researchers to work on the concept, which today includes Product Manager Damian Hickey, formerly Product Manager at AR Pioneer Blippar; and Denis Islamov, co-founder and CTO, whose background is in applied mathematics and physics.
After its official foundation in 2017, beem (previously HelloMe) was not entirely aimed at the consumer market. Instead, he developed augmented reality projects and campaigns for brands, businesses, and other organizations. Examples They are campaigns for Vogue, Carolina Herrera, Hermes, Louis Vuitton, H&M, Forever 21, Warner Music, LADBible (in association with KSI and Craig David), the British Army, TEDx and many others.
During this time, technology beem it was used to stream an artist's music in AR, deliver a live conference to users' homes in AR, create "virtual catwalks," and create other kinds of AR experiences for their customers.
This experience allowed beem be maintained as it tested and advanced its technology, but the company was never intended to be a development studio, Amstutz says, it wanted to offer a consumer experience.
With the mobile application beem, the goal is to put this kind of live AR, or 3D-like experience, in the hands of end customers.
To work, users download the app beem for iOS o Android and position themselves in the camera viewfinder either by mounting the phone to capture the full frame or by having a third party record them. beem takes the video, segments the human in the video away from the background using its proprietary computer vision algorithms and cloud infrastructure, processes the asset in real time, and packages it for the viewer.
The recipient receives the link that they open in their mobile device's browser, where they are directed to a microsite that is activated for each AR-style interaction. beem accesses the phone's accelerometer, allowing calculations on the server to skew and transform the sender's video stream into its own space to give it the illusion of three-dimensionality.
The recipient picks up their phone and places the "livestreamed" person in their own space by touching a point on the floor, as they do with other AR tools.
The end result is what appears to be a kind of hologram of the person in the viewer's space, speaking to them in real time. You can zoom in on the person or move them around, but you can't see them from all sides, as that aspect wasn't captured in the original video. The quality is not as good as, say, that cute AR on TikTok o Google's AR Animals but it's smart since it's live.
“We don't need to be super technical… We need the viewer's mind to believe that what they're seeing is a real human being and that's enough,” says Amstutz.
Users beem they can use the app to host "conferences," like AR Zoom calls, where they broadcast the holographic version of themselves to up to 25 people with two-way audio. They can also launch one-way live broadcasts to a larger audience or send pre-recorded video messages.
beem it has applied for patents on its engine infrastructure and specific patents on different pieces of its technology, such as video segmentation, its proprietary web experience, and its method of creating 3D-like experiences without "really" being 3D. Two, so far, have been achieved.
To date, the app has been adopted by some TikTokers, included Lev Cameron, Jake Joseph Everett Rosey Michael Fallon. In the United Kingdom, beem it also partnered with a group of schools that tested the technology for teacher-student communications, with plans for a rollout to 750 students. During Valentine's Day, users of beem they sent more than 1500 AR messages.
beem it now has a cadence of around 500 messages a day and has seen around 14,2 million "beem" views in the past six weeks, the company says.
However, the reality is that the consumer use case of beem it's still a bit of a hassle: you have to mount the phone or have someone hold it, record yourself, and when recording a message, there was a couple of minutes delay while it processed before the link could be shared.
However, Amstutz says that the ideal use case of beem it will not be the mobile phone.
“The ideal circumstance for two-way telepresence to occur is to wear a pair of augmented reality glasses and simply have a tracking webcam in the room,” he explains. "So you could basically give voice commands to the glasses... and then the tracking webcam will know it's time to track me and record me, send me to you, and I'll see you in my space at the same time and vice versa."
The founder says that beem has a prototype of an AR glasses experience that it is working on for a set of AR glasses, developed by an undisclosed company and a major chip supplier. But ultimately beem wants to bring this system to any AR glasses in the future.
Along with the launch of its consumer app, beem announces $4 million in seed funding from 5 Lion, Ascension Ventures, Grouport Ventures, Inertia Ventures, Lior Messika and other strategic investors. The funds will go towards growing the team and further developing their product to be ready for AR glasses in consumer markets.
“Innovation in AR is creating huge opportunities for a variety of different markets, and the potential impact of AR is colossal,” said the investor from beem, Lior Messika. “The technology of beem it creates the infrastructure through which AR can become truly accessible and viral. The global shift to digital is happening fast and beem It is at the forefront,” he added.
