Apple has barely mentioned augmented or virtual reality in its big keynotes lately, but at WWDC 2022 earlier this month, the company quietly released arguably one of the best 3D room mapping tools for mobile AR yet.
called RoomPlanthe ARKit Swift API uses the camera and LiDAR scanner on recent iPhones and iPads to create a 3D floor plan of a room, including key features such as dimensions and types of furniture.
However, it is not (yet) for consumers. Apple says it aims to appeal to professionals such as architecture and interior designers for conceptual exploration and planning, as well as developers of real estate, e-commerce or hospitality apps; developers can integrate RoomPlan directly in their AR-compatible apps.
When it was released earlier this month, Jonathan Stephens, Chief Evangelist at the spatial computing company said EveryPointtaken RoomPlan for a test drive to see what it could do. The results are quite surprising.
Follow along as I structured a series @Apple RoomPlan is testing and sharing my findings/notes in this thread.
First I tried to trick RoomPlan with a large mirror. Surprisingly, it wasn’t fooled! Also, it was far away from the height of French doors.#WWDC22 #AR #ARKit #AI @Scobleizer pic.twitter.com/R4hJbO57Km
— Jonathan Stephens (@jonstephens85) June 7, 2022
RoomPlan seems to be able to handle some traditionally difficult situations, including the mirror above, as well as cluttered spaces, open and closed doors, windows, and generally complex architecture. Still, Stephens’ house isn’t just a bunch of cubical rooms, so there are a few pieces that just don’t fit together.
Test #2 – Vaulted Ceilings. I noticed that the wall shapes should be rectangular. It couldn’t follow the angle of the ceiling line. As a result, parts of my walls have become much larger than in reality.
It did a great job picking out the desks and bedroom furniture. pic.twitter.com/fbu5B9L3Ds
— Jonathan Stephens (@jonstephens85) June 7, 2022
Vaulted ceilings, wall openings, multi-storey spaces like you might find in foyers were all a bit too difficult for RoomPlan to digest correctly. While it’s not perfect, it seems to be autocorrecting at least to some degree based on some assumptions about how things fit together best.
Here’s probably the coolest find yet. Looking down from the top, the walls correct themselves based on some of Apple’s assumptions. pic.twitter.com/KblqeLYm5x
— Jonathan Stephens (@jonstephens85) June 7, 2022
RoomPlan isn’t just for app integrations though. Apple says it outputs in USD or USDZ file formats that contain dimensions of each item recognized in the room, such as walls or cabinets, as well as the type of furniture detected.
If you want to fine-tune the scan, the dimensions and placement of each individual part can be adjusted when exported to various USDZ-compatible tools, such as Cinema 4D, Shapr3D or AutoCAD, Apple says.
We’re still no closer to learning when the company plans to release its rumored mixed reality headset or its full-fledged AR glasses, but an AR or MR headset would have extremely robust mapping capabilities. need the space. Seeing Apple make these kinds of moves using its existing platforms certainly shows that they are on the right track.
If you haven’t been following Apple’s rumor mill, check out some of the links below about the company’s mixed reality headset, codenamed N301:
What we (think we) know about the N301 Mixed Reality headset
A special thanks to Hrafn Thorisson to alert us to the news!