Create and Hold Anything with AI and a Merge Cube
Co-produced by Jaime Donally and Merge.
Using artificial intelligence (AI) and augmented reality (AR) together is a powerful combination, especially when combined with a Merge Cube! Because AI can now generate 3D models with text prompts, users can create virtually any 3D model and then hold and interact with it on a Merge Cube using AR technology.
What is a Merge Cube?
The Merge Cube is a soft, six-sided foam cube covered in unique patterns (known as AR markers) that make it look like an ancient alien artifact—ancient yet futuristic. When viewed through an AR-enabled device like an iPad with a Merge app, the patterns trigger the display of 3D holograms, transforming the cube into just about anything you can imagine. Users can hold, interact with, and explore the 3D models and simulations in real-time.
Creating 3D Models with AI Text Prompting
Traditionally, creating 3D models requires specialized software and a certain degree of technical expertise. But with the integration of text-based prompting powered by advanced AI algorithms, making 3D models is easier than ever. Users can generate intricate 3D models that can be uploaded, held, and explored on the Merge Cube by simply describing an object in natural language.
How It Works
Advanced Natural Language Processing (NLP) algorithms interpret and understand the user's text description. For example, a prompt like "a detailed model of a human heart with visible arteries and veins" is parsed to understand each component and detail.
The AI uses this interpretation to construct a 3D model. This involves sophisticated geometry processing and rendering techniques to ensure the model is accurate and visually appealing. Once the model is generated, it can be uploaded to the Merge Dashboard and viewed in the Object Viewer App on a Merge Cube. Learn how to upload objects on the Merge Dashboard here.
Tools for AI 3D Object Creation
Two of our favorite examples of software facilitating text-to-3D model creation are Meshy.ai and LumaLabs.ai. Meshy.ai employs sophisticated algorithms to convert text descriptions into detailed 3D meshes, providing users with precise and visually compelling models. Similarly, LumaLabs.ai/Genie leverages cutting-edge AI to interpret and render text prompts into 3D objects, streamlining the design process and ensuring high-quality outputs.
Both platforms integrate seamlessly with the Merge Cube, allowing users to upload their created models and bring them to life through augmented reality interactions.
Challenges
While the potential for AI 3D object creation is exciting, some challenges remain. The accuracy of AI in interpreting complex and nuanced descriptions needs constant refinement. Ensuring the fidelity and functionality of generated models is crucial, particularly for situations that require a high level of precision. As a relatively new technology, the software behind the text prompting of 3D objects is still evolving. It may not always perfectly interpret and execute every description, leading to models that require further tweaking and adjustment. This iterative process is part of the technology's growth and improvement, with user feedback playing a crucial role in its development.
It’s also important that the user interface for text prompting tools be intuitive and user-friendly, allowing individuals with no technical background to leverage the technology effectively. Continued NLP and 3D rendering advancements, combined with user feedback, will be key to overcoming these hurdles.
Final Thoughts
The ability to text-prompt 3D objects for the Merge Cube represents a significant leap forward in augmented reality. By simplifying the creation of complex models and making them accessible through a tangible interface, this technology has the potential to expand learning opportunities not only in education but also in medicine, design, and beyond. As we continue to explore and refine these capabilities, the boundary between the digital and physical worlds will blur, opening up new realms of possibility and innovation.
Steps to get started:
- Sign up for an account at Meshy.ai or LumaLabs.ai
- Add the text for an object you’d like to create.
- Select your 3D object from the choices available.
- Refine or make the object Hi-Res to finish your 3D object.
- Download the object as a gltf or glb.
- Open your Merge Dashboard and upload the 3D object. View your AR experience!
Do you want to get started uploading your Ai generated 3D Models to hold on a Merge Cube? Get a free trial at trymerge.com
For more information about AR/VR in the classroom, we highly recommend the books Learning Transported and The Immersive Classroom by ARVRinEDU founder, Jaime Donnaly.
