Text to glTF M3 Demo Day - HackMD (2024)

# Text to glTF M3 Demo Day[toc]---## Grant- cli tool- mirror posts - openvoxels.eth, m3org.eth, avirtualfuture.eth- dev logs - try text2gltf on hackmd.io/@xr/book posts- experiment on layouts - row - column - grid - circular - experimental / organic objects (trees of thought)---## Media### Layout Ideas- https://treeverse.app/view/esFYWzkd- https://twitter.com/graycrawford/status/1642695935565365252**Cascade with each page as separate glTF node**---## AI meeting summaryThe speaker discusses their work on an open source project related to the metaverse and virtual reality. They explain their goal of creating usable 3D documents without the need for extensions or standards. The speaker demonstrates various iterations of virtual documents, including a book-like format and a Wikipedia article converted into a virtual document. They highlight the challenges faced in rendering crisp text and share their solution of converting text into small polygons per character to optimize memory usage. The project aims to enable cross-platform document sharing and innovation without relying on extensions. The speaker showcases examples of 3D documents with clear text that can be viewed up close without high memory consumption. They mention the possibility of incorporating live data from the web into glTF files using HyperPi technology. The presentation concludes by highlighting the potential for spatializing documentation, facilitating learning, and creating immersive poster hubs through this project's procedural approach.{%youtube BfrDcUHW9x8 %}Watch: https://www.youtube.com/watch?v=BfrDcUHW9x8---## Action itemsBased on the transcript, there are several follow-ups and action items that can be identified:- Follow-up: Investigate existing efforts for creating documents in the metaverse. - In the transcript, the speaker mentions that there might be existing efforts to create documents in the metaverse, but none that they can currently use. It would be beneficial to explore these existing efforts to gather insights and potentially incorporate any relevant findings into the ongoing project.- Action item: Develop a tool to convert text into glTF models without extensions. - The speaker mentions that they have been working on a tool to convert text into glTF models, which can be used to create 3D documents in the metaverse. This tool aims to eliminate the need for extensions and provide a simple way to share and use documents across platforms.- Action item: Improve rendering of text in VR. - The speaker acknowledges that rendering text in virtual reality is challenging. They mention attempting different approaches, such as using JSON to load text into an interactive book-like structure or converting text into one-polygon-per-character models. The project would benefit from further research and development to enhance the rendering of text in VR and improve the readability of virtual documents.- Action item: Address material issues with HyperPi. - During the presentation, the speaker mentions encountering material issues with HyperPi, a component of the project. They need to investigate and resolve these issues to ensure the proper functioning of the tool and to enhance the visual quality of the documents.- Follow-up: Explore possibilities for incorporating live data into the documents. - The speaker mentions the potential to bring in live data from the web and have it refresh in the 3D documents created using glTF files. This opens up possibilities for dynamic and interactive content within the virtual documents. Further exploration and experimentation are needed to fully utilize this feature.- Action item: Continue improving and iterating on the project. - The speaker acknowledges that there are still many improvements to be made and mentions that the project has a lot of improvements in store. This implies an ongoing effort to refine and enhance the capabilities and features of the document creation tool and the virtual document experience.- Follow-up: Gather user feedback and address user needs. - The speaker mentions that if anyone has any questions or wants to know more about the process, they can reach out. Gathering feedback from users and potential users of the tool is crucial to understanding their needs and addressing any issues or limitations they may encounter.- Action item: Share the project for others to explore. - Towards the end of the presentation, it is mentioned that the project will be made available for people to check out. Sharing the project will allow others to explore its capabilities, provide feedback, and potentially contribute to its development.It is important to note that while these are the follow-ups and action items inferred from the transcript, the actual priorities and next steps may vary depending on the context and goals of the project.## OutlineBased on the transcript, I can create an outline with chapters and include timestamps. Here is the outline:- #### Chapter 1: Introduction to the Project - Timestamp: 00:18 - The speaker introduces their experience in working on metaverse projects. - Mentions the open-source project they are currently working on related to documents in the metaverse.- #### Chapter 2: Challenges in Sharing and Using Documents in the Metaverse - Timestamp: 00:50 - Discusses the current limitations in sharing and using documents across platforms in the metaverse. - Mentions the lack of existing usable solutions for documents in the metaverse.- #### Chapter 3: Goal of the Project - Timestamp: 01:26 - Describes the project's goal of turning blank text documents and markdown documents into usable 3D documents without the need for extensions. - Mentions the two main types of documents: horizontal and vertical.- #### Chapter 4: Rendering Challenges in VR - Timestamp: 01:45 - Talks about the difficulty of rendering text in VR. - Describes the initial attempt to create a virtual document using JSON and loading it into an interactive book.- #### Chapter 5: Creating a Tool for Text to 3D Conversion - Timestamp: 02:20 - Explains the need to create a tool that can convert text into glTF models without the use of extensions. - Shows examples of documents converted from markdown into 3D models.- #### Chapter 6: Crisp Text Conversion - Timestamp: 03:40 - Discusses the need to have crisp text in virtual documents. - Describes the engine created to convert text into small polygons per character, resulting in optimized and crisp-looking text.- #### Chapter 7: Project Overview - Timestamp: 04:44 - Summarizes the general idea of the project, which aims to provide cross-platform, open-source documents in the metaverse. - Mentions the ability to handle large documents without consuming excessive memory.- #### Chapter 8: Demonstration of the Project - Timestamp: 05:01 - Shows an actual export of the tool and demonstrates its functionality. - Mentions some material issues that need to be addressed.- #### Chapter 9: Progress and Future Improvements - Timestamp: 05:51 - Discusses the improvements planned for the project. - Shows another document to demonstrate the progress made.- #### Chapter 10: Questions and Answers - Timestamp: 07:27 - Invites questions from the audience.- #### Chapter 11: Potential Applications of the Project - Timestamp: 07:58 - Discusses the potential use of the project for writing, turning documentation into art, or using live data.- #### Chapter 12: Spatializing Documentation - Timestamp: 09:09 - Describes the concept of spatializing documentation, allowing users to interact with others and learn within the virtual environment.- #### Chapter 13: Conclusion and Summary - Timestamp: 09:39 - Summarizes the project's concept of generating a world from a book and the goal of creating a massive poster hub for documentation.Please note that the outline is based on the provided transcript and may not fully capture every detail or intention of the speaker.## Notes- Speaker is working on a metaverse project related to documents in virtual reality.- Speaker wants to share and use documents cross-platform without extensions or standards.- Speaker has not found any existing solutions for documents in the metaverse.- Speaker's goal is to turn text and markdown documents into 3D documents without extensions.- Rendering text in VR is difficult.- Speaker tried creating virtual documents using JSON and loading them into interactive books, but it required specific platform support.- Speaker developed a tool to convert text into glTF models without extensions.- The goal is to have horizontal and vertical documents for walking around and scrolling with hands.- Speaker created a virtual document based on a Wikipedia article with interactive images and simulated boxes.- Speaker needed to convert text into small polygons to achieve crisp text in large documents.- The project aims to provide cross-platform, open-source documents without extensions.- Speaker demonstrates the tool by showing exported documents.- There are still some material issues to be resolved.- The project allows for very large documents without using much memory.- The project works without any special glTF extensions.- Speaker invites questions and mentions that more information can be provided upon request.- The project is being showcased on the left side of the stage.- The project allows for interoperable text and can handle thousands of pages of documentation.- Hyperfi can retrieve live data from the web and refresh the glTF file in real-time.- Speaker forgot to include a screenshot showing a world generated from a book, where each white object represents a page.- The AI component allows users to see what others are reading and interact with them.- The goal is to spatialize documentation and create a collaborative learning environment.- The project aims to provide a library-like experience in a virtual environment.- The world generation is procedural based on the provided text document.---## Notes- https://github.com/MozillaReality/MOZ_text- https://www.youtube.com/watch?v=a_I1b9_x2LY## M3 NFT Collection ShelveEmpty space for NFTs not yet collectedCollect X amount to unlock something coolBookshelf / Trophy roomHave ability to hide something conditionally based on blockchain inventory, unhide when you have that asset. Need receiver app to hook up to the zone app.https://app.sablier.com/stream/LL2-1-84