This content originally appeared on Angular Blog - Medium and was authored by Ayşegül Yönet
If you missed the Google I/O Angular session, you missed not only the latest updates on Angular but also the Generative Art Gallery demo which I had so much fun building. The demo has all of my favorite technologies, including AI, Angular and WebXR. It also was a very pleasant developer experience. Let me tell you why!
Unleashing Creativity with AI
How do AI, Angular and WebXR come together in this project? Your interaction with the Gallery begins with an input prompt. Here, you describe the style and theme of your desired masterpieces.
While Vertex AI turns your vision into unique artworks, Gemini API provides insightful explanations about each of the generated artwork. But that’s not all, Gemini API can tailor its responses to your needs, crafting JSON responses that perfectly align with your API specifications.
Vertex AI’s Imagen API: This API serves as the creative engine, producing five unique images that embody your chosen style and theme.
Gemini Multimodal API: Ask questions about each artwork, and it responds with detailed explanations tailored to your queries. Even cooler, it can return these descriptions in a JSON format, perfectly suited for integration within your application.
Finally the artworks are displayed in a 3D gallery, which you can interact with using your 2D browser, or you can view it integrated into your environment as an Augmented Reality experience on your mobile phone, or even get fully immersed in the artwork using a Virtual Reality headset.
Google I/O Gemini AI & Angular WebXR Demo
WebXR Performance Requirements
Great features come with great performance requirements. Making any web app performant and reactive to user interaction and data can always be a challenge, and with WebXR in the mix, performance is even more of an issue. Loading and displaying 3D assets takes significant loading time, while making multiple API calls to generate the artworks is also time consuming. Another big challenge is reacting to user input and movement in AR and VR experiences. When you are holding your phone in your hand or wearing a headset, your position and movement is used to render a stable image in your environment. We humans move ever so slightly all the time, even when we think we are completely still. That means, to look stable, the 3D virtual objects around you constantly have to adapt to your movements. What’s more, also all of this animation has to happen at least four times faster than a normal browser animation for your brain to perceive it as smooth. That’s why we need Angular’s help to deliver a performant app in the first place and respond instantly to user interactions.
The Power of Angular: Building a Seamless Experience
Several new Angular features played a crucial role in bringing this project to life:
- Deferrable Views: 3D models of a WebXR experience are especially load intensive and take longer to render the first view. Deferrable views ensure a smooth user experience by allowing you to load any large component in the background while the browser is in the idle state. Once everything is ready, the full gallery seamlessly replaces the placeholder. Additionally, you can leverage this feature to conditionally load the gallery component, avoiding unnecessary processing when the user isn’t actively interacting with the artwork creation process.
<art-image-gen (newArtworksEvent)="onArtworks($event)"
[class.hidden]="isHidden()" #gen />
@defer (on interaction(gen); prefetch on idle) {
<art-gallery [artworks]="artworks()" />
} @placeholder (minimum 5s) {
<art-loading />
} @error {
<p>Failed to load the gallery</p>
}
- Signals: With Signals, both the mental model of the data flow and the boilerplate are significantly simplified. The image-gen component, responsible for artwork generation and captioning, emits an event to the museum component when new images are available, triggering an update in the gallery view. Meanwhile, the museum component, acting as the parent, manages the state of both gallery and image-gen. When new images arrive, the gallery component updates its frames, while the image-gen component hides away. Since canvas and svg elements needs to be updated outside the regular DOM update, angular team gave us a way to let the developer know of the changes by using effects.
effect(() => {
this.frameService.updateFrames( this.artworks() );
});- WebXR Components: Creating scenes, cameras, lighting, and renderers often involves substantial boilerplate code. This project tackled this challenge by introducing an abstract scene class. The abstract class served as a foundation for building new components that extended its functionality. Imagine the time saved during the development phase, where numerous test components were created to experiment with layouts, lighting, and other aspects! This saved significant development time and allowed me to experiment with ease.
- Server Side Rendering and Hydration: By running `ng add @angular/ssr`, I was able to add Server Side Rendering and Hydration. Without doing anything else, the first contentful paint improved by 10 seconds and the largest contentful paint improved by more than 20 seconds. That is the fastest I have ever improved any codebase. It is truly magic.
A Refreshing Return to Angular
For developers who haven’t actively engaged with Angular in recent years, this project serves as a testament to the framework’s significant advancements. The ease of getting started, coupled with the new and improved APIs, fosters a delightful development experience.
You can try out the code on Github: https://github.com/google-gemini/angular-webxr-art-sample. I hope it inspires you to build your next Angular, WebXR and AI project. Happy coding!
Immerse Yourself in Generative Art: An Angular WebXR Adventure was originally published in Angular Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Angular Blog - Medium and was authored by Ayşegül Yönet
Ayşegül Yönet | Sciencx (2024-07-16T12:47:13+00:00) Immerse Yourself in Generative Art: An Angular WebXR Adventure. Retrieved from https://www.scien.cx/2024/07/16/immerse-yourself-in-generative-art-an-angular-webxr-adventure/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.