Sketch To AI - Oman Comix

An interactive public installation developed in Unity 3D during the initial launch of Stable Diffusion. The application connects a C# drawing panel interface with an Automatic1111 API backend to convert real-time user sketches into detailed AI-generated artwork. Using a dual-display setup, users draw on an interactive touchscreen panel while preset prompt templates and style parameters guide the Stable Diffusion generation pipeline. The resulting high-resolution output is dynamically rendered and presented on a large-format main display in real time.

Highlights

• This project was a major turning point for me. It was my first time working with Automatic1111, and it came with a steep learning curve. Initially, the team had agreed on a specific API with the client, but the client was completely unsatisfied with the initial output.
• Pivoting quickly, I discovered Automatic1111 and presented it as an alternative. The client loved it. However, this introduced a new technical challenge: there was no existing SDK to connect it to Unity, and I had never deployed to a cloud GPU like Vast.ai before.
• Under intense pressure, I worked right up to the final deadline. After countless hours of troubleshooting, I successfully routed the communication between Unity and the cloud GPU by exposing the necessary ports on Vast.ai. Despite a moment where the pressure made me want to quit, everything finally clicked, and the system delivered fantastic results.