Self-described “maker of issues” Christopher Moravec has turned the parable of individuals’s smartphones and voice assistants continuously listening in to their personal conversions into actuality — with the intention to routinely generate topical paintings.
“The WhisperFrame listens to conversations in our front room after which generates artwork primarily based on these conversations,” Moravec writes of his venture. “[It] generates a brand new picture after each 5 minutes of lively dialog. When there hasn’t been any speaking, it should revert to displaying randomly chosen pictures generated up to now.”
The core idea of the venture, which has an always-on microphone recording snippets of close by dialog, brings up the pervasive however always-unproven fable of corporations utilizing smartphones and voice assistants to observe close by conversations for matters which may very well be data-mined and monetized. This time round, although, the very-real conversational recordings are being mined for thematic content material which may be fed to a generative synthetic intelligence (AI) system to create synthetic artwork.
The recordings are made in 15-20 second loops, then submitted to OpenAI’s Whisper software programming interface (API) for computerized transcription into textual content. When 5 minutes has elapsed, these extracts are fed to OpenAI’s GPT-4 massive language mannequin (LLM) with the immediate to extract one key subject and switch it right into a immediate for an image-generating mannequin — which is, in flip, fed to Secure Diffusion, the ensuing image downloaded, and the show up to date.
The imagery generated by Secure Diffusion is keyed to a single subject, drawn from the final 5 minutes by GPT-4. (📷: Christopher Moravec)
“It’s a bit self-fulfilling in that as folks speak concerning the picture it drew, it turns into extra probably that it tries as an instance that one once more, as the subject is extra prone to be chosen by GPT-4,” Moravec admits. “But it surely’s nonetheless superior! I even created a second one for my workplace that generates pictures throughout conferences! It’d even be a brand new solution to make assembly notes, an inventory of pictures representing the assembly as a substitute of motion objects. It most likely gained’t catch on, although!”
The complete write-up is out there on Moravec’s web site; all generated pictures can be found to browse on a devoted website.
