A Monster for Every Season: Summer 2
You can get A Monster for Every Season: Summer 2 now at Gumroad
Page 1 of 3 123 LastLast
Results 1 to 30 of 67
  1. - Top - End - #1
    Firbolg in the Playground
    Join Date
    Dec 2010

    Default AI art generation

    There have been a number of advances in AI text-to-image synthesis this year - DALLE2, Imagen, Parti from big commercial labs in particular. However, alongside these, there's also been community pushes to make open versions of these things - one is a by-artists-for-artists project called MidJourney, but the other one that just released the weights and made everything publically available is something called 'Stable Diffusion'. This is the web interface they're working on: https://beta.dreamstudio.ai/home

    You can also run it in a colab, or on your own PC if you have a GPU with at least ~7gb of VRAM.

    So now you can make pretty high quality stuff (at 512x512 or 1024x1024 granted) by typing in a prompt and hitting generate. Great for having illustrations for stuff in a tabletop campaign - scenes, concept art for places, magic items, etc... My favorite mode isn't in the web-based on yet, but its basically something where you draw a really crude sketch and then it re-renders that as if a professional artist had illustrated it. So you can control precisely where different things in the image are, but then get a high-quality version of that.

  2. - Top - End - #2
    Titan in the Playground
     
    ElfRangerGuy

    Join Date
    Aug 2007
    Location
    Imagination Land
    Gender
    Male

    Default Re: AI art generation

    Thanks, looks interesting!
    "Nothing you can't spell will ever work." - Will Rogers

    Watch me draw and swear at video games.

  3. - Top - End - #3
    Ogre in the Playground
     
    gomipile's Avatar

    Join Date
    Jul 2010

    Default Re: AI art generation

    I've seen a fair number of YouTube videos that seem to be made with this or something similar.

    Here's a recent one I like which, for me, exists in the intersection of "fun" and "oops, cosmic horror."

    https://youtube.com/watch?v=nyD6g47DHQk
    "Mr. Blue Sky - But every lyric is an AI generated image"
    Last edited by gomipile; 2022-08-22 at 11:15 PM.

  4. - Top - End - #4
    Colossus in the Playground
     
    Eldan's Avatar

    Join Date
    Jan 2007
    Location
    Switzerland
    Gender
    Male

    Default Re: AI art generation

    I'm really into it. I have a paid account on Midjourney now, for unlimited art generation, after one of my roleplaying discord channels started to heavily use the artwork for monsters, landscapes and NPCs and we were really impressed.

    Here's some of the stuff we've made and are using in game. These are all real-time generated by a discord bot. (About a minute per picture.)

    Characters:
    Spoiler
    Show

    Cleric:


    Valenar Elf 1:


    Valenar Elf 2:


    Woman with Shotgun (It's clearly not, but I like her):


    Woman wearing clothes made out of rats:


    Harpy Sisters:



    Creatures:
    Spoiler
    Show

    Undead Spider:


    Vengeful spirits:


    Angel:


    Army invading heaven:


    Mystic:


    Crab Cowboy:


    Warforged:


    Silk moth with key:
    https://i.ibb.co/bLP6R6X/Eldan-silk-...7d2c43af25.png


    Places:
    Spoiler
    Show

    Chinese manor:


    Cyberpunk city:


    Ominous rift:


    Path to the Underworld:


    Fountain Square in Carcosa:


    Alchemist's Lab:


    Ornate Vault Door:

    Last edited by Eldan; 2022-08-23 at 04:02 AM.
    "And now I see, with eyes serene,
    The very pulse of the machine."

  5. - Top - End - #5
    Bugbear in the Playground
     
    Imbalance's Avatar

    Join Date
    Dec 2018

    Default Re: AI art generation

    I haven't really tooled with them myself, but have seen numerous results that have done a great deal to convince me that H.R. Giger was an advanced AI.
    “Rule is what lies between what is said and what is understood.”

  6. - Top - End - #6
    Ogre in the Playground
     
    Batcathat's Avatar

    Join Date
    Nov 2019

    Default Re: AI art generation

    Awesome, I love stuff like this (both AI in general and their weird art in particular).

    In the same area, I can recommend a blog called AI Weirdness, which gives good insight in both the possibilities and limitations of current AI, while being very amusing.

  7. - Top - End - #7
    Firbolg in the Playground
    Join Date
    Aug 2009

    Default Re: AI art generation

    Quote Originally Posted by Batcathat View Post
    Awesome, I love stuff like this (both AI in general and their weird art in particular).

    In the same area, I can recommend a blog called AI Weirdness, which gives good insight in both the possibilities and limitations of current AI, while being very amusing.
    Thanks for the blog reccomendation!

  8. - Top - End - #8
    Ettin in the Playground
     
    MindFlayer

    Join Date
    Mar 2015
    Gender
    Male

    Default Re: AI art generation

    Yeah, I use stable diffusion's dream studio for my art. I takes a bit of getting used to for all the right prompts - but works well.
    Last edited by MrStabby; 2022-09-24 at 12:30 PM.

  9. - Top - End - #9
    Pixie in the Playground
     
    Advent's Avatar

    Join Date
    Sep 2022

    Default Re: AI art generation

    I've been seeing more and more of this. I watched an AI generated music video and it was fantastic and trippy!

  10. - Top - End - #10
    Colossus in the Playground
     
    Eldan's Avatar

    Join Date
    Jan 2007
    Location
    Switzerland
    Gender
    Male

    Default Re: AI art generation

    So, a project I've been on for a few years just decided that the best solution to our constant lack of illustrators for our books is going to be that we'll be using AI art, though we still have a team of artists and editors that will be going over the images and clean them up. But we are already using AI for backgrounds to most of our images, with the artists mainly filling in foreground characters and cleaning up obvious mistakes. It's pretty damn cool. I now have the official job title of "head of AI art generation".

    One guy is trying to figure out how to make things like page decorations, that's going to be fun, too.
    Last edited by Eldan; 2022-09-26 at 02:49 AM.
    "And now I see, with eyes serene,
    The very pulse of the machine."

  11. - Top - End - #11
    Firbolg in the Playground
     
    Bohandas's Avatar

    Join Date
    Feb 2016

    Default Re: AI art generation

    I actually spent the entire weekend messing around with Artbreeder

    I was using it to generate fanart for a computer game I like called Honey I Joined A Cult. So far my pictures are the only true fanart for them on Steam. The other stuff in their fanart section is stuff that should've been under "screenshots").
    Last edited by Bohandas; 2022-09-26 at 03:13 AM.

  12. - Top - End - #12
    Firbolg in the Playground
    Join Date
    Dec 2010

    Default Re: AI art generation

    Quote Originally Posted by Eldan View Post
    So, a project I've been on for a few years just decided that the best solution to our constant lack of illustrators for our books is going to be that we'll be using AI art, though we still have a team of artists and editors that will be going over the images and clean them up. But we are already using AI for backgrounds to most of our images, with the artists mainly filling in foreground characters and cleaning up obvious mistakes. It's pretty damn cool. I now have the official job title of "head of AI art generation".

    One guy is trying to figure out how to make things like page decorations, that's going to be fun, too.
    Nice!

    Img2img works for allowing control of layout if you don't mind severely restricting the model from doing any large scale structure decisions. That'd be my go-to for things that need to have precise boundaries or page structures. It's unfortunately a fairly big quality hit at this stage though.

    I've been completely unable to get Stable Diffusion to do any font work yet. You can get text-like stuff but even asking for e.g. a single letter doesn't seem to work at all. Action scenes, swordfights, etc are horribly Escherian as well, with like ambient swords hanging in the air at odd angles.

    It's really good for item icons though, and I've more or less got the workflow down for desktop backgrounds. Character art has been hit or miss. There's a certain fantasy artsy window that's easy to hit and get variations within, but superhero stuff seems to collapse to superman if you use the word superhero directly, and supervillains seem pretty narrow as well ...

  13. - Top - End - #13
    Colossus in the Playground
     
    Eldan's Avatar

    Join Date
    Jan 2007
    Location
    Switzerland
    Gender
    Male

    Default Re: AI art generation

    Yeah, half the work our art team does is editing out floazing weapons and faces where there shouldn't be any. Main reason why we're mostly doing backgrounds, Midjourney does Landscapes quite well. Except clouds. There's often trees and mountains in the clouds.
    "And now I see, with eyes serene,
    The very pulse of the machine."

  14. - Top - End - #14
    Ogre in the Playground
     
    Batcathat's Avatar

    Join Date
    Nov 2019

    Default Re: AI art generation

    Quote Originally Posted by Eldan View Post
    Yeah, half the work our art team does is editing out floazing weapons and faces where there shouldn't be any. Main reason why we're mostly doing backgrounds, Midjourney does Landscapes quite well. Except clouds. There's often trees and mountains in the clouds.
    I find it funny how insane this sounds out of context. "Yeah, the art is good except there are too many faces where there shouldn't be and there are trees in the clouds." Today's reasonable AI criticism is yesterday's signs of demonic possession.
    Last edited by Batcathat; 2022-09-26 at 04:07 AM.

  15. - Top - End - #15
    Colossus in the Playground
     
    Eldan's Avatar

    Join Date
    Jan 2007
    Location
    Switzerland
    Gender
    Male

    Default Re: AI art generation

    Here's two good examples we aren't using.. each made in about five minutes, ten or so iterations.

    Spoiler
    Show


    Last edited by Eldan; 2022-09-26 at 04:19 AM.
    "And now I see, with eyes serene,
    The very pulse of the machine."

  16. - Top - End - #16
    Troll in the Playground
     
    Flumph

    Join Date
    Nov 2006
    Location
    England. Ish.
    Gender
    Male

    Default Re: AI art generation

    Quote Originally Posted by Eldan View Post
    Here's two good examples we aren't using.. each made in about five minutes, ten or so iterations.

    Spoiler
    Show


    Oh, that first one is prime Fata Morgana material right there...
    Warning: This posting may contain wit, wisdom, pathos, irony, satire, sarcasm and puns. And traces of nut.

    "The main skill of a good ruler seems to be not preventing the conflagrations but rather keeping them contained enough they rate more as campfires." Rogar Demonblud

    "Hold on just a d*** second. UK has spam callers that try to get you to buy conservatories?!? Even y'alls spammers are higher class than ours!" Peelee

  17. - Top - End - #17
    Ogre in the Playground
     
    gomipile's Avatar

    Join Date
    Jul 2010

    Default Re: AI art generation

    Relevant news today in Ars Technica:

    https://arstechnica.com/information-...ey-for-ai-use/

    James Earl Jones wanted to wind down his performance of Darth Vader, so he signed off on Lucasfilm using an AI to allow voice actors to perform the character.
    Quote Originally Posted by Harnel View Post
    where is the atropal? and does it have a listed LA?

  18. - Top - End - #18
    Firbolg in the Playground
    Join Date
    Dec 2010

    Default Re: AI art generation

    It'll be interesting when this sort of thing becomes controllable for music generation. Currently you can do reasonable quality music generation with neural nets (at least, similar to the relative quality of diffusion model outputs a year or so ago when compared to professional art), but there isn't something like a text-to-music or music captioning model yet that I'm aware of, and that was kind of the big thing needed to get all of the current generation of image generators working.

    I wonder how precise people tend to be when trying to describe music in text, versus when trying to describe an image in text?

  19. - Top - End - #19
    Ettin in the Playground
     
    MindFlayer

    Join Date
    Mar 2015
    Gender
    Male

    Default Re: AI art generation

    Quote Originally Posted by NichG View Post
    It'll be interesting when this sort of thing becomes controllable for music generation. Currently you can do reasonable quality music generation with neural nets (at least, similar to the relative quality of diffusion model outputs a year or so ago when compared to professional art), but there isn't something like a text-to-music or music captioning model yet that I'm aware of, and that was kind of the big thing needed to get all of the current generation of image generators working.

    I wonder how precise people tend to be when trying to describe music in text, versus when trying to describe an image in text?
    The thing I am geting impressed by is the structure at different scales that can be generated - AI learning to do movements, repeated phrases, variations on a phrase, ryhems and harmonies. Just a few years ago you could do music, but it had no progression. It was a piece that didn't go anywhere. Now... it is geting a lot better.

  20. - Top - End - #20
    Firbolg in the Playground
    Join Date
    Dec 2010

    Default Re: AI art generation

    Quote Originally Posted by MrStabby View Post
    The thing I am geting impressed by is the structure at different scales that can be generated - AI learning to do movements, repeated phrases, variations on a phrase, ryhems and harmonies. Just a few years ago you could do music, but it had no progression. It was a piece that didn't go anywhere. Now... it is geting a lot better.
    All of this comes from the switch from local convolutional models to non-local attention models. It basically lets you have arbitrarily long-range internal references at the same sort of 'cost' as close-range references.

  21. - Top - End - #21
    Ettin in the Playground
     
    MindFlayer

    Join Date
    Mar 2015
    Gender
    Male

    Default Re: AI art generation

    Quote Originally Posted by NichG View Post
    All of this comes from the switch from local convolutional models to non-local attention models. It basically lets you have arbitrarily long-range internal references at the same sort of 'cost' as close-range references.
    I think its more than that, the earlier outputs I was comparing to - just a year or two back - used both long range memory and attention (though I imagine attention has moved on from 2019(ish) when I was last delving into it. I need to be a bit cautious about this though, it wasn't my research bu a colleagues and I don't want to misrepresent it.

  22. - Top - End - #22
    Firbolg in the Playground
    Join Date
    Dec 2010

    Default Re: AI art generation

    Quote Originally Posted by MrStabby View Post
    I think its more than that, the earlier outputs I was comparing to - just a year or two back - used both long range memory and attention (though I imagine attention has moved on from 2019(ish) when I was last delving into it. I need to be a bit cautious about this though, it wasn't my research bu a colleagues and I don't want to misrepresent it.
    So WaveNet was still primarily recurrent/convolutional, though it had a sort of novel hierarchical convolution scheme. That was 2016 and probably the first plausible direct audio synthesis stuff. Before that was some recurrent MIDI generation stuff from Karpathy I think...

    The next big jump I can think of was the Project Magenta work, which maybe yeah used Transformers, but I think it had a relatively short memory anyhow especially for direct audio... maybe 10 seconds or so?

    The most recent stuff I've seen swaps full self-attention with cross-attention in order to push the context length much further, which allows attention over a full 2-3 minute song.

  23. - Top - End - #23
    Halfling in the Playground
     
    Lizardfolk

    Join Date
    Aug 2022
    Location
    the other Pacific coast
    Gender
    Male

    Default Re: AI art generation

    well, we don't exactly know what model midjourney uses, but I doubt the model itself is as important as the data infrastructure.

    Excluding radical changes in approach, getting a better dataset will give you more of an edge than tweaking some part of a model.

    One of the big advantages Tesla has over others in autonomous driving tech, is the seamless data collection.
    Iirc, the model itself is just an optimised HydraNet. Google could do the same if they wanted. (in fact, Google's model is more "sophisticated", on paper)

    Nowadays, it's become very easy to automatically collect large amounts of well-annotated image data. There's so much people don't know what to do with it.
    Developments in computer vision and image search really helped there.

    Similarly, progress in music ID or music search can help a lot.

  24. - Top - End - #24
    Firbolg in the Playground
    Join Date
    Dec 2010

    Default Re: AI art generation

    Stable Diffusion uses LAION-5B for training - a 5 billion image dataset.

    That said, some of the big differences between Stable Diffusion and Dalle and older stuff like CLIP-guided diffusion or VQGAN are architectural and you can see the consequences in the images. CLIP-guided diffusion works by using CLIP on the partially generated image and backpropagating a loss that is basically 'how different is the image embedding from the prompt embedding?'. So that gives direct gradients to shape the image, but it does so relative to a single 768-dimensional vector that comes out of CLIP. That means that when you start to add lots of objects into the prompt, they can get munged together or duplicated in the image in ways that makes the 'that object-ness' of the image stronger to match the prompt.

    The thing Stable Diffusion does differently is that it throws out the visual part of CLIP and only uses the text embedder for the prompt. Furthermore, rather than using the terminal embedding vector, it uses the entire sequence of hidden states one layer prior to that. So instead of having a single 768d thing guiding the image, you have 77 of them (the number of tokens the input prompt is padded to), each corresponding to a single word in the prompt (after self-attention, so these are sort of 'digested' versions of the words). Then, during reconstruction, Stable Diffusion uses a U-Net model with built-in cross attention layers that compute attention between patches of the image at different scales and the different tokens in the prompt. Upside is, this means that if you have a prompt like 'a balloon, a birthday cake, a mountain, and an orangutan' then the model is sort of encouraged to have all of those things in different places in the image but also in particular individual places rather than just loading the image with lots of orangutan-ness. Downside is, this means that Stable Diffusion has to be trained on the image dataset against a specific version of CLIP, and if you were to go and make a better CLIP (as they actually did just recently), you can't just swap it out. Whereas for CLIP-guided diffusion approaches, you can just swap it out modularly.

    The other thing is that since you aren't using gradients to try to match the CLIP visual embedding to the CLIP text embedding, Stable Diffusion is free to deviate from the prompt in the case where that would make the image a more natural example from the training distribution. The reason why that might actually be a strength is, prompts are based on the sorts of descriptions people write to caption images. So imagine you gave an overly non-specific prompt like 'A beautiful painting'. Real paintings will have lots of other details like the artist name, medium, etc. If you're doing CLIP-guided diffusion, the image generator is sort of pressured to come up with a painting that people would say is beautiful but also which they would stop there and not say all those other things. Whereas with Stable Diffusion, its just trying to use the information in the prompt to improve the likelihood of its reconstruction, meaning that stuff which isn't explicitly specified can still occur as long if that makes the image more natural. Basically it means there's a behavior of taking reasonable defaults for things you don't say, versus actually trying to force those things to be 'empty' somehow.

  25. - Top - End - #25
    Halfling in the Playground
     
    Lizardfolk

    Join Date
    Aug 2022
    Location
    the other Pacific coast
    Gender
    Male

    Default Re: AI art generation

    I might as well go ahead and read the paper at this point (and I will), but for the sake of discussion,
    what is the significance of using only CLIP text embeddings, but not image embeddings, if the point of CLIP is the joint training?

    Isn't it kinda throwing out the "joint" part that made CLIP special in the first place?

  26. - Top - End - #26
    Firbolg in the Playground
    Join Date
    Dec 2010

    Default Re: AI art generation

    Quote Originally Posted by MetroAlien View Post
    I might as well go ahead and read the paper at this point (and I will), but for the sake of discussion,
    what is the significance of using only CLIP text embeddings, but not image embeddings, if the point of CLIP is the joint training?

    Isn't it kinda throwing out the "joint" part that made CLIP special in the first place?
    The text embedder still captures visually relevant concepts from it's input as would correspond to the distribution of images. But rather than relying on an indirect fortuitous property (that CLIP knows what things look like in detail and not just enough to relate text and image), SD's UNet is trained directly to accurately reconstruct a noised image using the CLIP embedding as extra contextual information. So it's more directly trained to do what it's intended for

  27. - Top - End - #27
    Firbolg in the Playground
     
    Bohandas's Avatar

    Join Date
    Feb 2016

    Default Re: AI art generation

    Quote Originally Posted by MrStabby View Post
    The thing I am geting impressed by is the structure at different scales that can be generated - AI learning to do movements, repeated phrases, variations on a phrase, ryhems and harmonies. Just a few years ago you could do music, but it had no progression. It was a piece that didn't go anywhere. Now... it is geting a lot better.
    Now I just wish personal computers would catch up to the technology. Nobody would need to pay money for an album ever again. Or for a book, or for art.

  28. - Top - End - #28
    Titan in the Playground
    Join Date
    May 2007
    Location
    Tail of the Bellcurve
    Gender
    Male

    Default Re: AI art generation

    Quote Originally Posted by Bohandas View Post
    Now I just wish personal computers would catch up to the technology. Nobody would need to pay money for an album ever again. Or for a book, or for art.
    No, then you just need to pay $12/month (or only $100/year!) for the software license.

    I am not enthused by this. I remember the world before social media turned a huge portion of human communication into something they could pump money out of. There was a lot to like about that. I'm not sure why I should be excited about another vast realm of human endeavor and enjoyment getting turned into a corporate fiefdom.
    Blood-red were his spurs i' the golden noon; wine-red was his velvet coat,
    When they shot him down on the highway,
    Down like a dog on the highway,
    And he lay in his blood on the highway, with the bunch of lace at his throat.


    Alfred Noyes, The Highwayman, 1906.

  29. - Top - End - #29
    Firbolg in the Playground
    Join Date
    Dec 2010

    Default Re: AI art generation

    I'm currently running this stuff on a GTX 2080, so I'd say it's already at personal computing level.

  30. - Top - End - #30
    Colossus in the Playground
     
    Eldan's Avatar

    Join Date
    Jan 2007
    Location
    Switzerland
    Gender
    Male

    Default Re: AI art generation

    Quote Originally Posted by warty goblin View Post
    No, then you just need to pay $12/month (or only $100/year!) for the software license.

    I am not enthused by this. I remember the world before social media turned a huge portion of human communication into something they could pump money out of. There was a lot to like about that. I'm not sure why I should be excited about another vast realm of human endeavor and enjoyment getting turned into a corporate fiefdom.
    I can't draw stickmen I like. AI art finally lets me do something artistic I enjoy, I've never had that feeling before in my life. I'm all for this.
    Last edited by Eldan; 2022-09-27 at 11:18 AM.
    "And now I see, with eyes serene,
    The very pulse of the machine."

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •