Generative AI

Since this class was offered last year, generative AI tools have really burst onto the scene.  There are multiple open-source and commercial products that all use generative AI as their core technologies.  The one discussed the most in the news is ChatGPT.  When we did our class polls of topics, this came up very specifically as a product as well.  What we are going to do in this topic is to explore different aspects of generative AI, mostly from a high-level view.  

Remember that generative AI goes well beyond large language models.  It also includes many of the art generation as well!  I have also included a video focused on generating images.  Let’s jump in!

Some of the latest models have been developed since the book was released (that tells you how fast this is all moving also, as the book is really quite recent!).  However, a key part of the underlying technology for generative AI is transformers and these are covered in chapter 24 of the book (Deep Learning for Natural Language Processing).  

An overview of large language models and chatbots 

Let’s start this section with an overview of large language models (often called LLMs).  This overview comes from code.org  (we used one of their videos in the previous topic as well, if you recall).  This will give a good high-level overview.

Generative AI Introduction

This video gives an introduction to generative AI overall.  The beginning of the video should be review for you (since it introduces ML) but I thought the later steps might be useful on the overview level.

What is a generative AI model?

I’m including several points of view on introduction to generative AI because I like the variety and you get different views of all that is happening in this space.  So here is one more introduction.

Jumping into some more details

Ok, that’s plenty of overview!  For the remaining videos in this topic, we will jump into some more details about different generative AI models. This stuff is all so new that there are not textbooks easily available for it yet and so I found some good online videos of people jumping into different generative AI technologies. 

What happens when you ask ChatGPT a question

The first video is a deeper dive into what ChatGPT does behind the scenes.  This is a good overview talk and he also released his slides here.

There is also a good writeup with details on chatGPT here

How does Stable Diffusion work?

Computerphile is a fun YouTube channel diving into a variety of techniques.  This one explains how stable diffusion works with good examples.

Exercise

Complete the exercise on generative AI