Course Details

Pros: 1. Extremely relevant 2. Lectures are still solid as with DL 3. Tightly written assignments Cons: 1. Not so much a con but: get the Colab Pro license either for free as a student or just drop the $10 (I did) 2. No final project like DL has (or at least used to have) 3. Detailed Review: I took this as my summer course. I was considering doing a thesis (and stealth starting in the summer), but this material is just too relevant to not take the class. The material is extremely relevant and tightly taught. Excellent lectures and topics. My only gripe is the loss of the final project. I have found those very valuable for the courses that have them.

Pros: 1. You actually get to learn about LLMs finally in a deep learning class 2. Compared to NLP, I actually think I got a MUCH better understanding of LLMs because he really goes quite in depth with how to train them and how they work. I am sure NLP did cover part of this material, but it did not "stick" in my mind until now 3. Homeworks are surprisingly pretty straightforward and easy 4. 2nd class with Dr K, and I like his overall teaching style with how he structures his lectures, content, homework structure, etc 5. TAs answer questions in a very helpful manner on the discussion boards 6. You can use LLMs on the homework, and it is (mostly) quite helpful 7. Only 4 homeworks, no exams or major projects 8. I do not work as an MLE, but I feel like if you did or were to, the content in this class would be very, very helpful. Cons: 1. Tons of lectures, but a lot of it is to give you the development history behind specific models 2. Required quizzes which honestly felt like more busy work than anything 3. You definitely need a relatively modern GPU for this class Detailed Review: Feeling a bit lazy considering this is going to be my penultimate review in this program and I am feeling a bit sick while writing this: I think the numbered list above encompasses everything you should know about this class. Definitely one of the best ML/AI classes in the program and I think my placement of taking it (ML/DL -> RL -> NLP -> ADL) was perfect. I definitely get and appreciate LLMs a lot more now. Homeworks, given the course name "Advanced", I had thought would be quite difficult, but he gives you SO much boilerplate code, it's awesome. The READMEs generally do a pretty great job of holding your hand and letting you focus on the actual aspects you need to do and learn from. Really did not require much effort: I took it in the summer, watched the lectures passively while driving, and started on the homeworks maybe like the Thursday before they were due (Monday nights) and that worked out quite well. You could definitely take another class with this during a long semester. ***Forgot to mention: You definitely need a relatively modern GPU for this class. I had a 1070 and that was fine for two of the homeworks but I had to go use my 4070 for the other two homeworks. I personally hate the idea of paying for Collab since I got burned by it during Parallel Systems. I am sure as other reviews have mentioned, there are ways to complete the homeworks without having a personal GPU.

Pros: 1. Great content. Very useful. 2. Similar to DL, the assignments are genuinely interesting. 3. Mostly asynchronous/self-paced. Cons: 1. Less theory. More application. 2. 3. Detailed Review: This course has a similar structure to the plain DL course. The primary differences are: the content is more interesting for people who have progressed beyond the "novice" phase, and the later programming assignments (in particular, the last one) take substantially longer. The final programming assignment was tough, largely from issues in my code. The code itself was simple but very small mistakes can snowball into bad results, and the training runs required a lot of patience. I would suggest starting the programming assignments early. Waiting until the last minute will be very stressful (ESPECIALLY the last assignment).

Pros: 1. Novel topics used in industry 2. Not a full theory-based assignments (work-oriented class) 3. Projects are good for learning and related to the topics 4. Nice TAs, staff, and Ed communication Cons: 1. You need new GPU / CUDA 2. Problems with versioning, but not a big deal (see the comment below) 3. Detailed Review: I took this class because the facility offered it and I was intended to take Reinforcement Learning, but they tried to get more people to take this one. Honestly I planned to take it after Deep Learning, but I thought it was an elective, so it will be "easier", but I was wrong. Overall the class is a solid 10/10 if you are interested in AI / DL, and the professor makes the lectures easy to understand, and it's based on papers, but if you have time, you can check them out by yourself. The homework was a bit complicated, but you don't need to spend a lot of time coding compared to the time you need to invest in understanding the assignment, and adding to the item above in the "cons" section, this was the very first time they offered the class, so it's normal that they need some iterations to improve some points, and one of them was the homework versioning, for example in the second project they realized like 4 versions and the grader worked only with the last one, and with that many of us spent like 2 weeks looking for a typo in the code, again it's not a big deal but from now it seems to be solved. Even though the course says that you don't need CUDA, you will need a "decent" graphics card, like more than 4GB will suffice to train your models in less than 20 mins. You can use Collab, but if you are in the US, you can use lightning.ai, (In my country it is not available) so you can rent one in Vast.ai or in runpod.io since they are pre-paid, you will not spend more than $10 USDs per homework. Tip: If you have a basic GPU, you can write your code and debug it, and once you can train your model, you can upload it to one of these platforms and train it and spend less than $1 USD. To summary, a nice class that every ML/AI enthusiast should take :)

Pros: 1. Rigorous and novel topics backed by recent research papers in the field excellently curated by the professor. 2. Very similar simple class layout as the new required DL class (Summer 2024 or later DL) of 4 coding homeworks, plus 1 short quiz of 1-3 MCQ per lecture. 3. Professor is very active with the class (often on Ed Discussion + two live Ask Me Anything Zoom session for Spring 2025) - probably the most in the CDSO program. 4. Straightforward and good lectures for the most part (many of which run 15-20 minutes only, with a few long exceptions). 5. Three week time frame per module and per assignment is more than enough (or just right for full-time working professionals with heavy workload). Cons: 1. RAM / GPU-intensive assignments especially HW3-HW4. Probably doable with very high RAM and good NVIDIA GPU (cuda on Windows) but if you're on Mac, Apple Silicon and mps, most likely Colab Pro (or lightning.ai) is better. 2. Late release of course assignments for Spring 2025 (as it was the first time the course was offered), but this should not be a problem moving forward. 3. More difficult coding tasks than DL (obviously, as the intended sequel of DL) Detailed Review: New course, and one of the best in the CDSO Program, covering modern developments in deep learning, as backed by the research papers the professor of the class nicely curated., and refer to in the course website here: https://ut.philkr.net/advances_in_deeplearning/. These research papers also give the course a feeling of being truly a graduate course, and one can get the most out of the course if one takes time to browse through these research papers. As a side note, in the revised basic deep learning course in this program, the goal was essentially to learn and master Pytorch, and training deep linear models, convolutions, and transformers. That last topic on transformers gives that course also a modern touch, although generally basic DL now in the program may look like undergrad level for many. Going back to ADL, the course begins with a review of basic deep learning and GPU Architecture (3 lecture videos). Afterwards, the first main module begins and is on memory efficiency, covering topics like half-precision training, LoRa adapters, quantization, etc. The second part is on types of generative models. The third and fourth parts are the heart of the course and covers LLMs and Computer Vision (with emphasis on Vision LLMs). The course itself has a structure similar to the new basic DL (4 coding assignments, short quizzes), but topics are generally at a higher level (but still, the professor does a good job of simplifying). The assignments' focus will be on fine-tuning models, including generative models, an LLM, and a vision LLM. Expect that training them will be GPU intensive especially for the Vision LLM.