Reviews

Pros: 1. Lecture content is quite interesting and well-explained. 2. Assignments are a good challenge and are deep enough to get a good understanding of the relevant parallelization techniques without being excessive. 3. Cons: 1. Assignments (particularly assignment 5) can be a bit ambiguous, and those ambiguities aren't necessarily clarified. 2. Extra credit only applies to the assignment it is gained in, and this was not told to us until quite late into the semester. 3. Personal Background: Math + CS undergraduate degree. First semester in the program and also did NLP this semester. Got an A in this course. Detailed Review: Be warned that Assignment 2 took substantially longer than the rest of the assignments for lots of students. Assignment 3 & 5 were somewhat ambiguous, and 5 was particularly annoying because our different interpretations were 'design choices' (rather than them telling us what interpretation to choose), and yet we were expected to be close to their answer for correctness (but they didn't tell us what interpretation they chose). The above didn't have much of a negative impact on my course experience. Overall, it was a good course with interesting lectures and assignments.

Pros: 1. generally not too difficult 2. a reasonable first class to take in the program 3. Cons: 1. class material is divided and taught by two different teachers, with varying results 2. 3. Detailed Review: The first portion of this class isn't BAD, but it is probably what most students struggle with in this course: getting into probability, combinatorics, throwing in some calculus, etc. The first couple of homeworks are some of the hardest of the semester, but assignments get easier (and more interesting) once you're in the second half of the semester and working on simulation rather than probability. I think the general concensus is that the second teacher (who covers simulation) generally does a better job and makes the material more interesting than the first teacher (who covers probability). The first teacher isn't responsive to questions on piazza and his material is pretty dry, but still workable. The second teacher is much more responsive on piazza and was a generally involved teacher who was open to discussion or questions. If you already have a stats background, a good portion of this class will be review for you. If you're totally new to this material, you'll have to work more, but it's entirely reasonable for any person to get an A in this class with enough effort. Depending on your familiarity with the material, you could probably spend anywhere from 5-15 hours/week on this course to completely cover all the material and do assignments.

Pros: 1. Professors were very active on Piazza and encouraged students to discuss problem sets. 2. Reviews for midterms were made available to students. These combined with quizzes and homework meant the midterm was never a surprise. 3. Course is a great review/intro of probability and statistics that should set a strong foundation for continued success in the program. To be determined. Cons: 1. Statistics lectures didn't really add much on top of the provided notes and lacked many examples. 2. Sometimes, the hints harmed more than helped. I suggest attempting to work the problems before reading the hints. 3. Detailed Review: For what it's worth, I scored a 100 in the class, so I felt it was mostly easy. There was one quiz that I had a brain lapse, but it was dropped as part of their generous grading policy. I'm not sure what the distribution of grades was, but it felt like everyone else scored very high, too. This class is a great stepping stone from undergraduate classes to graduate classes. The professors are very responsive to both e-mail and Piazza and welcome questions. I never attended office hours, so I cannot speak to those or the TAs, but I did see a TA response on Piazza occasionally. Given the disparity of foundational knowledge in the student population, the professors have adapted the course content so that it meets the needs of all of the students. Enrichment opportunities exist if you find the entire experience droll. Assignments are all graded using a novel multiple choice scheme. This seemed to be a huge source of confusion for the students for a very long time. General rule of thumb: work the problem, get an answer, and then choose the number that is closest to your answer. Don't overthink it beyond this. I thought this method was a clever way of easily grading hundreds of assignments while keeping the students from working backwards from the solutions. The exception is the proof-based questions on the homework. Good news for future generations, we probably found most of the bugs, and they will already be corrected for you. Textbooks were useful if you knew how to use them. The probability text was less useful, but it did have some information that helped with a few homework assignments, but you likely could find this information in other probability textbooks. The statistics textbook helped me a lot with understanding some statistics principles that weren't thoroughly covered in the lecture/notes. As a supplementary textbook, I often referred to Statistical Inference by Casella and Berger.

Pros: 1. Doesn't dive into too much of complex topics (so recommended for beginner) 2. Introduction to RL and slightly to NLP Cons: 1. Final project is tough and might be stressful (subjective) 2. Some lectures might not be used at all in the homeworks/projects (especially the last) Detailed Review: I have very little exposure to AI before this course. Also, I am amateur to python. Given my above scenario, I initially struggled a little in completing HW1, HW2 as I was accustoming to python. Later HW3, HW4 or very challenging. But if I look back now, I do see that I am much more comfortable to python, DL concepts (especially on how to come up with a model) and pytorch. My Tips: 1. Always go through the grader code to get more insights if you stuck with any homework, and don't worry too much about the final project or with your peer's progress. 2. Most importantly, you need to plan a head like complete HW1, HW2 as soon as possible, and similarly try to complete all homework by 1 month before the course completion. 3. Try to beat the extra credits / extra assignments (I lost A relatively with 0.75 marks)

Pros: 1. A lot of practical knowledge, you really learn to write parallel algorithms and a lot of languages/tools/approaches 2. Zero exams 3. You get the environment required for each lab, so not needed to get a VM. Cons: 1. Lectures are very very long. It would be better to have 5-6 10 mins sections rather than one 50-65 mins session. 2. The instructions of the assignments are very ambiguous and are written in a poor manner. For most of them, you will spend a lot of time trying to figure out what it is required to implement. 3. The assistance through Piazza was horrible. Most of my questions were unanswered. Be ready to wait at least 2 weeks to get a response (even for critical topics, like assignments) Detailed Review: All the other reviews are right. Expect to put a LOT of effort in this class and consider to just take this one for the semester (I took NLP alongside and I suffered, even when NLP is a light-workload class). You will end having practical knowledge in parallel programming and learning new languages and frameworks (Rust, CUDA, Go, MPI). It is easy to work on the assignments, although you need specific hardware to it (like GPU) you get an environment already configured via Codio, which is not the best but you can get a lot of value by connecting through VSCode. I really enjoyed the assignment 3, which is the most difficult one. The support from both TAs and Teachers was horrible. I saw the teachers answer TWO questions during the course, and both where rants out of topic. The TAs never responded questions on Piazza, no matter what was the topic (Assignment, grading, lectures, nothing). I made a question about the instructions of an assignment two months ago and it haven't been answered yet. They need to put more effort on supervising the support.

Pros: 1. The projects altogether give you a sense of how a hypervisor works from a programming standpoint. 2. The exams are tight for time (tightest of all the courses I have done so far) but fair. The questions test your knowledge and conceptual understanding well. 3. The lectures explain concepts well, and the content is quite interesting and modern. The first half goes through the inner details of virtualizing CPU, storage, etc. The second half looks at the main virtualization technologies and research (Unikernels, containers, Kubernetes, etc.). The slides aren't the best for the second half, but it conveys the ideas well. Cons: 1. Projects are group work but are not structured well for group work. You can implement the various functions separately, but you likely won't understand what's going on unless you do them all. For later projects, that missing understanding will hurt. Also, you typically can't tell if isolated functions are working correctly; if 1 function is incorrectly implemented, all you can tell is that something is wrong. In some cases, everything can seem to work well and you still have mistakes (that you won't know of until they hit you in later projects). 2. Projects are marked overly pedantically harshly. Missing 1 or 2 error checks could result in losing half or more marks for a given function. 3. Other notes: Try to make sure your setup works as soon as you can. I ended up having to use the faculty-provided cloud resources after the 2nd project because there was something wrong with my setup causing the 3rd project to fail just for me. I didn't do AOS before taking this course, so it's definitely not required.

Background: I work as an ML Scientist at FAANG. I started this program in Fall'2020, and this review was written in Dec'2023. This review is very specific, so a motivated person can probably find me. Let me save you the trouble, here is my LinkedIn: linkedin.com/in/ardivekar/ . Feel free to reach out with questions. I'm usually also on the MSCSO Slack. I applied for the Thesis in Dec'2022, and completed the Thesis over 3 semesters (Spring'23 to Spring'24). My thesis was in the NLP domain, during 2023 when AI went mainstream via ChatGPT, and new LLMs were released every single week. As you can expect, this makes my experience unique. But I expect everyone's thesis experience will be unique since it depends on your personal strengths and interests. Another thing which I did different from other students is that I framed my own research problem. This is recommended only if you're really dead-set on one idea, like I was. Otherwise it's okay to go with an idea your Thesis advisor gives you. But make sure you're eventually doing your own research, not just contributing to someone else's project. I've written about the process I followed to reach out to MS thesis professors. I've tried to put this together in a step-by-step fashion, starting from the very basics: https://adivekar-utexas.github.io/files/work-with-professors.pdf If you have questions on the steps to find an advisor, please reach out to me here: linkedin.com/in/ardivekar/ Detailed Review: This is in response to a question on on #thesis-option: "What are the pros and cons of doing a thesis?". ==================== Personally, I've grown a lot as a researcher, both in skills and temperament. I was an industry researcher before the thesis, and had published ML papers internally (at Amazon we have an annual internal conference with ~1k submissions and acceptance rate of ~30%). I felt like I knew what I was doing, and the thesis would be a breeze. But I had never had the perspective of academic research, where you pit your ideas against the very best work. My thesis also focuses on LLMs, where the "best" changed weekly in 2023. The last year (2023) has been very humbling. I feel like I better understand what NLP research is about, and the challenges. My advisor is very good, but at the end its up to me to do the work and push forward the ideas I think are best. This has led me to learn a lot of skills, like how to read a lot of papers fast, how to strategically cut corners to get useful experimental results, and how to orchestrate code which runs LLMs in parallel on a 100+ GPU cluster. As to cons? Well, like I said it can feel humbling. The thesis itself is a long grind which does not always produce results. For example, imagine you and your advisor agree that in 2 weeks, you should get results for one set of experiments. You quickly realize 2 weeks is quite a short time to implement this fairly complex idea. But you have to get it done, so you ignore calls from your spouse, push deadlines at work, and hack till 4am at night. And finally, your experiment runs and you get some results. But they are not good results...not enough to publish. So, you pivot and try a variation of the experiment. This also does not work. In a panic you read a lot of the research papers, and you get some new ideas but you also get the feeling like people have been doing work which is much more in-depth than your own experiments. You pivot your approach and try again, and again, for 3 months. At the end, you've built an excel table of results from many, many variations of the same core idea. But none of the results are strongly positive, and you aren't sure why (was it a code bug? data not good enough? issue with analysis metric? so many options...). Your advisor was initially excited about your project, but now seems less excited and is trying not to show it. At this point, your self-confidence has been steadily taking a beating for 3 months. You feel like your ideas are not good, that you are stupid, and (most importantly) that you've wasted time which you and your advisor could have given to something more productive or enjoyable. This is the reality of doing research. To continue pushing forward, you have to be very motivated by the idea you are working on, or just be bull-headed enough (or desperate enough) that you keep working on it regardless. At different points in time I have experienced all these feelings. What set me straight was, after one set of failed experiments, I attended ACL conference. ACL is the top NLP conference in the world, and in 2023 it was attended by true experts: I saw Geoff Hinton in-person, delivering the keynote (where he was publicly criticized by Emily Bender). I sat next to Chris Manning at a talk, and rode an elevator with Luke Zettlemoyer. I met a director at FAIR from whom I heard about LLaMa-2 a week before it was launched. But what stuck with me was the vibe. I did not see the bravado and showmanship you get at tech conferences. Everyone here (especially the students) were aware that they did not know some ultimate truth about NLP; they had just explored an idea, sometimes for years, and found something interesting to show. Some of the presentations were very under-whelming and barely seemed like a contribution, but they were being projected to everyone on a 50-foot screen. Most of the authors were nerds (with various levels of grooming) who stammered occasionally, but were excited about their findings; it was impossible to feel intimidated by them. I left with the impression that while some of the research I saw was truly very deep, most research is ordinary. And thus, I felt confident that with effort and guidance, I could at least produce "ordinary" work (as per the caliber of a conference like ACL). And that hopefully someday, with enough effort and some luck, I might produce very deep work. The Thesis is the journey which facilitates that feeling. EDIT in Nov 2024: this story has a happy ending. After 1.5 years of work, the paper resulting from my thesis was accepted at EMNLP 2024 Main (a top-tier NLP venue). You can read the paper here: https://aclanthology.org/2024.emnlp-main.1071/

Pros: 1. Solid lectures 2. Very helpful team of TA’s/LF’s 3. Shortened schedule Cons: 1. Exam requirements are kind of tedious 2. Loose peer grading guidelines Detailed Review: For context, I’m an MSDS student, and this is my 4th class after probability, data structures, and regression. I thought all three classes helped prepare me for this course, but if you’re not familiar with machine learning, taking probability as a prerequisite is a must, and you should have a good grasp of linear algebra and calculus, as well as Python for the programming assignments. There is a lot of overlap with other MSDS classes, especially in Liu’s half. I thought the lectures were solid. Both professors write out their lecture notes (Klivans in front of a classroom of students). The textbook for Kilvans’ section is pretty dense, though some of it does tie in directly to the homework. Liu provides his own supplementary textbook, which we were allowed to print out for the exam and more or less tracks with the lectures. The class was heavily theoretical, with a little bit of application mixed in here and there. Personally, I liked being able to learn how machine learning algorithms work under the hood and coding them out in Python. Parts of the programming assignments were plug-and-chug like others have noted, though other parts involved building algorithms from scratch, which helped me better understand them. The first couple of homework assignments are really intense, but the class eases up significantly after that. The LF’s were also extremely helpful during office hours and very responsive on Ed. Kudos to them, because it was a very large class (around 400 I think?) with students from MSDS, MSDS, and MSAI, and there were many technical kinks from the program transitioning to Canvas. When it came to peer grading, the staff didn’t really have any guidelines, other than to be lenient, so peer grades were all over the place for some people, and the Canvas system only did averages instead of medians. LF’s were willing to do regrades, with the caveat that they could either add or take away more points. The exams were pretty tough, especially the first one. In addition to that, we had to record ourselves taking them via webcam (understandable) and only use printed notes instead of electronic ones (tedious). The curve for final grades ended up being pretty generous. A raw score in the low 70’s was curved to a B-.

Pros: 1. Textbook/website is well organized and detailed 2. Instructor and TAs were responsive and helpful 3. Explains well how linear algebra relates to other CS concepts Cons: 1. Can be time consuming 2. Uneven assignment difficulty 3. Detailed Review: I enjoyed this course overall and thought the content was well-presented and extremely relevant for those in this program. I have a strong math background (my undergrad degree is in math), so I felt well prepared for this course despite it being my first course in this program. I think some of the kinks from Spring 2023 have been worked out -- grading was not too slow for Summer 2023 in my opinion, with assignments coming back around 1 week after they were due. TAs were present on Piazza and most questions got answered within a day or so. The textbook and lectures were awesome and the assignments directly related to the chapter content, making them not too difficult to figure out most of the time. However, because it is a proof based class, the weekly homework could sometimes take a significant amount of time. Some weeks, I spent just 5 hours on the course but other weeks it was up around 20 (the week of midterm 2 was pretty brutal time-wise). The end of chapter assignments were visible all semester, so one could work ahead to mitigate this. Overall pretty good, just be prepared to have a few very long/more difficult assignments that may overlap with exams and plan accordingly.

Pros: 1. The assignment is very close to the lecture 2. 3. Cons: 1. A lot of formality stuff 2. 3. Detailed Review: I think we I really like is the demo of the prof is actually helpful and useful in the assignment. I can apply what I just learned in my homework and that feels great. The difficulty of the assignment is medium and doable but it involves lots of presentations, paper writting, and recording. This has not much to do with the technology itself. I understand as a data scientist, story telling skill is super important but in the course, it is little too much in my opinion