Professor of Strategic Studies and founding director of Alperovitch Institute writes about using ChatGPT for 5 days in the Malware Analysis and Reverse Engineering classroom.

ChatGPT’s “involvement”: “chat.openai.com/chat/-tab was open on most student machines at all times.”

Benefits

The first use-case is that the machine “filters mundane questions,” as one of our students put it quite eloquently. Meaning: you can ask the dumb questions to the AI, instead of in-class.

The second benefit follows: “you no longer disrupt the flow of the class,”

ChatGPT will give you the response in 5 to 15 seconds, literally. That response speed was game-changing last week, because we could keep up with the instructor in real time, reading ChatGPT’s explanation of embedded resources while listening to Juan Andres talking about the same thing. 

The machine even did some of the script writing for us. “I think the coding was huge,” said Lee Foster, one of our adjunct professors who took the class as well.

Limitations

the system is good at some things but not as good at others: it created fake names in literature reviews, and has no concept of accuracy. “To scale it in the classroom we need to better understand its strengths and weaknesses,” said Hiromitsu. “Often the code was out of date,” added Martin, and pointed out that the language model had its knowledge cutoff in 2021, so about two years ago. Code often had evolved since. Don’t ask it to explain cryptonyms. Don’t trust book recommendations. It will hallucinate. It will make mistakes.