Source: Eastern Institute of Technology
27 seconds ago
An EIT researcher has contributed to a global study testing the limits of artificial intelligence, with the work published in Nature, one of the world’s most prestigious scientific journals.
Dr Syed Shahid, a Senior Postgraduate Lecturer in Health Science based at EIT’s Auckland campus, is among more than 1000 researchers involved in the international collaboration, which includes contributors from more than 500 universities worldwide.
The co-authored paper, titled A benchmark of expert-level academic questions to assess AI capabilities, explores how artificial intelligence performs when faced with complex, expert-level academic questions across disciplines.
Dr Syed Shahid, a Senior Postgraduate Lecturer in Health Science based at EIT’s Auckland campus, has been published in Nature.
Syed contributed questions in biomedical science, health and medicine, working on the project over more than a year.
“It’s a dream come true, because everyone in science wants to publish in a journal like Nature.”
He says the process required multiple rounds of refinement after early submissions were successfully answered by AI.
“Initially I created more than 50 questions, but all of them were answered correctly by AI, so they were rejected.”
“Then I took it as a challenge and started developing more complex, scenario-based questions that AI could not answer correctly.”
The scale of the project was significant, with more than 70,000 questions submitted globally before being narrowed down to around 6,000 that formed the final benchmark.
Syed says the research has important implications for education, particularly as artificial intelligence tools continue to evolve rapidly.
“AI is learning every day, every minute. But we also need to learn, and we need to learn in a different way.”
He says the work has directly influenced how he approaches teaching and assessment at EIT.
“We need to develop assessments that are AI resistant. It’s very difficult, but at least some components need to challenge AI so we can properly assess student learning.”
Syed, who has been at EIT for six years, says the collaboration also provided valuable insights into global research and teaching practices.
“I got a lot of information from researchers around the world about what they do and how they do it, and I will use those strategies in my classroom.”
Alongside his work in artificial intelligence and education, Syed is also leading research into health outcomes for New Zealand’s South Asian communities, focusing on the risk of conditions such as diabetes and cardiovascular disease.
He says the growing population and unique health risks within these communities make the research increasingly important.
“South Asian population is increasing in New Zealand, but it is still under-investigated and under-reported.”
He says EIT has played an important role in supporting his research journey.
“My managers and team have always provided full support, and that has helped me continue and grow my research.”
He hopes the Nature publication will contribute to ongoing conversations about the role of artificial intelligence in education and society.
EIT Auckland Campus Director Cherie Freeman says: “We are incredibly proud of Dr Syed Shahid’s remarkable achievement in contributing to a Nature publication, and we deeply value the vital role he plays in bringing innovation, excellence, and global impact to EIT”.