Researchers at India’s IIIT Hyderabad have published a paper showing existing AI technology can be used to view and interpret a cricket match reliably, providing text-based commentary that could conceivable be used to voice match-time play-by-play.
The researchers fed the system several hours of videos of cricket matches, leveraging the accompanying commentary to help the AI understand what was happening in each frame.
“In the first stage, the video is segmented into “scenes”, by utilizing the scene category information extracted from text-commentary. The second stage consists of classifying video-shots as well as the phrases in the textual description into various categories. The relevant phrases are then suitably mapped to the video-shots… This solution yields a large number of labeled exemplars, with no manual effort, that could be used by machine learning algorithms to learn complex actions.”
Cricket proved to be a good study, as the key to understanding the match is tracking the movement around the pitch. As the Washington Post pointed out, “A computer would struggle to make sense of a botched Michigan punt returned to the end zone in the final seconds by Michigan State or the bizarre sequence of events in the final seconds that led to Miami defeating Duke.”
Read the research paper: Cornell University Library
Read more: Washington Post