First published in the Daily Maverick 168 weekly newspaper.

The journalism fraternity is barely managing to make it out of the great disruption – digital publishing – when artificial intelligence (AI) is already being mooted as the follow-up wave that is about to hit. For many, this will trigger the human survival instinct to fill the void of uncertainty with the worst possible outcome.

Fears generally revolve around job losses in an already ravaged industry and the loss of editorial decision-making at the hands of journalists and editors. As with all innovation, a clear strategy and a focus on the potential benefits can help navigate this inevitable next chapter that is about to be written.

There are, of course, other professions that are further down the AI line, which we can look towards for inspiration. Pilots, for example, can flick the auto-pilot switch once the more technically demanding aspects of taking off and landing have occurred. This feature allows for more rest on long-haul flights and fresher, sharper pilots who can fly passengers to safety.

Investment professionals use AI to scour the investment universe, ploughing through more balance sheets than humanly possible and presenting a filtered list to managers so that they can decide on suitable targets. In the same way, we can apply machine learning and the power of AI to the way we create, distribute and analyse journalism.

With big data comes big fraud, reams of contracts and paperwork that are approved by lawyers and accountants. Systems already exist where entities (people, places and things) that are common in different sets of documents and datasets, can be linked together and analysed. It joins the dots to make analysis easier and, in some cases, possible. DocumentCloud and Google’s Pinpoint are examples of systems that enable journalists to do that entity analysis on large and varied document sets.

The Associated Press has, for many years, used a template to turn company earnings statements into written reports. As a result, many smaller companies are covered than would be possible if humans did the work. New jobs have also been created where these reports are checked for accuracy before publishing.

A/B testing is becoming an increasingly used tool in journalism to help improve chances of success. An independent analyst found that roughly 30% of New York Times headlines on their homepage had more than one headline, and some had up to eight variations. (An A/B test is where two or more headlines are pitted against each other within a time period and the best-performing one becomes the assigned headline.)

Although some were minor, some were pretty dramatic, and undoubtedly training an algorithm that will offer improvements to the production team as they seek to extract maximum impact from the more than 200 stories per day they publish. Articles with tested headlines were 80% more likely to feature in the most-read lists – and along with it came the risk that the process might be misused.

For years, publishers have been at the mercy of algorithms of social media. Some media businesses were built and destroyed in no time off these black boxes. It makes sense that the best chance of success against a machine is another machine.

We can scratch our heads, hoping that our social media posts will be sent out with enough time between each post, that they don’t contain too many words on images or any myriad reasons to be penalised. Or we can pay a service provider with smart machines that have analysed terabytes of data to do that for us.

And, of course, there are personalised recommendations. If you’ve ever let your kids loose on your Spotify or Netflix account, you’ll know why there’s a sudden flood of JoJo Siwa and PAW Patrol recommendations. (That’s my story and I’m sticking to it.)

Based on past behaviour and audience segmentation, content is being recommended to us by machines that know us better than we know ourselves. The trick is to design and train a feed that understands our needs and not just our wants (insert any one of many Facebook disasters here).

In a world where hundreds of thousands of people from different backgrounds, locations and socioeconomic circumstances visit the homepage of a news site daily, we can use AI to help us recommend and prioritise the important and relevant journalism that is likely to serve their needs and our goals.

Some publishers, like the Globe and Mail in Canada, relieve editors of the constant burden of updating the homepage by allocating the three most important slots and letting the machines do a more productive and efficient job – while never sleeping. Propensity modelling allows them to identify the readers who are most likely to get a subscription, and design a dynamic paywall strategy and/or show the readers who are unlikely to subscribe to a different offer.

As with any powerful tool, there are risks and benefits. But as I’ve described, these can be harnessed in ways that can make journalism and the product of journalism a better service than it would be without it. The gap between big publishers and smaller ones is being widened by their data-science budgets and their ability to leverage AI. Whether we like it or not, the use of AI will soon be table stakes at the high-stakes table of journalism. We should rather define our own strategy than let the AI tail wag the dog. DM168

This story first appeared in our weekly Daily Maverick 168 newspaper which is available for R25 at Pick n Pay, Exclusive Books and airport bookstores. For your nearest stockist, please click here.