AI analytics: Welcome to the Internet of Recognition
Posted on Apr 9, 2019 by Neal Romanek
Software company Micro Focus provides AI analytics tools to media companies and government agencies to analyse videos of all kinds – and that’s just the beginning
It’s easy to think of artificial intelligence as being something newly discovered, something new in 2018. But there are companies that have been steadily working at the coal face of AI for years and have been providing AI-enhanced service to a multitude of customers worldwide.
“Micro Focus is the seventh largest pure software company in the world,” says David Humphrey, CTO of rich media. Headquartered in the UK, but with offices globally, the company’s services cover everything from cybersecurity to cloud management to file and network access.
In increasing demand are its video AI analytics tools. The services run along two main strands. One is security, which includes video analysis for public safety, law enforcement, and worker health and safety. The other is media analysis of broadcast content for businesses, governmental organisations or brands wanting to track the occurrence of specific types of subject matter.
“It could be a government or enterprise wanting to know what people are saying about them or what situations are developing that they may need to react to,” explains Humphrey. “Or, to a lesser extent, tracking which logos are appearing on the television – not in a pure advert, because that’s well known and well-documented, but for product placement or in a Grand Prix how many times the Peroni logo is seen. It’s quite a wide gamut.”
The company’s video analysis can capture information about all aspects of the video content – who is appearing in the video, what is said, what music is playing, what products are appearing – and is offered as a set of tools for customers to build into their own applications.
Micro Focus applications include speech to text and object, facial and optical character recognition and audio classification. Although they may be straightforward and easy to incorporate from the user end, the technology behind them is the result of years of dogged research by developers in Micro Focus’ Cambridge labs.
“To use the current terms, our work is heavily based in AI, machine learning, neural networks. Those seem to be the buzzwords of the time, but we’ve been doing it for many years. Consequently, there is some stuff it automatically learns and interprets and understands better and there some parts that we allow you to train. For example, we will provide you with different models for different languages, but you might be analysing a documentary library on oil rigs and there could be some very industry-specific terms. You can improve the recognition rate by using more specific language models for that industry, or you might want to train the system to recognise some completely new object.”
Trending Micro Focus AI’s can automatically analyse and display trends in news coverage with automatic categorisation
Machine learning becomes more useful the more data it’s exposed
Educating the AI
Machine learning becomes more useful the more data it’s exposed to. Ongoing and ever-developing data sources like news reports can provide an excellent field for machine learning to work in.
“We continually monitor news so that we can update our language models. Certain names and places are not in common vocabulary, but when they come up on the news we want to be able to understand what they are.
“Unlike certain competitors who have a language model that’s based on actually understanding each word and how each word fits together – which is very difficult to keep track of – we use a probabilistic language model. We focus on the context of the language rather than the individual meanings…The classic example is if I tell you about a flightless black and white bird that lives in the sea in cold climates, you know what it is, even though I haven’t said the name.”
Using context, the AI analytics can tell whether an instance of the word “bush” refers to a large plant or a former president of the USA.
“Or we might look at how it would approach the appearance of the words “Daesh” and “ISIS”. A simplistic way to do this it is to run synonyms and say, “If you ever hear ISIS, then ISIS equals Daesh”. But that way takes an awful lot of upkeep. The way our system learns, in the probabilistic use of language, it will put together itself that Daesh and ISIS are the same thing.”
The underlying architecture of the Micro Focus AI supports about 150 languages, with some 30 languages supported in its speech to text application and its ability to recognise meaning and sentiment, all of which was acquired through machine learning.
AI analytics: The above image displays real time speech analytics transcribing broadcast audio to text
Massive metadata
Micro Focus works with major media brands in radio, newspapers and journalism, both as an archive tool and a newsgathering facility.
“We have worked with TV news companies, helping them understand what is said in other languages and to improve their workflow from getting footage to broadcast. We also do pure newsgathering with our media monitoring where we can learn who said this, who said that.
“We also work with companies who are monitoring their own programmes and generating a huge amount of metadata about those programmes, and then they’re comparing that to Nielsen and other audience analytics. And they’re trying to find a pattern between people turning off a programme and the content. What was said on-screen when people started to turn off that programme?”
The uses for the metadata produced by the Micro Focus analytics AI are bounded only by the imagination of the user. Consequently, the tools are designed to be flexible and easy to integrate into customer’s systems.
“Everybody is ever so slightly different, and we have to cater for those changes. It’s all API based and we have open-sourced the applications, so people can add on to them, and you can build or modify your pages to incorporate our solution.”
“One of the problems we have is that we can give you all this great metadata, but it will be hard for you to produce an application that exposes the metadata as well as we can. So, we provide these free open-source widgets that can nicely expose our metadata in your application. You’ve got timelines and scrollable text, and when someone’s speaking the words are highlighted. All that plumbing, we provide.”
Topic Map: The multi-coloured graphic below shows a set of key concepts automatically derived by Micro Focus’s AI technology from a massive set of data
The Internet of Recognition
Well aware of the booming media consumption on personal devices, Micro Focus is now focusing its efforts on mobile.
“We’re working on analysing rich media that comes from mobile devices, whether that’s drones, body-worn cameras or iPhones, and being able to index the huge amount of information provided by them. Using motion from a video we can effectively use it to create 3D models.”
The potential for 3D modelling of anything shot on a mobile camera is jaw-dropping. It effectively turns the camera on every mobile device into a 3D scanner.
“Once you know that you’ve got that 3D map from multiple devices in an area, then you get something called the Internet of Recognition. You’ve heard of the Internet of Things, but that’s basically dealing only with static devices and points. But when I can look at an incident on the street with these multiple 3D points of view, I can get a fully rounded sense of what’s occurring.”
The company is also releasing a new tool called Dynamic Corpus Connector. Rather than bringing the video to the AI analytics to be analysed, the Connector AI actively searches for the content to analyse.
“The thing that goes out and collects the data is now empowered with artificial intelligence, so it’s effectively looking at a repository to find the things you need. My analogy is, where previously, when open-cast mining for gold, you dug a hole in the ground and filtered extracted material to find the gold, now, instead, you find the vein and follow it. So your use case will be empowered to be faster and better.”
The Dynamic Corpus Collector will be useful on the dark web, which is not indexed and cannot be searched. It can search for things like pirated content. “It will be incredibly powerful,” says Humphrey.
If the progress of AI over the last couple years is anything to go by, that may be a big understatement.
This article first appeared in the July 2018 issue of FEED Magazine.
AI and machine learning archives
