By Joe Kenny
Microsoft says that its app ‘Seeing AI “narrates the world around you”.
Now there’s an appy snappy strapline if ever I heard one! But does it?
Whether you live with less than 20/20 vision or you’re like me and as blind as it gets, any app that’ll tell you stuff you need to know, is worth a look. Before I get stuck in, I do want to say that I think research and development such as what lies behind Seeing AI, is very welcome and it’s great that a tech giant such as Microsoft, are throwing their considerable resources behind such a project. Well, it’s not before time! My only issue is, how reliable is it. Will it do what it says it’ll do when you actually need it to do it.
Seeing AI was launched in 2017 by Microsoft and makes no bones about the fact that it is a long-term experiment that relies on the end-user to inform the future evolution of the product.
The Seeing AI app uses your phone’s onboard camera and the device’s own screen reader. What’s a screen reader? Well, it’s a software that reads, in a funny little voice, what you light dependent people see on the screen. If there’s one thing that has changed the world for the better for people who can’t see too well, it’s screen reading technology. I simply couldn’t work, bank, shop or travel as I do if it wasn’t for software that converts text to speech.
When you launch Seeing AI on your device, you have a set of tools at your disposal. Seeing AI calls these channels. You simply select what you want to use from a list.
From the top, we have a light detector. This is awesome if you can’t see anything. The amount of times I leave my house with the lights still on and so they shine all day and maybe all week until someone tells me to turn them off. I know what you’re thinking, If you’re blind why would the lights be on in your house? Ah, I live with other people and their eyes do work.
The Colour Preview is handy. Yes, you guest it, it identifies colours detected by the camera. I actually do use this for deciding what shirt to wear and whether I am actually wearing odd socks again.
An experimental feature called Scene Preview provided a few moments of intrigue. Point the phone’s camera at something, take a picture of it and the app will attempt to describe what it sees. This is surprisingly accurate and reveals even more surprises as you can explore the photo by running your finger around the screen and the app will read out the individual elements within the picture. Really interesting but haven’t quite found an everyday use for this yet but great at parties for a bit of a giggle.
Next up, ‘Currency’. A very handy feature that identifies bank notes. This works well but can take a while for the app to come up with the answer. I wouldn’t want to be in a lunch time queue in Belfast city centre trying to use it! You’d be lynched.
The next channel is probably the most interesting. It’s called Person. The idea being that you point the camera at someone, take a picture and it speaks a brief description. So of course the first thing I did was to point the camera at myself and see what it said. The response? Apparently it was looking at a 47-year-old man who looks happy. Uh, actually I’m a youthful 42 and now I don’t look that happy! Still for the life of me trying to figure out what you’d really use this for. At the risk of being controversial, it strikes me as an idea dreamed up by a sighted person so that us blind folks know who is shouting at us across the dark abyss. I’m probably being unkind.
The Product channel is an interesting one. The idea here is that you use the phone’s camera to look for a bar code on an item. Say a tin of beans in your cupboard. This is very useful and it did correctly identify my Uncle Ben’s microwave rice. Though it took that long that I was all for giving up and reaching for the oven chips.
The last two channels are all about reading text. One is for reading large documents like a page from a newspaper and the other is for reading short texts like a label or a receipt. Again, good practical ideas. The document reader even does its best to give instructions so you can line the camera up with the page before taking a photo for processing.
Is there downsides? Of course there is! Seeing AI absolutely hammers your phone battery, I couldn’t get the Hand Writing channel to work and you often have to close the app and reopen to get it to work properly but as Microsoft keep telling us, it’s still an experimental project and it’s always improving. I like it though. It does help if you can’t see too well as long as you’re not in a hurry. Oh and don’t use the Person feature if you’re at all sensitive about your age.
You can find out more about the Seeing AI app at https://www.microsoft.com/en-us/ai/seeing-ai