image recognition

4 Sources Of Functional Illiteracy That Technology Can’t Fix (Yet)

Posted by

5

Most blind people are perfectly literate. We may need screen readers and/or braille dots to do it, but most of us can read as well as any sighted person. Further, much of the reading material that was once unavailable to us—magazines, newspapers, pamphlets—can be accessed online. It’s much easier to be a bookworm in 2016 than it was in, say, 1995. The world of the written word is, more often than not, accessible now. There will always be exceptions, though, and those unfortunate little exceptions can conspire to create a lot of grumbling, at least in my life. You see, no matter how accessible the world becomes, blind people will remain functionally illiterate when it comes to…

1. Signs

Signs: helpful little things, which do a lot more than indicate street names and business establishments. GPS and a healthy knowledge of the city was not helpful to me when I nearly trailed the delicate sleeve of my favourite blouse in wet paint because I couldn’t read the sign on the railing. I’ve nearly ruined a cherished skirt while trying to sit on a newly-painted bench. I’ve slid on wet floors, only finding (and knocking over) the helpful wet floor sign after the fact. (Those really do lend themselves well to being loudly and conspicuously toppled, don’t they?) I’ve tried to use elevators and toilets that were out of order. I’ve tried to walk through emergency exits when there was no emergency in sight. All the many helpful hints signs can provide are lost on me, and it is only the boundless kindness of strangers that has saved me from many an embarrassing mishap. (Thanks to the odd passers-by, I still own both blouse and skirt!)

2. Subtitles and Captions

So there I am, watching some powerful video or other, when suddenly the actors switch language. The nice video editors have thoughtfully provided subtitles, but I’m left feeling totally lost. If I’m lucky, the video comes with description, so at least the describer can read the subtitles to me, though this is quite distracting and really takes away from the flow of dialogue. Mostly, I’m unlucky, and nearby sighted people are subjected to eyelash-fluttering and relentless entreaty until they agree to read me the subtitles. It’s frustrating, and while it doesn’t come up very often—I’ve memorized the Elvish bits in LOTR, so that at least is no issue—it’s a real thorn when it does.

3. Handwriting

No matter how skilled we become at inventing and using technology that can read printed material from menus, books, and photos, I don’t know if we’ll ever progress to the point where the blind can access handwriting. Everyone’s handwriting is unique, some more readable than others, but even the neatest penmanship is essentially inaccessible to anyone who can’t see it for themselves. I’ve only a rudimentary understanding of printed letters as it is, so when someone leaves handwritten notes, or uses fancy calligraphy on a bottle of perfume, I’m left wondering. Reading about how personal and intimate handwritten letters are does not help with morale, either. Excuse me while I go shed a few tears over the fact that I’ll never receive a handwritten love letter. I’ll never even take a Buzzfeed quiz on what my handwriting says about me.
Okay, I’m done now.

4. Packaging

It’s getting easier to read labels on packaging now that we have image recognition apps. If you’re able to snap a clear photo of the object in question, it’s possible to have your smart phone rattling off the information in seconds. This assumes you, unlike me, are any good at taking good photos on the first or seventh try, of course. No matter how intelligent the technology, no matter how clear the photo, no matter how strong your desire to read the packaging, however, the fact remains that some companies just don’t make it easy for us. The print on some items is so miniscule even fully-sighted people struggle to read the finer points. Try reading an expiration date or ingredients list without a microscope. And, if you can find and read the instructions without five minutes of fiddling, come talk to me. It would take less time to read a five-page forum on how to open that stubborn bottle of toilet cleaner than it would to find the convoluted instructions printed in tiny lettering on the back. Besides, you meet cool new people while trying to open things. If that fails, you can always resort to more eyelash-fluttering, obviously.

I’m glad to report that, as with so many issues, functional illiteracy for blind people is diminishing. We’re able to access so much material online now that the need to read conventionally is lessening every day. I am seldom reminded of my disability when it comes to reading material, and maybe that’s why it’s so jarring when I am. If you become accustomed to accessing something, and are suddenly and definitively unable, it stands out even more sharply for its rarity. Nothing transports me back to childhood faster than having something read to me, and that’s not the type of childhood nostalgia I welcome. My hope is that strangers will stay kind, and friends will stay patient. Just remember, while you’re rereading that piece of paper for the fifth time, I’m just as frustrated as you are.

Advertisement

In Praise Of TapTapSee

Posted by

0

I’ve always been skeptical of image recognition apps that try to compensate for a pair of broken eyes. I remember, rather too vividly, a CNIB demonstration of a colour indicator. The thing was outrageously priced, and in any case it really didn’t work. The salesperson didn’t do a very good job of hiding her dismay when it failed, during multiple attempts, to get the colour right—or even close to right. Since then I’ve been, perhaps unfairly, disenchanted with image recognition technology.

an image recognition app called TapTapSee came on the scene and encouraged me to think differently. Sure, it had a few kinks to be worked out, and even today, it’s not always spot on. (During one memorable session, it informed me that a teabag I was photographing said “tips about relationships.”) Despite its occasional mistakes, and its apparent inability to master colour indication, its uses cannot be quantified. It recognizes labels on packaging, articles of clothing, and almost anything else you’d need help to identify. Sometimes, it’s so descriptive that it scares me a little: it once told me that my profile picture included a “woman in a black tank top smiling in a field of yellow flowers.” The detail (and accuracy) was enough to make my jaw drop. It’s worth noting, however, that the magic happens largely because of the efforts of sighted volunteers. Without their insight, the app would be just as clumsy and ineffectual as all the others. Those volunteers, in particular, are what make TapTapSee shine.

It’s still best to label everything and keep my belongings organized. However, it’s nice to know that a clever app like TapTapSee has my back. It has only improved with time, and I can’t wait to see where image recognition technology goes from here.

What Is This, Anyway?

Posted by

3

I’ve sometimes pondered how much of my waking life I’ve spent simply identifying stuff. Being blind necessitates devising elaborate strategies to keep track of things like canned goods, important paperwork, and even clothing, depending on the complexity of your wardrobe. There are a lot of tools out there to help us, from free apps like CamFind to very costly Pen Friends—devices that allow you to “tag” certain items and have the pen tell you what it is later. The latter is efficient, but not everyone has a couple hundred bucks to shell out for it.

As I’ve said, there are some free (or inexpensive) apps designed to help us out, but they’re often difficult to use properly. I, perhaps more than many blind people, am hopeless at taking good pictures of things; the other day I was going through my tea collection, trying to find a certain bag, and I had to take them all out and sniff them to find what I was looking for. I eventually lost patience (I have a lot of tea), and pulled out my cell phone to use image recognition. One little problem: the pictures were either too blurry, or taken too close up, or taken too far away, or entirely inaccurate because I’d photographed my lap, or the floor, or anything else but the bags I was trying to identify. Generally, you just point and shoot, but I swear there are days when the apps just don’t want to cooperate. In fact, when photographing a particularly stubborn bag, my app cheerfully informed me that the tea was called “tips about relationships”. Gee, thanks.

Lighting and placement are other concerns that I struggle with. I don’t always consider how dark it is, or how many shadows might shield the product in question. I also sometimes accidentally place the product among others, which makes it harder for the app to know what I’m trying to photograph. I get better at it the longer I play with it, but I’m still a long way from perfect.

There are some relatively unusual tips and tricks I’ve picked up over the years. I can, for example, identify a can of green beans by shaking the can and listening to the squeak the beans make. No other canned vegetable that I know of makes that particular sound. When I tell sighted people about this, they don’t believe me until they try for themselves. I’m also good at finding minute differences between, say, a matching set of shampoo and conditioner, though I have had some interesting experiences when the bottles are identical.

Despite all the technology, organizational systems, and detail-oriented planning, I’ve definitely gotten myself into some very strange situations. I once found what looked like a candy dish on my living room coffee table. Thinking that my mother had thoughtfully put out candy, I picked up one of the curiously smooth treats and popped it into my mouth. All I tasted was dust: I’d tried to eat a decorative rock. I’ve put conditioner (and body wash) in my hair instead of shampoo; I’ve nearly put frozen berries in a dish instead of peas (luckily the smell tipped me off); I’ve poured instant oatmeal into my travel mug instead of hot chocolate mix. All this, and much more.

Gregg is fond of telling the “bath bead story”. When he was a little boy, he found a strange-looking dish in the bathroom which appeared to be filled with candy. Having disassembled (and failed to reassemble, naturally) a gumball machine the day before, he assumed that his mother had transferred the gumballs to this dish. Undeterred by the fact that people don’t usually store candy in the bathroom, he grabbed a candy and took a big bite. He immediately found his mouth filled with scented foam. he’d eaten one of his mother’s bath beads. He’s also had a hair mishap; he got a shampoo bottle and mayonnaise bottle mixed up, and nearly washed his hair with the latter. I’ve since informed him that mayonnaise is actually good for the hair now and then, but I’m not sure he believes me.

If there are any blind people out there who have good stories of this nature, please share them in the comments; we’d all love to indulge in a little schadenfreud…uh, I mean, we’d all love to share our compassion and sympathy. Yeah, let’s go with that.