navi

summer 2024. research internship


in the summer of 2024 i joined prof. rohan paul's lab at iit delhi. he handed me a problem that sounded simple: indoor navigation for the visually impaired. google maps gets you to the building, then leaves you at the door. everything after that, the corridors, the signs, the second entrance nobody mentions, is a blank.

i spent my first days just understanding how people actually deal with this. guide rails exist, but they're expensive and only help where someone bothered to install them. whatever we built had to work in any building, using nothing but what a person already carries.

so navi started as a mobile app. point your camera anywhere and it tells you what's around you, with directions woven in: your location plus the image, turned into context-aware guidance. the first version came together in a week. flutter for the app, node.js and mongodb behind it, aws underneath.

navi

the interesting part was choosing its eyes. i tested gpt-4o, gemini, and llava on 12 images, each rated by blind and sighted users. gpt-4o was precise about the things that matter when you can't see, doors and distances, where the others got conversational or missed the cues entirely. with detailed prompts it scored 7.01/10, the best of the lot, and i spent the next stretch tuning it against 113 real-world images.

july 23rd was field testing day: five spots across campus. rni park, rajdhani restaurant, the sit and bharti buildings, sbi bank. i still remember standing outside rajdhani rotating my phone like an antenna, trying to get the entrance into frame, thinking there has to be a better way. every location taught us something. the park punished sloppy camera angles, sit exposed the limits of text recognition, and bharti's o-shaped entrance simply would not fit in a phone camera's field of view.

navi

that day made the real problem obvious. people held the phone too high or aimed it at the floor, side entrances slipped past the narrow lens, and holding a phone up while navigating defeats the entire point of hands-free help. navi worked. the form factor didn't.

that realization became vyaakhya: smart glasses that carry the same intelligence without asking anything of your hands.