Ruofei Du, CS Ph.D. Student

Introduction


What if you have a micro camera mounted under the finger nail?

We propose FingerView, a vision-augmented touch device which supports activities of daily living by capturing and feeding back visual information about the physical world as the finger touched. Some sort of color identification, audio feedback combining with speech recognition have already been explored. Potential and ongoing directions could be: text identification and recognition (OCR); +accelerometer for gesture recognition onto different textures or 3D gesture classification; haptic feedback with different position and power; daily activity classification and output from ambient displays; daily objects recognition (including hand, clothes, textures) & touch but this one is hard to propose something new...

References



HandSight: A Glove for the Blind to Feel Shapes and Navigate Obstacles: http://www.instructables.com/id/HandSight-A-Glove-for-the-Blind-to-Feel-Shapes-an/. Accessed: 2014-01-10.

Yang, X.-D., Grossman, T., Wigdor, D. and Fitzmaurice, G. 2012. Magic Finger: Always-available Input Through Finger Instrumentation. Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (New York, NY, USA, 2012), 147–156.


Nanayakkara, S., Shilkrot, R. and Maes, P. 2012. EyeRing: A Finger-worn Assistant. CHI ’12 Extended Abstracts on Human Factors in Computing Systems (New York, NY, USA, 2012), 1961–1966.

Nanayakkara, S., Shilkrot, R., Yeo, K.P. and Maes, P. 2013. EyeRing: A Finger-worn Input Device for Seamless Interactions with Our Surroundings. Proceedings of the 4th Augmented Human International Conference (New York, NY, USA, 2013), 13–20.

Chan, L., Liang, R.-H., Tsai, M.-C., Cheng, K.-Y., Su, C.-H., Chen, M.Y., Cheng, W.-H. and Chen, B.-Y. 2013. FingerPad: Private and Subtle Interaction Using Fingertips. Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (New York, NY, USA, 2013), 255–260.

Butler, A., Izadi, S. and Hodges, S. 2008. SideSight: multi-“touch” interaction around small devices. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST’08) (2008), 201–204.

Hettiarachchi A., Nanayakkara S. C., Yeo K.P., Shilkrot R. and Maes P. ”FingerDraw: More than a Digital Paintbrush”, ACM SIGCHI Augmented Human, March, 2013.

Ransiri S. and Nanayakkara S.C. “SmartFinger: An Augmented Finger as a Seamless ‘Channel’ between Digital and Physical Objects”, ACM SIGCHI Augmented Human, March, 2013.

Wobbrock, J.O., Wilson, A.D. and Li, Y. (2007). Gestures without libraries, toolkits or training: A $1 recognizer for user interface prototypes. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST '07). Newport, Rhode Island (October 7-10, 2007). New York: ACM Press, pp. 159-168.