Project KinTrans Hands Can Talk uses AI to learn and process the body movements of sign language.
Over 30 Deaf participants of different races, ages, education, and geographic location in the US recorded the first American Sign language (ASL) 3D dataset to capture body movements of the the language. Project KinTrans was funded by Microsoft AI for Accessibility.
Why 3D is important...
We use machine-learning (ML), a software development tool that allows a computational system to learn and analyze volumes of data.
Since human body movement happens in 3 dimensions, ML facilitates computation of all this data. With computation and annotation of data, now the machine learns and analyzes for new discoveries from data.
We discovered....
The power of this project was not exclusively about analyzing body movements for new sign-technology.
Among the many learnings of this project, we discovered that differences in signs span not only geography in the form of dialects, but also across generations of signers! This highlights the importance and value of this diverse Deaf contributor base: 4 regions, 30 native signers spanning ages 18-50+, representing 5 different races.
Sign up for the dataset will be located here October 2020
Initial project documentation can be found on GitHub
Project KinTrans 2019-2020 Partners
Washington DC
We are an app publishing and creative productions company that specializes in developing community and educational resources related to American Sign Language and Deaf Culture. Matt Malzkuhn, Co-Founder Ink & Salt LLC. Visit Us Here
California
DCS is an “…of, by, and for Deaf and Hard-of-Hearing” agency. DCS’ mission centers on meeting the social, economic, educational, and behavioral health needs of the Deaf and Hard of Hearing, DeafBlind and Late Deafened Community. Visit Us Here
Colorado
Ktquiet LLC is a Deaf-owned boutique consulting firm specializing in tailoring solutions to strengthen and grow organizations. Visit Us Here
Texas
Deaf-owned marketing and web development agency. Visit Us Here
Project KinTrans Early Partners
Origins of the Project
KinTrans Inc dba linedanceAI was founded by Mohamed Elwazer in 2013 in Dubai. He was inspired to build the first automatic, machine-learning technology that can learn the movements of sign language.
What has evolved is a machine-learning platform for full body human movement.
Today the HQ is in Dallas, Texas USA.
Technology patented in the US, pending in Europe, Canada and Israel.
Technology development was inspired by Deaf User requirements of a future sign language translator:
-
Natural use of sign language - NO wearables, no gloves
-
Build a flexible application - so it can fit in different places, or on the mobile
-
Recognize signs when they are continuous
-
Accommodate different body types and signing styles
After joining the startup ecosystem in Austin, Texas, KinTrans Inc dba linedanceAI won the 1776 Challenge Cup 1 minute pitch competition for Texas in 2017.
Also in 2017, linedanceAI was selected as one of 6 startups at the WSJ Laguna Beach TECH D.LIVE event.
KinTrans won a grant for 2018 to support projects like the image above at Texas School for the Deaf. The grant enabled us to hire interns to build part of our ASL 3D dictionaries & conduct a Deaf user experience survey. Other sign languages were also recorded as part of the grant.
Today, KinTrans Hands Can Talk 3D database project is being modeled for greater scalability to digitize the movements of global sign languages.
Founders
Mohamed Elwazer, Founder/CTO
Mohamed Elwazer, from Cairo, Egypt, system architect for linedanceAI's human movement analysis platform. Computer systems engineer focused in machine-learning & image processing coupled with over 10 years of technical & leadership entrepreneurial experience.
THE VISION
We believe human movement is the next data frontier. Like those cognitive services that have come before such as voice, image, and object recognition, NOW human movement allows even greater depth into data analytics.
Visit linedanceAI to learn more about how we are brining this vision to life.
Catherine Bentley, Co-founder/Biz Dev
Catherine Bentley, from Dallas, Texas, manages partnerships, new business & operations for linedanceAI. A seasoned business consultant in human resources, strategy and innovation for businesses and governmental entities in the US and Middle East.
TECHNOLOGY
Human centered design meets machine-learning:
-
KinTrans Hands Can Talk 3D database project is built upon the linedanceAI technology platform.
-
This machine-learning platform hosts human movement recording, analysis software and various APIs.
-
The database project was Deaf-led and provides ~2820 sign movement. It is designed to compliment other pillars of sign-technology development