iPhone X and Becoming Eric Neuman
It is only a few months since the release of iPhone X by Apple and their introduction of Animoji and “face scan”.
I have some wild ideas regarding the future of “FACE” value and personal identity. The idea of having a “clone” and being able to be inside other avatar like in a game is suddenly becoming a little bit easier.
There are all kind of reactions from people in regards to “Face ID” and actually having an (expensive) iPhone that scans their face and able to mimic your facial expressions rather accurately:
- It’s creepy (some even said that Apple cutre Animoji is creepy)
- It’s amazing tech!
- It’s “funny”, but what is the big deal??
PS: Many also avoid it because it is Apple and they say it along this line: I am not developing VR or AR apps that only works within Apple ecosystem 🧐 – Well we cannot rely on apps all made by Google, right?
As a CG artist and animator, I am okey with Apple-ness and especially interested with making my own Animoji type of apps and maybe use it as some kind of story telling tools. And so I purchased the iPhone X and really want to embrace this Face Augmented Reality (AR) and explore it inside out. Maybe having our face data as interface is really something.
I wonder what would happen 10 years from now if many people embrace their face thing and scan their face more.
I am not a developer or programmer, but I tried my best to learn a bit of Swift and make at least a couple of simple apps that canenable me to use my face as controller to make my some kind of face animation like the Animojis.
Surprisingly, thanks to the Internet collaborative-ness, there are a few smart people out there sharing some informations about iPhone X, like Elisha Hung iPhone X Face Capture and also there is another guy who did the Face Baby Animation with full body mocap, everything recorded in realtime.
I studied some of those materials and make my own setup that works with Blender 3D open source. I do not use Unity much, but maybe eventually will try it as well. So far, I am onlyusing XCode and make my own simple Animoji apps, testing them with a couple of different 3D head mesh.
Also, I tried a face model borrowed from Puppeteer Lounge (AJAR) and make a couple of realtime animation test. Trying to acting it out. My setup is perfect, but good test and can improve. Translating face into blendshape that simple works is not going to happen, it needs extra care like tuning the weighting and also the shapes for each person.
Few days ago, a guy called Eric Neuman appears on my YouTube stream and giving a presentation about “Becoming Eric Neuman”, he bravely uploaded a medium quality body scan of his, with face blendshapes for everyone on the Internet to make something out of it. I decided to give it a quick test myself, just hooking up my iPhone X to his head morphs.
This starting to be a little bit more interesting, I even have some Quantum Leap moment right when I use this “Eric” body avatar head. The model eyeballs might need some tweak and also this rig seems to have kind of generic teeth, but other than that the motion of the face being generated on the fly of my face interfacing as Eric is pretty unreal.
Apart from testing the head inside Blender, I also make a simple iPhone X app to test the Eric 3D head. There is a slight complication when exporting DAE from Blender into XCode. Especially the fact that bone and face morphs are not translating correctly. So, I have been so far just using only the mesh without bones. It would work a lot nicer if I could use bone properly and having Eric Head with moving body, even just a simple pose.
Currently there is a lot of “limitations” where iPhone X can only read either just front camera (for Face AR) or just the back camera. Perhaps it is possible to first record the face animation, and then augment the animation and apply it into body animation. There is a lot of mystery surrounding iPhone X and what comes next.
I am hoping that iPad X, apart form having front camera that mirror and mimic the user face, also could have special true depth back camera of some sort that can work together providing user a glass experience.
Anyhow, I will continue working and testing some ideas and improve my AR Eric and hopefully can turn it into a working app that everyone can test.
I was thinking, what will happen if people actually makes Instagram or YouTube account using Eric head and then maybe Google Images will start to get confused between “artificial” Eric and real Eric Neuman?
To be continued…..
