Feature Based Image Metamorphosis

Morphing is an image processing technique. Metamorphosis a.k.a Morphing is the transformation from one image to another. The main steps in morphing can be summarized as:

      Image warping + cross dissolving

This technique has many interesting applications. Animation effects like making animals talk are the outcome of this technique. This effect has also been successfully applied in the entertainment industry. e.g: Michael Jackson's Black or White video (http://www.youtube.com/watch?v=IisX9N6-dBc), an Indian movie where a song shows ageing effect via morphing (http://www.youtube.com/watch?v=BXvWy4pdLcs).

The morphing technique that I have implemented is called Feature Based Image Metamorphosis, a technique published by Thaddeus Beier and Shawn Neely in 1992. The main idea behind this is to create a transformation between source image and destination image. This transformation is created using inverse transform where every pixel in the destination image is mapped to some pixel in the source image. This creates a distortion of the source image with respect to the destination image. The distortion is controlled by specifying pairs of control lines (one line for each image). These control lines are vectors i.e they have direction.

Let us look at a single pair of lines and understand how the distortion happens.
Let PQ be the vector in destination image and P'Q' be the vector in the source image. For each pixel X in the destination image, the distance 'u' from P along PQ and the perpendicular distance 'v' to PQ is calculated. The pixel X' is obtained by applying values 'u' and 'v' to P'Q'. The color at pixel X' is applied to X.

For getting finer control, more than one pair of lines can be defined for the images. Instead of performing a simple averaging to combine the effect of each line on each pixel, following weighting parameters are defined in the paper:

a: determine position of pixels on the line
  Large value => smooth warping, lesser control
b: Strength of relative lines with distance
  Large value => each pixel affected by line nearest to it.
  0 => all lines affect equally.
p: Effect of length of the line
  0 => all lines have same weight
  1 => longer lines have greater relative weight

The pseudo-code for the algorithm is as follows:

For each pixel X in destination
  DSUM = (0,0)
  Weightsum = 0
  For each line PiQi
     Calculate u,v based on PiQi
     Calculate X'i based on u,v and P'iQ'i
     Calculate displacement = Di = X'i – Xi for this line
     Calculate dist = shortest distance from X to PiQi
     weight = (length^p / (a+dist))^b
     DSUM += Di * weight
     weightsum +=weight
     X' = X + DSUM / weightsum
   DestinationImage (X) = sourceImage(X')

Feature Based Metamorphosis described above is also called Field Morphing. This is because the multiple pair of vectors and the weighting parameters create a field like effect.

This algorithm has the advantage of being intuitive and simple to implement. The other plus point is that it allows the user to define control points. However this technique is computationally expensive. Each pixel needs to be compared against every control line. As the number of lines and size of image increases, so does the computation time. This makes the algorithm slow. Also, to get a good morphing effect, large number of control lines need to be specified.

While this technique has its disadvantages, the advantages and the simplicity have made it state of the art. Some interesting effects I was able to create are presented below:

FACIAL MORPHING



ANIMATION EFFECT




Reference
[1] Thaddeus Beier, Shawn Neely, "Feature-Based Image Metamorphosis", SIGGRAPH July 1992.

Music speaks!

I am sitting in a Zakhir Hussain concert called Masters of Percussion, my eyes closed and my head bent back. As the instruments play, I feel these sensations. I feel as though the beat of the drums, the hum of the string are saying something. As I fall deeper and deeper under the spell .... I can hear them talk to me.

The drums start playing. Each beat is distinct from the other. Its like our thoughts, our decisions .. our mind. The rhythm, the precise beat are akin to the activities of the brain. It starts out as a simple beat, a single thought. Then the beat becomes complex, more than one thoughts mingle. As the tempo builds we get a rhythm consisting of a myriad of simple beats. An idea is born, a decision is made, a realization germinates ... all these are the amalgamation of multiple threads of thought. These instruments - the Tabla, Western drums, Doyra, Nagada, Dholak .... all THINK!

As I think about these instruments that think .... I feel something stirring inside me. Thats when I realise that my ears are picking up a soulful strain of music. The Sarangi and Sitar have joined in the melody. The beats settle down to the background and the strings take over. I feel open and exposed as though my soul has been bared. I feel emotional. They start out with a frisky melody that makes me feel bright and happy. As the piece ends and the next one starts, the change in mood is apparent. Because I feel tears sting my eyes and my heart going heavy. Its as though the string instruments are expressing pain. And this is where the brain butts in again. I think .... these string instruments are actually a form of expressing feelings.

The grandeur of the music reaches its peak when the beat joins the sound of the strings. On a similar level, its when thought meets emotion that we feel a confluence that brings about harmony. The music wraps up around me, cocooning me, making me feel relaxed and blissful. Not heady nor drenched with emotion, but a comfortable balance of both. I take in these magical moments, the music heightens and comes to a stop. I sit back and take a deep breath, as the echoes fade away and I am pulled back to reality.

Right .... Left ... or just plain Wrong?

Its about a month since I have been here,
And my mind has become a muddle, hard to clear.
Read on ahead and you will know why,
I have cooked up a hot n spicy, bheja fry!

Top of the list is the much know right hand drive,
Mixing left with right may not see you alive.
Changing orientation is easier said than done,
Driving here is hard work and not mere fun.

Lets enter the room and turn on the lamp,
The switch is turned down, but the room is dark and damp?
Thoughts race ahead, "I think, the bulb might have gone".
And then I see a finger turn the switch off to have the lights come ON.

"Never talk to strangers!", since my childhood I had been told,
But the way here is much different than this saying rather old.
People will chat up without each other knowing,
Of the weather, city or just a "How are you doing?".

BIG is the word, and I mean THE word that thrives,
Nothing is small or little, its either Large or a Mega size.
The Indian logic of little first and then if you like it some more,
Goes out of the window in a restaurant or a store.

Variety is the spice of life, and I can say that its true,
My head has stopped swooning and I see the sky, blue.
Its not "right? left? or just plain wrong?"!
Its a different tune to the same life's song.

Daylight Saving Time - A Limerick!

I woke up late on a Sunday morning
Rubbing my eyes, stretching and yawning,
My face turned sour
I had lost an hour
The clock was ticking to Daylight saving!

First Step!

I am sitting, with nothing much to do ..... lots of time on my hands, and lots of thoughts in my head. So here I am converting my nothing to something.

As I take my first step in blog-o-sphere, I think ..... what would be an appropriate first blog? And, my thoughts drift ............. my mind is recounting, reliving the first step events of my life.

My memories pull me back as far as they can, and I can see the first day of school. I remember feeling a "little" grown up, just enough to take that first step as I let go of my mother's hand. I find myself thinking of the first time I dipped my toe in the swimming and cried wild - "I am going to drown!!!". Oh! and I also just thought of the first time I walked on stage - this little guiding light of mine, I'm going to let it shine ..... we sang. The small footsteps became bigger, but the first step remained the same, atleast, it felt the same. I can still feel those moments of awe when I first understood what the word Earth meant, when I first held that regular onion peel under the microscope and saw something I had never seen before, when I first touched a newborn and lost touch with time. In the same breath I am flooded with random first timers: memories of my first drawing, first day at my first job, first time I climbed a tree , first award, first failure, first plane journey, first time I wronged, first time I forgave, first date (though he and I disagree on that one), first time I went para-sailing and more recently my first step over the threshold that gave me my new life, my new home, my new family (read marriage!).

There are a countless other memories, which can be tagged "the first" and I realize, I do something new everyday!!! Everyday I am having a first step experience. If only I had pondered on this a little earlier ... maybe I would have cherished them a little longer. Many of those firsts are indelible footprints that will always stand the test of time, some others will be overwritten while some may be left so far behind, its difficult to turn back and even see them. But no matter what we remember and what we don't, they have left a mark on us, for its our first steps that have made us come as far as we have.