First of all, and maybe this is just me, but I never have had this problem. Maybe for like a second ("was he talking to me? oh no he's on his phone") but usually the funny phone thing is when someone is on a hands free thing and you don't see it so you just think they are very very angry and very very schizophrenic.
The graph is too much. Look Randall, I know you like taking things like emotions and making them empirical but this is just a little much. No reason for you to jump from "sometime THIS awkward thing happens" to "thus, I will graph it!" And adding the fact that apparently your second relationship was...what, a very long conversation? The graph doesn't even make sense. It's way out to the right, so I guess that means it was a very long conversation, and it blips up a little bit, so I guess that means your relationship didn't happen very often? Better just to leave the joke out...
Also, while there's nothing wrong with the stick figure example in theory, I think the "Bernanke" reference is also too much. No relation to the comic at all, just totally random and smacking of "oh man I want to totally be current let's see now who is in the news oh goodness yes, that will work just fine." It seems way too much like the guy at the party trying and failing to look like he knows what's going on in the world.
So, in short, a comic full of parts that are slightly off, adding up to a whole that just feels all wrong and not funny. THE END.