Marrying the mother of ex girlfriend

Is it permissible for a man to marry the mother of a girl he used to have a relationship with? The relationship was in the past and involved physical touching but no penetration. I understand from [4:23](http://quran.com/4/23) that marriage to the wife's mother is permanently prohibited from the time a man enters into a marriage contract with a woman, whether he and his wife have engaged in sexual intercourse or not. The act of marriage itself gives the mother-in-law the same status as the mother. However in this case, the man never married the girl. Is her mother haram for him to marry?