This is the one loose end! Here's an explanation to fill the plot hole: When Walt was vacationing with Grethen and her family, he saw correspondence (e.g., letters--no texts back then) between Gretchen and Elliot and discovered that they were having an affair. So, he abruptly packed his stuff and left (as seen in the episode), and also left Gretchen, Elliot, and Gray Matter for good.
Walt left Gretchen because he found out that Gretchen is his biological sister, they have the same father, Gretchen's father. Walt did not know Gretchen was his sister until the 4th of July weekend when he went off with Gretchen and her family. In a kind of a Luke Skywalker/Darth Vader moment, Gretchen's father revealed the truth to Walt, that Walt and Gretchen were having an incestious relationship without knowing it. When Walt learned this, he took off, left Gretchen and Gray Matter, and sold all his interests in the company.
This whole story starts before Walt is born, a woman married a man who had a fatal disease, Huntington's Disease. The man was already suffering from the disease when she married him, after a few years married to this man, the woman decides she wants a child. The woman and her husband try to concieve, but because of the disease, they are unable to become pregnant. The wife still wants a child, so she takes matters into her own hands and goes out and finds a man and becomes pregnant by him, and then passes off the baby as her husbands who has the Huntington's Disease. When the child grows older and asks if he will get Huntington's Disease his mother tells him that she took him to the doctor and had him tested and he was OK, no disease. This is impossible, the test for Huntington's did not exist when Walt was a child, his mother told him this because she knew for a fact that he would never get the disease because she knew that the man with Huntington's disease was not his biological father. The timelines of the man with Huntington's Disease being Walt's father do not match up, the average life expectancy of someone who is diagnosed with Huntington's is 20 years after diagnosis, so if Walt was 7 years old when his father died, then his father was already in bad shape physically when Walt was conceived, he could not have done it, he was probably in about the same shape as Hector Salamaca.
OK, so back to Walt's biological father, this guy goes on with his life and has no contact with Walt or his mother after their "one night stand" takes place. A few years later this guy meets a woman and marries and starts a family of his own, he has sons and a daughter and is living the family life. Decades pass, and then one day his daughter comes to him and tells him of this wonderful new guy that she is in a relationship with and is thinking about marrying, the more she decribes this guy, the more her father realizes who this guy really is, he is the out of wedlock son that he had fathered with a woman years before. Gretchen's father knows he has to put a stop to the relationship immediately, so when they go off with Walt during the 4th of July holiday, Gretchen's father and Walt have a meeting where the truth is revealed.
After the meeting, Walt takes off, he will not tell Gretchen because he still loves her and does not want her to endure the shame of the relationship, Gretchen's father will certainly will not tell Gretchen either for the same reason. So Walt just gets away and starts a new life for himself, as time passes, he sees Gray Matter become a great success building on his work. Walt becomes more and more bitter as he sees Gretchen and Elliott become billionaires from his work and has nothing to show for it. Walt feels like he made a sacrifice to save Gretchen from the shame of having a sexual relationship with her brother only to have Gretchen and Elliott stab him in the back.
Years later when Walt meets Gretchen at the restaurant Gretchen still has not been able to figure out why Walt left her years before, it is because no one is talking, Walt has never told her, nor has Gretchen's father ever told her anything. You would think that by then Gretchen would have been able to figure out why Walt left if the reason was something ordinary.
Walt still blames his mother for his whole situation, that is why they are estranged. Walt never told his mother he had cancer, he told Skyler he was going to see his mother and tell her, but instead he went with Jesse into the desert and cooked meth. Walt blames his mother for not telling him who his biological father actually was, if he had known this, he could have stayed at Gray Matter, avoided a relationship with Gretchen, and become a billionaire, so he is bitter at his mother as well.
I think it is basically a retelling of the Star Wars story, Walt is Luke Skywalker, Gretchen is Princess Leia, Elliott is Hans Solo, and Gretchen's father is Darth Vader.
It is a crazy theory, it is my stab at it, but it seems to cover the bases on what has happened in the story. I hope that one day they reveal the reason Walt left.