Why doesn't the speaker return in The Unhappy One by Pablo Neruda?

The speaker notes that he is leaving the woman at the beginning of the poem. From the text, we can infer that he might have been leaving to serve in the war. The war, however, isn't mentioned until the fourth stanza..... he may have simply left her. All we really know is that the woman never stopped waiting for the man who never returned.