When Elie and his father got to the concentration camp their roles reverse: Elie becomes the father who's actually taking care of the child who is depended on him, Elie's father. Until Elie's father's death, Elie always had to look after his father and basically keep him alive in the hostile environment of the concentration camp. Elie felt like there was no one looking out for him, taking care of him or making sure that he was alright, he felt like he was all alone. Elie was just a boy in his teenage years and the circumstances forced him to grow up all at once and be the adult instead of his father.

In my opinion, Elie felt relived when his father died for 2 reasons:
1.    He no longer had the responsibility of taking care of his father and looking after him. The only person he had to take care of and worry about now was himself. He didn't have to be an adult anymore, he could go back to being child, maybe not a normal, regular child, but still he was released from the burdens of being an adult.
2.    Elie's father's death had put him out of his misery and suffering. Elie's father constantly told Elie to leave him to die and go on without him, but Elie kept on forcing him to go on and kept him among the living. When Elie's father died he was actually released from all the burdens of the horrible place he had to survive in and from his suffering.