Originally, In the book, the rhino is actually just a normal rhinoceros that had escaped from the London Zoo, though it nevertheless was responsible for killing James' parents (with the book erroneously claiming it ate them).
In the film, the rhino appeared out of nowhere and ate James' parents, causing him to live with his two abusive aunts Spiker and Sponge, who used the rhino to scare James. It is unknown whether it was once a real rhinoceros when it killed James' parents or if it is just a spirit haunting James (more likely the second one since rhinoceroses are herbivores and do not eat meat).
When James and his bug friends finally reach New York City, the rhino appears and tries to slow down their progress. Slowly, it approaches the giant peach, using the power of fear. When James manages to muster up the courage to say he isn't afraid of the rhino, it ultimately dissolved, killing the rhino after striking its final lightning bolt at giant peach (despite that James survived).