Mahabharata. The Mahabharata and the Ramayana are the two ancient epics of Hindu mythology. There are a lot of stories encompassed by them, but there is a tale of King Revaita in the Mahabharata, where he goes to another world to meet Brahma (the Creator) and when he returns, hundreds of years have passed.
Talmud. In the Talmud (the central text of Rabbinic Judaism), there is a story about a man who sleeps for 70 years, waking to find that his grandchildren had become grandparents.
Nihongi (Chronicles of Japan). This is the second oldest book of Japanese history. In it, there is a story of Urashima Taro, a fisherman who saves a turtle, and is rewarded with a trip to see the Dragon God under the sea. He stays for three days, but when he goes home, he has been gone for 300 years.
A Christmas Carol. In Charles Dickens' 1843 story, Scrooge's whirlwind tour of Christmasses past, present, and future are a form of time travel, whether you consider them a dream, a ghostly vision, or literal.
A Connecticut Yankee in King Arthur's Court. Mark Twain's 1889 classic is a humorous tale of an engineer who is transported back to the age of King Arthur and convinces the folks he finds there that he is a magician.
The Time Machine. Do I really need to tell you that this 1895 H G Wells book is about time travel?
Slaughterhouse-Five. This 1969 Kurt Vonnegut novel is about a man named Billy Pilgrim who becomes "unstuck in time." It's told in nonlinear order, because Billy's life jumps around randomly in his personal timeline.
After this, sci-fi abounded, and time travel was everywhere in books, TV shows, films, video games... everywhere.