I prefer an MP3 player over my phone. Here is the one I use. Why I like this one:

• Dedicated device designed for music.
• Hardware designed to play high quality music. (Think using Ubuntu vs Ubuntu Studio for music production)
• Dedicated buttons instead of all touch screen.
• More options for integration with other devices or systems
• No distractions. Phones nowadays demand our attention for every little thing. Every app, no matter what it is, has notifications.
• The Bluetooth is better.
• You can literally hear the difference in the quality of the music if you use good quality headphones/ear buds. The same song, same file, will not sound the same if it's a good quality FLAC.

My phone doesn't have a 3.5mm headphone jack.

While I use my phone for music I can certainly see some advantages:
physical buttons
smaller and lighter
less distraction than a phone
cheaper to replace if stolen or broken

I have a shuffle just for running.

You can replace your phone with dedicated electronics, which has some advantages, such as better battery life and generally better performance. Like I bet this thing has a better amplifier than a phone has.

And phones have their downsides. They don't last as long and are expensive. Privacy issues. And can be too stimulating and intrusive. For example sometimes you just need to know the time and before you know it you're emailing someone.

When I go hiking I can just as well take a dumb phone, a GPS and an mp3 player with me. Maybe a camera too.

Having something that doesn't connect to the internet makes for a better device in the long run. Plus, as others have mentioned, less distractions.