A turntable
Until recently there weren't any terrific budget turntable options out there. Cheaper tables would destroy your records, while nicer ones were out of most people's price range. Thankfully, that's no longer a problem thanks to U-Turn.
For my full thoughts, you can check out my review, but as far as I'm concerned, U-Turn's Orbit is the only budget turntable worth considering. It's well constructed, plays records well, and should last you or whoever you're buying it for a lifetime. If records don't sound good on this table, it's another component of your stereo system.
A ticket holder
Sticking them on your wall is one option, but they can get easily damaged by whatever you stick them on with. I used to slip mine into record sleeves, but that's also kind of a lame, inelegant solution. You want people to know you were there.
This shadow box is a cool way to highlight all the shows a person has been to without risking masking tape-related damage. After the show, the ticket goes in with the rest of them, where it can be easily plucked for reminiscence or bragging rights.
Bluetooth headphones
Even huge vinyl fans listen to digital music, be it at work, while exercising, doing chores, or running errands. A good pair of Bluetooth headphones can help turn a cumbersome commute into a more manageable one.
For their price, it's hard to beat 1More's over-ear option. They're pretty comfortable, sound very nice — there's a little extra emphasis on treble — and look pretty cool. You can read my full review here, but these headphones have a lot going for them.
Earbuds
They're a little pricey, but worth it. I haven't heard an earbud yet that matches their clarity, and with four different-sized earbud tips, they'll fit almost any ear.
My favorite part about these headphones (besides their sound) is the fact that they ship with a clip. That might seem like a small thing, but when you walk with earbuds on, gravity is working against you. As you walk, your headphone cable moves and puts pressure on the earbud, which can then pop out. The clip stops this from happening, and it makes a big difference.