Just updated the Stack Exchange app on my Nexus 5 today running Lollipop and I have to slowly and very deliberately swipe to the left to horizontally scroll items on the feed. 95% of the time it auto snaps back to it's initial position.
This is fixed in 1.0.55 out for beta users now, out for all users within a few days.
I'm really sad about what I had to do to fix this. balpha correctly pointed out that it wasn't reproducible if you scrolled even a tiny bit down the feed, so it only happened at the top of the feed when pull to refresh kicks in. Turns out what's happening is that the SwipeRefreshLayout we use for handling pull-to-refresh doesn't check the x-offset of a swipe at all, it simply triggers a pull to refresh if it detect any change in the y-offset from a swipe (e.g. from when your finger first touches the screen to when it goes off the screen).
As of version 1.0.55 we now use a custom version of SwipeRefreshLayout everywhere which checks to see if a swipe's y-offset is at least 2x bigger than the x-offset before it triggers a pull to refresh. This causes sliding in the feed to be smooth in this specific use case, and also makes it so you can't swipe to refresh by going from the top left of the screen to the middle right of the screen.