Yes, you should change the tag to the "root" or generic ruby-on-rails tag when you come across questions that use a version-specific tag, but are not version-specific.
You're completely correct in your assessment that tags with an explicit version number should be reserved only for questions that concern details applicable to only a particular version of that language/application.
Users will sometimes use the wrong tag, but short of eliminating version-specific tags altogether, there isn't much that we can do to fix this problem. In some cases, it doesn't make sense to have version tags, and in those cases we can make the version-specific tags a synonym of the "root" tag (effectively eliminating them altogether). But in lots of cases, it does make sense to keep these around. I'm not knowledgeable about Ruby on Rails, but I know that we have a c#-4.0 tag (among others) that is quite useful in categorizing content specific to version 4 of the language. Sometimes it gets misused as well, by users who just happen to be using version 4 of the language, but are asking a question about something that is not specific to version 4. But that's just the cost of doing business. There are plenty of helpful users who will retag those questions when they appear.
There has been some talk about tag hierarchies and/or "implied" tag relationships, where adding a ruby-on-rails-3 tag would automatically add or imply the ruby-on-rails tag. It's gotten various amounts of traction. I'm rather torn: in some ways, I think it would be useful. In other ways, I'm afraid that it would add unnecessary complexity to the tagging system and ultimately increase the amount of needless or "noise" tags that get attached to questions.