If you click Log out on the page that looks similar to this one:

You'll notice the text at the bottom:
If you're on a shared computer, remember to log out of your Open ID provider (Facebook, Google, Stack Exchange OpenID, etc.) as well.
Clicking Logout only logs you out from the Stack Exchange sites but NOT from the Stack Exchange OpenID provider. That behavior is similar to what you get when using Google or Facebook. You don't expect Stack Exchange to log you out from Gmail or from the facebook timeline you're watching.
To get to your Stack Exchange OpenID page isn't that obvious though.
Its direct url is https://openid.stackexchange.com/
To get to it from the UI follow these steps. First go to My Logins, for example in your Network profile:

Now click change password (trust me, you're not going to change your password). After clicking you'll be on your OpenID profile page:

Now you can click on the Logout tab which will bring you to the actual logout page of the Stack Exchange OpenID provider:

You can click the Log Out button here. That will log you out from the OpenID provider.
After clicking that button you'll notice that you have to re-enter all your credentials if you want to login again in any of the Stack Exchange sites.