I have absolutely no problem with the principle of proper adverts (i.e. Adverts that are relevant to the audience, and only appearing in the small sidebar like existing ones etc.) however the concern I have is the same one Joe H articulates in his response to the Meta post regarding the new advertising model.
The problem with allowing advertisers to "own" space on a page is that they could serve anything there, from adverts through to malware. There are numerous examples of adverts on sites considered secure ending up serving fairly malicious payloads, and a far as attackers go, sites like SE with a huge footprint are great targets.
This is what we call a Watering Hole Attack, where you identify where your targets go and catch them there. For example, if I wanted to compromise magento in some way, having adverts on the magento.se site would give me the opportunity to attack magento developers, and magento e-commerce sites!
The info SE sends to the advertiser doesn't concern me - this seems to protect the user's privacy well - but other than trusting the advert provider to fully vet every advert served, is there anything SE is doing to gain assurance over adverts?
This example described at arstechnica shows how attackers can stealthily hijack advert space, with seemingly innocuous JavaScript combined with code steganographically hidden in pixels in the banner ad.