We broke a critical barrier with Lucene.Net internally with respect to file locking earlier. While we've deployed some optimizations now and will need to make a larger shift long-term, our reduced web farm out in Oregon is feeling a bit of pain.
We're hitting limits that don't have an easy solution as certain elements on Lucene were not ported that would be very useful at the moment (
NIOFSDirectory). Since those aren't quick to slap in and test, we're in a bit of a bind deciding what to do...the community and support for Lucene.Net may not be at a level we're comfortable with continuing on. That's partly because we may be the largest volume user of it (I'd love to talk to anyone with an equivalent volume level, if you know anyone send them my way).
Currently, you'll notice search results are only 1000, not 5000, search quotas for backend processes have been reduced, some additional tuning on the code side has also been done.
For now we're back under 30% CPU on 5 web servers and that still makes us a bit uneasy. Some more hardware is on the way and we're making performance improvements across the board, we'll try our best to ensure the sites remain online until the move back to New York approximately a month from now.