So I went to the filters page and hit 'delete this filter'. The filter disappeared, but the emails are still coming through!
I've tried navigating to the specific filter by clicking the link in the filter confirmation email, but I can't edit it anymore.
Are they supposed to still send? How can I stop them?