How to catch request to http:// and https:// in nginx?

How to catch request to http:// and https:// in nginx?

I have a few websites each in its own server{} block in nginx, with their
domains defined in server_name directives. One of the websites also has an
https server{} block. They're all on the same IP.
How do I create server blocks that would catch the requests to
http://1.2.3.4 and https://1.2.3.4, but not any of the domains? I've tried
creating variations of "default" servers, but if nginx sees a default
https block, it just uses that block for all https connections, including
the website that has its own https server block.
nginx 1.4.2 on Ubuntu 12.04.
Thanks for any help.