I've the same problem as described here:

www.artio.net/support-forums/joomsef-pay...r-category-blog-view

Instead of a blog layout you'll see the category list...

URL on my site is also:
index.php?option=com_content&id=5&layout=default&view=category

I've set

Add layout to URL - Only when different from default
Default layout - Blog

This doesn't work correctly - after a few page hits you'll see the category layout again - when I disable JoomSEF or I clear the SEF URL's all works correctly.

The URL for a working URL's is the same as above:
index.php?option=com_content&id=7&layout=default&view=category

I don't know if this is JoomSEF or Joomla related...

Daniel