My stored procedure was running sp_OACreate which is an extended stored procedure which requires sysadmin priviledges.

I wish I understood why those are nailed down so tight.