Are you sure your drill was turning at the correct direction? Thumb of rules. No brick will be harder than drill bits. I am pretty sure your drill was turning the opposite direction
try masonry bit first. most likely you were using general purpose bit, with largest torque. if not working, try hammer drill. last time my neighbor said it was very laboring to drill in the wall only to find out he was using a general bit. I used masonry bit and it took almost no effort to do the same job.