I have 2 or 3 loops that I train on.
I like training by time, always trying to beat my best time.

My avg heart rate falls around 155.

I am 32 so around 150 to 160 is the zone that I guess your not suppossed to train in.

Should I slow down?

Your age is not the issue, as the formula for calculating max HR from age is notoriously useless for fit people. You need to find your maximum sustainable HR, which is bascially a number you get when doing a 20 minute full-on effort after a complete warmup. Then you base your training on this, if you want to do HR training. There are a number of books and references available to give you training programs based on the number, often referred to as your AT or LT (anaerobic threshold or lactate threshold). Neither of these terms is really accurate, but you can structure your training off this number, regardless of what you call it. All that said, you can also go by perceived effort.

Slow down ??? why don't you speed up ? You don't say how long you ride for.
My max is 195ish, but I can spend an hour at 175+ after warming up adequately.