1. Siento haber llegado tarde.
2. Siento por llegar tarde.
(In any case, "lo siento por llegar tarde", but that's it's an ugly sentence.)
3. Lo siento que he llegado (llegué) tarde. (Present Sunjunctive not allowed here)
(We don't make this kind of construction. You can separate both parts: "He llegado tarde. Lo siento/lamento". If you don't invert them, you're rather explaining more than apologizing.)
If you don't use "lo", you get a sentence like: "siento que haya llegado tarde". In that case, it's someone else who arrived late, not you. When you have the same subject for both verbs, then the infinitive is necessary.
- Lamento haber llegado tarde. (I arrived late)
- Lamento que yo haya llegado tarde. (Clumsy and inelegant; not only could you have used a shorter sentence like the first one, but you also had to add your pronoun to avoid an ambiguity.)
- Lamento que haya llegado tarde. (That's a third person who was late and I'm the one who's sorry.)
I hope I didn’t bother anyone:
• Espero no haber molestado a nadie.
• Espero que no
haya molestado
(molesté) a nadie. (Imperfect Sunjunctive not allowed here)
(It's not the subjunctive per se; it's the negative form of the verb in constructions like "esperar/desear que". Here, the use of the subjunctive doesn't sound so clumsy because there is a third person involved: "nadie", so it may be used. I'd still keep the first construction instead.) 
I'll be curious: Why do you say the subjunctive is not allowed in your sentences? 