Is praying towards kabah idolatry?

Assalam Ailaikum. I hope you are having a good day. I just have a quick question. Sometimes when I pray, and there is something in the way, instead of thinking about the object that is in the way I think about the kabah, I think about the kabah when I bow down while praying if this happens. However, I just started realizing that by doing this I may be committing idol worship by mistake, as I was thinking of the kabah and not Allah. However, my intention was to pray to allah and not to worship the kabah. Should I pray for repentance? Thank you, peace be upon you and have a good day.