As long as there is light and some minimal nutrients algae will grow on surfaces.Its inevitable .As long as is not fuzzy I quite like it as gives age appearance .If U dont like it scrub it off with toothbrush or small wire brush or spot dose it with liquid carbon and the scrub it.All those spotles aquasscapes with clean hardscape are achieved with good tank maintenance.
Spotlessly clean tank hardscape is a bit OCD for me.lol
You're right, nothing to worry about at all. I think that a patina of green algae covering rocks can add to the naturalness of a scape and also add a sense of age.
I also think that scrubbing rocks is kind of like Cnut trying to turn the tide...and eventually you'll give it up as a waste of time.