Dmoore's proposal does not work because variables cannot be expanded first (without some nasty hack), as that would replace $relative with an empty string and leave the brackets untouched.
I didn't claim a nasty hack wouldn't be required, but that's what happens when you try to be too cute with macro syntax
(don't get me wrong, I like cute). I guess I am not sure how general these absolute/relative functions need to be. For instance, should this work:
(1) $relative($(LIBNAME($LIBTYPE)))
or only this:
(2) $relative($(LIBNAME))
and this
(3) $relative($LIBNAME)
(there's also the % syntax to worry about, or was that removed?)
if all of (1), (2) and (3) are required then you can see why I proposed what I did (because (1) wouldn't make any sense until all other substitutions had been made), albeit with the side effect that you would need to avoid substitution of "$relative" or "$absolute". If only (2) and (3) are required then you could capture what you need with:
([^$]|^)(\$(absolute|relative)\((\$\([^\(\)]+\)|[^\(\)]+)\))
(you may also want to allow for whitespace in the brackets with a few " *" scattered about)
PS: I'm not sure how non-greedy search would help in this case. Isn't the problem that the regex isn't greedy enough when there are sub-braces?