Doesn't sound like it's worth the hassle to me. By the time that you (or whomever you are paying to do this work) are finished fabricating brackets for the E60 M5 calipers to fit the factory mounting points, you would have spent some pretty serious bucks on it. Then you have to buy wheels (19") to accommodate the new rotors that you have installed. And for what reason? When it comes to stopping the car, "cheap" generally doesn't exactly enter my mind in the list of adjectives for what I would want. But then again, expensive isn't at the top of my list either.
But what I can say is that UUC has developed a cost-friendly Big Brake Kit for the E39 M5 and 540i cars. Having spoken with him at length last week, I can attest to this statement. Even though I am not sure if it has been released yet. No, don't take the website's word for it either, because that hasn't been update for a while.
Aren't the E39 wheels hub centric anyway? So would it even be possible to use E60 M5 rotors on an E39 M5?