It is always a tough call on mods ... you will never recoup all the time and money spent on modifications (but you likely knew that before getting into the endeavor). They can hurt your resale value to some customers, and increase it to others. Depending on what you modified and whether you have the original parts, you can likely get full resale value (Edmunds / Kelly Blue book) + 30% of the modification costs ... though in modification costs, very few will consider the cost of installation.
You have a few options:
return the car to stock, sell it for whatever you can get & part-out the modifications
-- or --
set a reasonable price to entice the buyer to buy your modded M6 (I recommend the aforementioned value + 30% of the resale value of the mods ... as a starting point for negotiations).
Either way, good luck!
others on the boards may help with more information if you explain what your $5,000 in modifications bought you.