If nothing else, take this away from your experience.......
If you ever find yourself judge and jury on some guy who works for you who "dipped his pen in the company ink" have some sympathy for him and let it go. Nothing worse in the world than a hypocrite.
You tap this honey on the side, after you retire. Keep it on the sly from your SO, just like the hobby. See how you feel about her after a few years, it may fizzle before then and you can exit without loss.