I'm sorry for you to be going through this. For any mother to have her children being taken away by a control freak ex is an extremely daunting thing to endure. I should know as I've been through it too.
Back in november 2008 I went through the same thing. I was in financial hardship, though I was working and my ex partner (whom I will call "J") had given up the gardening business we started up the previous year, I was paying too much tax - £1,300 more per annum than I was supposed to - I didn't know at the time but that was the reason we were struggling to make ends meet. He was disillusioned by Scotland not being able to provide the type of job he was looking for (other gardening jobs or garden centre jobs) and he wanted to move back to where he was born, about 50 miles west of London. I didn't want to because my youngest daughter ("K") who was three at the time wouldn't see her dad (whom I will call "S") and unlike my eldest's dad (who to this day only came up once to see my eldest daughter in 2003) "S" lavishes affection on "K". I'd already moved a few hundred miles away from "S" to be with "J" and I didn't think it was fair to keep moving away and expecting him to follow. At that time he lived a 45 minute drive away.
"J" kept insisting for about a year that I tell "S" we were thinking about moving back south. Something in my gut would make up an excuse not to do so and I'd normally tell "J" I forgot to tell "S". Anyway, In Nov 2008 J pressurised me so much I gave in and mentioned it to "S" via text message (our normal way of communicating) and I got asked "You're leaving "K" here?" where I replied "No she's coming with us. maybe in 5 months, maybe in five years, maybe in ten. But eventually “we” will move back south nearer family". Of course, I said this during his time with her on a weekend, and he never brought her back to me. He told me I wasn't going to get her back until I sign a legally binding document stating that she would not be removed from Scotland without his knowledge/consent. I should have called the police as I was her primary care giver. But I didn't want "K" to have to go through being removed from one parent and given to another. So I went through legal channels and every week that passed was a week that she'd settle into her Dad's home more. I was so scared she'd forget me.
After the new year, trying to sort out the custody, I was told she was "settled" into her new playgroup now and had lots of friends. He'd only gone and signed her up to a school, after knowing she was already registered to start in the same one that her sister goes to - and that was how it was decided that he got primary care of her and I only see her at weekends, plus the fact that things at my place were extremely unsettled and constant arguments were occuring over finances, not wanting to be here etc so I decided that for the time being that I would only see her at weekends, as I could work a longer shift and increase my hours without having "K" pine for me at home, as it was better than not seeing her at all.
So I signed the damned document. I did not see her between November 2008 and February 2009. I missed her 3rd Yule and it broke my heart. But when she arrived back, her gifts still in the same place but the tree removed and she had a belated Crimbo.
Three years on, I do admit I blame "J" for the whole thing. If he hadn't insisted I do such a thing, "S" wouldn't have reacted in such a panic and gone to such measures. "J" is now where he wants to be with his parents and I, though I too am missing family in Wales and have given up my job for health reasons, couldn't be happier. I'm with a new man, pregnant and I see "K" who is now 6, at the weekends. She's happy and healthy and I couldn't ask for anything more than that really.
I really understand what you're going through, truly. I know you want to stay where you are but ask yourself this - what is more important to you, seeing your children again regularly and coming to a compromise where you live, or seeing them very rarely as you are living somewhere far away because you love to be there? For you to be truly happy you must find a compromise somewhere. Don't think of it as an "I must win this" scenario, show that you are the more mature one of the parents, as if you do decide to live in the same state, I'm guessing there may be an even bigger chance of you getting them back. Judges like to see people coming to an agreement like this and sometimes will rule in your favour.
However, I'm not telling you what to do, just giving advice to you based on how I handled my situation. Things between me and K's dad have never been better - we even have even gone beyond mere pleasantries and become more friendly. Much more civil and no hostilities.
Do what ever you feel is right and fight for your children, just don't let selfishness get in the way like your ex has done!
Much Love and Prayers