OK, so I got a suggestion on reddit that worked smoothly, I have to limit the fee in my config but in a different way than someguy123 suggested
You should be able to start litecoind up with maxtxfee=0.01 in your litecoin.conf and it will not send transactions with more than 0.01 LTC fee. This will abort transactions that attempt to pay more than that. Additionally, you can set paytxfee=0.001 and that will pay by default 0.001 LTC / kB. You have some large transactions of several kB, if they are larger than 10kB these settings will abort those.
It's still interesting that your transactions were being set outside of parameters defined in litecoin.conf
It sure seems like something is happening software side to increase the tx fee.
Glad to hear you have a fix in place.