The last few girls I've dated all seemed to have some degree of low self-esteem(LSE), and a main sticking point is knowing how much to compliment.

My first main GF was seriously low SE, I complimented her and she definitely lost attraction, however I think this was because I also failed to demonstrate sufficient attractive qualities e.g. I stayed in a lot, I came across as being generally unattractive to girls.

This issue has come up time and time again in my game, and once again with one of the girls I'm gaming at the moment. She requires a lot of validation, and asks if I like her. I've recently turned to complimenting her and showing her why I chose her. She often asks things like “Why did you choose me over all the other girls?”. I like to make her feel sexy, but also try to draw attention to her personality traits, especially ones I want to reward and thus for her to continue doing e.g. She doesn't try to control me like my last GF.

This time around I've made an effort to maintain an attractive lifestyle e.g. I show myself to be having a large social circle, being active, doing interesting things. But I still worry that with these girls I will overshoot the balance when it comes to complimenting and fail to be a challenge.

Any tips of keeping this balance?

Shy girls are different. Don't over compliment her for two reasons.
1. Obviously you've seen evidence that this is not working.
2. You are putting her above you when you compliment her too often, and this comes off as AFC which is unattractive and will slowly push her away from you.

You are doing good by telling her why you chose her over other girls by talking about her personality over her physical features. Except don't use things she doesn't do,EXAMPLE: not controlling like my last girlfriend (bringing up your x's is usually just a bad idea anyways). Say nice things about her personality that she does have, make sure you're not just saying bullshit, she'll know if you are. If you like her for reasons other than sex you should be able to think of something.
The thing with shy girls is they have barriers that take longer to take down, so you need to make her feel comfortable. Sounds like she has self-esteem issues, so don't just say she's sexy either, show/make her feel sexy with Kino so she knows you mean it.

_________________
PUSSY, MONEY, WEED.

I get the feeling that PUA community shuns away from complements like they are poison.

Complements feed to one of the basic human needs: a need to be desired and appreciated. Take a moment to realize how much power that holds!

However, start handing them out like mints and they will quickly lose that impact you are hoping for. I'm assuming this is where you are. I hate myself for saying this, but use them as a reward system for when she does something you approve of. Let her know how disappointed you are when she misbehaves.

You can bee a sweet and caring guy, but too much sweetness will turn her off.

Hope this makes sense.
Al