I just fiid that too much caster also can make it a bit darty, if your steering abilty is not on point.
Never experienced that but I find it best for straight line runs and it helps return to center for the small bumps without my input. I definitely wouldn't recommend it for anything that need to turn. When speed running I also turn down my steering input way down either through D/R and/or Steering speed. I believe my Gryo is off as well, not exactly sure of the state, it's been on the shelf for a bit.
One time I had a run where I twitched and the car got pitched a bit at about 100mph. Between the aero and the caster, it straighten out and I was fine. My underwear was a code brown though.