You can't fully counteract not SRSing. This is true for every use case of the SRS, especially when learning things like math, programming, physics, anatomy, biochemistry, etc. In language learning it is less noticeable, which leads to the idea that one can stop SRSing and experience no negative effects. This idea is more common among language learners.
Now, if you learn a language to some level of proficiency, and then stop using it completely, including immersion and the SRS, one day you'll forget all of it. If you immerse a little, it will happen later. If you immerse a lot, you will be improving. This all is pretty obvious. Nonetheless, language consists of domains of various sizes. Much like if you go to a math class, learn something, and then stop contacting or using that information, you will forget it, without the SRS you have to regularly come back to every domain you've immersed with, or you'll forget vocabulary specific to that domain.
There are many corner-case words that could be easily lost without repetition. These are words that come up maybe just a few times a year or less. Things like names of different animals, plants, food, fish and birds, stars and planets; words related to various skills, fields science and art; and many more. It's true that they are not very important, but they still come up occasionally. Every time they come up, you're going to feel like you're a child who's lost.
The SRS is the reason why AJATTers can have passive vocabulary larger than an average native their age. They're able to surpass natives in 5-7 years. A native speaker has spent their entire life using the language. When you take out the SRS, you start forgetting words you don't run into often enough, and as a result it takes you way longer to catch up to a native. Without enough immersion every day you may never be able to do it.
That's why it's not possible to counteract not SRSing — there are too many domains to worry about. Even if you read all day, there's always some vocabulary you're not contacting. When you stop using the SRS, the rules change, and you enter the game of fighting a leaking bucket. The SRS becomes especially important when there's not enough repetition of any kind of information.
Khatzumoto has an article on this topic, If You’re Fluent, Why Do You Still SRS. In the article he says that once he stopped SRSing, he started forgetting words that normally you don't see very often, words that you maybe don't see every day but need once in a while. Khatz was frustrated with his inability to improve without the SRS, so he started SRSing again.
Throughout my Japanese learning journey I always knew the consequences of dropping the SRS from reading about what people before me, including Khatz, experienced, so I never stopped SRSing. Once you've used SRS for a while, you start applying it to all aspects of your life, not just language learning. You start SRSing your exams, for example. When you have a variety of different topics in your SRS, it becomes harder to quit it.
If you have too many due cards each day, consider optimizing your settings and retiring mature cards. You could even delete some decks that you're sure you don't need anymore, though I'm not a fan of deleting entire decks.