Drained the trans fluid by mistake, refilled it all, how worried should I really be?

I'd keep driving and stop worrying. I say that as someone who made a similar mess on my old Tacoma years ago. My nephew was helping me and we drained the front diff instead of the transfer case, drove it a couple days short before I caught it. Refilled, and that truck kept going another 80k before I sold it, diffs still quiet. A short run with low fluid isn't great, but a few minutes of funny shifting on a 2018 with low miles isn't going to kill a Toyota transmission. The overfilled crankcase part I'd worry about even less, honestly. Modern engines spit out the excess thru the PCV system or you'd see it at the dipstick level, and a week of normal driving with no smoke or leaks tells you it settled fine. If it helps you sleep, do one drain and fill of the transmission fluid in a month or two and call it done. On that RAV4 it's a cheap afternoon with a couple quarts of WS fluid and a transfer pump. I wouldn't pay a shop for a line check unless it starts slipping or flaring between gears, which at this point I seriously doubt it will. Watch it for another month, check the driveway again, and then just enjoy the car.

I'd do the drain and fill in a month, just because it's cheap peace of mind. When I confused the plugs on my wife's Camry a few years back, we refilled and it drove fine, but I still dropped the pan fluid again about six weeks later and what came out looked darker than it should have for fluid that fresh. Could have been leftover stirred-up stuff, could have been nothing, but seeing it drain clean the second time let me stop thinking about it. One thing to keep in mind on that RAV4: the amount you drain from the pan is only part of what the whole unit holds, so one drain and fill doesn't swap everything. That's fine for this situation since nothing actually got contaminated, you just want to dilute whatever got churned up while it ran low. The engine oil overfill sounds like the smaller issue to me. They caught it and drained it back to level, so the crankcase never ran unprotected. A transmission running short for even a few minutes is the part I'd want to rinse out, and doing it yourself with a cheap hand pump and a few quarts of WS fluid is an easy afternoon job. No lift needed, just a drain, refill, run it through the gears, done.

So I'm in the middle camp: not a dealer visit, not ignoring it either.

The thing that settled my nerves on a similar mixup was doing the drain and fill myself a couple weeks later instead of waiting a month. When I topped off a trans on my old Corolla after a botched change, the part I kept getting wrong was checking the level, since it has to be done warm. A cheap bluetooth OBD dongle that reads the fluid temp fixed that for me, and I set the level right at the temp the manual calls for. Fluid that came out looked a little cloudy the first time, normal the second. If it shifts fine cold, I would not lose sleep over it.

One trick from way back, take a paint stick or even a nail file and scratch a mark on the engine drain plug after this. Saved me more than once when I was under a car half asleep and couldn't remember which hole was which. And when you do that second drain, wipe a little of what comes out on a white paper towel. If it smells burnt or looks glittery, that's your sign to dig deeper. If it just looks like normal WS fluid, you're done thinking about it.