There are no absolute rules about this. Typically I try to schedule Saturday with an adventure, then Sunday is a down day to catch up on things around the house.
If we have had a really successful day out (and by successful I mean tiring), this makes Sunday a great recovery day. Being tired with good reason after bonding on a successful day out can set you up for a pretty stellar day at home.
Today we got dressed late, watched a bit of TV in the morning, made some sandwiches for lunch, the kids created a bead project while I took care of some chores. Then we went to grab some takeout and play in the park by their moms house before I have to take them back.
It wasn't what you would call a day we accomplished a lot, but I would consider it highly successful.
Part of the point of adventuring isn't necessarily getting out, it's that getting out creates a stronger bond so that when we are home we can live a more peaceful life.