If you go with someone else people usually don't have a issue you if you leave to use a restroom as long as your buddy is still there.
Same thing goes for when it is getting close to going onto store, lugging your gear back to your vehicle. But maybe thats just a northern thing southerners may not be so kind!