I have always recommended to do the kind of physical activities that are fun. During the spring and summer months at least here in the Northeast it is easy to get involved in enjoyable outdoor activities that use your major muscle groups and raise your heart rate. So the short answer to the above question is no, you do not have to go to the gym.
For me I love gardening. And do not kid yourself it is exercise especially when you protect your back and squat when picking up garden waste to place in your leaf bag. All that up and down off the ground uses the large muscles of your legs. Or try hauling a wheel barrow load of mulch across your garden, then raking it out. That will get your heart rate elevated. No doubt about it this is exercise. In fact a 170 pound women will burn ~400 calories for 1 hour of gardening. Not bad. And I get to be outside and feel very satisfied when I am finished with my garden project.
Here is a good website that will calculate the calories burned for tons of different activities. All you have to do is enter your body weight. Check it out and have fun this summer.