Creating actionable plans has been shown to be helpful in promoting physical activity. However, little rèsearch has been done on how best to support the creation and execution of plans. In this paper, we interviewed 16 participants to study the role that context plays in the formulation and execution of plans for physical activity. Our findings highlight nuanced ways that contextual factors interact with each other and with individual differences to impact planning. We propose the notion of sweet spots to encapsulate how particular contextual factors converge to create optimal states for performing physical activities. The concept of sweet spots helped us to better understand the creation and execution of plans made by our participants. We present design guidelines to show how sweet spots can help support physical activity planning and guide the design of context-based tools for planning support.