Lemon Cake with Fruit

2 sticks unsalted butter, room temperature, plus more for pan
2 cups all-purpose flour, plus more for pan
2 tsp. baking powder
1 1/2 tsp. kosher salt
1 lemon
1 2/3 cups plus 3 Tbsp. granulated sugar
3 large eggs
1 large egg yolk
3 Tbsp. pear, orange, or apple brandy (optional)
2 lb. pears or apples, sliced, stone fruit, cut into thick wedges, blueberries, and/or blackberries
Powdered sugar (for serving)

Place a rack in middle of oven and preheat to 350°F. Butter and flour a 13x9" baking pan. Whisk baking powder, salt, and 2 cups flour in a medium bowl. Finely grate lemon zest into another medium bowl. Slice lemon in half; set aside. Add 1 2/3 cups granulated sugar to zest and work with your fingers until mixture is fragrant and looks like wet sand, about 1 minute.
Using an electric mixer on medium speed, beat 1 cup butter in a large bowl until smooth, about 20 seconds. Add sugar mixture, increase speed to medium-high, and beat until very light and fluffy, about 5 minutes. Reduce speed to medium and add eggs one at a time, beating well between additions; mix in egg yolk. Reduce speed to low and gradually add dry ingredients. Beat just until incorporated, then add brandy and beat just until smooth. Fold batter a few times with a rubber spatula; scrape into prepared pan and smooth top. Scatter fruit over (do not press down). Squeeze a reserved lemon half over fruit (save other half for another use) and sprinkle with remaining 3 Tbsp. granulated sugar.
Bake cake until sides are golden brown and top is golden, 55–65 minutes. Let cool; cut into squares. Serve dusted with powdered sugar.