In a medium mixing bowl, whisk together flour (2 1/2 cups), baking powder (1 teaspoon), and salt (1/2 teaspoon). Set aside.
In the bowl of a stand-up mixer, cream together butter (3/4 cup) and sugar (1 cup) until light and fluffy. Add egg (2), vanilla (1 teaspoon), and lemon zest (1 1/2 tablespoons) and mix until combined.
With the mixer on low, gradually add the flour, just until incorporated (do not overmix). Flatten dough into a disk and wrap in plastic. Refrigerate for at least 1 hour or overnight.
Preheat oven to 350ºF (177ºC). Line baking sheets with parchment paper or silicone baking mats.
Roll dough on a lightly floured surface to 1/2 inch thick and Use cookie cutters to cut into desired shapes. Place on baking sheet 1 inch apart.
Bake in preheated oven for 12-14 minutes. Cool on baking sheets for about 7 minutes. Remove and finish cooling on a baking rack.
When cookies are completely cooled, whisk together the confectioners' sugar (1 cup), lemon zest (1 tablespoon), and 1 tablespoon of milk, at a time, until the glaze reaches the drizzling consistency.
Pour the glaze into a small ziplock. Cut off a small piece of one corner.
Place cookies on a wire rack set over a sheet of foil and drizzle with glaze. Sprinkle with jimmies or colored sugar as desired. Allow cookies to sit on the rack until the glaze is fully set, about 3 hours.