Preheat oven to 375 degrees. Line baking sheets with silicone baking mats or parchment paper.
Whisk together flour, baking powder, and sugar until combined. Make a well in the center and add oil, milk 1 tablespoon almond extract, and eggs. Using a sturdy wooden spoon, mix together until all ingredients are incorporated and smooth.
Using a small cookie scoop, scoop up a one-inch ball of dough, and place on the prepared cookie sheet, with 1 inch between cookies. Slightly flatten tops. Bake for 8-10 minutes in the preheated oven until cookies are cooked. They will not brown.
Meanwhile, make, glaze. In a small mixing bowl, whisk together powdered sugar, almond extract, and hot water until smooth. The glaze should remain thick like the consistency of molasses.
Place a wire baking rack over a rimmed cookie sheet. Dip warm cookies in glaze, then set on a wire rack and allow to cool. Dip cookies in glaze a second time then sprinkle with rainbow nonpareil sprinkles immediately, while glaze is still wet. Allow to set completely on the rack.