In a large bowl fit with a mixer, beat butter on high speed until light and fluffy, about a minute or two. Add sugar, and vanilla/almond extract and beat on medium speed. Slowly, add flour (1/2 cup at a time) and continue to beat on low until Soft dough forms. Compact dough and cover it with plastic wrap for at least 2-4 hours; dough should be firm when working with. I refrigerated my dough over night.
Preheat oven to 350 degrees F. Line a cookie sheet with parchment paper. Scoop 1 inch balls of dough and roll them until they are smooth. Place on cookie sheet and press thumb into dough on each ball. Dough may crack, simply smooth over with finger. Fill each thumbprint center will jam and bake for 13-14 minutes. Do not over bake, cookies will harden but will be soft. Place dough refrigerator after each use. Allow cookies to cool before placing on a wire rack. Allow cookie sheet to completely cool before using again or cookie dough may spread.
To make the glaze, combine powdered sugar, water and flavored extract into a medium bowl and stir to combine. Using a spoon, drizzle glaze over cookies once cool.
Cookie dough can be stored in freezer for up to 2 months. Baked cookies can be stored for 2-3 months covered, in freezer.