Preheat oven to 350°F (175°C). Grease and flour two 9-inch round cake pans.
In a large mixing bowl, whisk together the sorghum flour, sweet rice flour, tapioca flour, millet flour, xanthan gum, baking powder, baking soda, and salt.
In a separate large bowl, using a handheld or stand mixer, cream together the unsalted butter and granulated sugar until light and fluffy, about 3-4 minutes.
Add the eggs one at a time, mixing well after each addition.
Stir in the vanilla and almond extracts (or extra vanilla for nut-free version).
Gradually add the dry ingredients to the wet ingredients, alternating with the whole milk and sour cream, in three additions, beginning and ending with the dry ingredients.
Divide the batter evenly between the prepared pans and smooth the tops.
Bake for 25-28 minutes, or until the cakes are golden and a toothpick inserted in the center comes out clean. Let the cakes cool completely in the pans on a wire rack.
To make the vanilla buttercream, in a large bowl, beat the butter until smooth and creamy.
Gradually add the sifted powdered sugar, about 1 cup at a time, mixing well after each addition.
Stir in the vanilla bean seeds or paste and vanilla extract.
Add salt and then add heavy cream one tablespoon at a time until the desired spreading consistency is achieved.
Once the cakes are cooled, use a serrated knife to level the tops gently if needed. Place one layer on a cake stand or serving plate.
Spread a thick layer of buttercream on top. Place the second cake layer on top and frost the top and sides of the cake
Serve at room temperature, ideally after refrigerating for at least 30 minutes.