Preheat the oven to 350 degrees F and grease a 6-inch round cake pan.
In a mixing bowl, cream the softened butter with the granulated sugar until light. Add the eggs one at a time and mix until combined.
Stir in the chopped rosemary and the chopped figs. In a separate bowl, whisk together the flour, baking powder, and salt. Alternate adding the dry mixture and the milk to the wet ingredients, mixing just until smooth.
Pour the batter into the prepared pan and bake for 30 to 35 minutes, or until a toothpick inserted in the center comes out clean. Let the cake cool completely in the pan before turning it out.
While the cake cools, melt the remaining butter in a saucepan over medium heat. Cook it until it turns golden brown and smells nutty, then remove it from the heat and let it cool to room temperature.
In a clean mixing bowl, beat the browned butter with the powdered sugar, heavy cream, and vanilla extract until smooth and spreadable. Frost the cooled cake and top with the sliced figs and rosemary sprigs.