Preheat the oven to 400 degrees F. Boil the potatoes in a large pot of salted water until soft, about 15 minutes. Drain and mash with 2 tablespoons butter and the milk until smooth. Season with salt and pepper.
While the potatoes cook, heat the remaining 2 tablespoons butter in a large skillet over medium heat. Add the onion and carrots and cook until the onion softens.
Add the ground lamb and cook until browned, breaking it up with a spoon. Stir in the garlic, tomato paste, Worcestershire sauce, rosemary, and thyme.
Pour in the beef broth and add the peas. Simmer until the mixture thickens, about 5 minutes. Taste and adjust salt and pepper.
Spread the lamb filling evenly in a baking dish. Spoon the mashed potatoes over the top and smooth with a spatula. Bake for 20 minutes until the potato topping starts to brown.