Place the potatoes in a large pot and cover with cold water. Add a pinch of salt and bring to a boil. Cook until the potatoes are tender when pierced with a fork, about 10 to 12 minutes. Drain and set aside to cool slightly.
Bring a separate pot of water to a boil. Add the green beans and cook for 3 minutes until bright green and still crisp. Drain and transfer immediately to a bowl of ice water to stop the cooking. Drain again once cooled.
In a large mixing bowl, whisk together the lemon zest, lemon juice, olive oil, Dijon mustard, minced garlic, salt, and black pepper until smooth.
Add the slightly cooled potatoes and green beans to the bowl. Toss gently to coat everything in the dressing.
Stir in the capers, parsley, and dill. Taste and adjust salt or lemon if needed. Serve at room temperature.