Preheat the oven to 400°F. In a mixing bowl, combine the halved baby potatoes with half the olive oil, minced garlic, lemon zest, salt, and pepper. Toss until the potatoes are evenly coated.
Spread the potatoes in a single layer on a large baking sheet. Place the sheet in the oven and bake for 20 minutes to start softening the potatoes.
While the potatoes bake, return the cod fillets to the same mixing bowl. Drizzle them with the remaining olive oil and the lemon juice, then season with salt and pepper. Turn the fillets gently to coat.
After 20 minutes, remove the baking sheet from the oven. Push the potatoes to the sides and nestle the cod fillets in the center. Arrange the lemon slices over the fish and potatoes.
Return the sheet to the oven and bake for 12 to 15 minutes more, until the cod flakes easily with a fork and the potatoes are golden and tender.
Sprinkle the chopped parsley over the finished dish before serving.