Heat the oil in a large Dutch oven over medium heat. Add the lamb neck pieces and brown them on all sides, about 8 minutes total.
Add the onion and garlic to the pot and cook until the onion softens. Stir in the allspice, salt, and black pepper.
Pour in the water and drop in 4 sprigs of thyme plus the whole scotch bonnet pepper. Bring to a simmer, then cover and cook for 1 hour 15 minutes.
Add the yam chunks and continue simmering uncovered for 30 more minutes until the yams are tender and the broth has thickened slightly. Remove the thyme stems and scotch bonnet before serving.
While the stew simmers, rinse the rice until the water runs clear. In a medium saucepan combine the rice, coconut milk, kidney beans, remaining 2 sprigs of thyme, and 1/2 cup water. Bring to a boil, then reduce to low, cover, and cook for 18 minutes. Remove from heat and let sit covered for 5 minutes. Fluff with a fork and serve alongside the stew.