Rinse lamb and pat dry with paper towels. Trim away excessive fat. Place lamb in a grill pan. Sprinkle ½ tsp salt, ½ tsp ground pepper, and garlic powder all over the lamb.
Mash anchovies and fresh rosemary in a small bowl. Spread rosemary anchovy mixture all over the lamb. Make sure the fatty side is up.
Roast the lamb in the oven for 15 minutes. It will start to brown quickly. Open the oven and tent the top of the lamb with a piece of aluminum foil to prevent it from burning.
Lower oven temperature to 350°F (180°C) and continue to roast lamb for 30 minutes.
While lamb is roasting in the oven, peel the potatoes. Cut each potato into half lengthwise and then into quarters. Make several cuts on the rounded side of each piece of potato at 1/8 inch interval but do not cut all the way through.
Place potatoes in a large bowl. Sprinkle remaining ¼ tsp salt and ¼ tsp ground pepper over the potatoes. Add extra virgin olive oil and mix well with a spoon.
Open the oven and pull out the grill pan. Arrange potatoes around the lamb.
Push grill pan back into the oven and continue to roast for another 1 hour 15 minutes to 1 hour 130 minutes or until internal temperature of lamb is 135°F (57°C) to 140° (60°C).