Peel and cut the potatoes in 2-inch chunks. Place in a large pot, cover with water by 1-inch and bring to a boil. Cook until fork tender, about 20 minutes.
Drain the potatoes and return to the pan. Add cream cheese, butter and milk. Mash until smooth and creamy. Season to taste with salt and pepper.
Preheat oven to 350°F. Lightly grease a 2-quart baking dish. Layer half of the potatoes in the bottom, then top with half the cheese. Layer the remaining potatoes and cheese. Bake for 20-30 minutes, until cheese is melted and bubbly.