Place cubed potatoes in a large pot and cover with cold salted water. Bring to a boil and cook for 12-15 minutes until fork-tender but not mushy.
Drain potatoes and spread them on a baking sheet to cool slightly for about 10 minutes while you prepare the dressing.
In a large bowl, whisk together mayonnaise, mustard, pickle relish, vinegar, sugar, salt, pepper, and celery seed until smooth and well combined.
Add the warm potatoes to the dressing and gently stir to coat. The warm potatoes will absorb the dressing beautifully.
Fold in the diced celery, red onion, and chopped hard-boiled eggs until everything is evenly distributed.
Cover and refrigerate for at least 4 hours or overnight to allow the flavors to meld together.
Give it a gentle stir before serving and adjust seasoning with additional salt and pepper if needed.