How can I convert a Control to a specific type like Button?

Hello. I am building controls dynamically on a form based on some config data I read. Originally I had code like:

public Button GetElementButton(string elementID, string text) {
Element elem = GetElement(elementID);
if (elem == null) return null;
Button ret = new Button();
ret.Text = text;
ret.Location = elem.location;
ret.Size = elem.size;
return ret;
}
public Label GetElementLabel(string elementID, string text) {
Element elem = GetElement(elementID);
if (elem == null) return null;
Label ret = new Label();
ret.Text = text;
ret.Location = elem.location;
ret.Size = elem.size;
return ret;
}
Button myButton = GetElementButton("btn.score.p1", "Score");

myButton.Click += (sender, e) => { Game.Score(1); };

Controls.Add(myButton);

I hate repeating myself though. It occurred to me I could (I thought) do:

public Control GetElementControl(string elementID, string text) {
Element elem = GetElement(elementID);
if (elem == null) return null;
Control ret = new Control();
ret.Text = text;
ret.Location = elem.location;
ret.Size = elem.size;
return ret;
}
Button myButton = (Button)GetElementControl("btn.score.p1", "Score");
myButton.Click += (sender, e) => { Game.Score(1); };
Controls.Add(myButton);

But they all appear blank, no text. Then when I click the button, it crashes with:

System.InvalidCastException: 'Unable to cast object of type 'System.Windows.Forms.Control' to type 'System.Windows.Forms.Button'.'

So I guess one can go Button -> Control but not Control -> Button. Do I have to go back to a method for each type, or does someone know a way to achieve the dynamic creation so I don't have to repeat?

Thanks!