Well, the exception sounds as though you're trying to delete the row used for adding new data at the bottom of the grid. However, that makes no sense, because if you started with 10 rows (including the 'new' row) and then inserted another row at index 0 at runtime, you'll have 11 rows and the 10th (with an index of 9) will be the last one with any data in it. It should therefore be deletable.
Possibly, therefore, you didn't in fact have 10 rows to start with or the Rows collection is not getting updated for some reason.
You can always check whether a row is a 'new' row or not before attempting to delete it, by using the DataGridViewRow.IsNewRow property.
You could therefore try a different approach: