About the Factory Design Pattern

I have a question about the factory design pattern in C#.

I read this article
http://www.c-sharpcorner.com/Language/FactoryPatternsinCSRVS.asp 

It's not the best article ever which explains the Factory Design Pattern I've seen.
But that's another story. Anyway, it could be useful.

What I wonder is... How do we prevent client code from directly
creating instances of the concrete classes which impement the Base interface ?
We have to put some access modifier for the constructors of Derived1 and Derived2, I guess.
But none of the access modifiriers do the desired job (public, protected, internal,
protected internal). If we make the Derived1 and Derived2
constructors private than even the factory won't be able to instantiate them.

As I am coming from the Java world ... Here is how the Factory Design Pattern is
usually implemented in Java: In Java we have non-static inner classes and normally
you use them in such scenarios like the one with the factory pattern ...

I mean, you make your concrete classes (Derived1, Derived2) to be non-static inner classes
for the factory class. This means if one does not have a reference to an instance of the
factory that someone may not create instances of the non-static inner classes too.

Then the key step is to make the factory's constructor private.
Now noone except the factory can instantiate the factory class.
So noone except the factory can instantiate Derived1, Derived2 , etc as you need
an instance of the factory class in order to instantiate the Derived1 and Derived2 classes.

I know that in C# there're no non-static inner classes.
The inner classes in C# are analogue of static inner classes in Java.

So how do you prevent badly written client code from directly instantiating
Derived1 and Derived2 ( i.e. how to forbid client code from surrounding the
factory class and thus compromising the Factory Design Pattern ) ?  

Thanks in advance.