Memory management - limitation

I'm trying to test some heuristic algorithms. I can't limit memory inside algorithms but I don't know how to start them (as processes ot threads) from my application with resource limits

Why do you want to limit it? All controls are in the programmers hands :P