Parallel.ForEach (is there any thing more faster) ?!

Abdalla Omran

In this method reading a big data from a text file it's lines about milionen and it takes between 6 until 7 min to read and filter that . so is there anyway to make it more faster ?

private static List GetWebShopDataAccesses(string path)
{
List elements = new List();
List<string> lines = File.ReadAllLines(path).ToList();
var filterIPs = lines.Where(x => x.Contains("google")).ToList();
Parallel.ForEach(filterIPs, line =>
{
// do some staff
});
return elements;
}

3 Replies

Know the answer? Post it — somebody with the same question will find it here.

Abdalla Omran6y ago
so i have already until now 400 Text file in each file there is miliones of lines like :

109.88.147.202 - - [31/Dec/2018:16:23:31 +0100] "GET /json.php?service=setSessionParameter&key=newsletter_popup&value=false HTTP/1.1" 200 4 "https://www.fahrrad-xxl.de/angebote/angebote-fahrraeder/faltraeder-klappraeder/" "Mozilla/5.0 (Linux; Android 6.0; XT1068) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.99 Mobile Safari/537.36"
after some filtring i have around 350000 until 800000 line and to read all that its takes really very long time
0
Amit Gupta6y ago
In addition to @Ashutosh's answer, it also depends upon the content of the text file. Debug your code and see which line is taking much time and your solution is right there.
+1
Ashutosh Gupta6y ago
abdalla,
first thing, normal Foreach loop is faster than Parallel.Forach and that is proven, normal foreach runs upon a single thread and processing takes place sequentially one by one .
Parallel.ForEach loop in C# runs upon multiple threads and processing takes place in a parallel way.

so you have answer, use normal foreach for faster processing.

but you are not going to use this List of element immidiatly and going to add new loop later on this. i would suggest to use c# yield which will be faster and and get to each element when caller ask for.
+2