How to customize .net getfiles() method

amShekar

MyDear Techies,

How to write getfiles() method to fetch only few (100 or 1000... ) files from a directory .
Please provide the solution ........:)

11 Replies

Know the answer? Post it — somebody with the same question will find it here.

Vulpes13y ago
I can't think of any obvious reason why the Take extension method might be ignored in a Windows Service.

I assume you're testing it on the same directory as you used in the Console application?

You could try using TakeWhile or Skip to see if they behave peculiarly as well.
amShekar13y ago
Am facing some issue with .take() in enumeration
I have developed one console application there the EnumerateFiles().Take(intval) working as expected i.e only intVal no of file its extracting from the folder but when i merged this changes in existed windows service i.e replace getfiles with Enumeratefiles throught the Windows services application the seems .Take(intval) not working i.e instead of intVal no of file the service (my enumeratefiles code with take() ) fetching the all file that contains in folder what could be the reason.

NOTE: This is my actual requirement where iam planning to customise getfiles. Actually my application (window services) is multithreaded application having several getfiles() methods
Vulpes13y ago
It depends how many cores (or processors) you have.

If you only have a single processor, then splitting the work up amongst different threads will probably increase the overall processing time because of the cost of switching between threads.

However, if you have multiple cores (preferably 4 or more) and are using .NET 4.0 or later, then parallelizing the work may well speed things up.

You don't need to do this manually. If you use PLINQ's AsParallel extension method, then it will split the work up for you into chunks using the available cores and then recombine the results after completion.

Here's a quick example of using AsParallel which is applied to the collection to be parallelized:

Imports System.IO
Imports System.Linq
Imports System.Linq.ParallelEnumerable

Module Test

Sub Main

Dim dirPath As String = "c:\windows\system32"
Dim fileNames As String() = (From filePath In Directory.EnumerateFiles(dirPath).AsParallel
Select Path.GetFileName(filePath)).ToArray
For Each fileName As String in fileNames
Console.WriteLine(fileName)
Next
Console.ReadKey

End Sub

End Module

Notice that there are issues in using Take, Skip etc. in a parallel query. Doing so, may result in the work being done sequentially rather than in parallel. However, it should be OK to use them on the collection before you apply the AsParallel extension method.
amShekar13y ago
Is this works correctly in multithreaded application
i.e i have one folder containing 50000 files every thread calling 1500 files
will it achieve my requirement
Vulpes13y ago
Unfortunately, the My.Computer.FileSystem object doesn't support the EnumerateFiles method, or anything similar, even in versions 4.0/4.5.

It only has GetFiles which returns a ReadOnlyCollection rather than an array of the file names. The former is effectively a read-only wrapper for the latter.

So, if you want to use this functionality, you'll have to use the System.IO.Directory.EnumerateFiles method instead.
amShekar13y ago
If i use Directory then i can use Directory.EnumerateFiles(dirPath).Take(max)
But in My.computer.Filesystem the enumeratefiles didnt find there i can find getfiles only
how can we tackle
thanks in advance vuples
Vulpes13y ago
The Directory.EnumerateFiles method was introduced in .NET 4.0 to provide better performance than Directory.GetFiles where you're dealing with directories containing a large number of files.

However, it depends on the LINQ infrastructure (introduced in .NET 3.0) which enables you to work with enumerable collections rather than waiting for arrays to be filled.

.NET 2.0 has no equivalent to this and so you're stuck with using Directory.GetFiles. If you're not already doing so then you could try using filters (get, say, all the files beginning with 'A' then 'B' and so) as the internal code for these may be more efficient than getting all the files first and then filtering them.
amShekar13y ago
Hi Vulpes,
Thanks for your reply it is working in 4.0 frame work.
My existed application in 2.0. How can we handle plz reply me :)
Vulpes13y ago
I just tested it using a simple console app (.NET 4.0). It's working fine:

Imports System.IO
Imports System.Linq

Module Test

Sub Main

Dim dirPath As String = "c:\windows"
Dim max = 10 ' or whatever
Dim files As String() = Directory.EnumerateFiles(dirPath).Take(max).ToArray
For Each file As String in files
Console.WriteLine(file)
Next
Console.ReadKey

End Sub

End Module
amShekar13y ago
Please can you send the whole method(sample i.e moving files from one folder to another folder) am getting some compilation errors

For older frameworks ?
Vulpes13y ago
If you're using .NET 4.0 or later, you can use LINQ to get a limited number of files from a directory:

Imports System.IO
Imports System.Linq

' ....

Dim max = 10 ' or whatever
Dim maxFiles As String() = Directory.EnumerateFiles(dirPath).Take(max).ToArray