-
Nov 22nd, 2020, 08:28 AM
#1
Thread Starter
Lively Member
-
Nov 22nd, 2020, 10:27 AM
#2
Re: Extracting specific information from an IEnumerable using LINQ?
What was wrong with the code you were given in SO? You said it didn't work, but you didn't say how it failed.
-
Nov 22nd, 2020, 10:34 AM
#3
Thread Starter
Lively Member
Re: Extracting specific information from an IEnumerable using LINQ?
I think I did put why it failed - "it is missing some directories where Attributes contains more than just Directory e.g. C:\ProgramData has Attributes shown as "Hidden Or Directory Or NotContentIndexed" so think the bit where it says "n.Attributes = Attributes.Directory" needs to change somehow to cover those that include .Directory rather than just ones that ONLY have .Directory"
-
Nov 22nd, 2020, 10:40 AM
#4
Re: Extracting specific information from an IEnumerable using LINQ?
OK. Well the normal pattern is to use the Enum as a mask, and compare the result to the mask itself. So instead of
Code:
Return n.FullName = rootDir & n.Name And n.Attributes = Attributes.Directory
you'd use something more like:
Code:
AndAlso ((n.Attributes And Attributes.Directory) = Attributes.Directory)
-
Nov 22nd, 2020, 10:41 AM
#5
Thread Starter
Lively Member
Re: Extracting specific information from an IEnumerable using LINQ?
So to explain further
It correctly gets directories that just have Attributes of Directory e.g.
But it doesn't get those which has multiple Attributes that include Directory e.g.
-
Nov 22nd, 2020, 10:44 AM
#6
Thread Starter
Lively Member
Re: Extracting specific information from an IEnumerable using LINQ?
@Inferrd - sorry posted my explanation at same time as you replied - thanks I will give that a try
-
Nov 22nd, 2020, 11:31 AM
#7
Thread Starter
Lively Member
Re: Extracting specific information from an IEnumerable using LINQ?
Okay from initial testing that may well have solved my problem. How would I amend below so the resulting list is in alphabetical order please?
Code:
Dim dirs As List(Of String) = nodes.Where(Function(n)
Return n.FullName = path & n.Name AndAlso ((n.Attributes And Attributes.Directory) = Attributes.Directory)
End Function).Select(Function(n) n.FullName).ToList
Dim files As List(Of String) = nodes.Where(Function(n)
Return n.FullName = path & n.Name AndAlso ((n.Attributes And Attributes.Directory) <> Attributes.Directory)
End Function).Select(Function(n) n.FullName).ToList
-
Nov 22nd, 2020, 11:41 AM
#8
Re: Extracting specific information from an IEnumerable using LINQ?
dirs (and files) is a List(Of String). The List(Of T) Class has a Sort Method you can use (or you can use OrderBy in your LINQ before calling ToList, as you were doing in your earlier code).
On a different note, do you find NTFSReader to be reliable? It seems to miss a lot of files/folders, and also mangle some of the path fragment names to a DOS like 8.3 scheme. First time I've played with it and I've not found any documentation, so maybe I'm doing it wrong?
-
Nov 22nd, 2020, 11:54 AM
#9
Thread Starter
Lively Member
Re: Extracting specific information from an IEnumerable using LINQ?
Yes so far I have found it to be quite reliable, certainly so much quicker than using any of the System.IO methods. I spent ages testing all of the different ways of enumerating my C: drive, some methods tooks minutes to complete whereas this is a matter of seconds. From testing so far I haven't noticed any of the issues you mention. Below are some links to two great projects that use it and articles the chap wrote explaining it all - worth a look
https://github.com/bsonnino/NtfsDirectories
https://github.com/bsonnino/NtfsFileEnum
https://blogs.msmvps.com/bsonnino/20...fs-structures/
https://blogs.msmvps.com/bsonnino/20...fs-structures/
Hope they help
I tried using OrderBy but couldn't get the code correct, and can't find the earlier code you mentioned? I would rather do it with OrderBy if possible
-
Nov 22nd, 2020, 12:12 PM
#10
Re: Extracting specific information from an IEnumerable using LINQ?
OK. Thanks for the links. Will check them out later.
You're actually using the OrderBy in your GetFileNodes method to order the IEnumerable(Of INode) returned by the NtfsReader:
Code:
.OrderBy(Function(n) n.FullName).ToList()
So the list of INode is already ordered alphabetically by the full path name, and so the files and dirs Lists should also be already ordered alphabetically.
If you want to further use OrderBy on a List(Of String) then you'd just use:
Code:
.OrderBy(Function(n) n)
although you might want to add some logic to deal with ignoring character casing order.
-
Nov 22nd, 2020, 12:30 PM
#11
Thread Starter
Lively Member
Re: Extracting specific information from an IEnumerable using LINQ?
No problem at all, have spent weeks working with it now and those project and the author were very helpful
Ah yes, sorry forgot I had used it in example project
Oddly enough even though the original list of INode was sorted alphabetically the files and dirs Lists were NOT coming out in alphabetical order
Thanks for that
As it happened it wasn't sort order which was causing the problem I had, it was actually because I was replacing a method that had used Directory.GetDirectories and Directory.GetFiles so this automatically knew to assume the "path" variable need to end with "" whereas the LINQ method didn't so I had to check for and add a "" to the end of the "path" variable and it then worked fine
I just did below before the Dim dirs As List(Of String) / Dim files As List(Of String) lines
Code:
If path.EndsWith("\") = False Then path = path & "\"
But if there is a way of doing this automatically in the LINQ statement then even better - but it is at least now working as I needed
Thank you for your help, was not getting very far on SO and I prefer being able to reply fully in posts rather than the SO method of short comments
-
Nov 22nd, 2020, 12:34 PM
#12
Re: Extracting specific information from an IEnumerable using LINQ?
Aye, you can't beat a good forum
-
Nov 22nd, 2020, 12:38 PM
#13
Thread Starter
Lively Member
Re: Extracting specific information from an IEnumerable using LINQ?
To explain fully (for future reading) my last comment
The below worked fine when "path" was "C:\" or "C:\Windows\" but doesn't work if it is "C:\Windows" e.g. if no \ on end
Code:
Dim dirs As List(Of String) = nodes.Where(Function(n)
Return n.FullName = path & n.Name AndAlso ((n.Attributes And Attributes.Directory) = Attributes.Directory)
End Function).Select(Function(n) n.FullName).OrderBy(Function(n) n).ToList
Dim files As List(Of String) = nodes.Where(Function(n)
Return n.FullName = path & n.Name AndAlso ((n.Attributes And Attributes.Directory) <> Attributes.Directory)
End Function).Select(Function(n) n.FullName).OrderBy(Function(n) n).ToList
-
Nov 22nd, 2020, 01:42 PM
#14
Re: Extracting specific information from an IEnumerable using LINQ?
Originally Posted by wingers
To explain fully (for future reading) my last comment
The below worked fine when "path" was "C:\" or "C:\Windows\" but doesn't work if it is "C:\Windows" e.g. if no \ on end
Yeah, I was a bit puzzled there.
With a little refactoring, you can use the Path.Combine Method from the Path Class to build your paths and sort that out for you automatically inside the LINQ:
Code:
Dim yourSearchPath = "G:\Windows"
Dim dirs As List(Of String) = nodes.Where(Function(n) n.FullName = Path.Combine(yourSearchPath, n.Name) AndAlso
(n.Attributes And Attributes.Directory) = Attributes.Directory).
Select(Function(n) n.FullName).
ToList
Dim files As List(Of String) = nodes.Where(Function(n) n.FullName = Path.Combine(yourSearchPath, n.Name) AndAlso
(n.Attributes And Attributes.Directory) <> Attributes.Directory).
Select(Function(n) n.FullName).
ToList
Note that because Path is the name of a Class, I'd normally avoid using path as a variable name, so I've renamed your path variable to yourSearchPath, but you should use whatever name makes sense to you (and those reading your code ;-).
Last edited by Inferrd; Nov 22nd, 2020 at 01:47 PM.
-
Nov 22nd, 2020, 01:48 PM
#15
Thread Starter
Lively Member
Re: Extracting specific information from an IEnumerable using LINQ?
Thanks
Doing it that way seems to be much slower than just adding \ to end of path if not already got one
Running it with your new LINQ code took 21 seconds to run and populate my TreeList, running it the other way took 11 seconds
I can only assume something to do with the Path.Combine - but thank you as is handy to know both methods
-
Nov 22nd, 2020, 01:55 PM
#16
Re: Extracting specific information from an IEnumerable using LINQ?
Yeah, it will be slower as that method is called for every node in the list, whereas adding the \ to the end of the search path is only done the one time. That said, I wouldn't have expected an extra 11 seconds! Using the Path.Combine, it takes me about 250 milliseconds to search through about 200,000 nodes, twice.
-
Nov 22nd, 2020, 01:58 PM
#17
Thread Starter
Lively Member
Re: Extracting specific information from an IEnumerable using LINQ?
I think it is because it is looping through that sub many times in my code to then add folder structure to a tree
-
Nov 22nd, 2020, 03:26 PM
#18
Re: Extracting specific information from an IEnumerable using LINQ?
Originally Posted by wingers
I think it is because it is looping through that sub many times in my code to then add folder structure to a tree
Do you mean that you are fetching the dirs and files lists for many search paths? If so how many?
I notice that the Path.Combine approach does not scale well. For 100 search paths, I'm seeing times of about 10 seconds for your append the \ method, and about 26 seconds for the Path.Combine method. Kinda worse than I was expecting!
However, you might be able to speed things up, depending on what you are doing.
I created a LookUp from the nodes, and that took about 9 milliseconds for the same 100 search paths.
(Populating TreeViews might slow things down, though. Not tried.)
-
Nov 22nd, 2020, 03:33 PM
#19
Thread Starter
Lively Member
Re: Extracting specific information from an IEnumerable using LINQ?
I am using a third party control - DevExpress TreeList as more customizable for my needs.
It is loading the levels of the filesystem "on demand" so to speak using what DevExpress called Virtual Mode - so yes I get all the node information using NTFSreader, and then get first levels of folders/files so it can create tree, it then loops though same sub x number of times depending on how many folders it found in root (or chosen start location) to retrieve next level of folder/file information ready to show when you expand the tree.
Works quite well and using the Dev control I can customize the look to suit my requirements - which in this case is an equivalent of TreeSize so sorted by total size of directories (including sub folders / files) and then a bar graph to illustrate sizes
Last edited by wingers; Nov 22nd, 2020 at 03:48 PM.
-
Nov 22nd, 2020, 03:48 PM
#20
Re: Extracting specific information from an IEnumerable using LINQ?
Looks good, even with the evil this forum does to images (resizes them to max 600px wide).
You must be recursing through all the folders on the drive to get the sizes? Is that what takes the time?
-
Nov 22nd, 2020, 03:52 PM
#21
Thread Starter
Lively Member
Re: Extracting specific information from an IEnumerable using LINQ?
Yes it displays the tree and then in the background it calculates the sizes of each folder (and its contents) from the information held in the nodes IEnumerable
Code:
totalsum = nodes.Where(Function(n) n.FullName.StartsWith(d.FullName & "\")).Sum(Function(n) n.Size)
It then gets size of parent folder and then the percentage for each folder is shown relevant to size of parent
Other information such as last access, last write time, file count, dir count is also gathered in background and added to grid as and when it gets results. That way it doesn't affect the loading of the tree nodes etc
-
Nov 22nd, 2020, 04:05 PM
#22
Re: Extracting specific information from an IEnumerable using LINQ?
Sounds like you've got it sussed
-
Nov 22nd, 2020, 04:09 PM
#23
Thread Starter
Lively Member
Re: Extracting specific information from an IEnumerable using LINQ?
I hope so, has taken some time, and getting this bit working was hopefully the last major hurdle
-
Nov 23rd, 2020, 10:04 AM
#24
Thread Starter
Lively Member
Re: Extracting specific information from an IEnumerable using LINQ?
Quick related question
I am struggling (for some reason) to be able to retrieve a specifc value from the IEnumerable when matching another value
In english based on image in first post - I want LastAccessTime when FullName equals the value I provide
Basically want to perform a lookup in the IEnumerable to get values for specific items
Thought it would be something like this but not working
Code:
Dim lastwritetime = nodes_test.Where(Function(n) n.FullName.Equals(dr.FullName)).Select(Function(a) a.LastChangeTime)
-
Nov 23rd, 2020, 10:30 AM
#25
Re: Extracting specific information from an IEnumerable using LINQ?
Originally Posted by wingers
In english based on image in first post - I want LastAccessTime when FullName equals the value I provide
Try something like:
Code:
Dim lastChangeTime As Date? = nodes.FirstOrDefault(Function(n) n.FullName.Equals(mypath))?.LastChangeTime
Note the 2 question marks. lastChangeTime is a Nullable Date, and will be set to Nothing if the path you provide isn't found.
-
Nov 23rd, 2020, 10:58 AM
#26
Thread Starter
Lively Member
Re: Extracting specific information from an IEnumerable using LINQ?
That doesn't seem to work as just returning Nothing
-
Nov 23rd, 2020, 11:07 AM
#27
Re: Extracting specific information from an IEnumerable using LINQ?
Originally Posted by wingers
That doesn't seem to work as just returning Nothing
I've tested it and it does work. If it's returning Nothing, then the path you are looking for doesn't exist in the node collection. You're using .Equals(filename) in your code. Make sure you are passing a full path, not just a file name.
-
Nov 23rd, 2020, 11:59 AM
#28
Thread Starter
Lively Member
Re: Extracting specific information from an IEnumerable using LINQ?
Ignore me - glitch my side, should learn to read properly!!
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|