Loosly compare string

[Starf0x]

Hey,

Subject sound strange, but let me explain;

I have a list of filenames, 99% of them look like each other, and the 1% is the one I need to capture.
How can I compare the list and get the ugly duckling?

This is the list:

uxm_543_001.jpg
uxm_543_001a.jpg
uxm_543_001b.png
uxm_543_002.jpg
uxm_543_003.png
uxm_543_004.jpg
uxm_543_005.jpg
uxm_543_006.jpg
uxm_543_007.jpg
uxm_543_008.jpg
uxm_543_009.jpg
uxm_543_010.jpg
uxm_543_011.jpg
uxm_543_012.png
uxm_543_013.jpg
uxm_543_014.jpg
uxm_543_015.jpg
uxm_543_016.jpg
uxm_543_017.jpg
uxm_543_018.jpg
uxm_543_019.png
uxm_543_020.jpg
uxm_543_021.jpg
uxm_543_022.jpg
prettypretty.png

I really appreciate the help, thank you

Kind Regards, Starf0x

[ShadowTzu]

'Add every filename in a list of string
list_files = New List(Of String)
'for ...
'   list_files.add filename...
'next
 
'find Ugly_duckling
 list_files = list_files.FindAll(AddressOf isUgly_duckling)
 
Private Function isUgly_duckling(filename As String) As Boolean
    If Not Regex.IsMatch(filename, "^[uxm_]") Then
        Return True
    End If
    Return False
End Function

[Starf0x]

That would be nice ShadowTzu, if the files where alway te same.
But I did my own research and found this, I'm not sure how waterproof this is:

        For Each strFile In strFileList
            Dim strMatch As Match = Regex.Match(strFile.ToString, "([00-99])\d+")
            If strMatch.Success = False Then
                strWrongOnes.Add(strFile.ToString)
            End If
        Next

kind Regards, Starf0x

[dbasnett]

That would be nice ShadowTzu, if the files where alway te same.
But I did my own research and found this, I'm not sure how waterproof this is:

        For Each strFile In strFileList
            Dim strMatch As Match = Regex.Match(strFile.ToString, "([00-99])\d+")
            If strMatch.Success = False Then
                strWrongOnes.Add(strFile.ToString)
            End If
        Next

kind Regards, Starf0x

All of this depends on the specifics which you did not state. Specifics for the file name guess:

1. must start with lowercase uxm_
2. followed by three numbers
3. followed by _
4. followed by three numbers
5. followed by a lowercase letter - optional
6. followed by a period
7. followed by either jpg or png

If those are the specifics then here is a match string that meets those rules:

uxm_\d{3}_\d{3}[a-z]*\.(jpg|png)

This will match all of the names in the list you provided except for the last.
It will not match UXM_123_456.jpg or uxm_12_345.jpg.

[Starf0x]

Hi dbassnet,

As I said, the names are not always the same e.g.:

Legacy v2 015-000.jpg
Legacy v2 015-013.jpg
Legacy v2 015-014.png
Legacy v2 015-015.jpg
Legacy v2 015-016.jpg
Legacy v2 015-017.png
Legacy v2 015-018.jpg
Legacy v2 015-019.jpg
Legacy v2 015-020.jpg
Legacy v2 015-021.jpg
dropletter12.jpg

I could be anything, I need to find that one or 2 files that are not the same as the rest.

Kind Regards, Starf0x

[SJWhiteley]

Hi dbassnet,

As I said, the names are not always the same e.g.:

Legacy v2 015-000.jpg
Legacy v2 015-013.jpg
Legacy v2 015-014.png
Legacy v2 015-015.jpg
Legacy v2 015-016.jpg
Legacy v2 015-017.png
Legacy v2 015-018.jpg
Legacy v2 015-019.jpg
Legacy v2 015-020.jpg
Legacy v2 015-021.jpg
dropletter12.jpg

I could be anything, I need to find that one or 2 files that are not the same as the rest.

Kind Regards, Starf0x

Well, you have 10 files not the same as the rest. Whichever file you choose, there are 10 files not the same as the rest. As DB noted, you need some rules.

I suspect what you are encountering, here, is that the human mind can make assumptions - a program cannot: presumably, the last file is 'not the same as the rest', but how is a program to know? How do you know the last file is not the same as the rest?

[dday9]

Here is my try at it:

Imports System
Public Module Module1
	Public Sub Main()
		'Get the contents of the file
		Dim contents() As String = { _
			"Legacy v2 015-000.jpg", "Legacy v2 015-013.jpg", "Legacy v2 015-014.png", "Legacy v2 015-015.jpg", "Legacy v2 015-016.jpg", "Legacy v2 015-017.png", "Legacy v2 015-018.jpg", "Legacy v2 015-019.jpg", "Legacy v2 015-020.jpg", "Legacy v2 015-021.jpg", "dropletter12.jpg"}

		'Sort the file
		Array.Sort(contents)

		'The most uncommon string will be either at the beginning or end
		If contents(0)(0) = contents(1)(0) Then
			Console.WriteLine(contents(contents.Length - 1))
		Else
			Console.WriteLine(contents(0))
		End If

		Console.ReadLine()
	End Sub
End Module

Edit -
That example assumes that there are at least 2 items in the array and that the first letter of the ugly duckling will be different from the rest.

[Starf0x]

Hi dday9,

That's not a feasable answer is it?
As I said, I do not know how the files are named, I only now there are 1 or 2 that are not part of the collection, those are the ones I need.

Kind Regards, Starf0x

[dday9]

I made an edit to my previous post but it assumes that there are at least 2 items in the array and that the first letter of the ugly duckling will be different from the rest.

[dday9]

Here is a more concise version in an array:

	Private Function FindUglyDuckling(ByVal contents() As String) As Integer
		Array.Sort(contents)
		
		If contents(0)(0) = contents(1)(0) AndAlso contents(contents.Length - 1)(0) <> contents(1)(0) Then
			Return contents.Length - 1
		ElseIf contents(contents.Length - 1)(0) = contents(1)(0) AndAlso contents(0)(0) <> contents(1)(0)
			Return 0
		Else
			Return -1
		End If
	End Function

It will return a -1 when there are no more ugly ducklings.

[IanRyder]

Hi,

Here is my contribution for what its worth. You are going to have to do some sort of File Name “Pattern Recognition” and work out a way to discern what is an “Ugly Duckling” and what is a “Pretty Pattern”.

As a starting point, please see below something I came up with. In this example I have assumed that if a Percentage of File Patterns (Grouped Together) is greater than or equal to 90% of the Total Number of files in a collection of File Names then there has to be some “Ugly Ducklings” in the remaining 10% of File Patterns.

There is quite a lot going on here so please do take the time to understand it and potentially refine it into a possible workable solution for yourself.

Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
  'Lets Set a Threshold to work with. Lets say 90%
  Const PercentageThreshold As Decimal = 90D
  'Get ths file names which ever way you want
  Dim fileNames As String() = IO.File.ReadAllLines("c:\temp\filenames.txt")
 
  'Get the shortest File Name length
  Dim shortestFileNameLength As Integer = fileNames.Min(Function(x) IO.Path.GetFileNameWithoutExtension(x).Length)
  'A couple of variables to hold information
  Dim fileGroups As IEnumerable(Of IGrouping(Of String, String)) = Nothing
  Dim prettyFileGroupFound As Boolean
 
  'Decrease the file length, one character at a time, to work with using a For Loop looking for patterns in the file names
  For fileNameLengthToUse As Integer = shortestFileNameLength To 1 Step -1
    Dim currentfileNameLengthToUse As Integer = fileNameLengthToUse
    'Group the file name patterns and look for Percentage Thresholds
    fileGroups = fileNames.GroupBy(Function(x) New String(x.Take(currentfileNameLengthToUse).ToArray))
    Dim maxGroupPercentage As Decimal = (fileGroups.Max(Function(x) x.Count) / fileNames.Count) * 100
 
    If maxGroupPercentage >= PercentageThreshold Then
      'If we get here then there is a common file pattern that has been recognised
      prettyFileGroupFound = True
      Exit For
    End If
  Next
 
  'If a common file pattern HAS been recognised and therefore is ALSO nore than 1 File Pattern Group then we have some "Ugly Ducklings"
  If prettyFileGroupFound AndAlso fileGroups.Count > 1 Then
    'Lest loop through the "Ugly Ducklings" nd display them
    For Each currentFileGroup As IGrouping(Of String, String) In fileGroups.Where(Function(x) (x.Count / fileNames.Count) * 100 < PercentageThreshold)
      For Each specificFileName In currentFileGroup
        MsgBox(specificFileName)
      Next
    Next
  End If
End Sub

Hope that helps.

Cheers,

Ian

[Starf0x]

[QUOTE=IanRyder;4849437]Hi,

Here is my contribution for what its worth. You are going to have to do some sort of File Name “Pattern Recognition” and work out a way to discern what is an “Ugly Duckling” and what is a “Pretty Pattern”.

As a starting point, please see below something I came up with. In this example I have assumed that if a Percentage of File Patterns (Grouped Together) is greater than or equal to 90% of the Total Number of files in a collection of File Names then there has to be some “Ugly Ducklings” in the remaining 10% of File Patterns.

There is quite a lot going on here so please do take the time to understand it and potentially refine it into a possible workable solution for yourself.

Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
  'Lets Set a Threshold to work with. Lets say 90%
  Const PercentageThreshold As Decimal = 90D
  'Get ths file names which ever way you want
  Dim fileNames As String() = IO.File.ReadAllLines("c:\temp\filenames.txt")
 
  'Get the shortest File Name length
  Dim shortestFileNameLength As Integer = fileNames.Min(Function(x) IO.Path.GetFileNameWithoutExtension(x).Length)
  'A couple of variables to hold information
  Dim fileGroups As IEnumerable(Of IGrouping(Of String, String)) = Nothing
  Dim prettyFileGroupFound As Boolean
 
  'Decrease the file length, one character at a time, to work with using a For Loop looking for patterns in the file names
  For fileNameLengthToUse As Integer = shortestFileNameLength To 1 Step -1
    Dim currentfileNameLengthToUse As Integer = fileNameLengthToUse
    'Group the file name patterns and look for Percentage Thresholds
    fileGroups = fileNames.GroupBy(Function(x) New String(x.Take(currentfileNameLengthToUse).ToArray))
    Dim maxGroupPercentage As Decimal = (fileGroups.Max(Function(x) x.Count) / fileNames.Count) * 100
 
    If maxGroupPercentage >= PercentageThreshold Then
      'If we get here then there is a common file pattern that has been recognised
      prettyFileGroupFound = True
      Exit For
    End If
  Next
 
  'If a common file pattern HAS been recognised and therefore is ALSO nore than 1 File Pattern Group then we have some "Ugly Ducklings"
  If prettyFileGroupFound AndAlso fileGroups.Count > 1 Then
    'Lest loop through the "Ugly Ducklings" nd display them
    For Each currentFileGroup As IGrouping(Of String, String) In fileGroups.Where(Function(x) (x.Count / fileNames.Count) * 100 < PercentageThreshold)
      For Each specificFileName In currentFileGroup
        MsgBox(specificFileName)
      Next
    Next
  End If
End Sub

Thanks Ian,

This is a whole different approach, I didn't think to look at this in this way.

What Imports are you using? Am I correct to assume:

Imports System.Linq

Kind Regards, Starf0x

[Starf0x]

Hey IanRyder,

This ihard to comprehend for me you code (never studied for it).
I'm trying, believe me

But it doesn't always work, i have issues with this pattern:

001.jpg
002.jpg
003.jpg
005.jpg
006.jpg
007.jpg
008.jpg
Zone50ft.jpg

I just doesn't recognize the "Ugly Duckling", and won't go past:

If prettyFileGroupFound = True AndAlso fileGroups.Count > 1 Then

Kind Regards, Starf0x

[IanRyder]

Hi,

I assume you got the LINQ question sorted so I will ignore that for the moment. With regards to your last post, don’t forget that the example I provided worked on the principle of 90% of files being a prettyFileGroupFound but in the last case that you presented you can clearly see that there are only 8 files and therefore the “prettyFileGroup” only accounts for about 87% of files which means the routine does not work.

This is where you need to expand on my logic to come up with something that works for you.

One suggestion could be to have a Tiered PercentageThreshold whereby if you have a total number of files of less than 10 then the PercentageThreshold should be 80%? If the total number of files is between 11 and 50 then the PercentageThreshold should be 85%? etc, etc...

Over to you to come up with something that works.

Hope that helps.

Cheers,

Ian

[Starf0x]

Ian,

Thank you for all your help, I will do my best to do so

Cheers,

Starf0x