-
Aug 2nd, 2018, 04:35 AM
#1
Thread Starter
Member
How can I remove text clones from the hosts file?
I have a program which writes websites to the hosts file from a .txt file which has a list of websites, so that next time a website is searched, the connection is closed.
It works great! However, because the program is told to keep writing websites to the hosts file, the file size is starting to grow. Since I have a large list of websites, if I keep writing those websites every time the application is ran, too much storage will be used up.
So, I thought that by removing the lines of text added until there a no more clones would be a good idea, so I tried using this code:
Code:
Dim text As String = File.ReadAllText("C:\Windows\System32\drivers\etc\hosts")
Dim index As Integer = text.IndexOf(line)
If index >= 2 Then
Dim delLine As Integer = 10
Dim lines As List(Of String) = System.IO.File.ReadAllLines("C:\Windows\System32\drivers\etc\hosts").ToList
lines.RemoveAt(delLine - 1) ' index starts at 0
MessageBox.Show("Clone Removed.")
End If
However, it doesn't work. The MessageBox shows up when the file has over 2 copies of the black-listed website, but it doesn't remove those new entries.
How can I remove text clones from the hosts file (or a .txt file) so that only one copy of the original text is stored, and not just text being written all the time?
-
Aug 2nd, 2018, 04:46 AM
#2
Re: How can I remove text clones from the hosts file?
I see that you deleted this question over at Stack Overflow. Is it still relevant?
-
Aug 2nd, 2018, 04:49 AM
#3
Thread Starter
Member
Re: How can I remove text clones from the hosts file?
Yes, it still is un-solved. I meant to post on vbforums but thought stackoverflow would be better.
-
Aug 2nd, 2018, 04:50 AM
#4
Re: How can I remove text clones from the hosts file?
vb.net Code:
Dim filePath = "file path here" File.WriteAllLines(filePath, File.ReadLines(filePath).Distinct())
Done!
Last edited by jmcilhinney; Aug 2nd, 2018 at 04:53 AM.
-
Aug 2nd, 2018, 04:54 AM
#5
Thread Starter
Member
Re: How can I remove text clones from the hosts file?
It's still not removing any cloned text. This is what my host file looks like:
Code:
127.0.0.1 www.badwebsite.com
127.0.0.1 www.badwebsite.com
127.0.0.1 www.badwebsite.com
-
Aug 2nd, 2018, 04:58 AM
#6
Re: How can I remove text clones from the hosts file?
If you need to only remove duplicates from part of the file, do something like this:
vb.net Code:
Dim allLines = File.ReadAllLines(filePath) Dim lineCount = allLines.Length 'The first 10 lines are not touched. Dim section1Lines = allLines.Take(10).ToArray() 'Remove duplicates from the middle of the file. Dim section2Lines = allLines.Skip(10).Take(lineCount - 20).Distinct().ToArray() 'The last 10 lines are not touched. Dim section3Lines = allLines.Skip(lineCount - 10).ToArray() File.WriteAllLines(filePath, section1Lines.Concat(section2Lines).Concat(section3Lines)
-
Aug 2nd, 2018, 04:58 AM
#7
Re: How can I remove text clones from the hosts file?
Rather than writing the duplicates to the file in the first place, check if they are in there before writing them.
To check you can use the first two lines you did above, followed by something like: If index = -1 Then writeToFile(line)
Alternatively you can use ReadAllLines (as you did above) and something like: If Not lines.Contains(line) Then writetoFile(line)
(this will take slightly longer to run)
-
Aug 2nd, 2018, 05:24 AM
#8
Thread Starter
Member
Re: How can I remove text clones from the hosts file?
I only want to remove , not just from random places of the file.
-
Aug 2nd, 2018, 05:28 AM
#9
Re: How can I remove text clones from the hosts file?
I had this first up:
vb.net Code:
File.WriteAllLines(filePath, File.ReadAllLines(filePath).Distinct())
and then changed it to this:
vb.net Code:
File.WriteAllLines(filePath, File.ReadLines(filePath).Distinct())
but that second one actually throws an exception because the file is still being read when it tries to write. The first is tried and tested and works as it should. If it doesn't work for you then you did it wrong.
-
Aug 2nd, 2018, 05:34 AM
#10
Re: How can I remove text clones from the hosts file?
Originally Posted by Modulus
I only want to remove , not just from random places of the file.
That doesn't make any sense.
-
Aug 2nd, 2018, 05:36 AM
#11
Thread Starter
Member
Re: How can I remove text clones from the hosts file?
Originally Posted by jmcilhinney
That doesn't make any sense.
is the piece of text that is added into the hosts file. So, for example, line could be: 127.0.0.1 test.com
So, in a simple way of explaining it, I want to remove 127.0.0.1 test.com
-
Aug 2nd, 2018, 05:48 AM
#12
Re: How can I remove text clones from the hosts file?
How about instead of removing the copy that is currently in the file (so you can add it again), you simply avoid adding the duplicate copy of it?
For suggestions on how to do that, see my previous post.
-
Aug 2nd, 2018, 05:50 AM
#13
Thread Starter
Member
Re: How can I remove text clones from the hosts file?
Could you leave me some code? I've read your post, and I understand what you're getting at. Some code would be really good thanks.
-
Aug 2nd, 2018, 05:55 AM
#14
Re: How can I remove text clones from the hosts file?
So you're saying, without actually saying, that you have some text in a variable and you want to remove lines matching that text from the file? In that case:
vb.net Code:
Dim filePath = "file path here" File.WriteAllLines(filePath, File.ReadAllLines(filePath).Where(Function(s) s <> line))
If you want to, say, remove all but the first instance of matching lines:
vb.net Code:
Dim filePath = "file path here" Dim allLines = File.ReadAllLines(filePath) Dim firstMatchIndex = Array.IndexOf(allLines, line) Dim section1Lines = allLines.Take(firstMatchIndex + 1) Dim section2Lines = allLines.Skip(firstMatchIndex + 1).Where(Function(s) s <> line) File.WriteAllLines(filePath, section1Lines.Concat(section2Lines))
-
Aug 2nd, 2018, 05:56 AM
#15
Re: How can I remove text clones from the hosts file?
Originally Posted by Modulus
Could you leave me some code? I've read your post, and I understand what you're getting at. Some code would be really good thanks.
Or, you make an attempt and then we can help you fix your code if it doesn't work? You're allowed to try and fail. It's part of learning.
-
Aug 2nd, 2018, 06:02 AM
#16
Re: How can I remove text clones from the hosts file?
Indeed it is... and all of the code you need (except for adding the new line to the file, which I assume you can do already) is either in post #7, or the parts of post #1 it mentions.
-
Aug 2nd, 2018, 06:12 AM
#17
Thread Starter
Member
Re: How can I remove text clones from the hosts file?
Still not deleting. Maybe there's a way to set a limit for the amount of times certain text is added?
-
Aug 2nd, 2018, 06:21 AM
#18
Re: How can I remove text clones from the hosts file?
What code did you use?
Were there any errors? (or anything else we should know)
-
Aug 2nd, 2018, 06:26 AM
#19
Thread Starter
Member
Re: How can I remove text clones from the hosts file?
Nope, no errors. I used this code:
Code:
If index >= 2 Then
MessageBox.Show("Test.")
Dim filePath = "C:\Windows\System32\drivers\etc\hosts"
Dim allLines = File.ReadAllLines(filePath)
Dim firstMatchIndex = Array.IndexOf(allLines, line)
Dim section1Lines = allLines.Take(firstMatchIndex + 1)
Dim section2Lines = allLines.Skip(firstMatchIndex + 1).Where(Function(s) s <> line)
File.WriteAllLines(filePath, section1Lines.Concat(section2Lines))
End If
-
Aug 2nd, 2018, 06:34 AM
#20
Re: How can I remove text clones from the hosts file?
Originally Posted by Modulus
Still not deleting.
Then you did it wrong. We can't know what you did wrong if we don't know what you did. Please put some thought into your posts.
-
Aug 2nd, 2018, 06:35 AM
#21
Thread Starter
Member
Re: How can I remove text clones from the hosts file?
I'm pasting in the code you've told me to!
-
Aug 2nd, 2018, 06:38 AM
#22
Re: How can I remove text clones from the hosts file?
Originally Posted by Modulus
Nope, no errors. I used this code:
Code:
If index >= 2 Then
MessageBox.Show("Test.")
Dim filePath = "C:\Windows\System32\drivers\etc\hosts"
Dim allLines = File.ReadAllLines(filePath)
Dim firstMatchIndex = Array.IndexOf(allLines, line)
Dim section1Lines = allLines.Take(firstMatchIndex + 1)
Dim section2Lines = allLines.Skip(firstMatchIndex + 1).Where(Function(s) s <> line)
File.WriteAllLines(filePath, section1Lines.Concat(section2Lines))
End If
And did you actually debug that? What was the value of 'index'? I just tested the code I posted in post #14 and it worked perfectly for me. I copied the data you posted in post #5 and the first code snippet deleted all three matching lines while the second code snippet left the first behind and deleted the last two.
-
Aug 2nd, 2018, 06:44 AM
#23
Thread Starter
Member
Re: How can I remove text clones from the hosts file?
My bad, I only tried the second code sample you provided, not the first of #14. Will try now.
-
Aug 2nd, 2018, 06:50 AM
#24
Re: How can I remove text clones from the hosts file?
Originally Posted by Modulus
My bad, I only tried the second code sample you provided, not the first of #14. Will try now.
Whether you should use either of them or something else similar depends on exactly what you want. They were intended as examples that you can modify as required. As I said, the first one will remove every line from the file that matches the value you specify, while the second will remove all but the first instance. If you don't either of those things exactly then you'll need to make modifications as required.
I should also point out that, while I have been answering the question that you seemed to be asking, I have to agree with si that prevention is generally better than cure, i.e. it would generally be better to not add duplicates at all rather than try to remove then after the fact. Is there any reason that you can't do that?
-
Aug 2nd, 2018, 07:02 AM
#25
Thread Starter
Member
Re: How can I remove text clones from the hosts file?
I probably could, but I’m just not 100% sure how too.
-
Aug 2nd, 2018, 07:13 AM
#26
Re: How can I remove text clones from the hosts file?
Who says that you have to be 100% sure about something to make an attempt to do it? si has given you pretty much all the information you need. Take some time to give it some thought and effort. The fact that you don't know immediately exactly what code to write is no reason to not bother and wait for someone else to write the code for you. The best way to learn is to do. We can help you if you run into an actual issue but if you don't try then you'll never know. We're more inclined to help those who help themselves.
Tags for this Thread
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|