-
Jul 25th, 2021, 05:37 AM
#1
Thread Starter
Addicted Member
[RESOLVED] Vb.net - Split large text file by number of lines
How can I split a single text file with 1000 lines into multiple smaller files of, for example, 300 lines ? I found two C# code, they feel good, but I'm not able to convert them to vb.net
Code:
using (System.IO.StreamReader sr = new System.IO.StreamReader("path"))
{
int fileNumber = 0;
while (!sr.EndOfStream)
{
int count = 0;
using (System.IO.StreamWriter sw = new System.IO.StreamWriter("other path" + ++fileNumber))
{
sw.AutoFlush = true;
while (!sr.EndOfStream && ++count < 20000)
{
sw.WriteLine(sr.ReadLine());
}
}
}
}
Code:
var reader = File.OpenText(infile);
string outFileName = "file{0}.txt";
int outFileNumber = 1;
const int MAX_LINES = 300;
while (!reader.EndOfStream)
{
var writer = File.CreateText(string.Format(outFileName, outFileNumber++));
for (int idx = 0; idx < MAX_LINES; idx++)
{
writer.WriteLine(reader.ReadLine());
if (reader.EndOfStream) break;
}
writer.Close();
}
reader.Close();
-
Jul 25th, 2021, 05:49 AM
#2
Re: Vb.net - Split large text file by number of lines
So what specific problems did you have while converting? Did you make any effort at all? That first code snippet has two using statements and two while loops, two declarations, a property setting and a method call. What exactly is preventing you doing the same thing in VB? Apart from the fact that you should be able to make a pretty good fist of a manual conversion for yourself if you bother to try, there are plenty of code converters available only that are of varying quality but can pretty much all handle simple code like that. There's also the downloadable Instant VB that can handle extremely complex code so will have no issue with that basic stuff. Basically, if you had made a reasonable effort then you could have done this yourself and you could have still done some of it yourself even with an unreasonable effort.
-
Jul 25th, 2021, 10:41 AM
#3
Thread Starter
Addicted Member
Re: Vb.net - Split large text file by number of lines
I don't know C#, I tried with some code converters without success, they can not convert them.
I have already solved the problem with a much longer code, I would like to understand how to reduce the code
Code:
Dim OriginalFilePath = "F:\Documenti\Visual Studio 2017\Projects\TextFileSplit.txt"
Dim lines = IO.File.ReadAllLines(OriginalFilePath)
Dim NumOfLines As Integer = lines.Length, LinesPerFile As Integer = 300
Dim NumOfFiles As Integer = NumOfLines \ LinesPerFile + 1
Dim StartIndex As Integer
Dim EndIndex As Integer
Dim sb As New System.Text.StringBuilder
For i = 0 To NumOfFiles - 1
EndIndex = StartIndex + LinesPerFile - 1
If EndIndex >= NumOfLines - 1 Then
EndIndex = NumOfLines - 1
End If
For index = StartIndex To EndIndex
sb.AppendLine(lines(index))
Next
Dim NewFilePath As String = "F:\Download\File" + i.ToString + ".txt"
IO.File.WriteAllText(NewFilePath, sb.ToString)
StartIndex = EndIndex + 1
sb.Clear()
Next
Last edited by patel45; Jul 25th, 2021 at 10:55 AM.
-
Jul 28th, 2021, 01:36 PM
#4
Re: Vb.net - Split large text file by number of lines
If you would like to see a proven method to chunk files (and there is a great deal more code as it's ported from a C# enterprise utility I wrote) see the following GitHub repository.
-
Jul 29th, 2021, 01:53 PM
#5
Thread Starter
Addicted Member
Re: Vb.net - Split large text file by number of lines
-
Jul 30th, 2021, 03:17 AM
#6
Re: Vb.net - Split large text file by number of lines
split a single text file with 1000 lines into multiple smaller files of, for example, 300 lines
Why? I would consider a 1000 line file as 'small' ? What problems are you getting with a file of this size that requires it to be split on number of lines?
All advice is offered in good faith only. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/
C++23 Compiler: Microsoft VS2022 (17.6.5)
-
Jul 30th, 2021, 08:47 PM
#7
Re: Vb.net - Split large text file by number of lines
This web site converted the code in the OP pretty comfortably.
It produced this:-
vbnet Code:
Using sr As StreamReader = New StreamReader("path")
Dim fileNumber As Integer = 0
While Not sr.EndOfStream
Dim count As Integer = 0
Using sw As StreamWriter = New StreamWriter("other path" & Threading.Interlocked.Increment(fileNumber))
sw.AutoFlush = True
While Not sr.EndOfStream AndAlso Threading.Interlocked.Increment(count) < 20000
sw.WriteLine(sr.ReadLine())
End While
End Using
End While
End Using
Though I'd probably refactor all that Interlocked.Increment code.
-
Jul 31st, 2021, 12:22 PM
#8
Thread Starter
Addicted Member
Re: Vb.net - Split large text file by number of lines
Thanks Niya, it runs well
-
Jul 31st, 2021, 06:30 PM
#9
Re: Vb.net - Split large text file by number of lines
Code:
Dim fileLines() as String = IO.File.ReadAllLines("path")
Dim AllLines(CInt(Math.Floor(fileLines.Length / 300)))() as String
For x as integer = 0 to AllLines.GetUpperBound(0)
AllLines(x) = fileLines.Take(300).ToArray
Next
Edit: Actually there's an error...
Code:
Dim fileLines() as String = IO.File.ReadAllLines("path")
Dim AllLines(CInt(Math.Floor(fileLines.Length / 300)))() as String
For x as integer = 0 to AllLines.GetUpperBound(0)
AllLines(x) = fileLines.Skip(x * 300).Take(300).ToArray
Next
Last edited by .paul.; Jul 31st, 2021 at 09:09 PM.
- Coding Examples:
- Features:
- Online Games:
- Compiled Games:
-
Aug 2nd, 2021, 12:24 PM
#10
Thread Starter
Addicted Member
Re: Vb.net - Split large text file by number of lines
Thanks Paul, very compact code, I modified it so
Code:
Dim OriginalFilePath = "F:\Documenti\Visual Studio 2017\Projects\TextFileSplit.txt"
Dim LinesPerFile As Integer = 20
Dim fileLines() As String = IO.File.ReadAllLines(OriginalFilePath)
Dim AllLines(CInt(Math.Floor(fileLines.Length / LinesPerFile)))() As String
For i = 0 To AllLines.Length - 1
AllLines(i) = fileLines.Skip(i * LinesPerFile).Take(LinesPerFile).ToArray
Dim NewFilePath As String = "F:\Download\File" + i.ToString + ".txt"
IO.File.WriteAllLines(NewFilePath, AllLines(i))
Next
-
Aug 2nd, 2021, 08:00 PM
#11
Re: Vb.net - Split large text file by number of lines
Replace this:
vb.net Code:
CInt(Math.Floor(fileLines.Length / LinesPerFile))
with this:
vb.net Code:
fileLines.Length \ LinesPerFile
-
Aug 2nd, 2021, 08:18 PM
#12
Re: Vb.net - Split large text file by number of lines
As you’re just writing it directly to new files…
Code:
Dim OriginalFilePath = "F:\Documenti\Visual Studio 2017\Projects\TextFileSplit.txt"
Dim LinesPerFile As Integer = 20
Dim fileLines() As String = IO.File.ReadAllLines(OriginalFilePath)
For i = 0 To AllLines.Length - 1
Dim subsetLines() As String = fileLines.Skip(i * LinesPerFile).Take(LinesPerFile).ToArray
Dim NewFilePath As String = "F:\Download\File" + i.ToString + ".txt"
IO.File.WriteAllLines(NewFilePath, subsetLines)
Next
- Coding Examples:
- Features:
- Online Games:
- Compiled Games:
-
Aug 2nd, 2021, 08:59 PM
#13
Re: Vb.net - Split large text file by number of lines
Given that we are supposedly talking about large files here, I'd be tempted to avoid calling ReadAllLines.
vb.net Code:
Dim inputFilePath = "file path here"
Dim outputFolderPath = "folder path here"
Dim linesPerFile = 20
Dim fileNumber = 1
Dim lines As New List(Of String)(linesPerFile)
For Each line In File.ReadLines(inputFilePath)
lines.Add(line)
If lines.Count = linesPerFile Then
Dim outputFilePath = Path.Combine(outputFolderPath, $"File{fileNumber}.txt")
File.WriteAllLines(outputFilePath, lines)
fileNumber += 1
lines.Clear()
End If
Next
If lines.Count > 0 Then
Dim outputFilePath = Path.Combine(outputFolderPath, $"File{fileNumber}.txt")
File.WriteAllLines(outputFilePath, lines)
End If
It's also worth noting that this:
vb.net Code:
Dim subsetLines() As String = fileLines.Skip(i * LinesPerFile).Take(LinesPerFile).ToArray
is going to enumerate the same array over and over again. It will only enumerate part of the array each time but that part will get larger and larger. It would be more performant to create the target array ahead of time and call Array.CopyTo and specify the section of the source array to copy. Not a big deal for small data sets but a potential issue for large ones.
-
Aug 2nd, 2021, 10:13 PM
#14
Re: Vb.net - Split large text file by number of lines
Originally Posted by jmcilhinney
Given that we are supposedly talking about large files here...
1000 lines is a large file???
- Coding Examples:
- Features:
- Online Games:
- Compiled Games:
-
Aug 2nd, 2021, 10:29 PM
#15
Re: Vb.net - Split large text file by number of lines
Originally Posted by .paul.
1000 lines is a large file???
Not really, although the OP did say "large text file". I guess it's just large in relation to the very small files they want to end up with. Still, if nothing else, it's good to know the different ways that you can approach something like this and what the pros and cons of each option are.
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|