Ok, let's see if I can offer some more productive comments.
The first question I have is about your sampling rate since you're taking in microphone sound samples. How much elapsed speech/sound time does 64 bytes represent? I'm assuming little or no compression, so each 64 bytes must pretty much represent the same amount of time.
This will help us understand the data arrival rate and thus help calculate the acceptable latency in bytes once we know what you can accept in seconds (or fractions)... so we'll need to know that too.
So far then I have two questions:
How much "time" does 64 bytes represent (or what is your sampling rate in bytes per second)?
What is the acceptable lag from microphone to output at the client?
Some lag is out of our control because there'll be some in converting bytes and bits into audio output at the clients as well of course.
Your VB program that is relaying this data to clients will encounter some inevitable buffering at levels you have little control over. Part of this is due to the Nagle algorithm, part of it due to TCP window optimization.
Your Java (or whatever) source program that's capturing sound to a bytestream and passing it to the relay/distribution program will be subject to these same factors. It has little to do with using Java, VB, C, or anything else.
It is possible to disable Nagling in a VB program. I don't know enough about your Java program's environment to comment about that.
Disabling Nagling is almost always a bad idea though.
http://en.wikipedia.org/wiki/Nagle's_algorithm
By disabling Nagling you may actually increase the latency because you're adding a ton of overhead. The presence of Nagling means that by sending 64-byte chunks you are actually fighting TCP's attempts to optimize throughput. Most likely much larger chunks are being strung together and sent by TCP right now, plus you have all of the other overhead of handling so many 64-byte chunks instead of handling fewer larger chunks.
Nagling basically buffers stuff up until it has "enough to be worth sending" or a certain amount of time has passed. Yet turning it off can result in The Tinygram Problem.
There is also a risk of "Silly Window Syndrome." This is independent of the nagling issue. It has more to do with sending rapidly streaming data in chunks of from 1024 to 2048 bytes. It is normally much better for TCP if you send fast-stream data in chunks (SendData calls) of 32K to just under 64K bytes. Multiple shorter SendData calls are fine, as long as they happen in extremely fast succession. On a 100M bps LAN it is almost impossible to fire the data fast enough with short SendData calls though. "Extremely fast" changes with the end-to-end net data rate of the network and gets shorter quickly as network data rates increase.
http://en.wikipedia.org/wiki/Silly_window_syndrome
Basically, those 64-byte chunks are killing you here! And most of the problem is at the source, not in this VB relay/distribution program.
But on top of all that your VB program is a victim of The Packet Fallacy.
The VB program will not reliably receive the same chunks that the source (Java?) program sent. It'll get all of the bytes and get them in the proper sequence, but they will not be in fragments received by GetData the same way they were sent by the source program:
Sent:
1234567890 1234567890 1234567890 1234567890
Received:
1234567890123456789012 345678 9012345 67890
In reality if the sending program is "slow" this unexpected fragment consolidation will be infrequent - but frequent enough to mess things up! If the end-to-end network is "slow" things get even worse though, since your program becomes relatively "fast" (relative to the raw network throughput).
Your code seems to expect something that "IsNumeric" (or "Not IsNumeric" - the logic is hard to follow) as a lead-in or something. If this is true, that is if you are expecting the "first bytes" of the result of a GetData to have special significance, your program will make mistakes, lose data, etc.
This is precisely why a TCP receiver needs to "buffer" inbound data at the application level. This buffer must be examined for something in the content to tell you where the boundaries are, then you split things out on the basis of that "something," and finally process the restored sent-chunks (messages or datagrams).
I'm not suggesting you can't make this work. It is exactly why a lot of streaming audio and most streaming video uses UDP though whenever possible. TCP will add delays and perform low-level buffering and windowing in an attempt to optimize the use of the connection and the network it operates over while guaranteeing delivery and sequence of the data stream. It was not meant as a real-time signaling system.