How to Send Audio Over RTP UDP VB.Net

[Bensley196]

Hey all,

Been a while, I've moved along in the projects i have been working on. And currenlty trying to send audio (and eventually video ect) over the network in real time. I've done a fair bit of research and i am fairly sure going down the RTP route is the correct option for me to do this.

Currently i have succesfully set up a mulitcast UDP sender and reciever, and from there, using a few forums, i was able to get audio coming through on the client. However it is extremely noisy and is playing at the wrong speed. I know i could fix the noise by adding a ulaw encoder/decoder but this add's high compression, and i need original quality (or as close to). And i think this is more a quick fix to the bigger problem.

So after looking everywhere and not finding anything helpful i've come here. I have some code (below) you can make out a song in it. But it's pretty rough. These are the steps i am trying to get through.

1. Release the data at a controlled rate
2. Structure the data into an RTP packet
3. Read the data and play it as soon as it is recieved (it's important all clients play the song as close to realtime and together as possible, hoping RTP's timing


I understand the photo from wiki (photo below), and what each chuck is doing, however i don't know how to actually implement something like that, so if anyone has some pointers that would be great.




Audio Code:

    Private BlockAlignedStream As WaveStream = Nothing

    Private BufferedProvider As BufferedWaveProvider = Nothing

    Private WaveReader As WaveStream = Nothing

    Private WaveChannel As WaveChannel32 = Nothing

    Private WaveOut As IWavePlayer

   Friend Function LoadAudioStreamer(ByVal Filepath As String, ByRef IPAddress As Net.IPAddress, ByRef Port As Integer) As Boolean

        Try

            [Stop]()

            Status = Statuses.Loading

             WaveReader = New Mp3FileReader(filePath)

            BlockAlignedStream = New BlockAlignReductionStream(WaveReader)

            ' Wave channel - reads from file and returns raw wave blocks
            WaveChannel = New WaveChannel32(BlockAlignedStream)

            WaveChannel.PadWithZeroes = False

            Dim UDPSender As New UDPCoreClass(AddressOf UDPCallback)

            If UDPSender.JoinMultiCast("0.0.0.0", IPAddress.ToString, Port) = True Then

                Dim alignment As Integer = WaveChannel.BlockAlign * 32

                Dim buffer As Byte() = New Byte(alignment - 1) {}

                Try
                    Dim numbytes As Integer

                    Do

                        numbytes = WaveChannel.Read(buffer, 0, alignment)

                        UDPSender.SendMulticast(buffer, numbytes)

                    Loop While numbytes <> 0

                Catch ex As Exception


                End Try

            End If

        Catch exp As Exception

            ' Error in opening file
            WaveOut = Nothing

            StatusMessage.Write("Can't open file: " & exp.Message)

            Status = Statuses.Error

            Return False

        End Try

    End Function

 Friend Function LoadAudioListener(ByRef IPAddress As Net.IPAddress, ByRef Port As Integer) As Boolean

        Dim udpListener = New UdpClient(Port)

        'udpListener.Client.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, True)

        Dim endPoint As IPEndPoint = New IPEndPoint(IPAddress, Port)

        Dim waveProvider = New BufferedWaveProvider(New WaveFormat(44000, 16, 2))

        waveProvider.DiscardOnBufferOverflow = True

        Dim b As Byte()

        Dim dso = New DirectSoundOut(100)

        dso.Init(waveProvider)

        udpListener.JoinMulticastGroup(IPAddress)

        dso.Play()

        Do

            b = udpListener.Receive(endPoint)

            waveProvider.AddSamples(b, 0, b.Length)

            'waveProvider.read(b, 0, b.Length)


            Threading.Thread.CurrentThread.Join(0)

        Loop While b.Length <> 0

        WaveOut.Dispose()

        WaveOut.[Stop]()

    End Function

Network Code:

Friend Delegate Sub ServerCallbackDelegate(ByVal bytes() As Byte, ByVal IPAddress As IPAddress, ByVal Port As Integer) ', ByVal sessionID As Int32, ByVal dataChannel As Byte)

    Friend ServerCallbackObject As ServerCallbackDelegate

    Friend errMsg As String

    Private UDPTalker As UdpClient

    Private UDPListener As UdpClient

    Private GroupEndPoint As IPEndPoint

    Private ListeningCue As MessageInQueue

    Private isListening As Boolean = False

    Private isTalking As Boolean = False

    Private isHushing As Boolean = False

    Private isDeafening As Boolean = False

    Private TargetPort As Integer

    Private TargetIP As String

    Private BindingIP As String

    Private BindingPort As String

    Private listenerThread As Thread

    Private TalkingState As CurrentState = CurrentState.Stopped

    Private ListeningState As CurrentState = CurrentState.Stopped

Friend Function JoinMultiCast(ByVal LocalIPAddress As String, ByVal IPAddress As String, ByVal Port As Integer, Optional ByRef ErrMessage As String = "") As Boolean

        Try

            If TalkingState = CurrentState.Running Then

                ErrMessage = "The server is already running."

                Return False

            End If

            BindingIP = LocalIPAddress

            TargetIP = IPAddress

            TargetPort = Port

            Dim GroupIP As IPAddress

            GroupIP = Net.IPAddress.Parse(IPAddress)

            UDPTalker = New UdpClient(AddressFamily.InterNetwork)

            Dim BindingEndPoint As New IPEndPoint(Net.IPAddress.Parse(BindingIP), 0)

            UDPTalker.Client.Bind(BindingEndPoint)

            GroupEndPoint = New IPEndPoint(GroupIP, TargetPort)

            UDPTalker.JoinMulticastGroup(GroupIP, 64)

            isTalking = True

            Return True

        Catch ex As Exception

            Return False

        End Try

    End Function

    Friend Sub SendMulticast(ByVal Text As String)

        If isTalking AndAlso isHushing = False Then

            Try
                Dim bteSendData() As Byte = Utilities.StrToByteArray(Text)

                'UDPTalker = New UdpClient()

                UDPTalker.Send(bteSendData, bteSendData.Length, GroupEndPoint)

            Catch ex As Exception

                Console.WriteLine(ex.Message)

            End Try

        End If

    End Sub

[Niya]

Seems the problem you're having is that the audio being received is of poor quality, is that right?

If so, my first question is whether it's the source quality that's poor. Have you verified this?

[Bensley196]

Hey Niya, thanks for getting back to me.

The original song is certainly good, you can play it fine in a player. It also sounds identical if i was to put a WaveOut block on that rather then send it over the network. So i'm either writing it badly to the network, or reading it badly from the network or both.

The fact that the song is playing so quickly to the person listening, makes me believe that the audio is been played/sent at 200% of the speed, and maybe i'm losing so many packets. Which is what i was thinking using the proper protocal would fix.

But one step at a time, how would i slow down the speed of which it is sending over the network?

[Niya]

UDP is an unreliable protocol. My first guess would be that you are losing a lot of UDP datagrams while transmitting the audio. You're going to have to account for that loss in your programs if this is indeed the cause.

[Bensley196]

Yes that is very likely, especially if the audio is being sent at a faster rate then needed. The RTP (Real Time Transport Protocal) does this, i just don't understand how to impliment something like it.

Here is the wiki on it, i believe if i can understand how to implement it, my problems will be reduced greatly

https://en.wikipedia.org/wiki/Real-t...sport_Protocol

[Niya]

Yes that is very likely, especially if the audio is being sent at a faster rate then needed. The RTP (Real Time Transport Protocal) does this, i just don't understand how to impliment something like it.

Here is the wiki on it, i believe if i can understand how to implement it, my problems will be reduced greatly

https://en.wikipedia.org/wiki/Real-t...sport_Protocol

That protocol looks very complicated. It might be a good idea to find out of someone has already implemented it in .Net. before you attempt this one on your own. If not then you have to prepare to roll up your sleeves and devote time to studying the protocol so you can implement it yourself.

However, do note that we are still assuming that the cause of your problem is packet loss. We don't know if that is the cause for sure. If I were you I'd try to find this out before doing anything else.

[Bensley196]

I've had a look, i found one application that uses a dll to do it. But i can't confirm where the dll is from, for copyright issues or compatibility checking. There are a few ones for video, but largely seem broken due to how old they are.

Do you know of any good articles (ones that use good examples or terminoligy won't blow my mind) that go into how to structure a packet? Currently when i'm sending commands over the network (this is TCP) i just covert some text into binary, send it, and then recieve it on the other end. How to create headers and fit correctly into a packet.

How would i prove that packet loss is the cause, or atlease a big cause of this problem?. Would a counter on every packet send from the server, and compare that to what the client clocks up be a efficient way of doing this?

It would determin if the packets were arriving badly out of order though.

[Niya]

How would i prove that packet loss is the cause, or atlease a big cause of this problem?..

Would I would have done is write a very simple program that breaks up the file into pieces and feed those pieces as a stream into another part of the program that tries to play it. The idea is to simulate what happens when the file is streamed from one endpoint to the next without all the complexity of actual UDP communication. You could then modify the program to artificially "drop packets" to see what would happen.

Do you know of any good articles (ones that use good examples or terminoligy won't blow my mind) that go into how to structure a packet? Currently when i'm sending commands over the network (this is TCP) i just covert some text into binary, send it, and then recieve it on the other end. How to create headers and fit correctly into a packet.

Are you talking about the Real Time Transport Protocol?

[Bensley196]

Ah ok so skip the network all together. And if the out come is similar then that's it.


Just dealing with packets in UDP in general that relate to .net, but i relise now that is something i can find pretty easy hahah. I'll see how i go with your suggestion

[Niya]

Ah ok so skip the network all together. And if the out come is similar then that's it.

Exactly.

Just dealing with packets in UDP in general that relate to .net, but i relise now that is something i can find pretty easy hahah. I'll see how i go with your suggestion

Yes that information is very easy to find. Winsock in .Net is well documented and the UDP protocol is also well documented.