[RESOLVED] Bits, Bytes and Boomerangs (without the boomerangs...)

[chris128]

If you want to put the RD at 1 and everything else at 0...

Just put the first byte a 1 and the second a 0
HeaderBytes(2) = 1 (0000 0001)
HeaderBytes(3) = 0 (0000 0000)

but that wont be in the correct place will it? This is what I dont get, you guys are saying that the order of the bits (high to low vs low to high) doesnt make a difference as thats just how us humans like to view the bits - but in that header table it shows that single bits affect specific settings (in this case the RD setting) so surely I have to get them in the exact right place?
Lets use a different example bit, lets say I wanted to set the TC bit, how would I do that? Again its just 1 bit I want to set so are you saying I should just do the same and set HeaderBytes to 1? If so then how will the DNS server that receives these bytes know I meant the TC bit and not some other bit in that word?

[chris128]

I thought you were trying to manage 0 to 15 bits - which is two bytes - a word.

But you don't care if the variable has more bits available.

Well yeah I guess I am but I was under the impression that I had to deal with each 8 bit section (ie a Byte) on its own. This is getting more and more confusing.
So lets say I use an Integer in place of the Byte then (which I didnt realise I could do), once I have somehow set the relevant bits in there, how would I add that to my byte array? The end result has to be a Byte array as I need to send it via UDP

[mickey_pt]

If you want to use a numeric data type use the Int16\Short (2bytes) that's a complete line of the header...

I think that your problem it's understanding the bits/bytes and how to read them...
My solution
Byte H(0) = 0000 0001 and Byte H(1) = 0000 0000
2Bytes = 0000 0001 0000 0000

The application that will read the header will read for example 2bytes (word) at once, so it will read all the line... so the value that you see like 1 will be 256 for the application. So everything it's related with the way that the data is read...

It's something like a string. You put:
H(0) = He
H(1) = llo
If you read the items separated you don't understand nothing, but if you read all the itens once you will see Hello.

I want to explain but my English isn't the best to explain... sorry...

[chris128]

Well I tried sending my DNS query to a DNS server like so:

Dim HeaderBytes(11) As Byte
        '--ID
        HeaderBytes(0) = 1
        HeaderBytes(1) = 1
 
        '--Parameters
        'First byte of parameter section
        Dim FirstParametersByte As Byte = 0
        FirstParametersByte = CByte(FirstParametersByte Or (1 * 128))
        HeaderBytes(2) = FirstParametersByte
        'Second byte of parameter section
        HeaderBytes(3) = 0
 
        '--QDCOUNT
        HeaderBytes(4) = 1
        HeaderBytes(5) = 0
 
        '--ANCOUNT
        HeaderBytes(6) = 0
        HeaderBytes(7) = 0
 
        '--NSCOUNT
        HeaderBytes(8) = 0
        HeaderBytes(9) = 0
 
        '--ARCOUNT
        HeaderBytes(10) = 0
        HeaderBytes(11) = 0
 
        'Build the question section
        Dim DomainnameBytes() As Byte = System.Text.Encoding.ASCII.GetBytes("microsoft")
        Dim DotComBytes() As Byte = System.Text.Encoding.ASCII.GetBytes("com")
        Dim QuestionBytes(6 + DotComBytes.Length + DomainnameBytes.Length) As Byte
        QuestionBytes(0) = CByte(DomainnameBytes.Length)
        Array.Copy(DomainnameBytes, 0, QuestionBytes, 1, DomainnameBytes.Length)
        QuestionBytes(1 + DomainnameBytes.Length) = CByte(DotComBytes.Length)
        Array.Copy(DotComBytes, 0, QuestionBytes, 2 + DomainnameBytes.Length, DotComBytes.Length)
        QuestionBytes(QuestionBytes.Length - 5) = 0
        QuestionBytes(QuestionBytes.Length - 4) = 0
        QuestionBytes(QuestionBytes.Length - 3) = 15
        QuestionBytes(QuestionBytes.Length - 2) = 0
        QuestionBytes(QuestionBytes.Length - 1) = 1
 
        'Combine the header and question
        Dim FinalBytes((QuestionBytes.Length + HeaderBytes.Length) - 1) As Byte
        Array.Copy(HeaderBytes, 0, FinalBytes, 0, HeaderBytes.Length)
        Array.Copy(QuestionBytes, 0, FinalBytes, HeaderBytes.Length, QuestionBytes.Length)
 
   
        Dim DNSServer As New IPEndPoint(IPAddress.Parse("192.168.0.254"), 53)
        Dim sock As New Net.Sockets.Socket(Net.Sockets.AddressFamily.InterNetwork, Net.Sockets.SocketType.Dgram, Net.Sockets.ProtocolType.Udp)
        'Send the data
        sock.SendTo(FinalBytes, 0, FinalBytes.Length, Net.Sockets.SocketFlags.None, DNSServer)
 
        'Receive the response
        Dim ReceivedBytes(512) As Byte
        Dim BytesRead As Integer = sock.Receive(ReceivedBytes, ReceivedBytes.Length, Sockets.SocketFlags.None)

and the server just did not respond at all. So I tried your suggestion Mickey and changed it to this:

'--Parameters
        'First byte of parameter section
        HeaderBytes(2) = 1
        'Second byte of parameter section
        HeaderBytes(3) = 0

and now the server responds but not with what I was expecting lol
It just sends back 12 bytes, all 0 apart from the first and second Bytes are the same as my ID that I set (in this case 1 and 1) and the third and fourth Bytes are both 129. The response that the server sends back starts with that same header format that I have been working with, so I have to somehow figure out what 129 129 means when applied to the fields that are in those 2 bytes...

 |QR|   Opcode  |AA|TC|RD|RA|   Z    |   RCODE   |

EDIT: I changed the order of the 4th and 5th bytes as someone suggested in an earlier post and now I get a 78 byte response which I thought was still not quite right because I am comparing my byte response (converted to hex) to the response shown here (about half way down the page): http://www.codeproject.com/KB/IP/dnslookupdotnet.aspx
But I just noticed that when I actually do an NSLOOKUP for microsoft.com's MX records now it does only bring back one result rather than the 3 that it brought up when that article was written... so maybe its working properly now I'm going to have to properly understand it though because I need to be able to work with the response it has sent me! I'm going out for the weekend now but I'll pick this up again next week, thanks for all the help so far guys

[namrekka]

Hi here is "someone again".

so I have to somehow figure out what 129 129 means when applied to the fields that are in those 2 bytes...

129 has a bit patern of: 10000001.
So the first byte will say:
Bit QR is set and RD is set. QR will say a response and that is correct. Not sure why you want bit RD set. The RFC mentioned "Recursive query support is optional."
The second byte:
RA is set which means that recursion is available and RCODE=8 which is strange.

Be careful with comparing the HEX codes. With the table presented LSB is on the left and normaly in a HEX viewer LSB is on the right.

Sorry for my english. Have some problems to put in right in english.

[chris128]

The RFC mentioned "Recursive query support is optional."

yes its an option that I want to use...
Basically if you dont tell a DNS query to be 'recursive' then the DNS server you send the query to will not pass that request on to any further DNS servers if it does not know the answer... so in 99% of cases you want the DNS query to be recursive.

Anyway if you read my post after that you will see that I no longer get that 129 129 response, I now get a 'proper' response from the server so I just need to work on reading that response. I might not get chance to do that for another couple of days now though but I will post back here if I have problems (which I undoubtedly will...)

Thanks
Chris

[chris128]

I know no one here has mentioned this so maybe its not the best way to do things but I had another look at the BitArray class last night and it seems very useful for what I am wanting to do. You can simply create a new instance of it and set the length, then set each item inside it to True or False to represent the bit being set or not set. Then when you are done you just copy it to a Byte array and there you go, all of the bits you wanted to set are set Same for reading a byte array as bits, you can pass in a byte array to the constructor of the BitArray class and then just loop through the bits and see what they are all set to - its much easier than messing about with bit masks and all that.
Having said that, I'm glad I read all of the things you guys have said in this thread as its given me a much better understanding of bits and bytes (which came in handy the other day actually when doing some networking work).
I'll post an example of how I'm using this BitArray later today

[szlamany]

It's the best way as long as it's easy for you to construct and maintain and also doesn't slow things down.

If you are processing lots of this stuff at a fast pace then the "old tried and true" method might serve you better.

[chris128]

Yeah you are probably right, I will try and get the hang of the AND and OR stuff for the final version of the app but first I just want to actually send a query to the DNS server and parse the response into something meaningful

I'm actually starting to wonder if this table is showing the bits in the wrong order...

The header contains the following fields:

                                    1  1  1  1  1  1
      0  1  2  3  4  5  6  7  8  9  0  1  2  3  4  5
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    |                      ID                       |
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    |QR|   Opcode  |AA|TC|RD|RA|   Z    |   RCODE   |
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    |                    QDCOUNT                    |
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    |                    ANCOUNT                    |
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    |                    NSCOUNT                    |
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    |                    ARCOUNT                    |
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+

For several reasons.
1. Every other example of bits on the web seems to do it the other way round, ie most significant bit first
2. When I tried to set the RD bit by setting that byte to 128, the server does not respond at all. So perhaps I am actually setting the first bit (bit 0) whcih is the QR bit. The QR flag is the bit that specifies whether or not this is a query or a response, and by setting it to 1 I would be indicating that this is a response so of course the server wont reply as it doesnt think it has received a question. When I change that byte to 1 instead of 128 then the server responds.
3. In the server response, I get the following bytes (as well as much more afterwards):
1, 1, 129, 128.
The two 1s are my ID. The 129 and 128 are the 2 bytes that relate to those parameter fields and 129 would mean bit 0 (QR) and bit 7 (RD) are set so thats fine, but 128 would mean the RCODE nibble is just set to a value of 1 - the RCODE nibble being set to 1 means the following:

Format error - The name server was
unable to interpret the query.

so maybe my request is incorrect but if that was the case then I wouldnt have thought I would then get another 70 bytes of data after that. So if my theory about the bits being the other way round (the normal way) is correct then what that actually means is that the RCODE nibble is set to 0 which is what I would expect, and its actually the RA bit flag that is set, which simply means that recursion is available and apparently is only set in a response.
4. If I change my request to use an invalid domain name, (microsoft.co instead of microsoft.com) then I get 129, 131 for the parameter fields, which would mean again the QR and RD bit flags are set which is fine, but then it would mean the RA bit flag is set and the 3 bit Z field is set to something but the RFC states that the Z field should always be 0 as it is reserved for future use. If we turn it round and look at it as if the bits are the 'normal' way round then it would equate to the RA bit flag being set which is fine, and it would mean the Z field is 0 and the RCODE nibble is set to 3 which means:

3 Name Error - Meaningful only for
responses from an authoritative name
server, this code signifies that the
domain name referenced in the query does
not exist.

So maybe even though they have labeled the bits from 0 to 15, this doesnt actually mean they are talking about the bits in the opposite order to how everyone else seems to talk about bits... what do you think?

[chris128]

OK after some more testing I am quite convinced now that the bits in that table are labelled in an 'odd' way and that to be easier to follow the table should actually look more like this:

The header contains the following fields:

                              1  1  1  1  1  1
      7  6  5  4  3  2  1  0  5  4  3  2  1  0  9  8
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    |                      ID                       |
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    |QR|   Opcode  |AA|TC|RD|RA|   Z    |   RCODE   |
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    |                    QDCOUNT                    |
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    |                    ANCOUNT                    |
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    |                    NSCOUNT                    |
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    |                    ARCOUNT                    |
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+

and I think I'm starting to get the hang of using AND, OR and AND NOT now as well Here is how I am setting the AA bit and the RD bit in that second field:

'--Parameters
HeaderBytes(2) = (1 Or 4) 'Sets the RD bit flag and AA flag
HeaderBytes(3) = 0

and then when I want to check to see if the QR bit flag is set I am doing this:

If (ReceivedBytes(2) And 128) = 128 Then
            'QR flag is set
End If

is that the right way to be doing it?

Thanks
Chris

[szlamany]

Yes - this is good code

HeaderBytes(2) = (1 Or 4)

And I see that those are the bit locations - RD is bit 0 and AA is bit 2 (which are values of 1 and 4)

This code can be written three ways

If (ReceivedBytes(2) And 128) = 128 Then
If (ReceivedBytes(2) And 128) <> 0 Then
If ReceivedBytes(2) And 128 Then

I prefer the last method myself - no reason to add the = 128 or <> 0 stuff.

[chris128]

Ah I see, thanks for that tip - with Option Strict on though I have to do the following:

If CBool(ReceivedBytes(2) And 128) Then

but I still prefer that over the way I was doing it

So one thing I am still a bit puzzled about (it may just be the terminology) is bit shifting. If I can test for any individual bit being set by just using AND then why/when would I use bit shifting? Is that more for when you want to set a bit rather than just checking to see if a bit is set?

Thanks
Chris

[RunsWithScissors]

I hope ALL of you are aware that you have just provided a college semester's worth of education for thos of us who can't afford to go to college (and are too old, and gotta work, and discovered programming as a "hobby" late in the game, etc . . .).

This string (EDIT: THis THREAD-too much code happening!)should be moved to the code bank, or a faq or something. I could not have found a better explanation of binary operations(?) by searching.

It will be a while before I actually need to USE this, and even longer before I get my head around it, but wow! Very cool stuff.

Thanks!

ps: One persons profession is another person's hobby is another's twisted obsession!

[chris128]

haha I'm glad my plethora of questions as I struggle to grasp this stuff might prove to be helpful to others

and in response to your PS - this is just my hobby

[szlamany]

So one thing I am still a bit puzzled about (it may just be the terminology) is bit shifting. If I can test for any individual bit being set by just using AND then why/when would I use bit shifting? Is that more for when you want to set a bit rather than just checking to see if a bit is set?

That OPCODE is 4 bits - right? That means it's a value from 0000 (0) to 1111 (15).

If you want to extract that value - and it's in the BYTE - you need to shift it down to position 0 and also MASK it to hide other bits that shift down with it.

So once again - the byte with the opcode - let's say it:

1101 0111

The 1010 bits is the 10 values I want to extract. The decimal value of 1101 0111 is not 10 [edit - oops] - so first thing is to shift it down

1101 0111 / 8

0001 1010 - now the 1010 is in bit 0, bit 1, bit 2 and bit 3
0000 1111 - this is the 15 mask

AND these and you get

0000 1010 - this is the 10 I want - it's in a byte - but the value of 10 (from the opcode) came through.

[chris128]

Ahh yeah I see, I hadnt considered checking anything other than individual bit flags
This brings me on to another question actually... (I know, will they ever end?)
For the fields that are 2 full bytes (ID, QDCOUNT etc) is there any way I can set them using the UInt16 type? Well, I know there is as UInt16 is just 2 bytes, but I just cant figure out how to get this to work with my byte array. The ID for example is bytes 0 and 1 in my byte array, and I would like a way to set both of these bytes in one go when given a UInt16 value - is that possible or do I have to break the UInt16 value down into 2 bytes and set byte 0 and byte 1 separately?

EDIT: and also am I using And Not in the correct way here? I just want it to clear a specific bit and leave all other bits as they are

Private _QR As Boolean
Public Property QR() As Boolean
        Get
            Return _QR
        End Get
        Set(ByVal value As Boolean)
            If value = True Then
                _HeaderBytes(2) = CByte((HeaderBytes(2) Or 128))
            Else
                _HeaderBytes(2) = CByte((HeaderBytes(2) And Not 128))
            End If
            _QR = value
        End Set
End Property

As you can tell, I am going to have a property that relates to each of this configurable bit flags so when someone sets the property I want it to be reflected in the Byte array

[szlamany]

You must shift before you OR - that's all.

Take you 2-bit value - doesn't matter if it's in a word, longword, quadword - it's in bits 0 and 1 - that's all.

SHIFT it up to where you want it to go with *2 or *4 or *8...

then set it with OR - you aren't OR's the "value" you think you want to because you shifted it - but where you shift it to is the 2 bits that it has to be.

[chris128]

I dont get why I have to shift before I do an OR because it seems to work without doing that..

For example if I want to set bits 3 and 4 (00011000) I can do this and it sets the correct bits:

HeaderBytes(2) = (HeaderBytes(2) OR 8 OR 16)

or is it just that I am doing the shifting mentally (e.g I'm working out that I would need to set 8 and 16 to set those bits) rather than using the code to just shift whatever value I want into whatever spot I want?

[szlamany]

...or is it just that I am doing the shifting mentally (e.g I'm working out that I would need to set 8 and 16 to set those bits) rather than using the code to just shift whatever value I want into whatever spot I want?

By Jove - I think he's got it.

Yes - that's exactly it!

If you are "extracting" the value you have no idea what it is - so shifting is needed.

If you shift out - then you shift in to put a value.

We should digress into a couple of posts about SI and SO (character 15 and 16? Ctrl/O and Ctrl/N? Is that right?)

[chris128]

We should digress into a couple of posts about SI and SO (character 15 and 16? Ctrl/O and Ctrl/N? Is that right?)

I have no idea what you are talking about lol

[chris128]

By the way, someone on a DNS specific forum confirmed that the bits in that table I keep referring to are in big-endian order, so bit 0 is actually the most significant bit contrary to what you would think when looking at it. Apparently all network communications are done using big-endian format bytes.
Also, just to make sure my DNS query was being asked properly, I used WireShark (www.wireshark.org) to monitor my network card just as I sent the query and it all looks like its doing what I want it to

My query capture from wireshark:



The capture of the response:



all looks as I wanted it to, so I guess I can finally mark this thread as resolved and start working on actually parsing the server's response

Thanks to all that helped!
Chris

[chris128]

Would help if I actually marked it as Resolved wouldnt it

This string (EDIT: THis THREAD-too much code happening!)should be moved to the code bank, or a faq or something. I could not have found a better explanation of binary operations(?) by searching.

I dont think it belongs in the codebank really, if someone is looking for similar information they can just do a search in this forum and will find it