Downloading binary files from Usenet (NNTP)

[Chris001]

Hi, I found a newsreader on pscode.com that was originally made for downloading text messages from Usenet, but I'm trying to change it to download binary files. I use a message-id to download a segment, but the problem is that only a very small part is downloaded. I hope somebody can help me with this.

For example a RAR file of 15 MB is split in 20 segments (about 775 KB each). All 20 segments need to be downloaded and together they are the 15 MB RAR file. All segments have their own message-id, which identifies them on the newsserver.

The problem is that when I try to download a 775 KB segment, it only downloads 5 KB and I can't figure out why.

Is there anybody here who knows how to work with NNTP and wants to take a look at the code?

Thank you.

(code removed - solution found)

[Al42]

A text box can only hold 64k so if you're putting it into a text box you're going to lose most of it. How you download a binary stream depends on how you're downloading it - INet, API, etc.

Next, you have to determine the method being used - is there a PAR file, is it encoded somehow, etc., or is it just a binary stream? If you ask in the Forte Agent newsgroup (sorry, I don't have a news reader on this computer, but just search for "agent" in your newsgroups - it's the one that's NOT for modifications), someone can probably point you to the information you'll need (and maybe even some code).

[Chris001]

I already figured out what the problem was. The data is not all coming in at once, but in smaller parts.

Public Sub Winsock1_DataArrival(ByVal bytesTotal As Long)
Dim sFinalData As String, sIncoming As String
 
    Winsock1.GetData sIncoming, vbString
    sFinalData = sFinalData & sIncoming
    'sFinalData eventually contains all the data
 
End Sub

Now I only need to find a real proper way of extracting:
the 'part' (73)
the 'name' (hnr-igs-cd2.rar)

And then I need to remove the first and the last part of the data, so all the "222 0", "=ybegin", "=ypart", "=yend" and empty lines are gone and only the binary data is left.

222 0 <[email protected]> body

=ybegin part=73 line=128 size=52480000 name=hnr-igs-cd2.rar
=ypart begin=46080001 end=46720000
‘Qn³ù;n’Amf½öÒÕDË-Ìc¤b…]»·÷²o2mõ¨Ò˜j=@BuŽ‘=JL$ú^3Ã("hÇnÄÅÌ71…8†ÎŸH‘ÌÙoö8Üxf¥t?/¹hú·¾©Ö¹H’¹aBr&Jg=}*¨î©Q²?Ъ"¨b=}
›ÀŒ•jK0,<jÌ–¡îÚm+ÆÇcÖqù¤þ_ph~ÌW!\›¬nÍîƒ@Å?_PVrz"«‡+rfdV̨åâó]žö©ê¢í¤ƒÌ>Muìån)DPJ}Ž£ªŽ¾r(<*<ñÏQè=MÃÑ=M)EÒ=M=MX³èõL”
~½Wå"‘n#ÒwîÌ=@B• Ž†1`ƹ:’Å”í…u6pD_å=M³û´FxÖÉ=@Û*Ï1W$ÿîDQ-:nîB¹¢ýªšß/ú†*SI1d÷Hßmª%÷nq»kø›Ð.¹'=J…9-ä]Žù—K4sõ
Qê™xëôyÌ›{Ê÷F¾?¯Ò ¨f€Önƒ=@ìø'k„[Šfþ)üB™-{f-„ñÆýâäU@K¶>5Äüæ|1Ë•²º**+4œp²»V¸—=}³²./:”þº’Å=@,rt½$ìß=}®t‹…ôoF´fcËãYòhvbl
ó´îLVZÿ jÔŠ?q¬q»=}Š\ºŽj\Ûw1{îr0¼Ú;=}7̘ÛÖ=@=JxŒÓŒïù¡~Ö_6\v=@wk3;ßÇx§Ù8ó‹hJÂó÷¶vx:ôÜk¾.Ó—n[]wÆýBsYÌ–OÎÖ„s™öGl-Ð(+:á̘[`
Êé[@Lû€[6‹*‡()?»³n(=@ª(|à?ƒpƦdœg=}›²~”Rjý¥²a÷s’Jªížë=@눰*æ>›H\Q*ø oê×+JÉ¡¥ƒ2OJy:ŠÒõ²aÜC{dQ0¿¡]‹›Zfãh•
›²K=}{D*r¿nO°K
Ÿ–]¯/=@§ýù-û) pôñöNqÄÅ <´{úˆ‡®=}±Å6ø;uÌÛlÓ'ØMˆçm’9M’¡PîCÛFV¡*̦A;`ðB˜ö¬îÈwîŸgÿ]vXtgÇD·/Ã6㧼æ)**+5äBáKš¶xÛðî3²ÁßUDœ
êÎ"³€ԍ33x‹°|Á¸CÃ9·{V¶ˆm¯@ª¦n¿óÓ<2ò^žê&Ý͝/ò=}H–fâ½KÜ®ÊE:OP‚V‹<>"ñù]û:ŸuÌØÓÖ.›:9ÊIîEþûJÓˆ ÀNƒ›ÜØoÒ×>ðcëpÀ¯N¾WòÿÚüì”ÄG†
¼o?_Áz¡*Du“ž¾ÕÖíÌOî5Ü**+än+¾+³í"**+ç1«ª/K*Ÿª,*ûtÇŠíƒA¢+¯®êÚÅ…¶zÔut\~ˆqCx³äˆ’*)ûJ͇‰*KZ4ÔÔÔÔÔÔÔÔÔÔÔÔÔÔÔÔÔÙû^4)½
‚šë×¢yå’õ-Åçw”ìÄÿ&6!ª¬.éJ7Ö䚢K*hr…›O*Þ~Ê®SèÙJJ+òÚ0B¶H|7·¢/â"|kÿ“O粄È‚81Ö•‘uEh_·l`æ ð¤p~”}‚ðÀßh7“Ï{ÖÖ{û…Љ€
Ù¶•¬¿ÙÍÄKPZó„ÛßOfÐËßßà7v Ê‹—ýp±ÜèzÃ…2÷Ó”’ïpûë3ÕSïð‘þgTFð;×ÄË‘`³>SÔò×ʍ…u„'Aà*C'¨/{ª·)Å3W(³ªD*4jG©‹È9ß2—(™Na_,×¼á!6+
€â:i=}oAîÓ–ÕÁ“÷¿–Õ…éŠ	ï·Õ³Ômp§*̶6˜ž°¿Ü×µ}i>¬1æ:jÖ+*Y*×-4¯p(X*]*¦N²*ñ·;¢¨„ˆ–,©ÒmìÅ𥻒-k‰z53 PY4r÷á#Á=Jf–:’@k+[Š
9ÙäÜŶÍþБ6—y—£d	›*U*–N+â-V*Öë#íipañ=J"³=Mà00á!ú+Ñõ«-hh&é–ãAaIßR}‡•—&.9ï*£ê-j0(Ñy\Åà…M¨2 åИ’û¯zð)‚È$* !Oß*³“Sé÷²,
ÒœM*u÷"J÷G<ÎÞ_קEÁj0×v&b‹EÉï(»ÑŒ¯I…·‰˜œ«^Ò·=MÉ<.á—X.àD}|)ŽªA¡£RW²,8Ø`¿7™îAµÄ„ƒ…›>z+E·À–dòcá»Ë7v(ª%˜©îC*¿pá2¹JœVÕ-¼í
?é=}ŠÌš-;N°£~±áð*C+¹·Ü’§º[UaÈ
U„¨Cñj,Y!ú€>e±rD_ÒþÞŒ¯á©¬^*~*þÜ…L£‹*®áG¦%t6„d¢o˜|Ü0
=yend size=640000 part=73 pcrc32=38fb8806

.

[Al42]

Look for "end=". From that point look for vbNewLine. Everything before that is the header.

Look, from the end, for "=yend". Look backward from that for vbNewLine. Everything after that is the fotter.

Between the end of the header and the beginning of the footer is your binary.

You can use InStr() and InStrRev() for the "looking".

[Chris001]

Thanks, I got that sorted now.

Is it possible that winsock returns corrupted data? Whenever I try to decode a file with a yEnc decoder it gives me an error message.

Trigger: =ybegin part=1 line=128 size=15000000 name=b-blatria.r04
DesFileName = (b-blatria.r04)
yDecoder started...
=ypart-line:  =ypart begin=1 end=249600
part-begin: 1
part-end  : 249600
Last line.
Endline (249609 bytes): =yend size=249600 part=1 pcrc32=36017bd5
Partial message corrupt: longer than (end-begin)!
yDecode failed (reason: 11)

1 error(s) during decoding.

Somehow the data between the header and the footer is 9 bytes too big, but that's the way I downloaded it. No matter what part I download it's always between 2 and 20 bytes too big. It seems like the data is not coming in correctly in the Winsock1_DataArrival event.

Here's one of those parts.

[Al42]

Either that or you're sending "invisible" characters (vbCR, vbLF, etc.) to the decoder. I doubt that the control is actually messing up the data.