PHP User Warning: fetch_template() calls should be replaced by the vB_Template class. Template name: bbcode_highlight in ..../includes/functions.php on line 4197
VS 2010 VB.NET Streaming Image Using Imag2Pipe Issue-VBForums
Results 1 to 9 of 9

Thread: VB.NET Streaming Image Using Imag2Pipe Issue

  1. #1

    Thread Starter
    Hyperactive Member
    Join Date
    Dec 2009
    Posts
    337

    Question VB.NET Streaming Image Using Imag2Pipe Issue

    i am trying so hard to stream images from my .net application to ffmepg using image2pipe.

    after a lot of trail and error.. i came to very close.. but now facing following issue:

    [bmp @ 02690240] not enough data (194700 < 2764854), trying to decode anyway

    here is the sample code that i used:

    Code:
            Dim objFProcess As System.Diagnostics.Process
            Dim o2 As System.Drawing.Bitmap
            Dim o1 As System.IO.BinaryWriter
            Dim strCLIText As String
            Dim strFiles() As String
            Dim strEachFile As String
            objFProcess = New System.Diagnostics.Process
            objFProcess.StartInfo.FileName = txtFMPEG.Text
            objFProcess.StartInfo.Arguments = "-y -f image2pipe -r 25 -s 1280x720 -vcodec bmp -i pipe:.bmp -crf 25.0 -vcodec libx264 -vf scale=1280:720 -coder 1 -rc_lookahead 60 -threads 0 g:\test.mp4"
            objFProcess.StartInfo.UseShellExecute = False
            objFProcess.StartInfo.CreateNoWindow = True
            objFProcess.StartInfo.WindowStyle = ProcessWindowStyle.Hidden
            objFProcess.StartInfo.RedirectStandardError = True
            objFProcess.StartInfo.RedirectStandardInput = True
            objFProcess.Start()
            o1 = New System.IO.BinaryWriter(objFProcess.StandardInput.BaseStream)
            strFiles = System.IO.Directory.GetFiles(txtIMGDir.Text, "*.bmp", IO.SearchOption.TopDirectoryOnly)
            For Each strEachFile In strFiles
                Me.Text = System.IO.Path.GetFileNameWithoutExtension(strEachFile)
                My.Application.DoEvents()
                Using imgStream As System.IO.FileStream = New System.IO.FileStream(strEachFile, IO.FileMode.Open, IO.FileAccess.Read)
                    o2 = CType(System.Drawing.Bitmap.FromStream(imgStream), System.Drawing.Bitmap)
                    o2.Save(o1.BaseStream, System.Drawing.Imaging.ImageFormat.Bmp)
                    o2.Dispose()
                    o2 = Nothing
                    'o1.Flush()
                End Using
            Next
            o1.Flush()
            o1.Close()
            objFProcess.WaitForExit()
            strCLIText = objFProcess.StandardError.ReadToEnd
            objFProcess.Close()
            objFProcess.Dispose()
            objFProcess = Nothing
            saveText2File(strAPPPath & "\***-13.txt", strCLIText)
    and this is the response i am getting: (from file ***-13.txt)

    Code:
    ffmpeg version 4.1 Copyright (c) 2000-2018 the FFmpeg developers
      built with gcc 8.2.1 (GCC) 20181017
      configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
      libavutil      56. 22.100 / 56. 22.100
      libavcodec     58. 35.100 / 58. 35.100
      libavformat    58. 20.100 / 58. 20.100
      libavdevice    58.  5.100 / 58.  5.100
      libavfilter     7. 40.101 /  7. 40.101
      libswscale      5.  3.100 /  5.  3.100
      libswresample   3.  3.100 /  3.  3.100
      libpostproc    55.  3.100 / 55.  3.100
    [bmp @ 0268fcc0] not enough data (194700 < 2764854), trying to decode anyway
    [bmp @ 0268fcc0] not enough data (194646 < 2764800)
    Input #0, image2pipe, from 'pipe:.bmp':
      Duration: N/A, bitrate: N/A
        Stream #0:0: Video: bmp, bgr24, 1280x720, 25 tbr, 25 tbn, 25 tbc
    Codec AVOption rc_lookahead (Number of frames to look ahead for alternate reference frame selection) specified for output file #0 (g:\test.mp4) has not been used for any stream. The most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some encoder which was not actually used for any stream.
    Stream mapping:
      Stream #0:0 -> #0:0 (bmp (native) -> h264 (libx264))
    [bmp @ 02690240] not enough data (194700 < 2764854), trying to decode anyway
    [bmp @ 02690240] not enough data (194646 < 2764800)
    Error while decoding stream #0:0: Invalid data found when processing input
    [libx264 @ 0267fc80] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
    [libx264 @ 0267fc80] profile High 4:4:4 Predictive, level 3.1, 4:4:4, 8-bit
    [libx264 @ 0267fc80] 264 - core 157 r2935 545de2f - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=25.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
    Output #0, mp4, to 'g:\test.mp4':
      Metadata:
        encoder         : Lavf58.20.100
        Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv444p, 1280x720, q=-1--1, 25 fps, 12800 tbn, 25 tbc
        Metadata:
          encoder         : Lavc58.35.100 libx264
        Side data:
          cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    frame=    0 fps=0.0 q=0.0 Lsize=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
    video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
    Conversion failed!
    any help please?

    thanks in advance..

    best regards

  2. #2

    Thread Starter
    Hyperactive Member
    Join Date
    Dec 2009
    Posts
    337

    Re: VB.NET Streaming Image Using Imag2Pipe Issue

    any help? any one?

  3. #3
    Sinecure devotee
    Join Date
    Aug 2013
    Location
    Southern Tier NY
    Posts
    5,911

    Re: VB.NET Streaming Image Using Imag2Pipe Issue

    Perhaps the problem is using the Save method.
    I haven't messed with this, but it seems like a lot of effort with various unnecessary conversions, since it looks like you just want to write the bitmap data to the stream, so I don't see a reason to read the data into a bitmap and then save it, since it should already be a bitmap in the file.
    Since I haven't done it, I could be completely wrong though.
    But, if I were to try it, what I would have tried would be to read the bytes from the file, and then write the bytes to the stream, something along the lines of:
    Code:
        For Each strEachFile In strFiles
          Me.Text = System.IO.Path.GetFileNameWithoutExtension(strEachFile)
          Dim FileContents As Byte() = IO.File.ReadAllBytes(strEachFile)
          o1.Write(FileContents, 0, FileContents.Length)
        Next
    I don't know if that would make any difference, but just sending the contents of all the files to the o1 stream would seem like what you are trying to do. Whether doing that is valid, I have no idea.
    "Anyone can do any amount of work, provided it isn't the work he is supposed to be doing at that moment" Robert Benchley, 1930

  4. #4

    Thread Starter
    Hyperactive Member
    Join Date
    Dec 2009
    Posts
    337

    Re: VB.NET Streaming Image Using Imag2Pipe Issue

    Quote Originally Posted by passel View Post
    ....
    I haven't messed with this, but it seems like a lot of effort with various unnecessary conversions, since it looks like you just want to write the bitmap data to the stream, so I don't see a reason to read the data into a bitmap and then save it, since it should already be a bitmap in the file.
    .....
    thanks for your reply sir, but the problem above code is just for example, as in real project bitmap object will be used and many things will be done on it. bitmap will not untouched after read from image file? hope that make sense?

    however.. thanks for your help/reply though..

    best regards

  5. #5
    Frenzied Member
    Join Date
    Jul 2011
    Location
    UK
    Posts
    1,282

    Re: VB.NET Streaming Image Using Imag2Pipe Issue

    You are saving your BitMap to the StandardInput stream with
    o2.Save(o1.BaseStream, System.Drawing.Imaging.ImageFormat.Bmp)

    If you look at the remarks section in the documentation for the BitMap.Save(Stream, ImageFormat) method you'll see it states:
    The image must be saved to the stream at an offset of zero. If any additional data has been written to the stream before saving the image, the image data in the stream will be corrupted.
    That could well be the cause of your problem.

    You could try saving the bitmap to a new MemoryStream object first, and then writing the contents of that memory stream to the StandardInput stream.


    I'll also mention that while testing this, I found that ffmpeg would freeze after writing several hundred images to the stream. The number of images before the freeze varied on each run; anywhere between 100 and 500 in my tests.

  6. #6
    Frenzied Member
    Join Date
    Jul 2011
    Location
    UK
    Posts
    1,282

    Re: VB.NET Streaming Image Using Imag2Pipe Issue

    Thinking about the freezing, it occurred to me that ffmpeg is probably outputting a lot of information to stderr as it processes the images. You only read that output data at the end of the process with:
    Code:
    objFProcess.WaitForExit()
    strCLIText = objFProcess.StandardError.ReadToEnd
    (By the way, those two lines are in the wrong order. You should call ReadToEnd before WaitForExit in order to avoid a deadlock. See the documentation for the Process.StandardError Property for more info.)

    Chances are that ffmpeg's output buffer is becoming full at which point it freezes and waits for you to relieve the pressure by reading data from the buffer.

    I think that theory is correct as you don't get the freezing if you handle the ErrorDataReceived event and read from StandardError as data becomes available.

  7. #7

    Thread Starter
    Hyperactive Member
    Join Date
    Dec 2009
    Posts
    337

    Re: VB.NET Streaming Image Using Imag2Pipe Issue

    Quote Originally Posted by Inferrd View Post
    Thinking about the freezing, it occurred to me that ffmpeg is probably outputting a lot of information to stderr as it processes the images. You only read that output data at the end of the process with.......
    first of all thanks a lot for your tries to help me out..

    1. sir, i don't think tehre isn't a proper way to pass thousands of images (every single frame) as pipe/stream to ffmepg, ffmpeg is very well known and it's been running for many years. so, i hope it support that function very well.

    2. sir, if i try with sir passel methods, then it seems it pass it pass the data but unfortunately it freeze even i try with only 50 image, but in real case i have to handle thousands of frames/images.

    3. after your last note about reading all data at end (StandardError), i have even tried skipping that part but it still freezes

    any help please?

    thanks in advance..

  8. #8

    Thread Starter
    Hyperactive Member
    Join Date
    Dec 2009
    Posts
    337

    Re: VB.NET Streaming Image Using Imag2Pipe Issue

    Quote Originally Posted by Inferrd View Post
    Thinking about the freezing,
    ...
    thanks a lot sir, you point me to the right direction..

    the main culprit StandardError, after you have pointed out, i tried to skipping following line:

    Code:
    Rem strCLIText = objFProcess.StandardError.ReadToEnd
    but that doesn't solve the problem..

    then i tried to handle the StandardError in a handler event.. using flowing codes:

    Code:
    	objFProcess.StartInfo.RedirectStandardError = True
    	AddHandler objFProcess.ErrorDataReceived, AddressOf consoleErrorHandler
    
        rem Catch the Error Output
        Private Sub consoleErrorHandler(ByVal sendingProcess As Object, ByVal errLine As DataReceivedEventArgs)
            If Not String.IsNullOrEmpty(errLine.Data) Then
                lstCLIErr.Add(errLine.Data)
            End If
        End Sub
    you see, i tried to get the data as it comes, but that also doesn't solve the problem..

    and then finally i tried with not redirecting StandardError, and that works like charm, even i tried with 900+ images/frames and that does not freeze at all

    thanks a lot for pointing me to right direction..

    but sir, now only a slight drawback with my solution, as i am not redirecting StandardError, i can't know what is happening in case there is any error

    can you please figure out a way to handle the standard error so that i can also have that data without freezing?

    once again thanks..

    best regards

  9. #9
    Frenzied Member
    Join Date
    Jul 2011
    Location
    UK
    Posts
    1,282

    Re: VB.NET Streaming Image Using Imag2Pipe Issue

    Quote Originally Posted by Shohag_ifas View Post
    1. sir, i don't think tehre isn't a proper way to pass thousands of images (every single frame) as pipe/stream to ffmepg, ffmpeg is very well known and it's been running for many years. so, i hope it support that function very well.
    I wasn't implying the fault was with ffmpeg. I was saying that ffmpeg was freezing because your code wasn't properly handling the output from ffmpeg, especially when processing many hundreds of images.


    Quote Originally Posted by Shohag_ifas View Post
    the main culprit StandardError, after you have pointed out, i tried to skipping following line:
    Code:
    Rem strCLIText = objFProcess.StandardError.ReadToEnd
    but that doesn't solve the problem..

    then i tried to handle the StandardError in a handler event.. using flowing codes:

    Code:
    	objFProcess.StartInfo.RedirectStandardError = True
    	AddHandler objFProcess.ErrorDataReceived, AddressOf consoleErrorHandler
    
        rem Catch the Error Output
        Private Sub consoleErrorHandler(ByVal sendingProcess As Object, ByVal errLine As DataReceivedEventArgs)
            If Not String.IsNullOrEmpty(errLine.Data) Then
                lstCLIErr.Add(errLine.Data)
            End If
        End Sub
    you see, i tried to get the data as it comes, but that also doesn't solve the problem..
    can you please figure out a way to handle the standard error so that i can also have that data without freezing?
    Yeah, commenting out objFProcess.StandardError.ReadToEnd won't help because you actually need something to read from standard error before it fills up and so make space for ffmpeg to write more data to it. You just need to read the data while it is being written, i.e. not wait until the end of the process.

    Handling the ErrorDataReceived event should do that for you. However, it's not enough to just register the event handler (AddHandler objFProcess.ErrorDataReceived, AddressOf consoleErrorHandler), you also have to call objFProcess.BeginErrorReadLine() in order to start the asynchronous reads:
    Code:
    AddHandler objFProcess.ErrorDataReceived, AddressOf consoleErrorHandler
    
    objFProcess.Start()
    
    objFProcess.BeginErrorReadLine()
    Ok, so based on your original code, the following should allow you to process thousands of images and log the output from standard error:
    VB.NET Code:
    1. Dim objFProcess As System.Diagnostics.Process
    2. Dim strFiles() As String
    3. Dim strEachFile As String
    4.  
    5. objFProcess = New System.Diagnostics.Process
    6. objFProcess.StartInfo.FileName = txtFMPEG.Text
    7. objFProcess.StartInfo.Arguments = "-y -f image2pipe -r 25 -s 1280x720 -vcodec bmp -i pipe:.bmp -crf 25.0 -vcodec libx264 -vf scale=1280:720 -coder 1 -rc_lookahead 60 -threads 0 g:\test.mp4"
    8. objFProcess.StartInfo.UseShellExecute = False
    9. objFProcess.StartInfo.CreateNoWindow = True
    10. objFProcess.StartInfo.WindowStyle = ProcessWindowStyle.Hidden
    11. objFProcess.StartInfo.RedirectStandardError = True
    12. objFProcess.StartInfo.RedirectStandardInput = True
    13.  
    14. AddHandler objFProcess.ErrorDataReceived, AddressOf consoleErrorHandler
    15.  
    16. objFProcess.Start()
    17.  
    18. 'initialise List used to store lines from StandardError (in Sub consoleErrorHandler)
    19. 'and start listening to StandardError
    20. lstCLIErr = New List(Of String)
    21. objFProcess.BeginErrorReadLine()
    22.  
    23.  
    24. strFiles = System.IO.Directory.GetFiles(txtIMGDir.Text, "*.bmp", IO.SearchOption.TopDirectoryOnly)
    25. For Each strEachFile In strFiles
    26.     Me.Text = System.IO.Path.GetFileNameWithoutExtension(strEachFile)
    27.     My.Application.DoEvents()
    28.  
    29.     'load bitmap from file
    30.     Using o2 As New System.Drawing.Bitmap(strEachFile)
    31.  
    32.         ' ##########################
    33.         'alter bitmap as needed here
    34.         ' ##########################
    35.  
    36.  
    37.         'save bitmap to memorystream first,
    38.         'then write MemoryStream to StandardInput's stream.
    39.         'We can't use the BitMap.Save(stream, format) method
    40.         'to write directly to the StandardInput stream
    41.         'because that method MUST write to position 0 in the stream.
    42.         'StandardInput stream won't be at postion 0
    43.         'when we need to save images to it.
    44.         Using ms As New MemoryStream
    45.             o2.Save(ms, System.Drawing.Imaging.ImageFormat.Bmp)
    46.             ms.WriteTo(objFProcess.StandardInput.BaseStream)
    47.         End Using ' ms
    48.  
    49.     End Using ' o2
    50.  
    51. Next
    52.  
    53.  
    54. 'finished writing images to StandarInput
    55. objFProcess.StandardInput.Close()
    56.  
    57. objFProcess.WaitForExit()
    58.  
    59. objFProcess.Close()
    60. objFProcess.Dispose()
    61.  
    62.  
    63. ' #############################################
    64. 'lstCLIErr now holds output from standard error
    65. ' #############################################

    The changes I made are:
    1. Got rid of BinaryWriter (o1) as not being used in code given.
    2. Create BitMap directly from file instead of explicitly creating a FileStream and then using Image.FromStream method.
    3. Wrapped BitMap creation in Using...End Using block so it is disposed of properly.
    4. Save Bitmap to MemoryStream then write MemoryStream to StandardInput's Stream as discussed above in Post#5.
    5. Removed call to StandardError.ReadToEnd because we are now handling the ErrorDataReceived event.
    6. Added line to close StandardInput when all images have been written. Was previously being closed when the BinaryWriter was closed.


    You should think about getting rid of the call to My.Application.DoEvents(). It can often cause you unexpected problems. Using a BackgroundWorker might be a relatively simple alternative, or an awaitable Task would be preferred and is even simpler to code at a basic level (but can be a concept that is more difficult to wrap your head around).
    Last edited by Inferrd; Oct 22nd, 2019 at 05:41 AM.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Featured


Click Here to Expand Forum to Full Width