Results 1 to 10 of 10

Thread: MySQL & ADO queries speed

  1. #1

    Thread Starter
    Member Dragokas's Avatar
    Join Date
    Aug 2015
    Location
    Ukraine
    Posts
    740

    MySQL & ADO queries speed

    Hi,

    why the speed is so low ~ 12 queries per sec.?

    I'm uploading ~ 5000 records doing sequential calls of ADODB.Command's Execute from this snippet.

    Yes, I'm re-using the existing connection (oConnect object above).
    Yes, I have good internet upload speed & ping.

    Is it due to Sync call? Does ADO have Async alternative?

    Or should I pack everything in a transaction?

    Thanks.
    Malware analyst, VirusNet developer, HiJackThis+ author || my CodeBank works

  2. #2
    PowerPoster Zvoni's Avatar
    Join Date
    Sep 2012
    Location
    To the moon and then left
    Posts
    4,444

    Re: MySQL & ADO queries speed

    You should always try to bulk-upload, and packing multiple SQL's into a Transaction is never a bad idea (Rollback!).

    Though, i have no idea if bulk-upload with parameters is possible.
    Last edited by Zvoni; Tomorrow at 31:69 PM.
    ----------------------------------------------------------------------------------------

    One System to rule them all, One Code to find them,
    One IDE to bring them all, and to the Framework bind them,
    in the Land of Redmond, where the Windows lie
    ---------------------------------------------------------------------------------
    People call me crazy because i'm jumping out of perfectly fine airplanes.
    ---------------------------------------------------------------------------------
    Code is like a joke: If you have to explain it, it's bad

  3. #3
    PowerPoster Arnoutdv's Avatar
    Join Date
    Oct 2013
    Posts
    5,906

    Re: MySQL & ADO queries speed

    Wrap them in a transaction, this should improve speed.
    https://docs.microsoft.com/en-us/sql...l-server-ver15

    An alternative could be BatchUpdate:
    https://docs.microsoft.com/en-us/sql...l-server-ver15

  4. #4

    Thread Starter
    Member Dragokas's Avatar
    Join Date
    Aug 2015
    Location
    Ukraine
    Posts
    740

    Re: MySQL & ADO queries speed

    Arnoutdv, thank you for the examples!

    How that "batch" works? Is that similar to the transaction, but without a rollback feature and not breaking at first failed commit?
    Malware analyst, VirusNet developer, HiJackThis+ author || my CodeBank works

  5. #5
    PowerPoster Zvoni's Avatar
    Join Date
    Sep 2012
    Location
    To the moon and then left
    Posts
    4,444

    Re: MySQL & ADO queries speed

    Quote Originally Posted by Dragokas View Post
    Arnoutdv, thank you for the examples!

    How that "batch" works? Is that similar to the transaction, but without a rollback feature and not breaking at first failed commit?
    Don't think so, since UpdateBatch is a Method of the ADODB.RecordSet-Object
    Basically it works like this:
    RecordSet is (implicitely?) set into "Edit"-mode (for lack of better word)
    You loop through the RecordSet(s), and edit Fields you want to edit

    Calling UpdateBatch --> Commit
    Calling CancelBatch --> Rollback

    Never used it, but should also work with Add (or is it Append?)

    Disadvantage: You don't use SQL, but you "edit" your RecordSet-Fields directly, so in that way: No sanitizing like if you use Parameters in a SQL-Statement
    Last edited by Zvoni; Tomorrow at 31:69 PM.
    ----------------------------------------------------------------------------------------

    One System to rule them all, One Code to find them,
    One IDE to bring them all, and to the Framework bind them,
    in the Land of Redmond, where the Windows lie
    ---------------------------------------------------------------------------------
    People call me crazy because i'm jumping out of perfectly fine airplanes.
    ---------------------------------------------------------------------------------
    Code is like a joke: If you have to explain it, it's bad

  6. #6
    PowerPoster
    Join Date
    Feb 2006
    Posts
    24,482

    Re: MySQL & ADO queries speed

    Not every DDBMS has a BULK INSERT or similar thing. But there are still things you can do to achieve better performance.

    Here's a demo (Project1) that creates a Jet MDB, creates one table, and then "firehose writes" 1000 rows and exits.

    Excerpt:

    Code:
        Dim Connection As ADODB.Connection
        Dim FieldNames As Variant
        Dim I As Long
    
        ChDir App.Path
        ChDrive App.Path
        LoadPics
        With CreateObject("ADOX.Catalog") 'Late-bind ADOX because unlike ADODB it doesn't
                                          'maintain binary compatibility between Windows
                                          'versions.
            .Create "Provider=Microsoft.Jet.OLEDB.4.0;Jet OLEDB:Engine Type=5;" _
                  & "Mode=Share Exclusive;Data Source='Demo.mdb'"
            Set Connection = .ActiveConnection
        End With
        Connection.Execute "CREATE TABLE `Data`(" _
                         & "`ID` IDENTITY CONSTRAINT pkData PRIMARY KEY," _
                         & "`TextData` VARCHAR(255) WITH COMP NOT NULL," _
                         & "`PicData` IMAGE NOT NULL)", _
                           , _
                           adCmdText Or adExecuteNoRecords
        FieldNames = Array("TextData", "PicData")
        With New ADODB.Recordset
            .CursorLocation = adUseServer
            .Open "Data", Connection, adOpenForwardOnly, adLockOptimistic, adCmdTableDirect
            For I = 1 To NUM_ROWS
                .AddNew FieldNames, Array(NewText(), NewPicture())
            Next
            .Close
        End With
        Connection.Close
        MsgBox "Data table in Demo.mdb created"
    Opening the database exclusively isn't always practical, but when you can do it you bypass a lot of locking and its overhead. Not every DMBS supports adCmdTableDirect either.


    There is also a Project2 included. This is a simple "browser" for stepping through the rows in the created database table. Notably, it has no code at all.
    Attached Files Attached Files

  7. #7
    PowerPoster
    Join Date
    Feb 2006
    Posts
    24,482

    Re: MySQL & ADO queries speed

    How much of that carries over to MySQL? I'm not sure who knows, I can't find documentation about it anywhere.

  8. #8
    Addicted Member
    Join Date
    Jul 2016
    Posts
    230

    Re: MySQL & ADO queries speed

    Quote Originally Posted by Dragokas View Post
    why the speed is so low ~ 12 queries per sec.?

    I'm uploading ~ 5000 records doing sequential calls
    I tested my own code. 5000 INSERTs using sequential calls to MariaDB running on localhost took 11465ms, that's 436 INSERT queries per second.

    Quote Originally Posted by Dragokas View Post
    Yes, I have good internet upload speed & ping.
    Test on localhost to eliminate external factors.

    Example with parameterized queries (I have since simplified the code as I understood the process better, but it will do for an example):
    https://www.vbforums.com/showthread....=1#post5479913

    Quote Originally Posted by Dragokas View Post
    Or should I pack everything in a transaction?
    Transactions won't help you with speed, they're for data integrity.
    https://mariadb.com/kb/en/transactions/
    Last edited by OldClock; Jun 7th, 2021 at 04:12 AM.

  9. #9

    Thread Starter
    Member Dragokas's Avatar
    Join Date
    Aug 2015
    Location
    Ukraine
    Posts
    740

    Re: MySQL & ADO queries speed

    Zvoni, thanks for an explanation!

    dilettante, thanks for code, I'll try MySQL connection string for ADOX a bit later and report.

    OldClock, I'm sure localhost will be much faster, but I need it for an external DB.
    Malware analyst, VirusNet developer, HiJackThis+ author || my CodeBank works

  10. #10
    PowerPoster
    Join Date
    Feb 2006
    Posts
    24,482

    Re: MySQL & ADO queries speed

    Quote Originally Posted by Dragokas View Post
    dilettante, thanks for code, I'll try MySQL connection string for ADOX a bit later and report.
    The ADOX portion of my example was just used to create a new Jet database. It only works for Jet and ACE.

    The code after creating the database and table might work though.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width