Results 1 to 16 of 16

Thread: [RESOLVED] How much can I load up into the DOM?

  1. #1

    Thread Starter
    MS SQL Powerposter szlamany's Avatar
    Join Date
    Mar 2004
    Location
    Connecticut
    Posts
    18,263

    Resolved [RESOLVED] How much can I load up into the DOM?

    Wow - thie jQuery section of the forum surely appears dead - no posts since Oct 1st - hope someone reads this!!

    At any rate - with jQuery it's become common for me to download objects of data and stash them somewhere in the DOM and then re-use them for auto-completes or dropdowns or slickgrid sources.

    I've got experience doing 2000 names in an auto-complete drop down - which means I stored all that data somewhere in the page for use as the page ajaxes away.

    How about doing 30,000 names in an auto-complete? I'm not concerned about the speed of the auto-complete - really worried about the available storage space in the browser apps out there and wondering if anyone has pushed a large amount to a browser and found a "breaking point" so to say.

    Thanks!

    *** Read the sticky in the DB forum about how to get your question answered quickly!! ***

    Please remember to rate posts! Rate any post you find helpful - even in old threads! Use the link to the left - "Rate this Post".

    Some Informative Links:
    [ SQL Rules to Live By ] [ Reserved SQL keywords ] [ When to use INDEX HINTS! ] [ Passing Multi-item Parameters to STORED PROCEDURES ]
    [ Solution to non-domain Windows Authentication ] [ Crazy things we do to shrink log files ] [ SQL 2005 Features ] [ Loading Pictures from DB ]

    MS MVP 2006, 2007, 2008

  2. #2
    Freelancer akhileshbc's Avatar
    Join Date
    Jun 2008
    Location
    Trivandrum, Kerala, India
    Posts
    7,652

    Re: How much can I load up into the DOM?

    Are you talking about the amount of data that can be stored using JavaScript ? If so, I think there is no limit. Since, it would depend on the user's PC.

    The more the number of elements or data, the more the RAM it would consume. And it might pose a lag in user's machine.

    That's what I think.

    If my post was helpful to you, then express your gratitude using Rate this Post.
    And if your problem is SOLVED, then please Mark the Thread as RESOLVED (see it in action - video)
    My system: AMD FX 6100, Gigabyte Motherboard, 8 GB Crossair Vengance, Cooler Master 450W Thunder PSU, 1.4 TB HDD, 18.5" TFT(Wide), Antec V1 Cabinet

    Social Group: VBForums - Developers from India


    Skills: PHP, MySQL, jQuery, VB.Net, Photoshop, CodeIgniter, Bootstrap,...

  3. #3
    Frenzied Member tr333's Avatar
    Join Date
    Nov 2004
    Location
    /dev/st0
    Posts
    1,605

    Re: How much can I load up into the DOM?

    I don't see any problems occurring on a desktop browser, but I would expect to see problems if this is on a device with low memory, eg. Smartphone or tablet device.

    If you're using Web storage then you can run into quota limits which are defined by the client browser, but these are usually at least a few megabytes.

    I guess the easiest way to find out is to just run some tests, loading up X objects into the dom and increase X until you start seeing problems. It's going to be different for every browser/machine that you're running on due to differing client hardware specs.
    Last edited by tr333; Nov 1st, 2012 at 06:19 PM.
    CSS layout comes in to the 21st century with flexbox!
    Just another Perl hacker,

  4. #4
    Hyperactive Member xxarmoxx's Avatar
    Join Date
    Mar 2007
    Posts
    378

    Re: How much can I load up into the DOM?

    I'm sure there is a tutorial out there for this. My hunch is NOT to store 30,000 items in the browser. I would make a new ajax call after each keystroke and display the results in the autocomplete.

  5. #5

    Thread Starter
    MS SQL Powerposter szlamany's Avatar
    Join Date
    Mar 2004
    Location
    Connecticut
    Posts
    18,263

    Re: How much can I load up into the DOM?

    Have you used the standard jquery UI autocomplete - where you type some letters and it pulls from the DOM to display matches? Very wicked fast.

    I've done 1000, 2000 or so names in a lookup like this.

    I was looking for people who have gone down this path - but with 30,000 or 50,000 rows of data - and looking for experience in this area. Seems no one has offered such - I'll eventually get to the point where I test this with some users out in the field and hopefully get back here with some meaningful results.

    *** Read the sticky in the DB forum about how to get your question answered quickly!! ***

    Please remember to rate posts! Rate any post you find helpful - even in old threads! Use the link to the left - "Rate this Post".

    Some Informative Links:
    [ SQL Rules to Live By ] [ Reserved SQL keywords ] [ When to use INDEX HINTS! ] [ Passing Multi-item Parameters to STORED PROCEDURES ]
    [ Solution to non-domain Windows Authentication ] [ Crazy things we do to shrink log files ] [ SQL 2005 Features ] [ Loading Pictures from DB ]

    MS MVP 2006, 2007, 2008

  6. #6
    Frenzied Member tr333's Avatar
    Join Date
    Nov 2004
    Location
    /dev/st0
    Posts
    1,605

    Re: How much can I load up into the DOM?

    Quote Originally Posted by szlamany View Post
    I was looking for people who have gone down this path - but with 30,000 or 50,000 rows of data - and looking for experience in this area. Seems no one has offered such - I'll eventually get to the point where I test this with some users out in the field and hopefully get back here with some meaningful results.
    I think that's the only real way to find out. Do lots of testing and see what you get out of it.
    CSS layout comes in to the 21st century with flexbox!
    Just another Perl hacker,

  7. #7
    Hyperactive Member xxarmoxx's Avatar
    Join Date
    Mar 2007
    Posts
    378

    Re: How much can I load up into the DOM?

    Yes I have seen the jquery autocomplete, and yes they use an array that has data in it, but that is only good for small data sets. You should make an ajax call that sets that array. The ajax call will return a small subset of the data making that array very small. Its the only way to do this for large data sets. Just use ajax that returns JSON and set that array with the JSON.

    http://stackoverflow.com/questions/3...on-large-array

  8. #8
    Hyperactive Member xxarmoxx's Avatar
    Join Date
    Mar 2007
    Posts
    378

    Re: How much can I load up into the DOM?

    Make sure your server call is fast and make sure your query is optimized, it should work well if you make that happen.

  9. #9

    Thread Starter
    MS SQL Powerposter szlamany's Avatar
    Join Date
    Mar 2004
    Location
    Connecticut
    Posts
    18,263

    Re: How much can I load up into the DOM?

    Ok - finally got around to loading this town's census data - 43,000 names of people with a corresponding CensusId.

    All of this loads at "boot" time for the page (initial ajax data call after un/pw authenication). This particular set of JSON data is 2.5 MB - and takes 1.6 seconds locally running here in the office on development machines.

    The jQuery autocomplete is handing 43,000 entries perfectly - I already extended it a long time ago to tell you to be more specific if more than 200 rows were being considered for display - so until then you simply get a record count of how many it's waiting to get under...

    Code:
    $.extend($.ui.autocomplete, {
        escapeRegex: function(value) {
            return value.replace(/[-[\]{}()*+?.,\\^$|#\s]/g, "\\$&");
        },
        filter: function(array, term) {
            var matcher = new RegExp($.ui.autocomplete.escapeRegex(term), "i");
            var arraytoreturn = $.grep(array, function(value) {
                return matcher.test(value.label || value.value || value);
            });
            if (arraytoreturn.length > 200) {
                arraytoreturn = [{ label: "Please be more specific - " + arraytoreturn.length + " matching entries!", value: ""}];
            }
            return arraytoreturn;
        }
    });
    I'll give further timings after we get the page up on a public facing server...

    I also want to try running this with a browser on a low-end / low-memory machine and see if eating 2.5 MB of Javascript object space is a bad thing...

    *** Read the sticky in the DB forum about how to get your question answered quickly!! ***

    Please remember to rate posts! Rate any post you find helpful - even in old threads! Use the link to the left - "Rate this Post".

    Some Informative Links:
    [ SQL Rules to Live By ] [ Reserved SQL keywords ] [ When to use INDEX HINTS! ] [ Passing Multi-item Parameters to STORED PROCEDURES ]
    [ Solution to non-domain Windows Authentication ] [ Crazy things we do to shrink log files ] [ SQL 2005 Features ] [ Loading Pictures from DB ]

    MS MVP 2006, 2007, 2008

  10. #10

    Thread Starter
    MS SQL Powerposter szlamany's Avatar
    Join Date
    Mar 2004
    Location
    Connecticut
    Posts
    18,263

    Re: How much can I load up into the DOM?

    Quote Originally Posted by xxarmoxx View Post
    Yes I have seen the jquery autocomplete, and yes they use an array that has data in it, but that is only good for small data sets. You should make an ajax call that sets that array. The ajax call will return a small subset of the data making that array very small. Its the only way to do this for large data sets. Just use ajax that returns JSON and set that array with the JSON.

    http://stackoverflow.com/questions/3...on-large-array
    Have you done lots of ajax calls for autocomplete's? How is the performance like going over the wire as you are typing characters? Does it appear instant??

    Do you set it for a minimum number of characters before you start polling the service to get you matching data?

    I've only ever done autocompletes with data that was loaded initially and just held for use later as needed.

    *** Read the sticky in the DB forum about how to get your question answered quickly!! ***

    Please remember to rate posts! Rate any post you find helpful - even in old threads! Use the link to the left - "Rate this Post".

    Some Informative Links:
    [ SQL Rules to Live By ] [ Reserved SQL keywords ] [ When to use INDEX HINTS! ] [ Passing Multi-item Parameters to STORED PROCEDURES ]
    [ Solution to non-domain Windows Authentication ] [ Crazy things we do to shrink log files ] [ SQL 2005 Features ] [ Loading Pictures from DB ]

    MS MVP 2006, 2007, 2008

  11. #11
    Serge's Avatar
    Join Date
    Feb 1999
    Location
    Scottsdale, Arizona, USA
    Posts
    2,744

    Re: How much can I load up into the DOM?

    I've used autocomplete with 500k user records (ajax calls). With proper indexes in the table, it works very well even if you set minChars: 1.

  12. #12
    Frenzied Member tr333's Avatar
    Join Date
    Nov 2004
    Location
    /dev/st0
    Posts
    1,605

    Re: How much can I load up into the DOM?

    If you "really" want to speed up things...

    I noticed a huge speed increase when switching from WebForms to MVC, because the amount of data sent on each request/response was reduced by not having to send through all the extra stuff that WebForms uses (ajax script methods, viewstate, etc.). Also, the new Bundling and Optimization stuff in MVC4 is a real winner. Moving all your scripts into external .js files allows you to concatenate/minify them to save loading time and also allows you to cache them properly, which a dynamically loaded .cshtml or .aspx view/page can't do as easily.

    Check out the web.config from the HTML5 Boilerplate template that includes settings for high levels of caching of static assets. Other useful tools include yslow and pagespeed.
    CSS layout comes in to the 21st century with flexbox!
    Just another Perl hacker,

  13. #13

    Thread Starter
    MS SQL Powerposter szlamany's Avatar
    Join Date
    Mar 2004
    Location
    Connecticut
    Posts
    18,263

    Re: How much can I load up into the DOM?

    Quote Originally Posted by tr333 View Post
    If you "really" want to speed up things...
    ...
    Other useful tools include yslow
    Point #15 - under "Web Performance Best Practices and Rules" of that link - "Make AJAX Cacheable" - is what this thread is all about.

    And I can happily report that you can in fact cache a huge amount of SOURCE data in the DOM at page-load time - making AJAX-calls for auto-complete's unneeded.

    I've got four autocompletes that each load 25,000+ rows of data - takes a couple of seconds to initially load (but the user is unaware - as it's an AJAX call to even load the initial data!).

    From that point forward the user can access those auto-completes and get full functionality without revisiting the server for appropriate data. Actually when I display a new bit of DOM with a similar autocomplete - that also needs that source data - I can effortlessly point to that object that is already in the DOM.

    I am going to mark this thread resolved

    *** Read the sticky in the DB forum about how to get your question answered quickly!! ***

    Please remember to rate posts! Rate any post you find helpful - even in old threads! Use the link to the left - "Rate this Post".

    Some Informative Links:
    [ SQL Rules to Live By ] [ Reserved SQL keywords ] [ When to use INDEX HINTS! ] [ Passing Multi-item Parameters to STORED PROCEDURES ]
    [ Solution to non-domain Windows Authentication ] [ Crazy things we do to shrink log files ] [ SQL 2005 Features ] [ Loading Pictures from DB ]

    MS MVP 2006, 2007, 2008

  14. #14
    Serge's Avatar
    Join Date
    Feb 1999
    Location
    Scottsdale, Arizona, USA
    Posts
    2,744

    Re: How much can I load up into the DOM?

    Quote Originally Posted by szlamany View Post
    Point #15 - under "Web Performance Best Practices and Rules" of that link - "Make AJAX Cacheable" - is what this thread is all about.

    And I can happily report that you can in fact cache a huge amount of SOURCE data in the DOM at page-load time - making AJAX-calls for auto-complete's unneeded.

    I've got four autocompletes that each load 25,000+ rows of data - takes a couple of seconds to initially load (but the user is unaware - as it's an AJAX call to even load the initial data!).

    From that point forward the user can access those auto-completes and get full functionality without revisiting the server for appropriate data. Actually when I display a new bit of DOM with a similar autocomplete - that also needs that source data - I can effortlessly point to that object that is already in the DOM.

    I am going to mark this thread resolved
    That goes against all rules in web page development. Your initial load will be slow(er). Also if you have a public website, you will be heavily penalized by search engines. Your rank is also based on the speed of your pages as well. What you should do is to cache your ajax calls on the server side.

  15. #15

    Thread Starter
    MS SQL Powerposter szlamany's Avatar
    Join Date
    Mar 2004
    Location
    Connecticut
    Posts
    18,263

    Re: [RESOLVED] How much can I load up into the DOM?

    My customers are running things like CENSUS and REAL ESTATE TAX collection apps here - all https secure - no need to worry about how my "page" gets ranked by search engines.

    The app uses jQuery TAB's to allow you to open dozens, if you want, tabs of data - for editing. I put huge amounts of effort into making sure that the "source" of my drop downs and autocompletes all come from "single" cached copies in the browser. I might have a dozen dropdowns all pointing to the same data object in javascript.

    All of these data sources get downloaded via ajax calls made in the READY event of the page - so the user is unaware (actually if a data source is not loaded when they open a tab that dropdown will display "Loading..." and be disabled - it becomes enabled in the ajax callback).

    Your arguments against this method don't seem to fit my situation - althrough I welcome your opinion!

    *** Read the sticky in the DB forum about how to get your question answered quickly!! ***

    Please remember to rate posts! Rate any post you find helpful - even in old threads! Use the link to the left - "Rate this Post".

    Some Informative Links:
    [ SQL Rules to Live By ] [ Reserved SQL keywords ] [ When to use INDEX HINTS! ] [ Passing Multi-item Parameters to STORED PROCEDURES ]
    [ Solution to non-domain Windows Authentication ] [ Crazy things we do to shrink log files ] [ SQL 2005 Features ] [ Loading Pictures from DB ]

    MS MVP 2006, 2007, 2008

  16. #16
    Serge's Avatar
    Join Date
    Feb 1999
    Location
    Scottsdale, Arizona, USA
    Posts
    2,744

    Re: [RESOLVED] How much can I load up into the DOM?

    I understand your reasons but until you get those arrays loaded the user will just sit there and wait. Although it might take a couple of seconds and you might only get one array of 25k records to share against multiple autocompletes (it might serve your purposes), what happens when you need more just 1 data source?

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width