Page 1 of 2 12 LastLast
Results 1 to 40 of 62

Thread: [RESOLVED] StringBuilder vs Concatenation

  1. #1

    Thread Starter
    Hyperactive Member
    Join Date
    Nov 2017
    Location
    Buckingham, UK
    Posts
    260

    Resolved [RESOLVED] StringBuilder vs Concatenation

    Quote Originally Posted by jmcilhinney View Post
    Also, don't keep appending text to the control like that. Create a StringBuilder and create the whole text, then set the Text of the control in one go.
    For my education ...

    Whilst I would not repeatedly append text to a control, what I would normally do is to build a string variable (using repeated &=) and then assign the final string to the control. However, I accept that the documentation says that StringBuilder is more 'efficient' than &=.

    Whilst I cannot argue with what the documentation says, I am a little surprised, because I would have thought that, as far as the compiler (hence the compiled code) is concerned, string concatenation is string concatenation, regardless of what words/syntax have been used in the source code to bring it about. However, that aside, I wonder if, in practice, there is going to be any noticeable difference in 'efficiency' (which I presume means speed), other than perhaps in the most extreme of circumstances. I ask mainly because I personally find code using &= to be somewhat clearer to read.

    Kind Regards, John
    i

  2. #2
    Addicted Member
    Join Date
    Jul 2017
    Location
    Exeter, UK
    Posts
    180

    Re: Viewing whole datagridview to rich text box

    The point is that each time you append to a string variable using &=, a copy of the original is made in memory which will impact on the resource available.

  3. #3

    Thread Starter
    Hyperactive Member
    Join Date
    Nov 2017
    Location
    Buckingham, UK
    Posts
    260

    Re: Viewing whole datagridview to rich text box

    Quote Originally Posted by bmwpete View Post
    The point is that each time you append to a string variable using &=, a copy of the original is made in memory which will impact on the resource available.
    Thanks. Fair enough, although I don't really understand why that is necessary. In any event, my question remains - other than in the most extreme of situations, is that usually going to have any noticeable effect on anything?

    Kind Regards, John

  4. #4
    Addicted Member
    Join Date
    Jul 2017
    Location
    Exeter, UK
    Posts
    180

    Re: Viewing whole datagridview to rich text box

    I suppose it depends on what you mean by extreme situations. I would imagine MS introduced a method called 'StringBuilder' which concatenates string efficiently to reduce possible side effects to a minimum.

  5. #5

    Thread Starter
    Hyperactive Member
    Join Date
    Nov 2017
    Location
    Buckingham, UK
    Posts
    260

    Re: Viewing whole datagridview to rich text box

    Quote Originally Posted by bmwpete View Post
    I suppose it depends on what you mean by extreme situations. I would imagine MS introduced a method called 'StringBuilder' which concatenates string efficiently to reduce possible side effects to a minimum.
    Yes, one has to imagine/presume that such is the case, but that doesn't help me to understand the reasoning . There seem to be countless such methods etc. in VB.NET (which, at first sight, appear to be 'not really necessary') and I can but presume that the reasonings behind their existence are similar.

    By 'extreme situations', I meant situations in which the number of strings being concatenated was extremely large, which I would have thought was pretty unusual. Certainly for me, it would usually be 'a handful' or maybe 'a few dozen' and I can imagine that it might occasionally be hundreds, or even thousands for some people, with anything more very being pretty rare. In the case that caused jmc to make his comment, there were 10 'appendings', involving just over 30 components (strings or constants).

    Kind Regards, John

  6. #6
    Addicted Member
    Join Date
    Jul 2017
    Location
    Exeter, UK
    Posts
    180

    Re: Viewing whole datagridview to rich text box

    The reasoning seems to be covered in the first paragraph of the documentation, good enough for me anyway.

  7. #7
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    38,988

    Re: StringBuilder vs Concatenation

    I split this discussion out to its own thread, because it wasn't on topic, yet it was a topic in its own right, and a good one.

    A few years back (may have been several, by now), this discussion came up, and people set about testing the difference between StringBuilder and concatenation. I forget the exact number, but I seem to remember that SB had better performance if there were going to be more than about half a dozen concatenations.

    The issue is that strings can't change size. So, when you concatenate, you don't just add to one string, you create a new string of the combined size, then copy both parts into that string, and finally discard anything that needs discarding (literals don't get discarded, and string variables...may or may not). So, that simple & hides a whole series of steps, some of which can be a bit costly. If you are concatenating a whole series of strings together, you are allocating and discarding loads of memory blocks that are likely to be quite tiny. This has an immediate cost, but could, potentially, have a greater cost down the road if memory gets tight.

    What the SB is doing is essentially making a mutable length string. It can't really do that, since memory isn't so lazy, but that's the essence. This avoids a bunch of the memory thrashing that concatenation can require.

    As to visible performance, it likely makes no difference until you get to thousands, and quite likely tens of thousands, of concatenations. Put that code in the right place, and the fact that one takes a tenth of a second longer than the other, won't be noticeable. Put it inside a loop, and it will add up to real time. Do enough concatenations and it won't be 100 ms, either, but far more than that small number. Re-paint a control after each concatenation, and it could take real time, too, though that probably won't really happen.

    Anyways, that's just kind of rambling background. The rule of thumb I remember from the earlier discussion is that if you are getting up around half a dozen concatenations, then a StringBuilder will simply be more efficient. Below that, the time difference won't matter at all. Furthermore, even if you are getting above half a dozen, you have to decide whether the gain matters, because there are times when a tenth of a second can be wasted, and times when it can not.
    My usual boring signature: Nothing

  8. #8

    Thread Starter
    Hyperactive Member
    Join Date
    Nov 2017
    Location
    Buckingham, UK
    Posts
    260

    Re: Viewing whole datagridview to rich text box

    Quote Originally Posted by bmwpete View Post
    The reasoning seems to be covered in the first paragraph of the documentation, good enough for me anyway.
    As I said at the start, it simply says that it is 'more efficient'. Whilst I have to assume that that is correct, it gives me no indication of whether it is 'more efficient' to the extent of being noticeable (to me) and hence potentially overcoming my personal preference for the source code one has when not using StringBuilder. I think I'm probably going to do some experiments .

    Kind Regards, John

  9. #9
    Sinecure devotee
    Join Date
    Aug 2013
    Location
    Southern Tier NY
    Posts
    6,582

    Re: Viewing whole datagridview to rich text box

    It depends on the size of the string you're building and how you're building it.
    This was an issue even with VB6.

    I recall someone building up a page of 8000 characters for display, and getting the data to append to the string a few characters at a time, i.e. perhaps converting a lot of small numbers into strings and building up a table type display.

    They were complaining about the inordinate amount of time it took to create a simple 8000 character string.
    The problem is if you're appending say three characters at a time to create the 8000 character string, then you have to concatenate about 2666 times.
    Each time, you have to allocate 3 more characters of memory space on top of what you originally had, so lets start a list.
    Length = Allocated
    3 = 3
    6 = 9
    9 = 18
    12 = 30
    15 = 45
    ...

    Each time you concatenate three characters you have to allocate new memory the size of the current string plus three characters. (and the characters usually take two bytes per character, so the memory allocation is twice the number of characters).
    So, it doesn't seem like much, but if you write a quick test to see the numbers, lets seem how much character space has to be allocated and disposed of to reach our 8000 characters (give or take).

    note: this was done in VB6, so the Longs are actually 32-bit integers.
    Code:
      Dim t As Long
      Dim a As Long
      Dim i As Long
      For i = 1 To 2667
        t = t + 3
        a = a + t
      Next
      Debug.Print t, a
    
    'result    8001          10673334
    The result is we would create a string of 8001 characters, and have allocated (and disposed of most of) over 10.6 million characters of memory.
    That took a significant amount of time in VB6. Since Net deals with immutable strings all the time, it might be more efficient in allocating and freeing up large amounts of memory so not be as slow.

    The answer for VB6 was to allocate a string of say, ten thousand characters, and then use Mid$ to insert your characters into the string, rather than concatenate.

    For .Net, that strategy won't work. Strings are immutable, so you can't modify the string by inserting into the middle of it. If you do an insertion, it really copies the whole string, with insertion, into a new string, so is not any better than concatenation.

    The string builder object gets around that by preallocating space, and when you append, it inserts the appended characters into the buffer it preallocated. If you exceed the space allocated, then it creates a new buffer double the size, copies what you have to the front half, and starts "appending" into the buffer again.

    Depending on the initial size allocated, the string builder may not have to reallocate more string space at all to do all the concatenations you need.
    That is why it is there. Perhaps your testing will confirm this.
    Last edited by passel; Nov 26th, 2019 at 02:10 PM.
    "Anyone can do any amount of work, provided it isn't the work he is supposed to be doing at that moment" Robert Benchley, 1930

  10. #10
    Smooth Moperator techgnome's Avatar
    Join Date
    May 2002
    Posts
    34,531

    Re: StringBuilder vs Concatenation

    copies what you have to the front half,
    Are you sure about that? Seems like it would be simpler if it maintained a linked list of sorts, and only did the final concatenation during the .ToString call.
    Regardless, either method is definitely preferable to the constant allocating/deallocating/smashing memory doing concats.

    -tg
    * I don't respond to private (PM) requests for help. It's not conducive to the general learning of others.*
    * I also don't respond to friend requests. Save a few bits and don't bother. I'll just end up rejecting anyways.*
    * How to get EFFECTIVE help: The Hitchhiker's Guide to Getting Help at VBF - Removing eels from your hovercraft *
    * How to Use Parameters * Create Disconnected ADO Recordset Clones * Set your VB6 ActiveX Compatibility * Get rid of those pesky VB Line Numbers * I swear I saved my data, where'd it run off to??? *

  11. #11
    PowerPoster PlausiblyDamp's Avatar
    Join Date
    Dec 2016
    Location
    Pontypool, Wales
    Posts
    2,458

    Re: StringBuilder vs Concatenation

    Quote Originally Posted by techgnome View Post
    Are you sure about that? Seems like it would be simpler if it maintained a linked list of sorts, and only did the final concatenation during the .ToString call.
    Regardless, either method is definitely preferable to the constant allocating/deallocating/smashing memory doing concats.

    -tg
    From the source at https://source.dot.net/#q=stringbuilder
    // A StringBuilder is internally represented as a linked list of blocks each of which holds
    // a chunk of the string. It turns out string as a whole can also be represented as just a chunk,
    // so that is what we do.
    So it looks like a linked list it is

  12. #12

    Thread Starter
    Hyperactive Member
    Join Date
    Nov 2017
    Location
    Buckingham, UK
    Posts
    260

    Re: StringBuilder vs Concatenation

    Quote Originally Posted by Shaggy Hiker View Post
    I split this discussion out to its own thread, because it wasn't on topic, yet it was a topic in its own right, and a good one.
    Yes, thanks. I should have done that myself, but it was too late by the time I'd thought of that!

    Quote Originally Posted by Shaggy Hiker View Post
    A few years back (may have been several, by now), this discussion came up, and people set about testing the difference between StringBuilder and concatenation. I forget the exact number, but I seem to remember that SB had better performance if there were going to be more than about half a dozen concatenations.
    Interesting. As I intimated would be the case, I'm currently undertaking some experiments myself, and will report back if I discover anything particularly interesting!

    Quote Originally Posted by Shaggy Hiker View Post
    The issue is that strings can't change size. So, when you concatenate, you don't just add to one string, you create a new string of the combined size, then copy both parts into that string, and finally discard anything that needs discarding ....
    Fair enough. I suppose that makes sense. I'm most used to a programming environment in which one can (essentially has to) declare the length of a string variable - which (provided one knows what maximum size one might need!) perhaps enables it to simply 'add to one string'.

    Quote Originally Posted by Shaggy Hiker View Post
    What the SB is doing is essentially making a mutable length string. It can't really do that, since memory isn't so lazy, but that's the essence. This avoids a bunch of the memory thrashing that concatenation can require.
    However it does it, is there some reason why concatenation can't do the same?

    Quote Originally Posted by Shaggy Hiker View Post
    As to visible performance, it likely makes no difference until you get to thousands, and quite likely tens of thousands, of concatenations. Put that code in the right place, and the fact that one takes a tenth of a second longer than the other, won't be noticeable.
    That's obviously what I was suggesting.

    Quote Originally Posted by Shaggy Hiker View Post
    Put it inside a loop, and it will add up to real time. Do enough concatenations and it won't be 100 ms, either, but far more than that small number. ...
    Fair enough, and I suppose I wasn't think of that situation. However, what you say is a general truth, that if something is going to be repeated a very large number of times, one has to do all one can to optimise the performance (speed) of what it is that's been repeated. If the 'something' is only going to be executed once, or a fairly small number of times, that obviously become far less important.

    Quote Originally Posted by Shaggy Hiker View Post
    Anyways, that's just kind of rambling background. The rule of thumb I remember from the earlier discussion is that if you are getting up around half a dozen concatenations, then a StringBuilder will simply be more efficient. Below that, the time difference won't matter at all. Furthermore, even if you are getting above half a dozen, you have to decide whether the gain matters, because there are times when a tenth of a second can be wasted, and times when it can not.
    Again, fair enough. I'll see what my experiments show!

    Kind Regards, John

  13. #13
    Smooth Moperator techgnome's Avatar
    Join Date
    May 2002
    Posts
    34,531

    Re: StringBuilder vs Concatenation

    However it does it, is there some reason why concatenation can't do the same?
    Here's the short of it: when you request some memory, you need to be able to specify how much you need. With most variable types, this is pretty easy since most things take up a specified defined size. Strings.... don't they don't have a predeterministic size inherit on their type. It only knows what it needs at that moment. I should note that some (a lot of) this discussion regarding strings is true for a large number of languages, and this is not unique to VB (and could also be why it is the way it is).

    -tg
    * I don't respond to private (PM) requests for help. It's not conducive to the general learning of others.*
    * I also don't respond to friend requests. Save a few bits and don't bother. I'll just end up rejecting anyways.*
    * How to get EFFECTIVE help: The Hitchhiker's Guide to Getting Help at VBF - Removing eels from your hovercraft *
    * How to Use Parameters * Create Disconnected ADO Recordset Clones * Set your VB6 ActiveX Compatibility * Get rid of those pesky VB Line Numbers * I swear I saved my data, where'd it run off to??? *

  14. #14
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    38,988

    Re: StringBuilder vs Concatenation

    You have the basic point well in hand, though: SB can be a nuisance compared to just plain &. Whether that is worth it or not depends entirely on the situation. Even if you are doing a dozen concatenations, it usually won't make any difference at all. All that really matters is that you know about this sufficiently that you know when SB would be appropriate for you.

    It's kind of like one of those drug ads: Ask your coder if StringBuilder is right for you!
    My usual boring signature: Nothing

  15. #15

    Thread Starter
    Hyperactive Member
    Join Date
    Nov 2017
    Location
    Buckingham, UK
    Posts
    260

    Re: Viewing whole datagridview to rich text box

    Quote Originally Posted by passel View Post
    The problem is if you're appending say three characters at a time to create the 8000 character string, then you have to concatenate about 2666 times. .... Each time, you have to allocate 3 more characters of memory space on top of what you originally had .... Each time you concatenate three characters you have to allocate new memory the size of the current string plus three characters. .... So, it doesn't seem like much, but if you write a quick test to see the numbers .... The result is we would create a string of 8001 characters, and have allocated (and disposed of most of) over 10.6 million characters of memory. .... That took a significant amount of time in VB6. Since Net deals with immutable strings all the time, it might be more efficient in allocating and freeing up large amounts of memory so not be as slow.
    Thanks for your interest. Yes, following what Shaggy Hiker wrote, I now understand that. I obviously didn't realise that a string could not be 'added to'.

    Quote Originally Posted by passel View Post
    The answer for VB6 was to allocate a string of say, ten thousand characters, and then use Mid$ to insert your characters into the string, rather than concatenate.
    That's one of the approaches I'm used to (with a string variable of defined length) - but it will often invoke a performance overhead due to the need to keep track of where to insert the characters. One of the issues with fixed length strings (at least, in terms of what I'm used to) is that one cannot simply 'add characters', since that will attempt to add them belong the defined length. Instead, one has to 'trim off' the trailing blanks of the initial value and then add to that (which I suspect would be a pretty inefficient process) - so that, in my world, I would have to code something like the following [ trim() removed the trailing blanks, and "||" is the concatenation operator - but this approach has, in any event, been superseded by a function which is probably mechanistically akin to SB ]

    Code:
       length Var1 $ 1000 ;
       Var1 = "" ;
       do i = 1 to 100 ;
          Var1 = trim(Var1) || "x" ;
       end ;
    Quote Originally Posted by passel View Post
    For .Net, that strategy won't work. Strings are immutable, so you can't modify the string by inserting into the middle of it. If you do an insertion, it really copies the whole string, with insertion, into a new string, so is not any better than concatenation.
    Fair enough.

    Quote Originally Posted by passel View Post
    The string builder object gets around that by preallocating space, and when you append, it inserts the appended characters into the buffer it preallocated. If you exceed the space allocated, then it creates a new buffer double the size, copies what you have to the front half, and starts "appending" into the buffer again. Depending on the initial size allocated, the string builder may not have to reallocate more string space at all to do all the concatenations you need.
    I suppose that there's no reason why the same approach could not be taken to concatenation - but I imagine that it would be inefficient if one was undertaking only one concatenation, or a small number thereof. I suppose they could have invented a different operator for that 'different type of concatenation'!

    Quote Originally Posted by passel View Post
    That is why it is there. Perhaps your testing will confirm this.
    Time will tell However, I have no reason to think that I will find anything different from what others have (but I like to prove it to myself!)

    Kind Regards, John

  16. #16

    Thread Starter
    Hyperactive Member
    Join Date
    Nov 2017
    Location
    Buckingham, UK
    Posts
    260

    Re: StringBuilder vs Concatenation

    Quote Originally Posted by techgnome View Post
    Here's the short of it: when you request some memory, you need to be able to specify how much you need. With most variable types, this is pretty easy since most things take up a specified defined size. Strings.... don't they don't have a predeterministic size inherit on their type. It only knows what it needs at that moment. I should note that some (a lot of) this discussion regarding strings is true for a large number of languages, and this is not unique to VB (and could also be why it is the way it is).
    Understood. As I've just written, some languages do allow defined string lengths but, as I've illustrated, that in itself comes with problems and, of course, requires that one knows the maximum length that might be required (which might be excessive in terms of what is needed most of the time).

    However, as I've said, I see no theoretical reason why concatenation could not do the same as SB does - but I have acknowledged that such an approach would probably be very inefficient for single concatenations (or a small number thereof).

    Kind Regards, John

  17. #17

    Thread Starter
    Hyperactive Member
    Join Date
    Nov 2017
    Location
    Buckingham, UK
    Posts
    260

    Re: StringBuilder vs Concatenation

    Quote Originally Posted by Shaggy Hiker View Post
    You have the basic point well in hand, though: SB can be a nuisance compared to just plain &. Whether that is worth it or not depends entirely on the situation. Even if you are doing a dozen concatenations, it usually won't make any difference at all. All that really matters is that you know about this sufficiently that you know when SB would be appropriate for you. ... It's kind of like one of those drug ads: Ask your coder if StringBuilder is right for you!
    Indeed so - and I think that, for me, the answer is probably going to be that I will have a need for SB only very rarely and that, for most of the time, I can (and will) stick with the concatenation syntax which I personally prefer.

    Kind Regards, John

  18. #18
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    38,988

    Re: StringBuilder vs Concatenation

    I would guess that EVERY language requires defined string lengths. I'm not sure how it could be written to do otherwise. Memory allocation can't be open ended. At best, there is something like SB, which others have said is using a linked list. In that case, you STILL have defined string lengths, you just have a few of those related using the linked list. The memory allocation for any item in the list is still defined and effectively immutable.

    The only alternative that I can see is a language that gives you some amount of memory for your string, and keeps track of whether the memory above that is used or not. If you run off the end of your allocated memory, the language would need to know whether or not you can freely do so, or whether it has to move your whole chunk of memory to some other place that is large enough to accommodate your new demands. That just seems absurd, these days, though theoretically possible back in the DOS days of real memory. Those were dark times.
    My usual boring signature: Nothing

  19. #19

    Thread Starter
    Hyperactive Member
    Join Date
    Nov 2017
    Location
    Buckingham, UK
    Posts
    260

    Re: StringBuilder vs Concatenation

    Quote Originally Posted by Shaggy Hiker View Post
    I would guess that EVERY language requires defined string lengths. I'm not sure how it could be written to do otherwise. Memory allocation can't be open ended.
    Yes, of course, but the difference is surely between those (probably most) languages which define the length of the string variable (hence the amount of allocated memory) to be that required to accommodate an assigned (or re-assigned) value, on one hand, and those (such as the one I illustrated) in which one can programatically define a fixed length, hence reserving a certain amount of memory (even if most of it will never be used), on the other hand, isn't it?

    Kind Regards, John

  20. #20
    Sinecure devotee
    Join Date
    Aug 2013
    Location
    Southern Tier NY
    Posts
    6,582

    Re: Viewing whole datagridview to rich text box

    Quote Originally Posted by JohnW2 View Post
    ...
    I suppose that there's no reason why the same approach could not be taken to concatenation - but I imagine that it would be inefficient if one was undertaking only one concatenation, or a small number thereof. I suppose they could have invented a different operator for that 'different type of concatenation'!
    ...
    Well the string isn't the only type that is immutable. If you read the documentation on the types, almost all of the instances of the classes of the .Net types are immutable.

    The primary reason for that is because of the easy multi-threading provided as part of the Net framework. All the types, even "Value" types are implemented through classes, and internally use a reference to the memory storage for the type. The code accessing the variable has no access to the internal reference used by the class, so doesn't know any different. When the code accesses the variable it just receives the current value, but the memory storage of the value of the type, i.e. byte, Integer, or String, that the thread is reading is guaranteed to not be modified by another thread while it is being read.

    That is because any write of the value of the type is written to a new location. Any future reads are from the new location, and if any thread modifies the value, the value is updated in a new memory location, and any future reads are from that location. So, the value you're reading can't be clobbered in the middle of a read of the value. Of course if you read the value twice, then it could have been updated by another thread while you were reading, or shortly thereafter, in which case you read the current/old value on the first read from one location, and the second read will read the updated value from the new location.


    for the various .Net types. If another thread modifies an instance of the type, the reference is updated to a new memory location to hold the new value. The first threads value in memory won't be modified, so it won't get a corrupted half changed value. Of course if the first thread accesses the instance again, it gets the updated value from the updated reference.

    Quote Originally Posted by From.Net_documentation
    Byte is an immutable value type that represents unsigned integers with values that range from 0 (which is represented by the Byte.MinValue constant) to 255 (which is represented by the Byte.MaxValue constant).

    Int32 is an immutable value type that represents signed integers with values that range from negative 2,147,483,648 (which is represented by the Int32.MinValue constant) through positive 2,147,483,647 (which is represented by the Int32.MaxValue constant.

    All members of this type are thread safe. Members that appear to modify instance state actually return a new instance initialized with the new value.
    Of course, the documentation goes on to say.
    As with any other type, reading and writing to a shared variable that contains an instance of this type must be protected by a lock to guarantee thread safety.
    So, while the memory you read won't get clobbered in the middle of the read, the instance can be updated to a new value while you're working with it, so when you modify the value and write it back to a new instance, you could working with an older version of the value and clobber an updated value with an older value.

    The common example is to have multiple threads incrementing a value. If you kick off four threads and have each increment the value by 1 (assuming you start with 0) 1000 times, in theory the value should end up being 4000, but will probably be less than 4000, as the reads and writes overlap. If you're working with a Double, you won't have to worry about reading an illegal value caused by a partial update of the value, but if you don't want the variable (not the memory) itself to be updated while you're accessing it, you need to lock it.
    Last edited by passel; Nov 26th, 2019 at 05:16 PM.
    "Anyone can do any amount of work, provided it isn't the work he is supposed to be doing at that moment" Robert Benchley, 1930

  21. #21
    PowerPoster PlausiblyDamp's Avatar
    Join Date
    Dec 2016
    Location
    Pontypool, Wales
    Posts
    2,458

    Re: StringBuilder vs Concatenation

    Quote Originally Posted by JohnW2 View Post
    Indeed so - and I think that, for me, the answer is probably going to be that I will have a need for SB only very rarely and that, for most of the time, I can (and will) stick with the concatenation syntax which I personally prefer.

    Kind Regards, John
    One thing to be careful of is what type of application you are writing, if you are mainly dealing with desktop apps then the overheads of concatenation are probably not worth the effort of worrying about. If you are writing a web app or a backend API then the concatenation is possibly happening for hundreds (thousands?) of concurrent users - the memory overheads might not be too much of an issue but the pressure on the GC can be a performance killer, a StringBuilder could save you a lot of unnecessary collections.

  22. #22
    Super Moderator jmcilhinney's Avatar
    Join Date
    May 2005
    Location
    Sydney, Australia
    Posts
    110,297

    Re: StringBuilder vs Concatenation

    I haven't read this entire thread so I may be repeating something that has already been said but I have read that straight string concatenation is actually more efficient up to about a dozen operations, at which point a StringBuilder becomes more efficient. I'd say that PlausiblyDamp makes a good point too, i.e. that an application servicing one user generally won't notice much difference either way but a multi-user application certainly could.

  23. #23

    Thread Starter
    Hyperactive Member
    Join Date
    Nov 2017
    Location
    Buckingham, UK
    Posts
    260

    Re: StringBuilder vs Concatenation

    Quote Originally Posted by PlausiblyDamp View Post
    One thing to be careful of is what type of application you are writing, if you are mainly dealing with desktop apps then the overheads of concatenation are probably not worth the effort of worrying about.
    Thanks. Desktop apps are, indeed, what I virtually always am involved with.

    Kind Regards, John

  24. #24

    Thread Starter
    Hyperactive Member
    Join Date
    Nov 2017
    Location
    Buckingham, UK
    Posts
    260

    Re: StringBuilder vs Concatenation

    Quote Originally Posted by jmcilhinney View Post
    I haven't read this entire thread so I may be repeating something that has already been said but I have read that straight string concatenation is actually more efficient up to about a dozen operations,...
    Yes, 'about half a dozen' has actually been mentioned - so it seems clear that there is a pretty small number of concatenations before one reaches the 'break even' point.

    Quote Originally Posted by jmcilhinney View Post
    I'd say that PlausiblyDamp makes a good point too, i.e. that an application servicing one user generally won't notice much difference either way but a multi-user application certainly could.
    Indeed. As you may have seen, on the basis of everything I've been told, it seems fairly clear that, in terms of the sort of things I do, it would be very rare for there to be enough difference to even notice, let alone concern me. I'll therefore probably stay with concatenation, since I personally favour the syntax, but will keep my eye out for any (I'm sure very rare, for me) situations in which performance issues would make SB a more appropriate approach.

    Kind Regards, John

  25. #25

    Thread Starter
    Hyperactive Member
    Join Date
    Nov 2017
    Location
    Buckingham, UK
    Posts
    260

    Re: Viewing whole datagridview to rich text box

    Quote Originally Posted by passel View Post
    ....Perhaps your testing will confirm this.
    I would be grateful for some advice regarding my 'testing' ....

    I realise that there are quite a lot of ways in which one determine the time that a piece of code has taken to execute. I thought that "Stopwatch" was the most sensitive one, but it appears to only go down to 1 millisecond - is that correct [no, that's not correct - see my answer to my own question in next post!] and, if so, is there some other method I can use to measure execution times less than 1 millisecond?

    I ask because, if I am doing things right, on my modest laptop (and in debug mode), anything less than about 1,200 (single character addition) concatenations seems to be taking less than 1 ms (i.e. Stopwatch.Elapsed.Milliseconds is returning zero).

    Thanks for any suggestions.

    Kind Regards, John
    Last edited by JohnW2; Nov 26th, 2019 at 09:35 PM.

  26. #26

    Thread Starter
    Hyperactive Member
    Join Date
    Nov 2017
    Location
    Buckingham, UK
    Posts
    260

    Re: Viewing whole datagridview to rich text box

    Quote Originally Posted by JohnW2 View Post
    I realise that there are quite a lot of ways in which one determine the time that a piece of code has taken to execute. I thought that "Stopwatch" was the most sensitive one, but it appears to only go down to 1 millisecond - is that correct ....
    As so often happens after one has typed outr and posted a question, I've discovered the answer to my own question! Although Stopwatch.Elapsed.Milliseconds produced just an integer number of milliseconds, Stopwatch.Elapsed.TotalMilliseconds appears to give a ('Double') figure to at least 4 decimal places - i.e. to at least 0.1 microseconds.

    Kind Regards, John
    Last edited by JohnW2; Nov 26th, 2019 at 10:06 PM.

  27. #27
    Super Moderator jmcilhinney's Avatar
    Join Date
    May 2005
    Location
    Sydney, Australia
    Posts
    110,297

    Re: Viewing whole datagridview to rich text box

    Quote Originally Posted by JohnW2 View Post
    Stopwatch.Elapsed.Total.Milliseconds appears to give a ('Double') figure to 4 decimal places - i.e. to 0.1 microseconds.
    That would presumably be TotalMilliseconds rather than Total.Milliseconds. That really has nothing to do with the Stopwatch. The Elapsed property is type TimeSpan so that's what you're actually dealing with. That type has Days, Hours, Minutes, etc, properties that give you just that part of the TimeSpan, e.g. TimeSpan.FromSeconds(90).Seconds would yield 30 because the first 60 represent a full minute. The corresponding properties starting with Total are the entire time period in that unit, so TimeSpan.FromSeconds(90).TotalSeconds would yield 90.0.

  28. #28

    Thread Starter
    Hyperactive Member
    Join Date
    Nov 2017
    Location
    Buckingham, UK
    Posts
    260

    Re: StringBuilder vs Concatenation

    Quote Originally Posted by jmcilhinney View Post
    That would presumably be TotalMilliseconds rather than Total.Milliseconds.
    Indeed so. However, it's a bit odd. I saw, and edited out, that typo (errant full stop) almost immediately after I posted it, so you must have been very quick off the mark in replying (but there seems to be a 6 minute difference between the timestamps). (I edited again much later to add the "at least" to 4 decimal places, becuase I wasn't certain that it would never be more) Anyway, apologies for the typo!

    Quote Originally Posted by jmcilhinney View Post
    That really has nothing to do with the Stopwatch. The Elapsed property is type TimeSpan so that's what you're actually dealing with. That type has Days, Hours, Minutes, etc, properties that give you just that part of the TimeSpan, e.g. TimeSpan.FromSeconds(90).Seconds would yield 30 because the first 60 represent a full minute. The corresponding properties starting with Total are the entire time period in that unit, so TimeSpan.FromSeconds(90).TotalSeconds would yield 90.0.
    OK, thanks. I understood most of that, but that wasn't really my main problem. I was only (at the time) interested in intervals less than 1 second, so I knew that I was only interested in the milliseconds part (since the potential seconds, minutes, hours and days parts would all be zero). What I didn't realise (I suppose because I hadn't looked!), and hence my question, was that, whilst the Milliseconds part was (like the Seconds, Minutes, Hours and Day parts) was integer, the TotalMilliseconds part (like all the other Totalxxxx parts) was Double.

    What about the 'sensitivity' though? I still haven't seen ...Elapsed.TotalMilliseconds ever produce more than 4 decimal places (whereas Double could obviously provide far more than that). Is that down to Stopwatch, or maybe even hardware?

    Kind Regards, John

  29. #29
    Super Moderator jmcilhinney's Avatar
    Join Date
    May 2005
    Location
    Sydney, Australia
    Posts
    110,297

    Re: StringBuilder vs Concatenation

    Quote Originally Posted by JohnW2 View Post
    What about the 'sensitivity' though? I still haven't seen ...Elapsed.TotalMilliseconds ever produce more than 4 decimal places (whereas Double could obviously provide far more than that). Is that down to Stopwatch, or maybe even hardware?
    Not sure but perhaps look at the Ticks property to see whether it provides higher resolution.

  30. #30
    Frenzied Member
    Join Date
    May 2014
    Location
    Central Europe
    Posts
    1,372

    Re: StringBuilder vs Concatenation

    To add some flesh to the bones and taking it out of the theoretical area: we had a poorly coded vb6 script where a string output was concatenated. the output was multiple lines with multiple fields so basically like a csv file. The code did add field by field to the output and when the line was complete, added a newline char and continued with the next line.
    This was working ok until the moment when a really huge output was to create. The code then failed tremendously by taking several hours to complete. The thing is: the bigger the output gets, the more memory must be copied to the new string and the longer it takes. this rises exponentially. fix was to concat each line independently and i recall we then even concated up to 100 lines into a separate buffer and then this to the output. instead of taking hours it completed within seconds.
    stringbuilder is made for exactly this scenario and is the recommended approach in this case.
    if you do just a few concats with a predictable small output (over the entire lifetime of the app), then concats are just fine.

  31. #31
    PowerPoster PlausiblyDamp's Avatar
    Join Date
    Dec 2016
    Location
    Pontypool, Wales
    Posts
    2,458

    Re: StringBuilder vs Concatenation

    Quote Originally Posted by JohnW2 View Post
    Indeed. As you may have seen, on the basis of everything I've been told, it seems fairly clear that, in terms of the sort of things I do, it would be very rare for there to be enough difference to even notice, let alone concern me. I'll therefore probably stay with concatenation, since I personally favour the syntax, but will keep my eye out for any (I'm sure very rare, for me) situations in which performance issues would make SB a more appropriate approach.

    Kind Regards, John
    The trick is to code appropriately and know what is available, concatenation makes for clearer and more obvious code so I would personally use it in preference to StringBuilder. If however I was getting performance issues and I could prove they were down to string concatenation then I would switch to StringBuilder because there was a need.

    I would always prefer clean, maintainable code over the fastest / most optimised code unless the cleaner code was a performance issue. Usual rules about premature optimisation etc apply here.

  32. #32

    Thread Starter
    Hyperactive Member
    Join Date
    Nov 2017
    Location
    Buckingham, UK
    Posts
    260

    Re: StringBuilder vs Concatenation

    Quote Originally Posted by PlausiblyDamp View Post
    The trick is to code appropriately and know what is available, concatenation makes for clearer and more obvious code so I would personally use it in preference to StringBuilder.
    Exactly the same here, and that's why I asked the question which started this discussion.

    Quote Originally Posted by PlausiblyDamp View Post
    If however I was getting performance issues and I could prove they were down to string concatenation then I would switch to StringBuilder because there was a need.
    Again, the same for me. However, this is just a specific example of a much more general issue. If I find that one step/part of my code is causing performance issues (and that's obviously most common with something which is executed a vast number of times), I would investigate to see if the situation could be improved by changing the algorithm and/or coding of it.

    Quote Originally Posted by PlausiblyDamp View Post
    I would always prefer clean, maintainable code over the fastest / most optimised code unless the cleaner code was a performance issue. Usual rules about premature optimisation etc apply here.
    At risk of sounding like a broken record, I again agree. Indeed, given that performance is very rarely an issue in terms of things I do, I will often deliberately write verbose 'brute force' code, even if I know that there are more 'elegant' (and 'concise') ways in which it could be done (and undoubtedly often when there even more elegant ways of which I'm not aware!), simply to make the code cleaner, clearer and more easily maintained (for/by me).

    Kind Regards, John

  33. #33

    Thread Starter
    Hyperactive Member
    Join Date
    Nov 2017
    Location
    Buckingham, UK
    Posts
    260

    Re: StringBuilder vs Concatenation

    Quote Originally Posted by jmcilhinney View Post
    Not sure but perhaps look at the Ticks property to see whether it provides higher resolution.
    I'm about to start a new thread ("Stopwatch Resolution"), but the bottom line appears to be that, as I suspected, the answer is system/hardware-dependent, but on my machine the resolution is, indeed 0.0001 milliseconds (0.1 microseconds, 100 nanoseconds).

    Kind Regards, John
    Last edited by JohnW2; Nov 27th, 2019 at 08:35 AM.

  34. #34

    Thread Starter
    Hyperactive Member
    Join Date
    Nov 2017
    Location
    Buckingham, UK
    Posts
    260

    Re: StringBuilder vs Concatenation

    OK, some preliminary and tentative experiment results (with some caveats) ….

    Quote Originally Posted by Shaggy Hiker View Post
    A few years back (may have been several, by now), this discussion came up, and people set about testing the difference between StringBuilder and concatenation. I forget the exact number, but I seem to remember that SB had better performance if there were going to be more than about half a dozen concatenations.
    Quote Originally Posted by jmcilhinney View Post
    I haven't read this entire thread so I may be repeating something that has already been said but I have read that straight string concatenation is actually more efficient up to about a dozen operations, at which point a StringBuilder becomes more efficient.
    Unless I’m doing something wrong/silly, there is a tremendous amount of run-to-run variation in execution time, with either concatenation or Stringbuilder. Even if I average 1,000 repetitions of executing the code (only practical for very small numbers of concatenations/’additions’) the results are still anything but repeatable. Anyway, with the caveat that something might be going wrong ….

    I have repeatedly added a single character to s string, either by concatenation or by using StringBuilder, and timed the two processes (with Stopwatch). Where appropriate, I have repeated that exercise (for a given number of concatenations/ additions) and averaged the resulting execution times.

    … I don’t really seem to be able to reproduce the behaviour mentioned in the above two quotes. For any number of concatenations/additions from 1 to 12, StringBuilder appears to be faster (increasingly so as the number of concatenations/ additions increases). In fact, with StreamBuilder the times taken for these 1-12 operations (about 150 - 400 nanosecs) is only just within the (100 ns) resolution of Stopwatch.

    Of course, if things aren’t ‘multiplied up’ (by looping or whatever), such there are just 12 concatenations, the time taken (a bit over 25 microseconds for 12) is trivial ….

    Name:  StringBuilder1.jpg
Views: 1067
Size:  24.8 KB

    Quote Originally Posted by passel View Post
    Each time you concatenate three characters you have to allocate new memory the size of the current string plus three characters. (and the characters usually take two bytes per character, so the memory allocation is twice the number of characters). ... So, it doesn't seem like much, but if you write a quick test to see the numbers, lets seem how much character space has to be allocated and disposed of to reach our 8000 characters (give or take). ... The result is we would create a string of 8001 characters, and have allocated (and disposed of most of) over 10.6 million characters of memory.
    As expected, when one starts increasing the total number of concatenations/ additions, SB becomes increasingly (and dramatically) faster than concatenation. Not being very patient, I gave up with concatenation at 200,000 concatenations (which took about 40 seconds), but I was able with SB to go up to 10^8 character additions before running into memory problems, and that only took about 0.9 seconds.

    Because of the log scales, the curves for SB and concatenation below may not at first sight look dramatically different - but, even with just 1,000 concatenations/ additions, SB is almost 100 times faster, and by 200,000 concatenations/ additions (where I gave up with concatenation), SB is around 10,000 times faster.

    Name:  StringBuilder2.jpg
Views: 1050
Size:  31.7 KB

    Kind Regards, John

  35. #35
    Smooth Moperator techgnome's Avatar
    Join Date
    May 2002
    Posts
    34,531

    Re: StringBuilder vs Concatenation

    Might help the rest of us if you posted the project (sans binaries) that you used...

    -tg
    * I don't respond to private (PM) requests for help. It's not conducive to the general learning of others.*
    * I also don't respond to friend requests. Save a few bits and don't bother. I'll just end up rejecting anyways.*
    * How to get EFFECTIVE help: The Hitchhiker's Guide to Getting Help at VBF - Removing eels from your hovercraft *
    * How to Use Parameters * Create Disconnected ADO Recordset Clones * Set your VB6 ActiveX Compatibility * Get rid of those pesky VB Line Numbers * I swear I saved my data, where'd it run off to??? *

  36. #36

    Thread Starter
    Hyperactive Member
    Join Date
    Nov 2017
    Location
    Buckingham, UK
    Posts
    260

    Re: StringBuilder vs Concatenation

    Quote Originally Posted by techgnome View Post
    Might help the rest of us if you posted the project (sans binaries) that you used...
    Apologies. I intended to include some code, but totally forgot. I should also have reminded you all (I mentioned this in a previous post) that all of my current experimentation has been in the debug environment, and I'm not sure how much difference that would make.

    The meat of the code I used is as below (anything else was just to output/display the results in the two arrays). It's meant to be functional, rather than in any way elegant, but I'll be interested to hear if any one can see any silly (or even subtle!) errors in what I've done!

    Code:
            Dim stpWatch1 As New System.Diagnostics.Stopwatch
            Dim ConcatString As String = ""
            Dim NumConcats As Integer
    
            Dim NumReplics As Integer = CInt(txtRepeats.Text)
            Dim SBtimes(NumReplics) As Double
            Dim ConcatTimes(NumReplics) As Double
    
            NumConcats = CInt(txtNumConcats.Text)
    
            For Replic As Integer = 0 To NumReplics
    
                stpWatch1.Reset()
                stpWatch1.Start()
                Dim builder As New System.Text.StringBuilder
                For i As Integer = 1 To NumConcats
                    builder.Append("x")
                Next
                Dim BigString As String = builder.ToString
                stpWatch1.Stop()
                SBtimes(Replic) = stpWatch1.Elapsed.TotalMilliseconds
    
                stpWatch1.Reset()
                stpWatch1.Start()
                For i As Integer = 1 To NumConcats
                    ConcatString &= "x"
                Next
                stpWatch1.Stop()
                ConcatTimes(Replic) = stpWatch1.Elapsed.TotalMilliseconds
    
            Next
            Dim SBaverage As Double = SBtimes.Average
            Dim ConcatAverage As Double = ConcatTimes.Average
    Kind Regards, John

  37. #37

    Thread Starter
    Hyperactive Member
    Join Date
    Nov 2017
    Location
    Buckingham, UK
    Posts
    260

    Re: Viewing whole datagridview to rich text box

    Quote Originally Posted by passel View Post
    .... Each time you concatenate three characters you have to allocate new memory the size of the current string plus three characters. (and the characters usually take two bytes per character, so the memory allocation is twice the number of characters). .... So, it doesn't seem like much, but if you write a quick test to see the numbers, lets seem how much character space has to be allocated and disposed of to reach our 8000 characters (give or take). .... The result is we would create a string of 8001 characters, and have allocated (and disposed of most of) over 10.6 million characters of memory.
    I'm not sure whether this would often/ever be of much practical importance, but I wonder if you know the answer to this?

    We have established that, with the following code, the variable MyVar (which, in some situations, might already be enormous) would be re-written (and the previous version deleted') N times - which could be an issue when N and/or any of the variables involved are large ...

    Code:
            MyVar &= Var1
            MyVar &= Var2
            MyVar &= Var3
            ....
            MyVar &= VarN
    
            Or
    
            For i As Integer = 1 To N
                MyVar &= Var(i)
            Next
    ... but what is the situation with the following code? Does MyVar again have to be re-written (and the previous version 'deleted') after each of the N concatenations, or can it somehow do all the concatenations 'in its head' and therefore only have to re-write MyVar only once? There's no point in my making any further comments, or asking any supplementary questions, until I know the answer to that.

    Code:
            MyVar = MyVar & Var1 & Var2 & Var3.... & VarN
    However, something else you wrote made me wonder about something which could be much more important to me, given that a lot of the programming I do is ‘mathematical’ in nature. You wrote ….

    Quote Originally Posted by passel View Post
    Well the string isn't the only type that is immutable. If you read the documentation on the types, almost all of the instances of the classes of the .Net types are immutable. .... The primary reason for that is because of the easy multi-threading provided as part of the Net framework. ... The common example is to have multiple threads incrementing a value. ...
    Do I take it that the immutability of the variables means that every time the value of a numeric variable changes, the variable has to be re-written (and the previous instance of it deleted)? If so, then, in analogy with my question above about string variables, what if there is an assignment statement which involves multiple mathematical operations and/or functions, such as ….

    Code:
            MyVar = Math.Sqrt(Math.Log(1 + ((x + y) / z))) + 25
    … in this case, does MyVar get re-written (and the previous instance deleted) after each and every mathematical operation/function, or is the entire calculation somehow ‘done in its head’ with just the final value of MyVar being re-written.

    The possible practical relevance of this to me is that, for convenience of code writing (and clarity of code reading), I will often split a complex mathematical calculation into several assignment statements. However, if that has a significant impact on performance (particularly given that I may well be doing this within loops that repeat the calculation millions of times), I may have to re-think my practices when using VB.NET.

    Kind Regards, John

  38. #38
    Super Moderator si_the_geek's Avatar
    Join Date
    Jul 2002
    Location
    Bristol, UK
    Posts
    41,929

    Re: Viewing whole datagridview to rich text box

    Quote Originally Posted by JohnW2 View Post
    ...all of my current experimentation has been in the debug environment, and I'm not sure how much difference that would make.
    Whether or not you run it in Visual Studio doesn't make a significant difference, what can be massive however is whether you are running a Debug build or a Release build (you can select in a drop-down list at the top of Visual Studio before running).

    The differences in speed can be enormous, and can be very unpredictable... some things will take virtually the same time in Debug and Release builds, while others will be hundreds of times slower in a Debug build.

    As you would generally distribute a Release build, that is the one where the timings should be done.

    I'll be interested to hear if any one can see any silly (or even subtle!) errors in what I've done!
    Inside the Replic loop you are creating a new StringBuilder each time but you aren't clearing ConcatString, so ConcatString will contain far more characters. To make it more equivalent, I would recommend moving the declaration of ConcatString to just before the loop that fills it.

  39. #39
    Frenzied Member
    Join Date
    Jul 2011
    Location
    UK
    Posts
    1,335

    Re: Viewing whole datagridview to rich text box

    Quote Originally Posted by JohnW2 View Post
    ... but what is the situation with the following code? Does MyVar again have to be re-written (and the previous version 'deleted') after each of the N concatenations, or can it somehow do all the concatenations 'in its head' and therefore only have to re-write MyVar only once? There's no point in my making any further comments, or asking any supplementary questions, until I know the answer to that.

    Code:
            MyVar = MyVar & Var1 & Var2 & Var3.... & VarN
    That results in just a single new string being created which is assigned to the MyVar variable; the existing contents of MyVar being discarded and garbage collected (at some point).

    The & operator when used to concatenate 2 (or more) strings is a wrapper for the String.Concat method.

    The String.Concat method has overloads that allow for concatenating 2,3 or 4 strings in one go, and also an overload that will concatenate an array of strings in one go.
    The compiler will optimise your code by placing all the variables on the right hand side of the expression into an array, and then calling String.Concat on that array.

    And yes, as far as I understand it, the Concat method works out 'in its head' how much memory is needed to store the resulting string, so reallocations won't happen when it concatenates multiple strings in one call.

  40. #40

    Thread Starter
    Hyperactive Member
    Join Date
    Nov 2017
    Location
    Buckingham, UK
    Posts
    260

    Re: Viewing whole datagridview to rich text box

    Quote Originally Posted by si_the_geek View Post
    Whether or not you run it in Visual Studio doesn't make a significant difference ....
    Thanks for your input. Yes, that's what I assumed (and it was obviously more convenient to do my playing in VS) - or, at least, that it shouldn't make a material difference to the relative speeds of concatenation and SB.

    Quote Originally Posted by si_the_geek View Post
    .... what can be massive however is whether you are running a Debug build or a Release build (you can select in a drop-down list at the top of Visual Studio before running). .... The differences in speed can be enormous, and can be very unpredictable... some things will take virtually the same time in Debug and Release builds, while others will be hundreds of times slower in a Debug build. ... As you would generally distribute a Release build, that is the one where the timings should be done.
    That I didn't realise, and I suppose it makes sense, I presume due to different 'optimisations'. As you will probably appreciate, at the point on the learning curve where I currently am, I have not yet 'released' anything - but, as you say, it sounds as if I should be using a Release build for my tests - so I'll do that from now one.

    Quote Originally Posted by si_the_geek View Post
    Inside the Replic loop you are creating a new StringBuilder each time but you aren't clearing ConcatString, so ConcatString will contain far more characters. To make it more equivalent, I would recommend moving the declaration of ConcatString to just before the loop that fills it.
    Whoops - silly indeed My failure to clear ConcatString was obviously a total oversight, and presumably will have had a dramatic effect on the results - what this space for some corrected results (and using Release build). 'Habit would reslt in my clearing it simply with [ ConcatString = "" ] - is there any reason why I shouldn't continue with that habit with VB.NET?

    Do I take it that it's OK to create a new StringBuilder during each iteration of the Replic loop? My doing this was essentially 'deliberate', since I was trying to achieve the situation in which each iteration of the Replic loop would reproduce the entire process that one would undertake when using SB, and that includes creation a new StringBuilder.

    In passing, I'd be interested to know if you have any observations in relation to my post #37 above.

    Finally, although I presume this is an unchangeable 'feature' of the forum software, I see that when Shaggy Hiker split this thread off from the one it started on, the Subject line in the first message in this new thread (and on the 'index' page) changed, but all the subsequent posts bear the Subject/title of the original thread!

    Kind Regards, John

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width