Results 1 to 12 of 12

Thread: Server Side Storage

  1. #1

    Thread Starter
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    38,988

    Server Side Storage

    I'm new to ASP.NET Core, and fairly new to web development in general. This leaves me a bit unfamiliar with web concepts, and best practices, hence this question:

    I am working on a program where some models will be stored in a database. The models have a variety of data, much of which may be needed by any particular user. The model could be seen as:

    Class P holds one Class U, one or more class B, zero or more class S, zero or more class N and maybe something else. In a desktop environment handling the same thing, I would load Class P such that all the members of P are available and accessible via P. In the web environment, there could be multiple users, and each will have zero or more P associated with them. When the user first logs in, they will see the set of P that are associated with them, but more importantly, they will see a name describing P, which is made up of the U, B, S, N, and other elements that are members of P.

    It would be entirely possible for the user upon login to get the set of P that they have, then when they want to see/edit one of the P that they have, they could submit the ID for that P, and that P would be loaded up for them, but while stateless, this seems horribly inefficient. Every time they go to a certain page, the server would be acting like they had never been there before, and get all the P for them from the database.

    What I'd prefer to do is to have a PM class that holds all the P objects for each user, and have that PM class persist for as long as that user is still working with the site. I could also have a UM class (User Manager) that would be essentially a dictionary of PM by User. This may be a bad idea, though, because that object could persist on the server for the life of the running application. It would be efficient from a memory standpoint, as there would be no need to load the PM for the user if it was already in the dictionary, but it would be sitting there in memory even when the site was not being used by anybody (the total number of users is likely less than 100, and they would likely use the site for brief bursts roughly once per year).

    If I were to persist that UM dictionary object, it looks like I could use the Cache, but am I right in thinking that would be bad form?

    If I just persist the PM class, that could exist for as long as the user kept coming back to the site. It looks like SessionState would be appropriate for this, with the understanding that the SessionState could be dispensed with by the server, so I would have to write code to make sure that the PM stored in the SessionState existed, and reload the PM if it had gone away.

    Does that sound right, and if not, what should I be doing?
    My usual boring signature: Nothing

  2. #2
    King of sapila
    Join Date
    Oct 2006
    Location
    Greece
    Posts
    6,597

    Re: Server Side Storage

    I'll admit you got me lost with the B S Z thing.

    But session and cookies are what options I see fit. Not cache.
    You can set a session timeout or a cookie timeout depending on the situation. Or I think you can maintain a value on a hidden control but that seems like a bad idea.
    If you are using asp.net authentication then the things are more robust, a login user cannot access another users data(it can if the design is not proper but I'm talking about proper authentication).
    If you are not using asp.net authentication and using a database and you want to go back and forth to check who is calling what (depending on probably a random assigned value on session or encrypted cookie) then you be better of doing this with javascript and web methods (jquery ajax or I bet other newer javascript platform support that) to avoid server Postbacks.
    Ah, by UM, you mean have the users persist on the server....Hmmm... I guess, since the data is on the server and cannot be located on the page that should be OK but how would you persist the data on the server?
    Probably with a session or you will have to reload the page every time for changes. So in that situation, for me, it would be better to to use an encrypted cookie or to use a web method that will server the UM but, it's been ages since I dealt with authentication so there may be better concepts.

    Edit. I've read a little more on ASP.NET MVC as I'm using simple asp.net. It supposed to make better use of the cache so if you are using 4.0 and up you can go that way also on the dictionary.
    Just remember that using cache pushes data to the server memory, so you need to make sure that it won't bloat the IIS. There are some settings you can specify on the IIS and more specifically on how much memory it will recycle but if you are not caching huge amount that won't matter (as I recall IIS application pools recycle at max server memory) but on huge amount, an IIS recycle will lose you all the cached data.
    Also notice that sessions timeout at some point (change settings on IIS) . I believe is either 20 or 30 minutes. After that you loose the session.
    Last edited by sapator; May 3rd, 2021 at 11:38 PM.
    ἄνδρα μοι ἔννεπε, μοῦσα, πολύτροπον, ὃς μάλα πολλὰ
    πλάγχθη, ἐπεὶ Τροίης ἱερὸν πτολίεθρον ἔπερσεν·

  3. #3

    Thread Starter
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    38,988

    Re: Server Side Storage

    What I do not want is all the data going to the client on every request.

    I'm trying to avoid needing to re-load a complex map of objects on each request to the server. In database terms, the user will be interacting with a permit. That permit would be one record in a DB table, but it would include one or more child records from each of several other tables. In a stateless system, every time the client requests something, the server is supposed to treat that request as if it was independent of any other, as I understand it. However, that would mean loading all that data from the database, and constructing the entire map of objects, for every single request.

    Any one request might require only bits and pieces of the total map of objects, so I might load just what is needed for each request, but that just seems like a really bad idea from a performance perspective.
    My usual boring signature: Nothing

  4. #4

    Thread Starter
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    38,988

    Re: Server Side Storage

    Another thing I've seen, which surprises me, is that session state can only be strings or integers. MS appears to recommend that I turn my map into a JSON object, store that to session state and restore from that as needed. I can do that, I'm just surprised to find that I need to.
    My usual boring signature: Nothing

  5. #5
    King of sapila
    Join Date
    Oct 2006
    Location
    Greece
    Posts
    6,597

    Re: Server Side Storage

    You can't avoid re loading as the session object is server side on standard asp. Will you be using MVC or asp?
    Ajax calls is to "trick" the page not to postback but eventually you will have to grab the info from the server, even if no reloading "flickering: is happening.
    That is why I suggested a cookie, BUT, cookie max size is 4096 bytes. If it is a huge object then I'm note sure what will happen to the page.
    In MVC you can use localstrorage or sessionstorage , I'm not the least sure about size or speed limitation as I'm not using MVC but here is an article:
    https://visualstudiomagazine.com/art...ving-data.aspx

    Personally I would have created Ajax calls and bring data continuously as the page progress. I wouldn't suggest "on big push" of data on the page. You will may find out later that the page will is very slow to respond.
    If you need a "table" data system, there are JS solution (szlamany) have utilized many of them, that do not cram the data.
    A new thread on AJAX, or JS thread would be advised as szlamany could help further.
    (note that this is called, xosimo in Greek, aka "pushing" ορ "jab" in English , is when you suggest another person for the work without him/her knowing it but szlamany shouldn't brag so much for his skills. Now he get's the "jab" ! )
    ἄνδρα μοι ἔννεπε, μοῦσα, πολύτροπον, ὃς μάλα πολλὰ
    πλάγχθη, ἐπεὶ Τροίης ἱερὸν πτολίεθρον ἔπερσεν·

  6. #6
    Smooth Moperator techgnome's Avatar
    Join Date
    May 2002
    Posts
    34,532

    Re: Server Side Storage

    Didn't read through all the replies... got through the main post though... here's my initial thoughts:
    On the server: cache the user's objects... there's no harm in that. If the data doesn't change, except when pushed by the user, cache it. But, don't send all of it to the client. Only send the P that the user is actively working on to the client. Then in the client, store the P object in the state, and work with it there. If you need to persist the changes back to the server, then either send it back, or only send the deltas, but it should be one-way, upon request.

    Hope that made some sense.
    In short, on the server you cache PM, and in the client you cache P, with the occasional trip to the server to deliver updates from P as needed. If the user then needs a new P, for context switching, that would then be the only time to send back the entire P back from the server (where it was cached in the PM) other than the initial one.

    -tg
    * I don't respond to private (PM) requests for help. It's not conducive to the general learning of others.*
    * I also don't respond to friend requests. Save a few bits and don't bother. I'll just end up rejecting anyways.*
    * How to get EFFECTIVE help: The Hitchhiker's Guide to Getting Help at VBF - Removing eels from your hovercraft *
    * How to Use Parameters * Create Disconnected ADO Recordset Clones * Set your VB6 ActiveX Compatibility * Get rid of those pesky VB Line Numbers * I swear I saved my data, where'd it run off to??? *

  7. #7
    King of sapila
    Join Date
    Oct 2006
    Location
    Greece
    Posts
    6,597

    Re: Server Side Storage

    Just to be clear.
    Session state that tech suggests, still stores data to the server, there is only an ID stored to the client so it can identify it's session equivalent when called.
    So you are not saving any server memory.
    The question here is why you need full data to the client and not calls to the server.
    As suggested there are other ways than bloating the full data of P, particular if you are requesting it through session, you are calling the server to retrieve the data anyhow.
    I'm under the impression that there are some caching mechanisms when you request the session data for the 2nd time from the server to lower the load but I haven't looked into that.
    ἄνδρα μοι ἔννεπε, μοῦσα, πολύτροπον, ὃς μάλα πολλὰ
    πλάγχθη, ἐπεὶ Τροίης ἱερὸν πτολίεθρον ἔπερσεν·

  8. #8
    MS SQL Powerposter szlamany's Avatar
    Join Date
    Mar 2004
    Location
    Connecticut
    Posts
    18,263

    Re: Server Side Storage

    There are several methods on the server to hold data - I use the APPLICATION pool mostly. Since I need that pool to persist data, I trap the APPLICATION end event and "write" that pool to a text file. When the APPLICATION start event fires, I re-populate the APPLICATION pool. Not sure if all that "management" work is done my MS if you are using MVC...

    If you use the SESSION pool instead you will need to handle the SESSION start and end events in a similar fashion if you want to persist over those moments.

    I use only AJAX - so my page never re-loads. I'll have a user logged in for several days, as long as I persist that APPLICATION pool successfully, I've got their security GUID and a ton of facts about them (alerts and multi-user controls and what not).

    With all that said, you have to be careful of how much you store in any of these memory pools. Read online about this before you store lots of data.

    It is not uncommon for my web app to pre-load all the addresses in a city on the initial page load. Browsers do not freak out about tons of data - I can have 30,000 addresses and client side performance is stellar. With all those addresses loaded, for instance, I can allow the user to find an address and drill down to the "next" level in the maintenance experience. The JS code to REGEX search these "arrays" is incredibly fast - making the user experience a really good one.

    The "web world" might be stateless, but us programmers are responsible for giving it "state".

    *** Read the sticky in the DB forum about how to get your question answered quickly!! ***

    Please remember to rate posts! Rate any post you find helpful - even in old threads! Use the link to the left - "Rate this Post".

    Some Informative Links:
    [ SQL Rules to Live By ] [ Reserved SQL keywords ] [ When to use INDEX HINTS! ] [ Passing Multi-item Parameters to STORED PROCEDURES ]
    [ Solution to non-domain Windows Authentication ] [ Crazy things we do to shrink log files ] [ SQL 2005 Features ] [ Loading Pictures from DB ]

    MS MVP 2006, 2007, 2008

  9. #9

    Thread Starter
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    38,988

    Re: Server Side Storage

    Quote Originally Posted by techgnome View Post
    Didn't read through all the replies... got through the main post though... here's my initial thoughts:
    On the server: cache the user's objects... there's no harm in that. If the data doesn't change, except when pushed by the user, cache it. But, don't send all of it to the client. Only send the P that the user is actively working on to the client. Then in the client, store the P object in the state, and work with it there. If you need to persist the changes back to the server, then either send it back, or only send the deltas, but it should be one-way, upon request.

    Hope that made some sense.
    In short, on the server you cache PM, and in the client you cache P, with the occasional trip to the server to deliver updates from P as needed. If the user then needs a new P, for context switching, that would then be the only time to send back the entire P back from the server (where it was cached in the PM) other than the initial one.

    -tg
    I'm sending less than one object to the client. All the client gets is the portion of the model that applies to the page in question. Using the in-memory cache in ASP.Net Core is looking like the way to go.

    @Sapator: I'm not trying to save server side memory, I'm looking to spend a bit of server side memory to speed up server side responses by not needing to go back to the database on every request. I'm not sending everything to the client, just what is needed on the page, but what is needed on the page is some portion of a semi-complex model, and I don't want the server to be loading that model on every request.
    My usual boring signature: Nothing

  10. #10

    Thread Starter
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    38,988

    Re: Server Side Storage

    @szlamany: Are you using ASP.NET Core or something a bit older? I seem to remember encountering some mention that Core dropped session state events, though I could very well be misremembering that.
    My usual boring signature: Nothing

  11. #11
    MS SQL Powerposter szlamany's Avatar
    Join Date
    Mar 2004
    Location
    Connecticut
    Posts
    18,263

    Re: Server Side Storage

    I am not using ASP.NET Core - I'm just using VS 2019 against IIS with .Net 3.5 (I'm using old SDK's for things like OpenXML).

    I have a Global.asax file that VS gave me that has all kinds of SUB's for the various APPLICATION and SESSION events. I do nothing with SESSION events - do it all with APPLICATION events. Although I did recently put some logging of an HTTPS SSL check in the Session_Start() event.

    Code:
        Sub Session_Start(ByVal sender As Object, ByVal e As EventArgs)
            LogOutput("Session Start")
            LogOutput("...HttpContext.Current.Request.IsSecureConnection=" & HttpContext.Current.Request.IsSecureConnection.ToString())
            ' Code that runs when a new session is started
        End Sub
    
        Sub Session_End(ByVal sender As Object, ByVal e As EventArgs)
            LogOutput("Session End")
            ' Code that runs when a session ends. 
            ' Note: The Session_End event is raised only when the sessionstate mode
            ' is set to InProc in the Web.config file. If session mode is set to StateServer 
            ' or SQLServer, the event is not raised.
        End Sub
    In the APPLICATION pool I store session guids and manage users - my LOGIN web method drops the info into the APPLICATION pool and subsequent AJAX calls check for the guid.

    Code:
    With Application
        .Lock()
        If .Item("g_GUIDS") IsNot Nothing Then
            g_GUIDS = DirectCast(.Item("g_GUIDS"), Dictionary(Of String, Dictionary(Of String, String)))
        End If
        g_GUIDS.Add(strMessage, New Dictionary(Of String, String))
        g_GUIDS(strMessage).Add("status", "success")
        g_GUIDS(strMessage).Add("logindate", Date.Now.ToShortDateString)
        g_GUIDS(strMessage).Add("logintime", Date.Now.ToLongTimeString)
        .Item("g_GUIDS") = g_GUIDS
        .UnLock()
    End With
    I also store SQL SPROC parameters in the APPLICATION pool so I don't have to keep going to the database for them. Otherwise I keep storage in the pool at a minimum.

    Code:
    Private Sub DetermineParameters(ByRef dcn As SqlConnection, ByRef cmd As SqlCommand, Optional ByRef errorText As String = "")
        Dim blnError As Boolean = False
        Dim blnDoClose As Boolean = False
        Dim blnWriteItBack As Boolean = False
        With Application
            .Lock()
            If .Item("g_prmSP") IsNot Nothing Then
                g_prmSP = DirectCast(.Item("g_prmSP"), Dictionary(Of String, SqlParameter()))
            End If
            Try
                If g_prmSP.ContainsKey(cmd.CommandText) Then
                    .UnLock()
                    Dim prm As New SqlParameter
                    For Each prmSP As SqlParameter In g_prmSP(cmd.CommandText)
                        prm = CType(CType(prmSP, ICloneable).Clone(), SqlParameter)
                        cmd.Parameters.Add(prm)
                    Next
                Else
                    blnWriteItBack = True
                    Using cmd2 As New SqlCommand
                        cmd2.CommandType = CommandType.StoredProcedure
                        cmd2.CommandText = "dbo.awcGetSP"
                        cmd2.Connection = dcn
                        If Not cmd.CommandText.StartsWith("dbo") AndAlso cmd.CommandText.Contains(".") Then
                            cmd2.Parameters.AddWithValue("@StoredProc", cmd.CommandText.Substring(cmd.CommandText.LastIndexOf(".") + 1))
                        Else
                            cmd2.Parameters.AddWithValue("@StoredProc", cmd.CommandText)
                        End If
                        If dcn.State <> ConnectionState.Open Then
                            dcn.Open()
                            blnDoClose = True
                        End If
                        Dim blnDidSetup As Boolean = False
                        Dim blnGotPassConnId As Boolean = False
                        Dim cntParams As Integer = 0
                        Using sdrReader As SqlDataReader = cmd2.ExecuteReader
                            Do
                                If Not blnDidSetup Then
                                    blnDidSetup = True
                                Else
                                    Do While sdrReader.Read()
                                        If sdrReader("Parameter_Name").ToString = "@PassConnId" Then
                                            blnGotPassConnId = True
                                        End If
                                        If blnGotPassConnId Then
                                            cntParams += 1
                                            If cntParams > 9 Then
                                                blnGotPassConnId = False
                                            End If
                                        End If
                                        If blnGotPassConnId Then
                                            If cntParams = 1 Then
                                                cmd.Parameters.Add(New SqlParameter("@PassConnId", SqlDbType.Int)).Value = 0
                                                cmd.Parameters.Add(New SqlParameter("@RetStat", SqlDbType.Int, 0, ParameterDirection.InputOutput, True, 0, 0, "", DataRowVersion.Current, 0))
                                                cmd.Parameters.Add(New SqlParameter("@RetText", SqlDbType.VarChar, 100, ParameterDirection.InputOutput, True, 0, 0, "", DataRowVersion.Current, ""))
                                                cmd.Parameters.Add(New SqlParameter("@RetMode", SqlDbType.Int, 0, ParameterDirection.InputOutput, True, 0, 0, "", DataRowVersion.Current, 0))
                                                cmd.Parameters.Add(New SqlParameter("@RetGrid", SqlDbType.Int, 0, ParameterDirection.InputOutput, True, 0, 0, "", DataRowVersion.Current, 0))
                                                cmd.Parameters.Add(New SqlParameter("@RetExtra", SqlDbType.VarChar, 100, ParameterDirection.InputOutput, True, 0, 0, "", DataRowVersion.Current, ""))
                                                cmd.Parameters.Add(New SqlParameter("@PassMode", SqlDbType.Int)).Value = 0
                                                cmd.Parameters.Add(New SqlParameter("@PassExtra", SqlDbType.VarChar, 100)).Value = ""
                                                cmd.Parameters.Add(New SqlParameter("@PassTest", SqlDbType.VarChar, 100)).Value = ""
                                            End If
                                        Else
                                            Dim prm As New SqlParameter
                                            With prm
                                                prm.ParameterName = sdrReader("Parameter_Name").ToString
                                                Select Case sdrReader("Data_Type").ToString()
                                                    Case "varchar"
                                                        prm.SqlDbType = SqlDbType.VarChar
                                                        prm.Size = CInt(sdrReader("Character_Maximum_Length"))
                                                    Case "datetime"
                                                        prm.SqlDbType = SqlDbType.DateTime
                                                    Case "money"
                                                        prm.SqlDbType = SqlDbType.Money
                                                    Case "int"
                                                        prm.SqlDbType = SqlDbType.Int
                                                    Case "table type"
                                                        prm.SqlDbType = SqlDbType.Structured
                                                        prm.TypeName = "dbo. " & sdrReader("User_Defined_Type_Name").ToString()
                                                        'prm.SqlDbType = SqlDbType.Udt
                                                        'prm.UdtTypeName = "dbo. " & sdrReader("User_Defined_Type_Name").ToString()
                                                End Select
                                                cmd.Parameters.Add(prm)
                                            End With
                                        End If
                                    Loop
                                    sdrReader.Read()
                                End If
                            Loop While sdrReader.NextResult
                        End Using
                    End Using
                    Dim prc(cmd.Parameters.Count - 1) As SqlParameter
                    For x As Integer = 0 To cmd.Parameters.Count - 1
                        prc(x) = CType(CType(cmd.Parameters(x), ICloneable).Clone(), SqlParameter)
                    Next
                    g_prmSP.Add(cmd.CommandText, prc)
                End If
            Catch ex As Exception
                blnError = True
                errorText = ex.Message.Replace("""", "'").Replace("\", "\\")
                LogOutput("DetermineParameters: " & cmd.CommandText & " " & ex.Message)
            Finally
                If blnDoClose Then
                    If dcn.State = ConnectionState.Open Then dcn.Close()
                End If
            End Try
            If blnWriteItBack Then
                .Item("g_prmSP") = g_prmSP
                .UnLock()
            End If
        End With
    End Sub

    *** Read the sticky in the DB forum about how to get your question answered quickly!! ***

    Please remember to rate posts! Rate any post you find helpful - even in old threads! Use the link to the left - "Rate this Post".

    Some Informative Links:
    [ SQL Rules to Live By ] [ Reserved SQL keywords ] [ When to use INDEX HINTS! ] [ Passing Multi-item Parameters to STORED PROCEDURES ]
    [ Solution to non-domain Windows Authentication ] [ Crazy things we do to shrink log files ] [ SQL 2005 Features ] [ Loading Pictures from DB ]

    MS MVP 2006, 2007, 2008

  12. #12
    Superbly Moderated NeedSomeAnswers's Avatar
    Join Date
    Jun 2002
    Location
    Manchester uk
    Posts
    2,660

    Re: Server Side Storage

    Another thing I've seen, which surprises me, is that session state can only be strings or integers. MS appears to recommend that I turn my map into a JSON object, store that to session state and restore from that as needed. I can do that, I'm just surprised to find that I need to.
    Yes I have done exactly this recently, You cant store complex objects in the session, however it is pretty easy to do by serialising your objects and storing JSON. I use a couple of helper methods to get and set the session variables

    Code:
            public static void SaveSessionVariable(HttpContext HttpContext, string item, string value)
            {
                HttpContext.Session.Set(item, Encoding.ASCII.GetBytes(value));
            }
    
            public static string GetSessionVariable(HttpContext httpContext, string name)
            {
                try
                {
                    httpContext.Session.TryGetValue(name, out byte[] bytes);
    
                    if (bytes == null)
                        bytes = new byte[0];
    
                    return Encoding.ASCII.GetString(bytes);
                }
                catch
                {
                    return "";
                }
            }
    Here is it in use saving and retrieving a list of objects

    Code:
    HelperFunctions.SaveSessionVariable(HttpContext, "BookingPaymentsCharges", JsonSerializer.Serialize(BookingPaymentsChargesList));
    Code:
    BookingCase.BookingPaymentsChargesList = JsonSerializer.Deserialize<List<BookingPaymentsChargesBo>>(HelperFunctions.GetSessionVariable(HttpContext, "BookingPaymentsCharges"));
    I am using the Json.Text Serialiser built into Asp.Net core rather than NewtonSoft here.
    Please Mark your Thread "Resolved", if the query is solved & Rate those who have helped you



Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width