Server Side Storage

I'm new to ASP.NET Core, and fairly new to web development in general. This leaves me a bit unfamiliar with web concepts, and best practices, hence this question:

I am working on a program where some models will be stored in a database. The models have a variety of data, much of which may be needed by any particular user. The model could be seen as:

Class P holds one Class U, one or more class B, zero or more class S, zero or more class N and maybe something else. In a desktop environment handling the same thing, I would load Class P such that all the members of P are available and accessible via P. In the web environment, there could be multiple users, and each will have zero or more P associated with them. When the user first logs in, they will see the set of P that are associated with them, but more importantly, they will see a name describing P, which is made up of the U, B, S, N, and other elements that are members of P.

It would be entirely possible for the user upon login to get the set of P that they have, then when they want to see/edit one of the P that they have, they could submit the ID for that P, and that P would be loaded up for them, but while stateless, this seems horribly inefficient. Every time they go to a certain page, the server would be acting like they had never been there before, and get all the P for them from the database.

What I'd prefer to do is to have a PM class that holds all the P objects for each user, and have that PM class persist for as long as that user is still working with the site. I could also have a UM class (User Manager) that would be essentially a dictionary of PM by User. This may be a bad idea, though, because that object could persist on the server for the life of the running application. It would be efficient from a memory standpoint, as there would be no need to load the PM for the user if it was already in the dictionary, but it would be sitting there in memory even when the site was not being used by anybody (the total number of users is likely less than 100, and they would likely use the site for brief bursts roughly once per year).

If I were to persist that UM dictionary object, it looks like I could use the Cache, but am I right in thinking that would be bad form?

If I just persist the PM class, that could exist for as long as the user kept coming back to the site. It looks like SessionState would be appropriate for this, with the understanding that the SessionState could be dispensed with by the server, so I would have to write code to make sure that the PM stored in the SessionState existed, and reload the PM if it had gone away.

Does that sound right, and if not, what should I be doing?

I'll admit you got me lost with the B S Z thing.

But session and cookies are what options I see fit. Not cache.
You can set a session timeout or a cookie timeout depending on the situation. Or I think you can maintain a value on a hidden control but that seems like a bad idea.
If you are using asp.net authentication then the things are more robust, a login user cannot access another users data(it can if the design is not proper but I'm talking about proper authentication).
If you are not using asp.net authentication and using a database and you want to go back and forth to check who is calling what (depending on probably a random assigned value on session or encrypted cookie) then you be better of doing this with javascript and web methods (jquery ajax or I bet other newer javascript platform support that) to avoid server Postbacks.
Ah, by UM, you mean have the users persist on the server....Hmmm... I guess, since the data is on the server and cannot be located on the page that should be OK but how would you persist the data on the server?
Probably with a session or you will have to reload the page every time for changes. So in that situation, for me, it would be better to to use an encrypted cookie or to use a web method that will server the UM but, it's been ages since I dealt with authentication so there may be better concepts.

Edit. I've read a little more on ASP.NET MVC as I'm using simple asp.net. It supposed to make better use of the cache so if you are using 4.0 and up you can go that way also on the dictionary.
Just remember that using cache pushes data to the server memory, so you need to make sure that it won't bloat the IIS. There are some settings you can specify on the IIS and more specifically on how much memory it will recycle but if you are not caching huge amount that won't matter (as I recall IIS application pools recycle at max server memory) but on huge amount, an IIS recycle will lose you all the cached data.
Also notice that sessions timeout at some point (change settings on IIS) . I believe is either 20 or 30 minutes. After that you loose the session.

What I do not want is all the data going to the client on every request.

I'm trying to avoid needing to re-load a complex map of objects on each request to the server. In database terms, the user will be interacting with a permit. That permit would be one record in a DB table, but it would include one or more child records from each of several other tables. In a stateless system, every time the client requests something, the server is supposed to treat that request as if it was independent of any other, as I understand it. However, that would mean loading all that data from the database, and constructing the entire map of objects, for every single request.

Any one request might require only bits and pieces of the total map of objects, so I might load just what is needed for each request, but that just seems like a really bad idea from a performance perspective.

Another thing I've seen, which surprises me, is that session state can only be strings or integers. MS appears to recommend that I turn my map into a JSON object, store that to session state and restore from that as needed. I can do that, I'm just surprised to find that I need to.

You can't avoid re loading as the session object is server side on standard asp. Will you be using MVC or asp?
Ajax calls is to "trick" the page not to postback but eventually you will have to grab the info from the server, even if no reloading "flickering: is happening.
That is why I suggested a cookie, BUT, cookie max size is 4096 bytes. If it is a huge object then I'm note sure what will happen to the page.
In MVC you can use localstrorage or sessionstorage , I'm not the least sure about size or speed limitation as I'm not using MVC but here is an article:
https://visualstudiomagazine.com/art...ving-data.aspx

Personally I would have created Ajax calls and bring data continuously as the page progress. I wouldn't suggest "on big push" of data on the page. You will may find out later that the page will is very slow to respond.
If you need a "table" data system, there are JS solution (szlamany) have utilized many of them, that do not cram the data.
A new thread on AJAX, or JS thread would be advised as szlamany could help further.
(note that this is called, xosimo in Greek, aka "pushing" ορ "jab" in English , is when you suggest another person for the work without him/her knowing it but szlamany shouldn't brag so much for his skills. Now he get's the "jab" ! :bigyello: )

Didn't read through all the replies... got through the main post though... here's my initial thoughts:
On the server: cache the user's objects... there's no harm in that. If the data doesn't change, except when pushed by the user, cache it. But, don't send all of it to the client. Only send the P that the user is actively working on to the client. Then in the client, store the P object in the state, and work with it there. If you need to persist the changes back to the server, then either send it back, or only send the deltas, but it should be one-way, upon request.

Hope that made some sense.
In short, on the server you cache PM, and in the client you cache P, with the occasional trip to the server to deliver updates from P as needed. If the user then needs a new P, for context switching, that would then be the only time to send back the entire P back from the server (where it was cached in the PM) other than the initial one.

-tg

Just to be clear.
Session state that tech suggests, still stores data to the server, there is only an ID stored to the client so it can identify it's session equivalent when called.
So you are not saving any server memory.
The question here is why you need full data to the client and not calls to the server.
As suggested there are other ways than bloating the full data of P, particular if you are requesting it through session, you are calling the server to retrieve the data anyhow.
I'm under the impression that there are some caching mechanisms when you request the session data for the 2nd time from the server to lower the load but I haven't looked into that.

There are several methods on the server to hold data - I use the APPLICATION pool mostly. Since I need that pool to persist data, I trap the APPLICATION end event and "write" that pool to a text file. When the APPLICATION start event fires, I re-populate the APPLICATION pool. Not sure if all that "management" work is done my MS if you are using MVC...

If you use the SESSION pool instead you will need to handle the SESSION start and end events in a similar fashion if you want to persist over those moments.

I use only AJAX - so my page never re-loads. I'll have a user logged in for several days, as long as I persist that APPLICATION pool successfully, I've got their security GUID and a ton of facts about them (alerts and multi-user controls and what not).

With all that said, you have to be careful of how much you store in any of these memory pools. Read online about this before you store lots of data.

It is not uncommon for my web app to pre-load all the addresses in a city on the initial page load. Browsers do not freak out about tons of data - I can have 30,000 addresses and client side performance is stellar. With all those addresses loaded, for instance, I can allow the user to find an address and drill down to the "next" level in the maintenance experience. The JS code to REGEX search these "arrays" is incredibly fast - making the user experience a really good one.

The "web world" might be stateless, but us programmers are responsible for giving it "state".

Quote:

---

Originally Posted by techgnome

Didn't read through all the replies... got through the main post though... here's my initial thoughts:
On the server: cache the user's objects... there's no harm in that. If the data doesn't change, except when pushed by the user, cache it. But, don't send all of it to the client. Only send the P that the user is actively working on to the client. Then in the client, store the P object in the state, and work with it there. If you need to persist the changes back to the server, then either send it back, or only send the deltas, but it should be one-way, upon request.

Hope that made some sense.
In short, on the server you cache PM, and in the client you cache P, with the occasional trip to the server to deliver updates from P as needed. If the user then needs a new P, for context switching, that would then be the only time to send back the entire P back from the server (where it was cached in the PM) other than the initial one.

-tg

---

I'm sending less than one object to the client. All the client gets is the portion of the model that applies to the page in question. Using the in-memory cache in ASP.Net Core is looking like the way to go.

@Sapator: I'm not trying to save server side memory, I'm looking to spend a bit of server side memory to speed up server side responses by not needing to go back to the database on every request. I'm not sending everything to the client, just what is needed on the page, but what is needed on the page is some portion of a semi-complex model, and I don't want the server to be loading that model on every request.

@szlamany: Are you using ASP.NET Core or something a bit older? I seem to remember encountering some mention that Core dropped session state events, though I could very well be misremembering that.

I am not using ASP.NET Core - I'm just using VS 2019 against IIS with .Net 3.5 (I'm using old SDK's for things like OpenXML).

I have a Global.asax file that VS gave me that has all kinds of SUB's for the various APPLICATION and SESSION events. I do nothing with SESSION events - do it all with APPLICATION events. Although I did recently put some logging of an HTTPS SSL check in the Session_Start() event.

Code:

---

    Sub Session_Start(ByVal sender As Object, ByVal e As EventArgs)
        LogOutput("Session Start")
        LogOutput("...HttpContext.Current.Request.IsSecureConnection=" & HttpContext.Current.Request.IsSecureConnection.ToString())
        ' Code that runs when a new session is started
    End Sub

    Sub Session_End(ByVal sender As Object, ByVal e As EventArgs)
        LogOutput("Session End")
        ' Code that runs when a session ends.
        ' Note: The Session_End event is raised only when the sessionstate mode
        ' is set to InProc in the Web.config file. If session mode is set to StateServer
        ' or SQLServer, the event is not raised.
    End Sub

---

In the APPLICATION pool I store session guids and manage users - my LOGIN web method drops the info into the APPLICATION pool and subsequent AJAX calls check for the guid.

Code:

---

With Application
    .Lock()
    If .Item("g_GUIDS") IsNot Nothing Then
        g_GUIDS = DirectCast(.Item("g_GUIDS"), Dictionary(Of String, Dictionary(Of String, String)))
    End If
    g_GUIDS.Add(strMessage, New Dictionary(Of String, String))
    g_GUIDS(strMessage).Add("status", "success")
    g_GUIDS(strMessage).Add("logindate", Date.Now.ToShortDateString)
    g_GUIDS(strMessage).Add("logintime", Date.Now.ToLongTimeString)
    .Item("g_GUIDS") = g_GUIDS
    .UnLock()
End With

---

I also store SQL SPROC parameters in the APPLICATION pool so I don't have to keep going to the database for them. Otherwise I keep storage in the pool at a minimum.

Code:

---

Private Sub DetermineParameters(ByRef dcn As SqlConnection, ByRef cmd As SqlCommand, Optional ByRef errorText As String = "")
    Dim blnError As Boolean = False
    Dim blnDoClose As Boolean = False
    Dim blnWriteItBack As Boolean = False
    With Application
        .Lock()
        If .Item("g_prmSP") IsNot Nothing Then
            g_prmSP = DirectCast(.Item("g_prmSP"), Dictionary(Of String, SqlParameter()))
        End If
        Try
            If g_prmSP.ContainsKey(cmd.CommandText) Then
                .UnLock()
                Dim prm As New SqlParameter
                For Each prmSP As SqlParameter In g_prmSP(cmd.CommandText)
                    prm = CType(CType(prmSP, ICloneable).Clone(), SqlParameter)
                    cmd.Parameters.Add(prm)
                Next
            Else
                blnWriteItBack = True
                Using cmd2 As New SqlCommand
                    cmd2.CommandType = CommandType.StoredProcedure
                    cmd2.CommandText = "dbo.awcGetSP"
                    cmd2.Connection = dcn
                    If Not cmd.CommandText.StartsWith("dbo") AndAlso cmd.CommandText.Contains(".") Then
                        cmd2.Parameters.AddWithValue("@StoredProc", cmd.CommandText.Substring(cmd.CommandText.LastIndexOf(".") + 1))
                    Else
                        cmd2.Parameters.AddWithValue("@StoredProc", cmd.CommandText)
                    End If
                    If dcn.State <> ConnectionState.Open Then
                        dcn.Open()
                        blnDoClose = True
                    End If
                    Dim blnDidSetup As Boolean = False
                    Dim blnGotPassConnId As Boolean = False
                    Dim cntParams As Integer = 0
                    Using sdrReader As SqlDataReader = cmd2.ExecuteReader
                        Do
                            If Not blnDidSetup Then
                                blnDidSetup = True
                            Else
                                Do While sdrReader.Read()
                                    If sdrReader("Parameter_Name").ToString = "@PassConnId" Then
                                        blnGotPassConnId = True
                                    End If
                                    If blnGotPassConnId Then
                                        cntParams += 1
                                        If cntParams > 9 Then
                                            blnGotPassConnId = False
                                        End If
                                    End If
                                    If blnGotPassConnId Then
                                        If cntParams = 1 Then
                                            cmd.Parameters.Add(New SqlParameter("@PassConnId", SqlDbType.Int)).Value = 0
                                            cmd.Parameters.Add(New SqlParameter("@RetStat", SqlDbType.Int, 0, ParameterDirection.InputOutput, True, 0, 0, "", DataRowVersion.Current, 0))
                                            cmd.Parameters.Add(New SqlParameter("@RetText", SqlDbType.VarChar, 100, ParameterDirection.InputOutput, True, 0, 0, "", DataRowVersion.Current, ""))
                                            cmd.Parameters.Add(New SqlParameter("@RetMode", SqlDbType.Int, 0, ParameterDirection.InputOutput, True, 0, 0, "", DataRowVersion.Current, 0))
                                            cmd.Parameters.Add(New SqlParameter("@RetGrid", SqlDbType.Int, 0, ParameterDirection.InputOutput, True, 0, 0, "", DataRowVersion.Current, 0))
                                            cmd.Parameters.Add(New SqlParameter("@RetExtra", SqlDbType.VarChar, 100, ParameterDirection.InputOutput, True, 0, 0, "", DataRowVersion.Current, ""))
                                            cmd.Parameters.Add(New SqlParameter("@PassMode", SqlDbType.Int)).Value = 0
                                            cmd.Parameters.Add(New SqlParameter("@PassExtra", SqlDbType.VarChar, 100)).Value = ""
                                            cmd.Parameters.Add(New SqlParameter("@PassTest", SqlDbType.VarChar, 100)).Value = ""
                                        End If
                                    Else
                                        Dim prm As New SqlParameter
                                        With prm
                                            prm.ParameterName = sdrReader("Parameter_Name").ToString
                                            Select Case sdrReader("Data_Type").ToString()
                                                Case "varchar"
                                                    prm.SqlDbType = SqlDbType.VarChar
                                                    prm.Size = CInt(sdrReader("Character_Maximum_Length"))
                                                Case "datetime"
                                                    prm.SqlDbType = SqlDbType.DateTime
                                                Case "money"
                                                    prm.SqlDbType = SqlDbType.Money
                                                Case "int"
                                                    prm.SqlDbType = SqlDbType.Int
                                                Case "table type"
                                                    prm.SqlDbType = SqlDbType.Structured
                                                    prm.TypeName = "dbo. " & sdrReader("User_Defined_Type_Name").ToString()
                                                    'prm.SqlDbType = SqlDbType.Udt
                                                    'prm.UdtTypeName = "dbo. " & sdrReader("User_Defined_Type_Name").ToString()
                                            End Select
                                            cmd.Parameters.Add(prm)
                                        End With
                                    End If
                                Loop
                                sdrReader.Read()
                            End If
                        Loop While sdrReader.NextResult
                    End Using
                End Using
                Dim prc(cmd.Parameters.Count - 1) As SqlParameter
                For x As Integer = 0 To cmd.Parameters.Count - 1
                    prc(x) = CType(CType(cmd.Parameters(x), ICloneable).Clone(), SqlParameter)
                Next
                g_prmSP.Add(cmd.CommandText, prc)
            End If
        Catch ex As Exception
            blnError = True
            errorText = ex.Message.Replace("""", "'").Replace("\", "\\")
            LogOutput("DetermineParameters: " & cmd.CommandText & " " & ex.Message)
        Finally
            If blnDoClose Then
                If dcn.State = ConnectionState.Open Then dcn.Close()
            End If
        End Try
        If blnWriteItBack Then
            .Item("g_prmSP") = g_prmSP
            .UnLock()
        End If
    End With
End Sub

---

Quote:

---

Another thing I've seen, which surprises me, is that session state can only be strings or integers. MS appears to recommend that I turn my map into a JSON object, store that to session state and restore from that as needed. I can do that, I'm just surprised to find that I need to.

---

Yes I have done exactly this recently, You cant store complex objects in the session, however it is pretty easy to do by serialising your objects and storing JSON. I use a couple of helper methods to get and set the session variables

Code:

---

        public static void SaveSessionVariable(HttpContext HttpContext, string item, string value)
        {
            HttpContext.Session.Set(item, Encoding.ASCII.GetBytes(value));
        }

        public static string GetSessionVariable(HttpContext httpContext, string name)
        {
            try
            {
                httpContext.Session.TryGetValue(name, out byte[] bytes);

                if (bytes == null)
                    bytes = new byte[0];

                return Encoding.ASCII.GetString(bytes);
            }
            catch
            {
                return "";
            }
        }

---

Here is it in use saving and retrieving a list of objects

Code:

---

HelperFunctions.SaveSessionVariable(HttpContext, "BookingPaymentsCharges", JsonSerializer.Serialize(BookingPaymentsChargesList));

---

Code:

---

BookingCase.BookingPaymentsChargesList = JsonSerializer.Deserialize<List<BookingPaymentsChargesBo>>(HelperFunctions.GetSessionVariable(HttpContext, "BookingPaymentsCharges"));

---

I am using the Json.Text Serialiser built into Asp.Net core rather than NewtonSoft here.