Quote Originally Posted by FWinds View Post
Let's just say that each row is an "agent" or "individual" and the columns are several attributes of that individual that can change, depending on the elapsed time, interactions with other individuals or due to an event happening. It is similar to a simulation and yes, all data are relevant at all times and may change at every iteration.
Reads like an "online-game"-scenario (with that amount of "2Mio. individuals, which can possibly interact").

So maybe we should get a bit more concrete - and look at the problem from an (Online-Clients) side.

A certain Client (one of the individuals) will have "a scene" on the screen -
and this scene involves coordinates which place it in the "game-world" ...

Now the entities the client-avatar in the game can "interact with" are limited already
by those scene-coordinates.

So you will only need to consider other "objects", which are "near the given coordinates" -
and perform appropriate (fast, indexed) Range-Selects against your "players-table" -
against your "monsters-table" - and against your "mystical-objects-table" - and select
only those, which are "near the scene-coords" - and you will end up with perhaps
10 or 20 entities of each of the three groups.

Now the interaction among those 20 other individuals, 20 monsters and 20 mystical objects
is quite possible to handle with a certain amount of CPU-Power on the client (even per JavaScript).

If your simulation is not like in the above scenario, you can surely find other "limiting factors"
by which you could reduce the amount of "interacting entities", which are considered
"to be able to interact at all".

Olaf